A data structure that extends pyspark.sql.DataFrame with metadata information.
from metaframe import MetaFrame
mf = MetaFrame(df=df, metadata={"columns": ["a", "b"]})
mf = mf.withColumn("new_col", F.lit(1))
mf = mf.set_metadata(columns=["a", "b", "new_col"])
mf.show()
assert mf.metadata["columns"] == ["a", "b", "new_col"]
ETL Pipeline with Airflow, Spark, s3, MongoDB and Amazon Redshift
Intake: A general interface for loading data Intake is a lightweight set of tools for loading and sharing data in data science projects. Intake helps
Using Streaming Twitter Data with Kafka and Spark Reading streams of Twitter data, publishing them to Kafka topic, process message using Kafka Stream
Epistemic Model Checker This is a model checker for verifying properties in epistemic models. The goal of the model checker is to check for Pluralisti
similarpanda A module to check for differences between pandas Dataframes, and generate a report in Excel format. This is helpful in a workplace settin
UC Module Python Package How To Install clone repo. cd UC-module pip install . How to Use uc.module.UC(measurment=str, dark=str, reference=str, heade
sportsdataverse-py See CHANGELOG.md for details. The goal of sportsdataverse-py is to provide the community with a python package for working with spo
Predictive Modeling & Analytics on Home Equity Line of Credit Data (Python) HMEQ Data Set In this assignment we will use Python to examine a data set
My group and I took Covid-19 outbreak statistics from ontario, and analyzed them to find different patterns and future predictions for the virus
BinTuner is a cost-efficient auto-tuning framework, which can deliver a near-optimal binary code that reveals much more differences than -Ox settings. it also can assist the binary code analysis rese
WeRateDogs Twitter Data from 2015 to 2017 Udacity - Data Analyst Nanodegree - Project 4 - Wrangle and Analyze Data Table of Contents Introduction Proj
tinder_likes_data Gathering data of Likes Sent on Tinder within the past 7 days. Versions November 25th, 2021 - Functionality to get the name and age
Transform application Lifecycle Data and Design and ETL pipeline architecture for ingesting data from multiple sources to redshift This project is composed of two parts: Part1 and Part2
TheMachineScraper 🐱👤 is a tool made purely for analysing machine data for any reason.
A set of procedures that can realize covid19 virus detection based on blood.
Streaming Data Pipeline - Kafka + ELK Stack Streaming weather data using Apache Kafka and Elastic Stack. Data source: https://openweathermap.org/api O
npy-converter Matrix Data Converter. Expand matrix for multi-thread, multi-process Divid matrix for multi-thread, multi-process Support: .mtx, .npy, .
Candlestick-Pattern-Recognition-with-Python-and-TA-Lib Goal Look at the S&P500 to try and get a better understanding of these candlestick patterns and
Linked Data are becoming more popular for user-created metadata in HDF5 files.
Feature engineering and machine learning: together at last! Lambdo is a workflow engine which significantly simplifies data analysis by unifying featu
214 Jan 02, 2023
851 Jan 01, 2023
1 Dec 06, 2021
2 Dec 22, 2021
9 Sep 15, 2022
1 Oct 20, 2021
37 Dec 27, 2022
1 Jan 09, 2022
0 Jan 20, 2022
42 Dec 16, 2022
1 Jan 12, 2022
12 Jan 05, 2023
1 Jan 19, 2022
5 Dec 01, 2022
3 Mar 07, 2022
2 Jan 20, 2022
1 Feb 07, 2022
11 Oct 07, 2022
8 May 17, 2022
14 Sep 15, 2022