data-processing-pipelines

Scalable data pre processing and curation toolkit for LLMs

Visual AI development framework for training and inference of ML models, scaling pipelines, and automating workflows with Python

convtools is a specialized Python library for dynamic, declarative data transformations with automatic code generation

A Python package & CLI tool that assists users create automated ETL pipelines for various types of real-time and archived weather data & supports GRIB2 -> NETCDF. WxData can access 20+ Weather Models, Observations (METAR & RAWS), Radar Data, Observed Soundings, Fuels Data and interfaces for Open-Meteo and AirNow APIs.

Making it easier to navigate and clean TAHMO weather station data for ML development

Streamlit app to export and bulk-update Plex music metadata and smart playlist creator.

A simplistic, general purpose pipeline framework.

Artifician is an event-driven framework designed to simplify and accelerate the process of preparing datasets for Artificial Intelligence models.

A pipeline that consumes twitter data to extract meaningful insights about a variety of topics using the following technologies: twitter API, Kafka, MongoDB, and Tableau.

Understanding the customer life cycle Acquiring customer data Applying big data concepts to your customer relationships Finding high propensity prospects Upselling by identifying related products and interests Generating customer loyalty by discovering response patterns Predicting customer lifetime value (CLV) Identifying dissatisfied customers …

Homework assignments for MFF UK course NDBI046 - Introduction to Data Engineering

An open-source Python library for processing and developing End-to-End AI pipelines for Time Series Analysis

Notebooks from finance, general practice and Jovian courses on data analysis, ML and DL

The Resume Application Tracking System uses Google Gemini Pro Vision to automatically parse, analyze, and categorize resumes for efficient recruitment. It integrates AI-driven vision capabilities to enhance resume processing and candidate selection.

A collection of workflows (R, python, bash) for retrieving and processing data from databases

Data Engineering & Software Blog

🎢 IaaS visual editor to create & deploy data processing pipelines - python, rmq, react, meteorjs

Interactive dashboard comparing ARIMA, Prophet, LSTM, and Temporal Fusion Transformer (TFT) on stock time-series data. Includes uncertainty intervals and RMSE/MAPE metrics. Built with Streamlit, PyTorch, and Prophet to explore where classical models excel and where deep learning adds value.

Automated, scalable data extraction solution transforming unstructured web info into structured, business-ready datasets.

Successfully established a machine learning model using PySpark which can accurately classify whether a bank customer will churn or not up to an accuracy of more than 86% on the test set.