Automated Data Pipeline for E-commerce Analytics
2018 - 2020
This project involved automating the data pipeline to collect, process, and analyze sales data, providing real-time insights for the e-commerce platform.
Designed and developed a data pipeline using Apache Airflow.
Utilized Python and Pandas for data cleaning, transformation, and aggregation.
Integrated AWS S3 for data storage and Redshift for data warehousing.
Implemented data validation checks.
Developed visualizations and dashboards.
Collaborated with data analysts and business intelligence teams.