Scalable Big Data Platform Development - Vietnam
2023
Developed and maintained a highly scalable and extensible Big Data platform aimed at enabling the collection, storage, modeling, and analysis of massive data sets from multiple channels. Created and upheld scalable and reliable data pipelines for ingesting data from diverse data sources into the Data Lake. Ensured data adherence to quality standards and optimal formats, facilitating swift access for downstream users. Developed both big data and batch/real-time analytical solutions leveraging emerging technologies.
Develop and maintain a highly scalable and extensible Big Data platform enabling collection, storage, modeling, and analysis of massive data sets from various channels.
Develop and maintain scalable and reliable data pipelines to ingest data from diverse data sources into the Data Lake.
Ensure adherence to data quality standards and guarantee quick access to data for downstream users.
Develop and enable big data and batch/real-time analytical solutions utilizing emerging technologies.
Technologies: Python, Data Vault 2.0, FastAPI, FlaskAPI, Scala, SQL, Big Data Ecosystem, Data Warehouse, Apache Hadoop, Apache Spark, Apache Kafka, Apache Airflow, Azure Synapses, Azure Databricks, AWS EC2, AWS Glue, AWS EMR, CICD, Jenkins, Docker, Linux Operating, PowerBI