Free projects you could work on
February 12, 2024 ยท View on GitHub
- End-to-end Uber Data engineering project with BigQuery
- Build a pipeline with LLMs
- Build a SQL query engine with LLMs and LangChain
- Extract Metadata from Youtube Videos in AWS with Free Tier
- Building a Practical Data Engineering Project: Web-scraping with real-estates, uploading them to S3, Spark and Delta Lake, adding Data Science with Jupyter, ingesting into Druid, dataviz with Superset and managing everything with Dagster.
- GitHub Project and DevOps
- An End to End Azure Data Engineering Real Time Project Demo: The use case for this project is building an end to end solution by ingesting the tables from on-premise SQL Server database using Azure Data Factory and then store the data in Azure Data Lake. Then Azure databricks is used to transform the RAW data to the most cleanest form of data and then we are using Azure Synapse Analytics to load the clean data and finally using Microsoft Power BI to integrate with Azure synapse analytics to build an interactive dashboard. Also, we are using Azure Active Directory (AAD) and Azure Key Vault for the monitoring and governance purpose.