Data Warehouse using Airflow
16346
portfolio_page-template-default,single,single-portfolio_page,postid-16346,bridge-core-3.0.8,qode-quick-links-1.0,qode-page-transition-enabled,ajax_fade,page_not_loaded,,qode-child-theme-ver-1.0.0,qode-theme-ver-29.5,qode-theme-bridge,disabled_footer_bottom,qode_advanced_footer_responsive_1000,qode_header_in_grid,wpb-js-composer js-comp-ver-6.10.0,vc_responsive

Data Warehouse using Airflow

Client

Category
Data engineering
About This Project

In cooperation with Lumosity, we improved their Data warehousing by introducing Airflow. The main goal of this project was to write, schedule and maintain ETLs for a part of a large data warehouse. These ETLs pull data from multiple information sources, transform it, usually store it to storage web service and write it to the warehouse database. To simplify deployment and speed up the overall set up, all components are packaged in the container.