Modern Data Engineering and The Rise of Apache Airflow - Seguridad Mania.com - España y América Latina

Storage and compute are cheaper than ever. As a result, data engineering is undergoing a generational shift and is no longer defined by star-schema modeling techniques on data warehouses. Further, downstream operations are not just BI reporting and now include emerging use-cases such as data science. This means that modern day ETL tools should be dynamic, scalable, and extensible enough to handle complex business logic. Airflow provides that level of abstraction today’s Data Engineers need. The Qubole Data Platform provides single-click deployment of Apache Airflow, automates cluster and configuration management, and includes dashboards to visualize the Airflow Directed Acyclic Graphs (DAGs). In this webinar we will cover: - A brief Introduction to Apache Airflow and its optimal use cases - How to remove the complexity of spinning up and managing the Airflow cluster - How to Scale out horizontally with multi-node Airflow cluster - Real-world customer examples