PyCon Nigeria Annual Conference

Python-Powered DataOps: Streamlining Data Management and Automation

speaker-foto

Khairat Ayinde

Khairat is experienced working with data sets and passionate about leveraging data to drive business decisions and improve overall performance. She applies her expertise to provide insights and recommendations that drive business strategy, identifying key factors that drive predictive consumer behaviour and develop data-driven strategies to improve business performance. Overall, she looks to make an impact using technology and leading tech communities where individuals can connect, learn, and collaborate. Through organizing meetups, workshops, mentorship, and networking events, She aims to create a more inclusive and diverse landscape where everyone has equal opportunities to thrive.

Description

This talk dives into DataOps, a method combining data management with automation. We'll explore how Python, a powerful and readable language, can be used with libraries like Apache Airflow to build and automate DataOps pipelines, streamlining your workflow.

Abstract

In this session, we will explore the concept of DataOps and how it can be implemented using Python. DataOps is a methodology that combines data engineering, data integration, and data management with DevOps practices. It emphasizes automation, collaboration, and improvement throughout the data lifecycle.

Python, with its readability and extensive libraries, is a powerful tool for building DataOps pipelines.

  • Target Audience: intended for Python developers who are interested in learning more about DataOps and how to use Python to automate data management tasks.
  • The presentation will be a mix of lecture and code demonstration. I will be using slides to explain the concepts and code snippets to illustrate how to use Python libraries for DataOps tasks. The presentation will be interactive as well.
Audience level: Intermediate or Advanced