In this talk, we'll focus on a hands-on example that illustrates setting up a CDC pipeline. Specifically, we'll demonstrate how to capture database changes from PostgreSQL in an upstream Python application, then stream these changes to a downstream Python application for immediate processing, utilizing Debezium for change detection and RabbitMQ for message queuing.

Attendees will leave with a practical guide on leveraging Python, PostgreSQL, Debezium, and RabbitMQ to create a robust CDC pipeline, empowering their analytics platforms to deliver insights with minimal latency.

This is what the outline would look like:

1. Introduction

• Briefly introduce the concept of real-time analytics and its importance in modern business operations.
• Highlight the role of Change Data Capture (CDC) in achieving real-time data processing.

2. Understanding Change Data Capture (CDC)

• Define CDC, discuss some of its benefits and practical use-cases.
• Explain how CDC works at a high level and demonstrate what typical CDC pipeline would look like.

3. Introducing the Demo CDC Pipeline

• Introduce the demo Python application we will be working with in our CDC pipeline
• Show a complete illustration of our CDC pipeline with all the different components pieced together: Python applications, PostgreSQL, Debezium server, and RabbitMQ
• Briefly introduce Debezium server, RabbitMQ and roles they play in our CDC pipeline

4. Running the CDC pipeline

• Here, participants get to see CDC in action — we will run the demo pipeline,

5. Conclusion

• A recap of everything