Explore how artificial intelligence is revolutionizing data engineering in this 16-minute conference talk from Conf42 DevOps 2025 presented by Srinivas Murri. Trace the evolution of data engineering from traditional methods to modern AI-powered approaches, understanding the limitations of conventional data pipelines and how AI addresses these challenges. Discover real-world case studies demonstrating real-time adaptability of smart data pipelines, the key technologies driving this transformation, and practical steps to implement AI in your data engineering workflows. The presentation concludes with insights into future trends, providing a comprehensive overview of how AI is creating more efficient, adaptive, and intelligent data processing systems.
Smart Data Pipelines: Revolutionizing Data Engineering with AI
Overview
Syllabus
00:00 Introduction and Speaker Background
00:21 Evolution of Data Engineering
02:22 Challenges in Traditional Data Pipelines
03:28 AI's Role in Transforming Data Pipelines
05:28 Real-Time Adaptability and Case Studies
10:44 Key Technologies Driving AI-Powered Pipelines
12:36 Getting Started with AI in Data Engineering
13:52 Future Trends and Conclusion
Taught by
Conf42
