Class Central is learner-supported. When you buy through links on our site, we may earn an affiliate commission.

YouTube

Introducing the New Python Data Source API for Apache Spark

Databricks via YouTube

Overview

Explore the groundbreaking Python Data Source API for Apache Spark™ in this 27-minute conference talk by Databricks. Discover how this new API simplifies big data processing for Python developers, eliminating the need for Scala knowledge when integrating custom data sources into Spark. Learn about key features, including streamlined reading and writing operations, and understand how this innovation makes the big data ecosystem more accessible. Gain insights from Databricks engineers Allison Wang and Ryan Nienhuis, along with a customer co-presenter, as they discuss the API's impact on expanding Spark's reach within the Python community. Delve into additional resources like the Big Book of Data Engineering and The Data Team's Guide to the Databricks Lakehouse Platform to further enhance your understanding of data engineering concepts.

Syllabus

Introducing the New Python Data Source API for Apache Spark™

Taught by

Databricks

Reviews

Start your review of Introducing the New Python Data Source API for Apache Spark

Never Stop Learning.

Get personalized course recommendations, track subjects and courses with reminders, and more.

Someone learning on their laptop while sitting on the floor.