Class Central is learner-supported. When you buy through links on our site, we may earn an affiliate commission.

DataCamp

Foundations of PySpark

via DataCamp

Overview

Master PySpark to handle big data with ease—learn to process, query, and optimize massive datasets for powerful analytics!

This course is perfect for data engineers, data scientists, and machine learning practitioners looking to work with large datasets efficiently. Whether you're transitioning from tools like Pandas or diving into big data technologies for the first time, this course offers a solid introduction to PySpark and distributed data processing.


Why Spark? Why Now?


Discover the speed and scalability of Apache Spark, the powerful framework designed for handling big data. Through interactive lessons and hands-on exercises, you'll see how Spark's in-memory processing gives it an edge over traditional frameworks like Hadoop. You'll start by setting up Spark sessions and dive into core components like Resilient Distributed Datasets (RDDs) and DataFrames. Learn to filter, group, and join datasets with ease while working on real-world examples.


Boost Your Python and SQL Skills for Big Data


Learn how to harness PySpark SQL for querying and managing data using familiar SQL syntax. Tackle schemas, complex data types, and user-defined functions (UDFs), all while building skills in caching and optimizing performance for distributed systems.


Build Your Big Data Foundations


By the end of this course, you'll have the confidence to handle, query, and process big data using PySpark. With these foundational skills, you'll be ready to explore advanced topics like machine learning and big data analytics.

Syllabus

  • Introduction to Apache Spark and PySpark
    • A General introduction to PySpark and distributed computing. This section introduces PySpark, PySpark DataFrames, and RDDs.
  • PySpark in Python
    • A continuation of DataFrames and complex datatypes. This section expands on what DataFrames offer in PySpark and introduces some Spark SQL concepts.
  • Introduction to PySpark SQL
    • Delve into leveraging Spark SQL and PySpark for scalable data processing, combining SQL's simplicity with PySpark's distributed computing power to handle large datasets efficiently.

Taught by

Ben Schmidt

Reviews

Start your review of Foundations of PySpark

Never Stop Learning.

Get personalized course recommendations, track subjects and courses with reminders, and more.

Someone learning on their laptop while sitting on the floor.