Explore stock prices with Spark SQL

Overview

Class Central Tips

In this 1-hour long project-based course, you will learn how to interact with a Spark cluster using Jupyter notebook and how to start a Spark application. You will learn how to utilize Spark Resisilent Distributed Datasets and Spark Data Frames to explore a dataset. We will load a dataset into our Spark program, and perform analysis on it by using Actions, Transformations, Spark DataFrame API and Spark SQL. You will learn how to choose the best tools to use for each scenario. Finally, you will learn to save your results in Parquet tables.

Syllabus

Explore stock prices with Spark SQL

Welcome to this project-based course on Exploring stock prices with Spark SQL! In this project, you will learn the basics of distributed programming using Spark and you will learn how to derive knowledge from data in an interactive way. This is a great hands-on experience to interact with Spark. You will learn how to optimally load data for analysis, and how you can explore it by using Spark RDD, Spark DataFrames. By the end of this project, you will be able to explore and perform statistical analysis on Stock prices datasets using Apache Spark SQL and Spark DataFrame API. Learners will be able to create parquet tables and store their results in them.