This course on Python web scraping using Scrapy aims to teach learners how to scrape websites at scale. By the end of the course, students will be able to create Scrapy spiders, crawl websites, scrape data, clean data using Items & Item Pipelines, save data to various formats, use fake user-agents and proxies to avoid blocking, and deploy scrapers to the cloud. The course is designed for beginners and covers essential skills for web scraping using Python and Scrapy.
Overview
Syllabus
⌨️ Part 1 - Scrapy & Course Introduction
⌨️ Part 2 - Setup Virtual Env & Scrapy
⌨️ Part 3 - Creating a Scrapy Project
⌨️ Part 4 - Build your First Scrapy Spider
⌨️ Part 5 - Build Discovery & Extraction Spider
⌨️ Part 6 - Cleaning Data with Item Pipelines
⌨️ Part 7 - Saving Data to Files & Databases
⌨️ Part 8 - Fake User-Agents & Browser Headers
⌨️ Part 9 - Rotating Proxies & Proxy APIs
⌨️ Part 10 - Run Spiders in Cloud with Scrapyd
⌨️ Part 11 - Run Spiders in Cloud with ScrapeOps
⌨️ Part 12 - Run Spiders in Cloud with Scrapy Cloud
⌨️ Part 13 - Conclusion & Next Steps
Taught by
freeCodeCamp.org
