Share Description
Web scraping has become a required skill for data analysts, developers and academics. This course focuses deeply on Scrapy (a popular and fast Python framework for web crawling) and Splash (a JavaScript rendering service for scraping dynamic websites). This course will provide you with the tools and information you need to collect data effectively and responsibly for personal projects, market research, job listings and business analytics. The course begins by providing an overview of HTTP, HTML structure and how the Scrapy framework operates. It then teaches you through the process of developing and operating your own web spiders, which crawl and extract data.
Topics Covered
- Introduction: Starting with a short introduction to web scraping and ethical scraping practices
- Scrapy Fundamentals: Understanding the HTML DOM and XPath/CSS selectors and setting up and configuring Scrapy projects.
- Project 1 Spiders: Creating and managing spiders and crawling through multiple pages (pagination).
- Building datasets: Data extraction and pipeline management with handling cookies, sessions, and headers
- Splash crash course: Scraping JavaScript-rendered content using Splash scripts for complex interactions.
- And many more topics to explore.
Who Will Benefit
- Python developers: Those who are looking to expand their skill set with data scraping.
- Data analysts and scientists: Who need to collect data from websites
- Digital marketers and SEO professionals: Professionals seeking competitive intelligence can also opt for this course.
- Researchers and students: Those who require automated data collection.
- Freelancers and entrepreneurs: Building data-driven applications or tools.
Why Take This Course
Manual data collection is time-consuming and error-prone. With the increased demand for web data, automated scraping skills are extremely valuable. This course will teach you not only how to use Scrapy and Splash, but also how to deal with the actual problems of scraping modern websites. By the end of the course, you'll be capable of creating strong, scalable web scrapers that can handle both static and dynamic sites. You will acquire practical knowledge, develop your Python skills and be ready to handle any data extraction work swiftly and responsibly.
