Exercise 1: Multi-Page Product Catalog Scraper
Create a scraper that: - Navigates through multiple pages of a product catalog - Extracts product name, price, description, and image URL - Handles pagination automatically - Saves data to both CSV and JSON formats - Implements rate limiting (1-2 seconds between pages)
Hint: Try scraping from a demo e-commerce site or use quotes.toscrape.com for practice.
Exercise 2: Search Form Scraper
Build a scraper that: - Submits search queries to a website's search form - Extracts all search results - Handles multiple search terms - Saves results with the search query as metadata
Hint: Use quotes.toscrape.com/search.aspx for practice.
Exercise 3: Data Pipeline
Create a complete data pipeline that: - Scrapes data from a website - Cleans and validates the data - Stores it in a database (SQLite) - Generates a summary report - Handles errors gracefully with retries
Exercise 4: Respectful Scraper with robots.txt
Build a scraper that: - Checks robots.txt before scraping - Respects crawl-delay directives - Implements exponential backoff for errors - Logs all requests and responses - Can be configured for different websites
Solutions
Try to solve these exercises yourself first! Solutions demonstrate best practices and can be found in the solutions directory (if provided).