Tutorial: Web data collection with Python

This tutorial is based on the content in the GESIS fall seminar Automated Web Data Collection with Python in 2023 and has two parts. In the first part we discuss the use of Web API as data source and use the MediaWiki API which powers Wikipedia as an example. In the second part we discuss how to collect data from static web pages with Python. There are lecture units and corresponding exercises with solutions for each part.

Table of content

• Part 1 - Wikipedia

  • Lecture 1 - MediaWiki API
  • Lecture 2 - Python packages for Wikipedia
  • Exercise 1 - MediaWiki API, solution
  • Exercise 2 - Python packages for Wikipedia, solution

• Part 2 - Static web scraping

  • Lecture 1 - Static web scraping 1
  • Lecture 2 - Static web scraping 2
  • Lecture 3 - Static web scraping 3
  • Exercise 1 - Static web scraping 1, solution
  • Exercise 2 - Static web scraping 2, solution
  • Exercise 3 - Static web scraping 3, solution