Harvesting data about a domain using the IA CDX API
Works with IA
In this notebook we'll look at how we can get domain level data from the IA CDX API. In most other notebooks using the CDX API we've harvested data into memory and then saved to disk later on. Because we're potentially harvesting much larger quantities of data, we're going to reverse this and save harvested data to disk as we download it.
Sherratt, Tim & Jackson, Andrew. (2022). GLAM-Workbench/web-archives (version v1.1.0). Zenodo. https://doi.org/10.5281/zenodo.6450762