Exploring subdomains in the whole of gov.au

Works with IA

Most of the notebooks in this repository work with small slices of web archive data. In this notebook we'll scale things up a bit to try and find all of the subdomains that have existed in the gov.au domain. As in other notebooks, we'll obtain the data by querying the Internet Archive's CDX API. The only real difference is that it will take some hours to harvest all the data. Once we have the data we'll do some analysis, and visualise the domain hierarchy as a dendrogram.

Run live on Binder

Other options

• Download from GitHub
• View using NBViewer

Related resources

• Harvest of unique urls from the gov.au domain
• Unique subdomains of gov.au split into components
• Unique subdomains of gov.au in SURT format
• Circular dendrograms of gov.au subdomains

Additional documentation

• Run these notebooks

Getting help

• Ask a question at OzGLAM Help
• Report a problem
• Book an appointment

Cite as

Sherratt, Tim; Jackson, Andrew & Bickford, Jake. (2023). GLAM-Workbench/web-archives (version v1.2.0). Zenodo. https://doi.org/10.5281/zenodo.7898218

Section sponsor

The Web Archives section of the GLAM Workbench is sponsored by the British Library.