Trove People & Organisations

Current version: v1.0.0 ¶

Trove's People and Organisations zone aggregates information about individuals and organisations, bringing multiple sources together under a single identifier.

See below for information on running these notebooks in a live computing environment. Or just take them for a spin using Binder.

Harvesting data¶

There are two methods of harvesting the complete set of data from the People and Organisations zone – using the OAI-PMH API, or using the main Trove API in conjunction with the SRU interface. The OAI-PMH method is much faster, but includes duplicate records that you'll need to filter out afterwards.

Harvesting the complete set of data from the People and Organisations zone ¶

This notebook demonstrates the API/SRU method for creating a complete data harvest.

Harvesting the complete set of data from the People and Organisations zone using OAI-PMH ¶

This notebook demonstrates the OAI-PMH method for creating a complete data harvest.

Analysing data¶

Extract some aggregated data from the complete harvest ¶

The complete harvest of records in the Trove People & Organisations zone is very large – more than 1.3 million records, almost 9gb of data. To do some analysis of its content, we'll extract some aggregate totals by looping through all the EAC-CPF records.

Harvest SRU API results as JSON ¶

You can query the People & Organisations data using the SRU (Search/Retrieve via URL) API, and retrieve data in a number of XML formats, the richest and most complex of which is EAC-CPF. However, the XML records are not easy to work with, so to simplify further processing, this notebook queries the SRU interface and then converts the EAC-CPF XML results into JSON.

Visualising intersections and overlaps between data sources ¶

The People & Organisations zone aggregates information from a range of data sources into individual records. In this notebook we'll explore connections between the data sources.

Get VIAF links relating to Trove People & Organisation records ¶

VIAF, the Virtual International Authority File, aggregates identifiers for people and organisations from a wide range of name authority systems, including Libraries Australia. Many records in Trove's People & Organisations zone have Libraries Australia identifiers attached to them. Using these LA identifers it's possible to query VIAF for links related to a Trove record in other name authority systems.

Data¶

Complete harvest of People and Organisations data ¶

This is a complete harvest of the EAC-CPF records in Trove's People & Organisations zone.

Aggregated data extracted from People and Organisations data ¶

These datasets provide aggregated data extracted from the complete harvest of data from Trove's People and Organisations zone.

Run these notebooks¶

There are a number of different ways to use these notebooks. Binder is quickest and easiest, but it doesn't save your data. I've listed the options below from easiest to most complicated (requiring more technical knowledge).

Using ARDC Binder¶

Click on the button above to launch the notebooks in this repository using the ARDC Binder service. This is a free service available to researchers in Australian universities. You'll be asked to log in with your university credentials. Note that sessions will close if you stop using the notebooks, and no data will be preserved. Make sure you download any changed notebooks or harvested data that you want to save.

See Using ARDC Binder for more details.

Using Binder¶

Click on the button above to launch the notebooks in this repository using the Binder service (it might take a little while to load). This is a free service, but note that sessions will close if you stop using the notebooks, and no data will be saved. Make sure you download any changed notebooks or harvested data that you want to save.

See Using Binder for more details.

Using Reclaim Cloud¶

Reclaim Cloud is a paid hosting service, aimed particularly at supported digital scholarship in hte humanities. Unlike Binder, the environments you create on Reclaim Cloud will save your data – even if you switch them off! To run this repository on Reclaim Cloud for the first time:

Create a Reclaim Cloud account and log in.
Click on the button above to start the installation process.
A dialogue box will ask you to set a password, this is used to limit access to your Jupyter installation.
Sit back and wait for the installation to complete!
Once the installation is finished click on the 'Open in Browser' button of your newly created environment (note that you might need to wait a few minutes before everything is ready).

See Using Reclaim Cloud for more details.

Using Docker¶

You can use Docker to run a pre-built computing environment on your own computer. It will set up everything you need to run the notebooks in this repository. This is free, but requires more technical knowledge – you'll have to install Docker on your computer, and be able to use the command line.

Install Docker Desktop.
Create a new directory for this repository and open it from the command line.

From the command line, run the following command:

docker run -p 8888:8888 --name trove-people -v "$PWD":/home/jovyan/work quay.io/glamworkbench/trove-people repo2docker-entrypoint jupyter lab --ip 0.0.0.0 --NotebookApp.token='' --LabApp.default_url='/lab/tree/index.ipynb'

It will take a while to download and configure the Docker image. Once it's ready you'll see a message saying that Jupyter Notebook is running.
Point your web browser to http://127.0.0.1:8888

See Using Docker for more details.

Setting up on your own computer¶

If you know your way around the command line and are comfortable installing software, you might want to set up your own computer to run these notebooks.

Assuming you have recent versions of Python and Git installed, the steps might be something like:

Create a virtual environment, eg: python -m venv trove-people
Open the new directory" cd trove-people
Activate the environment source bin/activate
Clone the repository: git clone https://github.com/GLAM-Workbench/trove-people.git notebooks
Open the new notebooks directory: cd notebooks
Install the necessary Python packages: pip install -r requirements.txt
Run Jupyter: jupyter lab

See [Getting started](https://glam-workbench.net/getting-started/#using-python-on-your-own-computer for more details.

Contributors¶

Tim Sherratt

Cite as¶

Sherratt, Tim. (2023). GLAM-Workbench/trove-people (version v1.0.0). Zenodo. https://doi.org/10.5281/zenodo.7645059