Trove books

Trove's 'book' zone includes books (of course), but also ephemera (like pamphlets and leaflets) and theses. You can access metadata from the book zone through the Trove API.

See below for information on running these notebooks in a live computing environment. Or just take them for a spin using Binder.

Harvesting data¶

Harvesting the text of digitised books (and ephemera)¶

This notebook harvests metadata and OCRd text from digitised works in Trove's book zone. Results of the harvest are available below.

Metadata for Trove digitised works ¶

In poking around to try and find a way of automating the download of OCR text from Trove's digitised books, I discovered that there's lots of useful metadata embedded in the page of a digitised work. Most of this metadata isn't available through the Trove API.

Getting the text of Trove books from the Internet Archive ¶

Previously I've harvested the text of books digitised by the National Library of Australia and made available through Trove. But it occured to me it might be possible to get the full text of other books in Trove by making use of the links to the Open Library.

Exploring harvested books¶

Counting words and phrases ¶

This notebook provides a simple example of extracting word and ngram frequencies from the OCRd text of a digitised book using TextBlob and Wordcloud. The text is downloaded from the Cloudstor repository created by the full harvest of Trove digitised books.

Recipe generator ¶

In this notebook we use TextBlob to extract nouns, verbs, and sentences from the OCRd text of a 19th century cookery book. We try to clean things up a bit, using regular expressions to discard likely OCR errors. Then we recombine the various parts in random combinations to create delicious recipes for all occasions. Enjoy!

Exploring the Digitised Books Collection from Trove by Adel Rahmani ¶

This notebook explores the 9,738 text files from digitised books available below. Adel notes: 'A particular feature of this book collection is that it is multilingual, therefore I'll be focusing a bit on that, and on the use of the topic model to figure out what the collection is about.'

Data and text¶

OCRd text from Trove books and ephemera ¶

Harvested: August 2021

I've harvested 26,762 files of OCRd text from digitised books and ephemera using the notebook above.

CSV formatted list of books available in digital form ¶

This file provides metadata of 21,218 digitised works with the format Book.

Government publications in digital form ¶

Harvested: 5 August 2021

This dataset combines records from the separate harvests of books and periodicals available from Trove in digital form that have the type 'Government publication'.

OCRd text from the Internet Archive of 'Australian' books listed in Trove ¶

I've harvested 1,513 text files from the Internet Archive of 'Australian' books listed in Trove using the notebook above. Trove's 'Australian content' filter was used to try to limit the results to books published in, or about, Australia. However, this is not always accurate and some of the harvested works don't seem to have an Australian connection.

CSV formatted list of 'Australian' books in Trove with full text versions in the Internet Archive ¶

Harvested: 24 May 2019

This file includes metadata of 1,511 'Australian' books listed in Trove that have freely available text versions in the Internet Archive.

Run these notebooks¶

There are a number of different ways to use these notebooks. Binder is quickest and easiest, but it doesn't save your data. I've listed the options below from easiest to most complicated (requiring more technical knowledge).

Using ARDC Binder¶

Click on the button above to launch the notebooks in this repository using the ARDC Binder service. This is a free service available to researchers in Australian universities. You'll be asked to log in with your university credentials. Note that sessions will close if you stop using the notebooks, and no data will be preserved. Make sure you download any changed notebooks or harvested data that you want to save.

See Using ARDC Binder for more details.

Using Binder¶

Click on the button above to launch the notebooks in this repository using the Binder service (it might take a little while to load). This is a free service, but note that sessions will close if you stop using the notebooks, and no data will be saved. Make sure you download any changed notebooks or harvested data that you want to save.

See the Using Binder section of the GLAM Workbench for more details.

Using Reclaim Cloud¶

Reclaim Cloud is a paid hosting service, aimed particularly at supported digital scholarship in hte humanities. Unlike Binder, the environments you create on Reclaim Cloud will save your data – even if you switch them off! To run this repository on Reclaim Cloud for the first time:

Create a Reclaim Cloud account and log in.
Click on the button above to start the installation process.
A dialogue box will ask you to set a password, this is used to limit access to your Jupyter installation.
Sit back and wait for the installation to complete!
Once the installation is finished click on the 'Open in Browser' button of your newly created environment (note that you might need to wait a few minutes before everything is ready).

See the Using Reclaim Cloud section GLAM Workbench for more details.

Using Docker¶

You can use Docker to run a pre-built computing environment on your own computer. It will set up everything you need to run the notebooks in this repository. This is free, but requires more technical knowledge – you'll have to install Docker on your computer, and be able to use the command line.

Install Docker Desktop.
Create a new directory for this repository and open it from the command line.

From the command line, run the following command:

docker run -p 8888:8888 --name trove-books -v "$PWD":/home/jovyan/work glamworkbench/trove-books repo2docker-entrypoint jupyter lab --ip 0.0.0.0 --NotebookApp.token='' --LabApp.default_url='/lab/tree/index.md'

It will take a while to download and configure the Docker image. Once it's ready you'll see a message saying that Jupyter Notebook is running.
Point your web browser to http://127.0.0.1:8888

See the Using Docker section of the GLAM Workbench for more details.

Contributors¶

Tim Sherratt

Cite as¶

Sherratt, Tim. (2019). GLAM-Workbench/trove-books (version v0.1.0). Zenodo. https://doi.org/10.5281/zenodo.3549481