Trove's 'book' zone includes books (of course), but also ephemera (like pamphlets and leaflets) and theses. You can access metadata from the book zone through the Trove API.
See below for information on running these notebooks in a live computing environment. Or just take them for a spin using Binder.
Harvesting the text of digitised books (and ephemera)¶
This notebook harvests metadata and OCRd text from digitised works in Trove's book zone. Results of the harvest are available below.
Metadata for Trove digitised works¶
In poking around to try and find a way of automating the download of OCR text from Trove's digitised books, I discovered that there's lots of useful metadata embedded in the page of a digitised work. Most of this metadata isn't available through the Trove API.
Getting the text of Trove books from the Internet Archive¶
Previously I've harvested the text of books digitised by the National Library of Australia and made available through Trove. But it occured to me it might be possible to get the full text of other books in Trove by making use of the links to the Open Library.
Exploring harvested books¶
Counting words and phrases¶
This notebook provides a simple example of extracting word and ngram frequencies from the OCRd text of a digitised book using TextBlob and Wordcloud. The text is downloaded from the Cloudstor repository created by the full harvest of Trove digitised books.
In this notebook we use TextBlob to extract nouns, verbs, and sentences from the OCRd text of a 19th century cookery book. We try to clean things up a bit, using regular expressions to discard likely OCR errors. Then we recombine the various parts in random combinations to create delicious recipes for all occasions. Enjoy!
Exploring the Digitised Books Collection from Trove by Adel Rahmani¶
This notebook explores the 9,738 text files from digitised books available below. Adel notes: 'A particular feature of this book collection is that it is multilingual, therefore I'll be focusing a bit on that, and on the use of the topic model to figure out what the collection is about.'
Data and text¶
OCRd text from Trove books and ephemera¶
Harvested: August 2021
I've harvested 26,762 files of OCRd text from digitised books and ephemera using the notebook above. You can browse the full collection in CloudStor. There's about 3.6gb in total. You can also use this searchable database running on Glitch to identify and download texts of interest.
CSV formatted list of books available in digital form¶
Harvested: August 2021
The CSV file includes the following columns:
title– title of the work
url– link to the metadata record in Trove
contributors– pipe-separated names of contributors
date– publication date
format– the type of work, eg 'Book' or 'Government publication', can have multiple values (pipe-separated)
fulltext_url– link to the digital version
trove_id– unique identifier of the digital version
language– main language of the work
rights– copyright status
pages– number of pages
form– work format, generally one of 'Book', 'Multi volume book', or 'Digital publication'
volume– volume/part number
children– pipe-separated ids of any child works
parent– id of parent work (if any)
text_downloaded– file name of the downloaded OCR text
text_file– True/False is there any OCRd text
Government publications in digital form¶
Harvested: 5 August 2021
This dataset combines records from the separate harvests of books and periodicals available in digital form that have the type 'Government publication'. You can explore it using a searchable database running on Glitch.
OCRd text from the Internet Archive of 'Australian' books listed in Trove¶
I've harvested 1,513 text files from the Internet Archive of 'Australian' books listed in Trove using the notebook above. Trove's 'Australian content' filter was used to try to limit the results to books published in, or about, Australia. However, this is not always accurate and some of the harvested works don't seem to have an Australian connection. You can browse the collection in CloudStor.
CSV formatted list of 'Australian' books in Trove with full text versions in the Internet Archive¶
Harvested: 24 May 2019
This file includes metadata of 1,511 'Australian' books listed in Trove that have freely available text versions in the Internet Archive. You can download the CSV file, or browse the metadata using Google Sheets.
This file includes the following columns:
creators– pipe-separated list of creators
date– publication date
ia_formats– pipe-separated list of file formats available from the Internet Archive (these can be downloaded from the IA)
ia_id– Internet Archive identifier
ia_url– link to more information in the Internet Archive
ol_id– Open Library identifier
text_filename– name of the downloaded text file
title– title of the book
trove_url– link to more information in Trove
version_id– Trove version identifier
work_id– Trove work identifier
Run these notebooks¶
There are a number of different ways to use these notebooks. Binder is quickest and easiest, but it doesn't save your data. I've listed the options below from easiest to most complicated (requiring more technical knowledge).
Click on the button above to launch the notebooks in this repository using the Binder service (it might take a little while to load). This is a free service, but note that sessions will close if you stop using the notebooks, and no data will be saved. Make sure you download any changed notebooks or harvested data that you want to save.
See the Using Binder section of the GLAM Workbench for more details.
Using Reclaim Cloud¶
Reclaim Cloud is a paid hosting service, aimed particularly at supported digital scholarship in hte humanities. Unlike Binder, the environments you create on Reclaim Cloud will save your data – even if you switch them off! To run this repository on Reclaim Cloud for the first time:
- Create a Reclaim Cloud account and log in.
- Click on the button above to start the installation process.
- A dialogue box will ask you to set a password, this is used to limit access to your Jupyter installation.
- Sit back and wait for the installation to complete!
- Once the installation is finished click on the 'Open in Browser' button of your newly created environment (note that you might need to wait a few minutes before everything is ready).
See the Using Reclaim Cloud section GLAM Workbench [for more details.
You can use Docker to run a pre-built computing environment on your own computer. It will set up everything you need to run the notebooks in this repository. This is free, but requires more technical knowledge – you'll have to install Docker on your computer, and be able to use the command line.
- Install Docker Desktop.
- Create a new directory for this repository and open it from the command line.
- From the command line, run the following command:
docker run -p 8888:8888 --name trove-books -v "$PWD":/home/jovyan/work glamworkbench/trove-books repo2docker-entrypoint jupyter lab --ip 0.0.0.0 --NotebookApp.token='' --LabApp.default_url='/lab/tree/index.md'
- It will take a while to download and configure the Docker image. Once it's ready you'll see a message saying that Jupyter Notebook is running.
- Point your web browser to
See the Using Docker section of the GLAM Workbench for more details.
Sherratt, Tim, (21 November 2019), GLAM-Workbench/trove-books (version v0.1.0), Zenodo, https://doi.org/10.5281/zenodo.3549481