Filling the MOH Gaps

The Wellcome Library has a great collection of Medical Officer of Health (MOH) Reports. These reports are stuffed full of grim and useful information from the 19th and 20th century, such as statistics on infant mortality. JISC is, very wisely, funding the digitisation of the London reports. There are some gaps in the Wellcome Library’s collection so, in order to make a really useful digital resource, we have been working out what is missing. This has not been straightforward.

First we needed to check what we held and then we needed to make sure that our gaps really were gaps. We didn’t want to waste time looking for reports that were never created. The very first MOH report in Britain was produced in Liverpool in 1847. The first London report was produced in the following year but the early reports do not cover the whole of London. The Public Health Act of 1848 permitted local authorities to employ MOHs but, since it was not obligatory, only a minority did. The Metropolis Management Act of 1855 required MOHs to be appointed in central London but the big change came with the 1875 Public Health Act. From then until 1972 the production of MOH reports was pretty solid.

Another challenge was getting to grips with the boundary changes. Over the years the administrative boundaries of London have altered several times. The current 32 London Borough boundaries date from 1965 when Greater London was established. Before that there were 28 metropolitan boroughs plus various boroughs, urban and rural district councils in what is now, outer London. Before 1899 much of what we now think of as London was part of Kent, Middlesex, Essex or Surrey. The City of London has long gone its own distinctive way and the tangle of parish boundaries there is particularly confusing. Old maps and a book on administrative units by Frederick A. Youngs helped us to make sense of all these changes.

We have decided to start from the centre and try to create a complete as record as possible for the 12 inner London Boroughs. We’ve got to the stage where we have a list of reports that we want to find. The next step is to track them down in other collections and ask if we can get them digitised.

Watson and Crick Letters

I've just had the privilege of reading a fantastic series of letters written in 1954 by James Watson and Francis Crick. They were written a year after they published their seminal article on the structure of DNA. In the letters the two men are exchanging ideas and their excitement shines through. They write about all sorts of things, for example, the importance of building space filling three dimensional models, confusion over how thymine fits into the helical structure and what the researchers at KCL are up to. In March 1954 Watson also expresses his frustration with the research process, “The whole thing is puzzling and paradoxical (for could DNA be wrong) and is slowly driving me to despair and to loath nucleic acids.” (PP/CRI/D/2/45)

I got to read them because last month the first batch of digitised material arrived from Cold Spring Harbor Laboratory in New York, one of the five external organisations contributing digitized material to the WDL pilot project. The James Watson archive is held at Cold Spring Harbor and contains the letters written to him by Francis Crick. The letters Watson wrote to Crick are held by the Wellcome Library.

Later this year, when the WDL is launched, Watson and Crick’s correspondence will be digitally united. Lots of people will be able to read these letters (and lots of other stuff) online while the originals stay safely tucked away in their archival homes. I am excited about that!

The Medical Officer of Health reports project begins.

I recently started working on the Medical Officer of Health digitisation project.

Spine with methylcellulose applied

Since the beginning of December 2011, I have been spending the majority of my time in the conservation studio. I have been carrying out disbinding, cleaning and rehousing of the late 19th century Medical Officer of Health (MOH) reports that are bound by year. This is so that the digitisers can scan or photograph them for the project.

The MOH reports in our collection are shelved in different sequences: Main, London and Provincial. However, all three sequences contain reports for London areas. Main and Provincial sequences are bound by geographical area and generally they're in a good condition but the London sequence reports are bound by year and tend to be in a poorer state due to their heavy use. There are about 80 bound volumes in the London sequence and they are all being disbound. Eventually we will be able to house all of the London reports together by geographical area.

Lining coming off

In order to take the bound reports apart, I first need to remove the cloth case and spine linings whilst keeping the pamphlets intact. I use a 4% methylcellulose solution to break down the binding animal glue. A major challenge with this is that each volume’s binding breaks down at a different rate so it requires constant checking to avoid damaging the paper. On average it takes a couple of hours just to remove the spine linings. We recently purchased a pink portable clothes steamer to try to speed up the process. I haven’t tried this yet but we are hopeful that this will work faster.





The aftermath of removal

Another important part of the project is creating separate bibliographic records for each report.  We have decided to catalogue these as monographs in order to improve searching and allow users to find reports by fields such as geographic area, Medical Officer's name and date of the report.  

Whilst I am going through the reports, I have been finding  some very interesting snippets. I will include some as I continue to blog so look out for them!

Guest blog post: Digitising early printed books at the Wellcome Library

Karine Larose and Jeremy Uhl are working on the Early European Books digitisation project at the Wellcome Library. They are empolyees of Diadeis, a digitisation company based in France, and work on site alongside the rest of the Wellcome's digitisation team. In this guest post, Karine and Jeremy explain what they do here at the Wellcome Library: 




Hello, Karine and Jeremy here from Diadeis. We are the production team currently working on the digitization project for ProQuest at the Wellcome library. The following will be a sort of 'day in the life' of our department. We hope you enjoy it and would love any comments, questions, or suggestions.

Each day, when we start, new books are delivered to us by Matt Brack, the Digitisation Support Officer, for digitisation. We then register them in the database (also known as the check-in process). First we assess the books to determine if they are suitable for scanning. We might need to refuse a book if

• the book is too big or thick for the scanning machines to handle (usually higher than 42 cm and wider than 30 cm) 
• the book cannot open more than 100 degree (later in the project these books will be scanned using different equipment) 
• The book is too fragile and we feel that we cannot scan it without causing more damage. 

Also during the check-in process we might notice other minor problems with the books but not so severe as to refuse them. In that case we usually put them on hold. An example would be if most of the pages are stuck/glued together. ‘Problem books’ on hold are placed on a separate shelf, and we later consult with Matt as to what action has to be taken. Simultaneously, the books we scanned on the previous days are given back to Matt and we update the database to show that they have been returned to the library.

Before scanning the pages of the books, the spine and edges are photographed first. This step is to show the condition of the book before we start the scanning. We do our best during the scanning process to be careful with the books, to return them in the state they were delivered. The books are scanned using two main scanners, one for restricted angles and one for opened books. A few problems we sometimes have when scanning books are:

• Huge fold-outs that cannot be scanned with the current equipment 
• missing pages 
• pieces of a book coming apart (usually the spine) 

Our most common problem is dust! This includes tiny book pieces, occasionally hair, once we even found tiny bits of cereals in a book. There are lots of surprises and we never know what we may find in between pages of a book. Sadly we have yet to find any money in any of the books ;). Several times during the day we have to clean the glass panel; for that we use a special cloth and a vacuum cleaner. The archive office kindly supplied us with alcohol wipes which has a disinfection effect. Also because some of the books have leather covers, our hands can become oily after handling them and we have to wash our hands regularly.

Once the scanning day is over, we pack all the images in a packaging software and send them off to the indexing team for quality control, cropping and indexing.

Working on this project at the Wellcome library has been fun and exciting because we get to see rare books every day and provide digital access to them to people all over the world. We are pleased to be given the chance to explain our daily routine to the followers of this blog and look forward to any comments they might have.

We would like to thank Matt for being so flexible with our timetable and bringing us lots of books everyday and Christy for her support and giving us the opportunity to write for the blog. For those of you who work for the Wellcome library do come along and say "hello" to us, the room may be dark but I assure you we are here!

Authors: Karine Larose and Jeremy Uhl 

NB: The first batch of 400 books can now be seen on the EEB website (do a Quick Search for Wellcome). These are freely available to Wellcome Library members and everyone in the UK.

Preserving our digital assets #2 - WORM storage

This post follows on from my earlier description of our DAM system (Safety Deposit Box  or SDB), which manages long-term preservation and access to our digital assets. Here we turn to the back-end of the back-end: storage.

There are a number of requirements that must be met to safely storing digital assets. The storage solution must be:

• Secure (behind a firewall)
• Robust (able to manage points of failure in the disks)
• Replicable (multiple copies on multiple sites)
• Scalable (able to handle tens of millions of files)
• Quick to access (the archived files are also used for delivery) 

After considering different systems and suppliers – including robotic tape back-up – we settled on a solution that gives us the confidence we need for long-term preservation, and fits well with the Trust’s existing storage infrastructure. Our existing storage suppliers, Pillar Data Systems, have extended the existing RAID5 enterprise storage system used for all the Wellcome Trust’s business needs by incorporating a “Write Once, Read Many” (WORM) back-up storage server for use by the Wellcome Digital Library. Associated management software copies files from the main storage server to the WORM, and monitors the main server for file errors that can be “healed” using the WORM copy.

To explain this in a bit more detail, related to our primary requirements:

Security means that only authorised users or systems are able to access the files, and that unauthorised deletions or changes are guarded against. Locking master files behind a firewall is the main form of defence from unauthorised external access (i.e. hackers, or the ability to download files by "guessing" the network path and filenames). However, we still faced the prospect of file deletion, changes or corruption due to system failures, and accidental or malicious actions by otherwise authorised users. In order to eliminate this possibility, the Trust IT Department recommended permanent, on-line WORM storage as our back-up solution. Files stored on the WORM can be accessed, but they cannot be overwritten or deleted. This means that we have a permanent back-up of every digital file that cannot be tampered with.

Robustness is tied up with the WORM system - although the RAID5 storage we normally use is highly robust. It distributes the bit stream of digital files in such a way that points of failure generally do not damage the entire file, allowing it to be reconstituted, while short-term back-ups allows complete recovery of damaged or lost files. The WORM system adds a further element of confidence. Once a new file is stored on the main servers, it is copied to the WORM drive. The software managing this process checks the files on the main servers periodically, and if it finds a mis-match with the WORM drive (a lost or corrupted file on the main server), it pulls a copy of the WORM'ed file to the main server, thus "self-healing" the damage.

Replicability means that there is more than one copy of the file. One lives on the main servers at the Wellcome Trust offices, and the other is stored outside of London. If either server is damaged in a serious accident, one server remains to keep the content safe.

Scalability is important, as we are creating millions of images during the pilot project, and will create up to 30 million images over the longer term as we digitise the Wellcome Library. All the systems that are in place must be able to increase capacity - both in terms of hardware and processing software. The system we selected is scalable – we simply need to add storage “bricks” and “racks” (additional hardware), and processing units to manage that additional hardware, as our data store grows over time.

Speed of access is also key. In order to keep our storage footprint as small as possible, we are using one file as both the master archive file (preserved in the long-term, and backed-up to the WORM) and as the dissemination file. Our DAM, SDB, will maki copies of the archived files available to the front-end delivery system as the user requests images via the Wellcome Digital Library (or via back-end administrative systems), and this must be handled as quickly as possible – something RAID5 is particularly well suited for.

Preserving our digital assets #1 – SDB4

The successful long term management of digital assets is a key concern for us as we build our digital resource. Until recently, we did not have a dedicated system in use for storing and managing master files for any digitised content, and our file backup system was not idea for dealing with large sets of data. Files were managed via simple filesharing on dedicated storage servers. Backups were only created for some content, and even then the backups were not permanent.

When the Wellcome Digital Library was initiated it quickly became clear that we needed a dedicated system to manage our digital masters - something that was scalable, robust, and could handle all the digital formats we create or procure - including born digital material. We also needed a secure storage system with offsite, permanent backup capability. This blog post describes our digital asset management system; a future blog post will provide details of our secure storage solution.

We already had an existing digital asset management system in place to manage born digital archives: Safety Deposit Box (SDB), developed by Tessella. This system incorporates a suite of tools designed to manage and preserve digital files. It provides a context in which administrative and descriptive metadata is associated with all ingested content. SDB can be combined with tools that can "migrate" files from one format to another to counter format obsolescence and therefore ensure the longevity of the data. In other words, when Word 2010 is no longer supported by Microsoft, SDB can help migrate these files to a current format that is supported by software available at that time.

However, in order to manage preservation of large sets of digitised content, SDB “out of the box” was not entirely able to meet the needs of the Wellcome Digital Library. We carried out a Feasibility Study in 2010 to determine whether it would be possible to use a modified version of the system. The research and prototype system we commissioned proved that it was indeed feasible to use SDB with certain software extensions.

In the spring of 2011, we commissioned Tessella to extend and install the newest version of the software (SDB4). This work is now complete and in the testing phase.

Key preservation functionality that SDB4 provides is listed here (some of this was “out of the box”, some were developed as extensions to the core system or as modules):

• Automated ingest: automated SDB workflow to create and ingest a “submission information package” (SIP) – a bundle of content files and metadata forming a complete “object”


• Multiple manifestations: ability to associate all the different manifestations of an object together (e.g. master video file, broadband versions, narrowband versions, transcripts, etc.)


• SQL Server database: stores and indexes administrative and descriptive metadata describing objects stored in SDB


• Characterisation: use of the JHOVE, DROID and PRONOM characterisation tools to extract essential technical information about digital files to be stored in the database


• Format Migration: SDB4 builds on the PLANETS framework to support preservation planning for mitigation of obsolescence


• Integrity checking: creates and stores a unique SHA-1 hash code for each file that can be used to test validity of the file over time


• Provide access to content: delivers content to external systems using an API


• Automatic export of administrative metadata: allows us to store metadata such as unique SDB identifiers in our workflow system in order to deliver files to the user


• Administrative interface: allows administrators access to the content and the database, and to generate reports

This is by no means a complete description of SDB’s functionality, but the above provides a flavour of the most important features for long-term asset management and access to master files. Much of the development work that was done to extend SDB4 is likely to have an application for other users, not just the Wellcome Digital Library. Where possible, extensions are generic – although system requests/commands and metadata mappings may be highly specific to us.

We hope that our efforts in specifying our own needs will benefit other SDB users who see the value in long term preservation for digitised materials, and the efficiencies of combining born digital and digitised content into a single system strategy.

In future, SDB will interface with our workflow system and the image/media servers employed by our digital delivery system. These latter two systems have not yet been implemented.

Arabic manuscripts online - Fihrist launch

Fihrist, an online catalogue for Islamic manuscripts held at the Bodleian and Cambridge University Library (CUL), was launched on 28 March. Developed by the the OCIMCO (Oxford & Cambridge Islamic Manuscripts Catalogue Online) project, the catalogue uses the TEI/XML schema created as part of the WAMCP (Wellcome Arabic Manuscript Cataloguing Partnership) project to structure the descriptions and to provide for future enhancement of those records.

Fihrist contains around 10,000 catalogue entries, retroconverted from hard copy lists and card catalogues at the Bodleian and CUL, providing an integrated online search tool for these large collections. Searching for a manuscript or a work shows you a list of works contained in the relevent manuscript, with separate descriptions provided for each work. The catalogue records are currently very brief, but the OCIMCO project "will eventually provide detailed manuscript descriptions that will include digital representations of the manuscripts themselves. The TEI/XML schema provides an extensible framework that will allow for these future enhancements" (About us).

The project recieved funding from the JISC’s Digital Resources for Islamic Studies programme - a scheme that also part-funded the WAMCP project.

At the Fihrist launch at Clare College, Cambridge, a series of presentations from the OCIMCO project managers and staff provided further details of the collections and the technical implementation of the project; a presentation from the JISC on the Islamic Studies programme provided background to the funding scheme; a joint presentation was given by Richard Aspin and Nikolai Serikoff of the Wellcome Library, and Gerhard Brey of King's College London on the WAMCP project; and we heard about the like-minded Yale/SOAS Islamic Manuscript Gallery.

The day then turned to a follow up project that Oxford is carrying out over the summer to create a union catalogue of Islamic manuscript catalogues (the Islamic Studies Gateway). This project was recently awarded JISC funding, and will seek to develop Fihrist further to "provide cross searching of existing online manuscript resources ... that at present do not have a significant internet presence."

"In addition to creating the gateway itself, the aim of the Fihrist is to create a sustainable user community of Islamic manuscript metadata standards and cataloguing tools to ensure a long term commitment by stakeholders to supporting and developing the Fihrist beyond the lifetime of the project."

The WAMCP partnership (comprising the Wellcome Library, Bibliotheca Alexandrina, and King's College London) will collaborate on this project by providing open access to metadata for 500 manuscripts. This metadata, with digitised manuscripts, is expected to be publically available online via the WAMCP website from Summer 2011.