Will more data lead to different histories being told?

New technology is making information more widely available and, when it launches later this year, the WDL will make it easier to access historical evidence about the foundations of modern genetics.  Will this democratize our understanding of the history of genetics and lead to different versions of the history being told?

There is an African proverb which says that history is written by the hunter not the lion.  History inevitably simplifies the past and the selection process can be subjective.  When it launches the WDL will start to put 21 archive collections and around 2,000 books on-line.  The project is to digitise as much as we can rather than cherry pick the highlights. This means that the building blocks used by historians to piece together the past will be made freely available to a wider audience.  A lot of this material may seem like mundane workaday stuff. Users will have to wade through a lot of material to reach the bits they are interested in but this is probably a more accurate reflection of the scientific research process.

Flashes of genius are essential but they do not happen in isolation. Thomas Edison’s phrase about invention being 1% inspiration and 99% perspiration applies to scientific research too. The discovery process needs both.

Watson and Crick were extremely clever to work out the helical structure of DNA but they did not get there simply because they were lone geniuses. Before they made their discovery a lot of people had spent years experimenting, writing and thinking about DNA. There had even been flashes of insight which ended up being wrong.I recently read a letter from Gerald Oster sent to Aaron Klug after Rosalind Franklin’s death, in which he recalled his time working in London. He reflects that even though he had much of the relevant information by early 1950 he lacked the insight to work out the structure of DNA.  This letter (FRKN/06/07/001-2) is held by the Churchill Archives Centre in Cambridge and a digitised version will become part of the WDL.

I am rather hoping that the WDL might help us to recognise that while flashes of inspiration are part of scientific discovery they are only possible because a team of other people paved the way.

Three Geneticists from the University of Glasgow

The Wellcome Digital Library isn’t just about collections held in the Wellcome Library. We are working with a number of other organisations that hold material on the history of modern genetics. One of the contributing partners, the University of Glasgow Archives Service, have just started to digitise the archives of three men who worked at the University’s Department of Genetics - Guido Pontecorvo (1907-1999), James (Jim) Harrison Renwick (1926-1994) and Malcolm Ferguson-Smith (1931 – ).

You can see photos of their brand new digitising suite here.

Watson and Crick Letters

I've just had the privilege of reading a fantastic series of letters written in 1954 by James Watson and Francis Crick. They were written a year after they published their seminal article on the structure of DNA. In the letters the two men are exchanging ideas and their excitement shines through. They write about all sorts of things, for example, the importance of building space filling three dimensional models, confusion over how thymine fits into the helical structure and what the researchers at KCL are up to. In March 1954 Watson also expresses his frustration with the research process, “The whole thing is puzzling and paradoxical (for could DNA be wrong) and is slowly driving me to despair and to loath nucleic acids.” (PP/CRI/D/2/45)

I got to read them because last month the first batch of digitised material arrived from Cold Spring Harbor Laboratory in New York, one of the five external organisations contributing digitized material to the WDL pilot project. The James Watson archive is held at Cold Spring Harbor and contains the letters written to him by Francis Crick. The letters Watson wrote to Crick are held by the Wellcome Library.

Later this year, when the WDL is launched, Watson and Crick’s correspondence will be digitally united. Lots of people will be able to read these letters (and lots of other stuff) online while the originals stay safely tucked away in their archival homes. I am excited about that!

100,000 image milestone, and how we did it

A significant milestone was reached this week, with 100,000 images of Crick papers photographed. The Crick collection, previously described, includes around 285,000 images - over half the total amount to be digitised as part of the Archives digitisation project over a period of 20 months by Laurie Auchterlonie and Tom Cox in the Imaging department.

Laurie and Tom use Canon Mark II digital SLR cameras mounted to columns attached to two copy stands. These allow the cameras to be automatically raised and lowered according to the size of the items being digitised. This is a fairly typical imaging set-up for this type of material. However, with a project this large, it is important to ensure that the minimum amount of time is taken to photograph, edit and manage the images. In spring 2010, the photographers spent time developing a workflow that would allow them to digitise the highest number of items possible, whilst not compromising on quality or care in handling. It was in fact found that time-saving measures actually resulted in higher quality images, and minimised the amount of handling required.

For example, "live view" screens allow the photographers to easily see and adjust the alignment of each item on the copy stand, and the degree to which it fits the frame of view. This saves time as the photographers do not have to look through the viewfinder (difficult when the camera is 6 or more feet above the ground), or take multiple shots to get it right. Post-processing work has also been almost completely eliminated, as has the need to reshoot items at a later date.

Purchasing higher columns limited the number of times lenses had to be changed. Larger items require the cameras to be raised quite high, and if the column isn't high enough, the photographer has to change to a different, shorter or wide-angle lens. The flexibility built into the workflow by these measures is highly advantageous when dealing with heterogeneous materials such as personal archives.

Other aspects of the workflow that had to be specifically tailored to archival collections was the storage and foldering of images so that they could easily be found and identified. Using the existing archive catalogue hierarchies, the foldering system allows the user to pinpoint the exact file or item to be viewed to create copies for users, or to carry out QA against the original items (a sample of images is checked against the originals by Julia Nurse, who prepares the items before photography, to ensure that filenaming is accurate and that items aren't being missed). Eventually, these folders will be rendered obsolete, as we implement a digital asset management system that will restructure the archive storage of our images on ingest. But it is important not to underestimate the need to access images during the pre-ingest process of digitisation and QA. And if you do not have a digital asset management system, it is even more important that ease of access is factored in from an early stage.

A bit of preparation and testing makes a huge difference when setting up a new workflow. Even a minute saved per item means a large overall time saving when spread over hundreds of thousands of images.

Preparing and planning a large archives digitisation project

Archives digitisation is currently underway in our Imaging Studio, with two full-time members and two part-time members of Library staff dedicated to preparing and digitising the items. We will talk more specifically about the work being carried out on these materials on this blog in the near future, but first we present an introduction to the setup and planning of the project.

Once the theme was chosen (Modern Genetics and its Foundations), and relevant collections identified (see previous blog post), we realised that we had quite a large job on our hands. The scope of the project was bigger than anything we had done before: 620 boxes of material, containing around 800 pages each, adds up to around half a million pages to be digitised.

Based on a series of tests, we estimated the project would take 2 years to complete – starting with the preparation of the material in advance, with photography coming into play a few months later. Two full-time staff would be focused on imaging the material, with two part-time member of staff preparing, tracking and assessing the items.

There are a range of logistical issues to bear in mind when planning and starting up a project of this nature. The boxes are stored in the basement stores, and had to be retrieved for a period of some months while the material was being worked on. We divided the collections into batches of a size that could be imaged in a period of 4-6 weeks and retrieve and return each batch as a unit, tracking all movements on a spreadsheet. The tracking spreadsheet also records information such as location of each box in the batch, notes from the preparer for the archivists, photographers, and/or conservation staff, and the percentage of items in each box that can be OCR’d among other things.

We put a notice on our website of the entire schedule of archives to be digitised, so readers could see at a glance what would be unavailable and when. The catalogue records are also amended to show where material cannot be reserved. Each time a batch is retrieved, checked out, checked in, or dates altered, this has an impact on the website and two different cataloguing systems (the Archives and Manuscripts Catalogue, and the Library Catalogue), so communication with the departments responsible for retrieval and metadata was key.

The preparation staff were trained in advance by the conservation team so they could carry out basic stabilisation and first-aid work on the materials if required for digitisation. The photographers ran multiple tests on different equipment and with different cameras to ensure the workflow was efficient and appropriate to the formats of the material, the anticipated end use of the material, and to ensure proper QA could be accommodated. Preparation and imaging takes place in the Imaging Studio - ensuring that all staff are in close proximity and able to communicate easily with each other. The Imaging Studio was refitted with desks, shelving and equipment to make sure all the boxes in process at any one time could be accommodated. A further planning issue was in determining how to assess and record different levels of sensitivity of information contained in the archives. We are currently developing a policy for access to archives that takes account of online display, and this has informed the workflow for assessment.

This project required liaison between several different departments and stakeholders in the Library in order to set up a suitable workflow. In future, we hope that workflow issues will be streamlined further by procuring a Workflow Tracking System that will serve to centralise tracking and monitoring of all digitisation projects. We anticipate that this pilot project will enable us in future to plan effectively for much larger digitisation projects as we work towards the digitisation of all suitable material held in the Wellcome Library.