In his project Old Ireland in Colour at the Insight SFI Research Centre at NUI Galway Professor John Breslin enables us to reimagine the past by adding colour to black and white historical images with the help of a tool called DeOldify. Here he explains just how he ensures he has the right shades.
DeOldify works by learning what colours should be applied to different textures, shapes and objects in black and white photos, using models that are trained on a large bank of millions of colour images. The accuracy of the colours is as good as what can normally be expected for a certain type of texture or shape encountered in the image bank.
Grass, trees, the sea, etc. usually come out really well. Roofs sometimes (but not always) come out with the wrong colour tiles. Quite often, clothes come out in an averaged blue or purple colour, whereas in reality you may know that they should be a particular hue.
In those cases, all you can do is carry out some post-processing where you touch up or manually adjust that colour in an image-editing package like Photoshop. Some photos are fine because you do not know if someone was wearing navy or brown, but there are specific incidences where there is either a historical record or some human intuition or knowledge that the colour is just not right.
For example, in various projects we have manually changed Constance Markiewicz's uniform from navy/purple to bottle green, the shawls of Claddagh women to red, and Tom Crean’s woolen jumpers to any colour but lilac! The most common change is where we have to manually touch up an ear, arm or leg that was not automatically colourised for some reason (e.g. if it was partially obscured so not recognised as such).
While some tweaking of photographs is necessary either before (brightness, contrast, cropping) or after colourisation, in general the heavy lifting is done by the computer, reducing the time to completion for a decent colourisation by a significant amount (from days to hours).
There are now various open-source and commercial frameworks that make use of DeOldify, each of which produces slightly different colourisations depending on their implementation. We also use other AI-based frameworks to enhance the resolution of faces and other features in the photographs where the image source quality is poor.
We think of the colourised, enhanced and restored result as being a bit like a Wikipedia page: the colourised photograph may be the first of many iterations that will continue to improve as one receives feedback; as processes, models and training improve; and as you can reference more relevant sources for greater historical accuracy. It is an interpretation of what the colours may be, because unless a parallel colour version taken at the same time exists, or someone has painted the scene exactly, one can never know the exact colours.