For the past month (Jan-Feb 2024) I have been experimenting with ChatGPT. I’ve subscribed to ChatGPT Plus to get access to the latest version at the time of writing – ChatGPT 4. Its extra features including vision (image analysis) capabilities and the ability to create custom “GPTs” (focus output on uploaded documents only rather replying on the very general knowledge of the main AI model) as well as the claim of better accuracy were enough to pique my curiosity.
The TL;DR (‘Too Long; Didn’t Read) of this post is that ChatGPT 4 is a seriously useful tool for researchers if you understand how to use it and are very aware of its limitations and biases.
Before I signed up and paid OpenAI, I made a list of features that I wanted to test. I had watched a few YouTube videos which had demonstrated them, and was itching to try them for myself.
These included:
- Custom GPTs – create a customised version of ChatGPT 4 that can be trained on uploaded documents (and told to only use that content in answers)
- ChatGPT Vision – the ability to upload images for analysis (Optical Character Recognition – transcriptions, descriptions for ALT tags)
- Translation – particularly little-known languages such as Kernewek (Cornish)
- Using it as a writing companion – seemingly its main use but I’d not tried it since some terrible ChatGPT 3.5 output a year ago
This has all been written by me – so far! I decided to ask ChatGPT 4 for some help in creating some sub-headings for this blog post. I got carried away and asked it to write this blog post, but the output was generally a bit too ‘peppy’ for my liking (“Hello, history enthusiasts and curious minds!“). Any AI generated text will be in italics, for interest. The featured image for this blog post was created with DALL-E using the blog post title as a prompt.
Custom GPTs: Tailored Time Machines
ChatGPT output: One of the most exciting features of ChatGPT 4 is the ability to create custom models that act like personalised time machines, giving us direct access to the insights and knowledge of specific eras. Imagine having a version of ChatGPT that has read every existing document from the Elizabethan period or has ‘lived’ through the Industrial Revolution. By uploading documents, letters, diaries, and scholarly articles to these custom models, researchers can generate a focused AI that draws purely on the content of these uploads. This means you could ask intricate questions about Victorian etiquette or the strategies in the Napoleonic wars and receive answers that feel like they’re straight from the pages of history.
ChatGPT actually described that quite well. One of my specialisms is in the folklore and traditions of West Cornwall and I am currently researching for a forthcoming exhibition that I am curating at a museum. The main source for folk traditions are the three volumes by William Bottrell in the 1870s-1880. Could I create a custom GPT (called BottrellGPT, naturally) and use it to interrogate the 1000+ pages? The answer is yes.
The first thing to remember about any Large Language Model (LLM) is that the training must have good data in the first place. The public domain scans of the three volumes of Bottrell’s works exist as PDFs (thanks to the Internet Archive) with a machine-readable text layer. However, copying and pasting some pages as plain text into a text editor revealed that the OCR hadn’t done a great job. They were littered with errors and would have been next to useless. So I used the command line tool OCRmyPDF to remove the existing text and OwlOCR (which uses new, more robust algorithms) on my Mac to create a new, more accurate, text layer. I spent some time checking the accuracy of the converted text, and while it wasn’t perfect, it was vastly improved. As explained further down, ChatGPT can perform OCR very well indeed, but cannot yet replace text in a PDF or do this for whole books.
Then I used ChatGPT 4 itself to create “BottrellGPT”. I asked it to only use information in the uploaded PDFs and exclude any of its general knowledge. I enabled DALL-E so that images to be created based on descriptions in the books. I also asked ChatGPT to enable citations, so that the volume and page numbers could be displayed after an answer.
It worked really rather well except for citations which appear to be generated but are immediately deleted as the output appears on the chat transcript. I’ll be reporting this as a bug to OpenAI. However, I was able to ask it explicitly from which volume and which page that it used for each answer, which was an adequate workaround.
I was able to ask questions about topics that I knew well from the books and get excellent answers. Sometimes I didn’t get the information that I knew was there in the books, so asked it to expand upon its answer which generally included the information that I felt was missing. Each answer took about 20 seconds to generate. Not bad considering there’s over a 1000 pages of text to consider.
I admit to being quite excited about custom GPTs. It helped me to make many connections between some of the traditions that I am researching in a very short amount of time. At the moment (February 2024) a custom GPT is limited to 20 documents each with a maximum file size of 512MB. This will eventually be incredibly useful for organisations wishing to better use their own information, especially when more files are allowed. I’m thinking of combinations of museum catalogues, for example, combined with the usual copious folders (analogue and digital) of notes on different topics, or even internal policies and strategies.
Top tip: ask ChatGPT to expand upon its answers, or regenerate a new one.
GPT Vision: Bringing the Past into Focus
ChatGPT output: ChatGPT 4’s vision capabilities are a game-changer for historians. With GPT vision, you can breathe life into old newspapers and book scans. Traditionally, OCR (Optical Character Recognition) has been used to digitize texts, but it often stumbles over smudged ink, cursive handwriting, or antiquated fonts. Enter GPT vision, which not only reads these challenging texts but can understand them in context. This allows for the transcription of even the most stubborn historical documents, transforming them into searchable, editable text.
Researching for my museum exhibition had led me to make extensive use of the British Newspaper Archive. The quality of the OCR on the BNA is pretty good considering that many of the newspaper scans are hard to read. However, it’s far from perfect either (I’m sure they’ll redo them all at some point). I often have to manually retype articles. So I tried screenshotting some of them and uploading them to ChatGPT 4. The simple prompt “transcribe this” yielded fantastic results, better than OwlOCR on my Mac.
Take this excerpt from the Royal Cornwall Gazette on 11 May 1811 about Furry Day (widely called Flora Day today) in the Cornish town of Helston:
This is the unedited OwlOCR output:
The spirit oftbe Furry-day at Helston on Wedresday last, was somewhat damped in the morning by the rain, which indeed fell in torrents; but the god of fire, determitied to have
4.
before ae blink o’ the honny burdies,”
clouds, he retired to res, dispersed the and the usnal feativities began. The streets echoed to the good old tunt. ‘The lasses appeared in white and flowers : and with their favored vouths, threaded the strects in mazy columns, while Age in nn eestaev bobbling along, Beat timne with bis crutch to the tane of their sonT But ’twas soon over in the streets. The sun
went down–to report progress to his old dame Theti-leaving sister Lona to light to the Assembly-room at the Angel, all that is beadtifal. brilliant and fa hionable within
a cirele of some miles round Helston-whet, we are gratifed nt hearing, The festivities
were kept UD with degree of taste and spi- rit thet has sellom been exceeded in that so-
cial efrele. – Next morning, the sun having shook the spray from his wig, invited the fa-
shionables, gay and smiling like himself, to : horse race on the downs, where Gallant Grey and Highover re declared tbe winners.
This is the unedited ChatGPT 4 OCR output:
The spirit of the Furry-day at Helston on Wednesday last, was somewhat damped in the morning by the rain, which indeed fell in torrents; but the god of fire, determined to have
“ae blink o’ the bonny burdies,”
before he retired to rest, dispersed the clouds, and the usual festivities began. The streets echoed to the good old tune. The lasses appeared in white and flowers; and with their favored youths, threaded the streets in mazy columns, while
Age in an estsy hobbling along,
Beat time with his crutch to the tune of their song
But ’twas soon over in the streets. The sun went down—to report progress to his old dame Thetis—leaving sister Luna to light to the Assembly-room at the Angel, all that is beautiful, brilliant and fashionable within a circle of some miles round Helston—whence we are gratified at hearing, The festivities were kept up with a degree of taste and spi- rit that has seldom been exceeded in that so- cial circle.—Next morning, the sun having shook the spray from his wig, invited the fa- shionables, gay and smiling like himself, to a horse race on the downs, where Gallant Grey and Highover were declared the winners.
As you can see from the screenshot above, the text isn’t particularly clear and there are some blobs of ink and other marks that obscure text. ChatGPT’s version, whilst not perfect, was the best I’ve tried and was much quicker to verify and correct than any other OCR that I have tried. And it was much quicker to ask ChatGPT than to retype it manually.
Object Identification: A Picture’s Worth a Thousand Words
ChatGPT output: GPT vision doesn’t stop at texts; it’s also an ace at identifying and describing objects within historical photographs or illustrations. Whether it’s distinguishing between types of ancient pottery or detailing the components of a steam engine from the industrial age, ChatGPT 4 can provide descriptions and context that might take a human expert hours to compile.
ChatGPT is really trying to sell itself here. It can recognise many things in images, and it is impressive. But really only for popular and generic items. It’s hit and miss.
I uploaded an image of a 2nd century Roman Samian Ware bowl and it told me that it was Ancient Greek. A second Samian bowl image was correctly described. As the ‘old’ internet saying goes: Your Milage May Vary (YMMV).
However, that one of the answers was correct is impressive and we know that with better training accuracy will improve. Specialist archaeological pottery AIs are already showing huge promise. Perhaps in the next version of ChatGPT there might be a method to teach it to recognise specific objects in a scene (tip: this can already be done on the desktop using Liner).
As a complete aside, I uploaded a cartoon and asked ChatGPT to describe it, which it did rather well:
ALT Tags: Describing History for All
ChatGPT output: Inclusivity in historical research means ensuring that everyone has access to our shared heritage. ChatGPT 4 can automatically generate descriptive ALT tags for images on project websites and social media. These tags are essential for visually impaired users who rely on screen readers to access web content. By providing rich, descriptive ALT tags, ChatGPT helps bring historical visuals to life for everyone, ensuring a broader understanding and appreciation of the past.
In January I wrote about 40 ALT tags for a website so that they are meaningful descriptions for people using assistive technologies to browse the web. It can sometimes be a struggle to fluidly describe a scene. ChatGPT 4’s vision capabilities have proven to me that it is useful in this area, at least to provide a start.
In this example, using a photo of the view from the medieval St Levan Chapel, I explicitly asked for text suitable for an ALT tag. I felt that its output was a rather meagre description, so asked for a longer one. I would be comfortable using the longer description on a production website.
Crossing Language Barriers with Translations
ChatGPT output: Historical research often involves documents in multiple languages, some of which may be archaic or have evolved significantly over time. ChatGPT 4 can translate these texts, making them accessible to modern-day researchers. This capability not only breaks down language barriers but can also preserve the nuances of historical languages, providing a more authentic glimpse into the past.
Sometimes my current folklore research leads me to short passages of text written in Kernewek (the Cornish language). I understand bits of it, but I’ve never learned formally. Could ChatGPT understand Cornish? Well, it seems that it’s better at it than I am.
ChatGPT has taken the Late Cornish text written by Nicholas Boson in the 17th century and worked out that it is a version of the Lord’s Prayer, and presented a translation. I’m sure it’s not perfect – ChatGPT even admits that it “attempts to convey the general meaning”, but it’s not awful either. I was very surprised. Late Cornish is a fairly niche language.
The Caveat: Beware of Missing the Details
ChatGPT output: Now, while ChatGPT 4 is an extraordinary tool, it’s important to use it wisely. When summarising articles or documents, there’s a risk of missing out on vital information. AI models are designed to condense information efficiently, which means they might overlook details that a historian would consider crucial. Always double-check the summaries against the original documents to ensure no precious historical nugget is left behind.
Wise use of ChatGPT can be beneficial for researchers. The automated output above sums up some of the caveats quite well.
Ask it to regenerate answers, to clarify them, and even to walk you through an answer. The way we use language, and language transformation into machine instructions, isn’t perfect. Maybe I’ve read too much folklore, but the term “be careful what you wish for” certainly applies. We might ask a question or write a ‘prompt’ in such a way that the AI answers literally or allegorically. Think carefully about how you ask a question. The answer could have a powerful effect either way on your research. And do undertake some due diligence with some of the answers using the source materials if using a Custom GPT.
ChatGPT output: In conclusion, ChatGPT 4 stands as a powerful ally in historical research, offering an array of tools that can transform how we interact with the past. However, it’s important to remember that AI is a supplement to, not a replacement for, meticulous human analysis. Use it to explore, to expand your horizons, but let’s not forget the irreplaceable value of a historian’s discerning eye. Happy time traveling!
Is ChatGPT Plus worth it? It is to me. It’s far from perfect. As the AI wrote above, a human eye is still required to check any output or answers. In my tests this was especially apparent when I discovered that William Bottrell’s work contains racial stereotypes that were reinforced in its answers. But that’s for a post all of its own.
As a general purpose tool ChatGPT 4 is useful. I will think about keeping the Plus subscription and review it monthly. I’m excited to track its developments in terms of accuracy and natural language processing. Custom GPTs will enable powerful knowledge bases for museums to understand and make better use of their institutional knowledge. The technology will improve, and judging by the leap from ChatGPT 3.5 to 4, it will happen quickly. We truly live in exciting times.