8 minute read

Guidelines for the Assignment (Draft):

The Assignment

The Spatial Data Assignment is an assignment in one step. It builds upon the work we have done in class since 6 October. You will take a portion of either the Zanzibar Gazette or Lorimer’s Gazetteer of the Persian Gulf, Oman and Central Arabia, and “wrangle” a dataset from one of them by using the LLM(s) of your choice.

  • Format: Individual or pairs (maximum 2 people)
  • Length: Approximately 1500 words (about an 8-minute read), plus visuals (screenshots or saved images from the notebook) & a dataset published to your assets folder in Github.
  • Due Date: Monday, 30 October 2026.
  • Medium: a post in Github pages, with screenshots and an interactive map

If you work in pairs, it can be written together, but should be posted on each of the team member’s course site.

We will work with the pdfs of either the Zanzibar Gazette or Lorimer’s Gazetteer of the Persian Gulf, Oman and Central Arabia (in Drive) on a project to map some aspect of the historical Gulf region or Zanzibar that appears in them. You will use an LLM to generate data from the OCR’d digital copies we possess. Although we will not combine all of our collective datasets together, imagine that the datasets that you generate could be assembled by the instructor to create a “thick mapping” of a Gulf region or Zanzibar.

Main Elements

This assignment has seven main elements: (1) choosing which text you would like to work on, (2) choosing a portion of the source material and identifying the spatial elements of this humanities source material (3) modeling a spatial dataset based on those elements as they are represented in the sources, (4) prompting the LLM to extract and organize the data according to the desired structured, assessing the output and perhaps re-prompting (5) correcting the data manually, enriching with any other relevant metadata, especially the spatial data (6) visualizing the data on a map, and (7) discussion of both process and product using an article on the humanities and genAI.

Step 1: Choose the topic that you would like to work on from within the pdfs of either source. Do not use both sources. We explored in class the sources to understand some of the information they contain–you will have to do more exploration on your own.

You should aim to create a dataset which has about 300 data points total. There is a folder in Drive of what other students have done in the past for Zanzibar. In GitHub there are other datasets that were made by the OpenGulf research team. You can choose to build on any of these expanding them in time or space, or you can choose an entirely new idea.

Important to know about Lorimer is that there are 800+ chapters describing many different tribes and places in the Gulf and Arabia. All of them contain geographic information. General areas covered here include Najd (Saudi), Eastern Arabia, Iraq, Kuwait, the Persian coast, Qatar, the Trucial States, and Oman.

Important to know about the Zanzibar Gazette is that there is much more data that is collected over time, but only a smaller amount of which has concentrated geographic information.

Here are some ideas to choose from:

  • Imports into or Exports From Zanzibar over several months or several years
  • Shipping routes
  • kind of licenses issued in Zanzibar (for commercial purposes) over several months or several years
  • Produce imported from Africa in the ZG
  • the specific individuals, companies or merchants having “enemy ties,” thereby disqualifying them from trade with Zanzibar - over the years
  • the number of fighters per tribe in Lorimer (either several tribes for few regions or a single tribe across many regions)
  • the number of houses per tribe in Lorimer (either several tribes for few regions or a single tribe across many regions)
  • kinds of boats (Mashuwahs, Ghunchahs, Baqarahs, Baghlahs, Shashahs [date stick boats], pearling boats, fishing boats, steamers) found in different regions in Lorimer
  • the depth of wells and quality of water (bad, sweet, indifferent quality, brackish) in Lorimer
  • the width of river beds in Lorimer
  • routes from place X to place Y and the wells along the way in Lorimer
  • crops map (barley, tobacco, pomegranates, mangoes, maize, etc) for a few regions in Lorimer (either several crops for few regions or a single crop across many regions)
  • sects and religious groups in Lorimer (Shi’ah, Sunni, Jews, Sabians, Christians, Hindu)
  • animal map for a few regions in Lorimer (either several crops for few regions or a single crop across many regions)
  • tribes map for a few regions in Lorimer
  • other

Step 2: There is one main difference between the Zanzibar Gazette and Lorimer. Lorimer was compiled over a few year period and does have yearly installments. The Zanzibar Gazette, as a newspaper, spans many years. If you choose Lorimer, you will need to look over multiple sections of the Gazetteer that contain similar information to capture geographic space.

If you choose the Gazette, you may need to screenshot multiple pages from the Gazette, this might mean choosing information in a similar format longitudinally in the Gazette (that is, across years).

Step 3: Modeling the data. You will need to choose what data you are going to extract from the source and imagine a structured data table into which it is set. It could be helpful to review some of the principles from an article we read earlier this term: Data Organization in Spreadsheets.

Step 4: you will semi-automate the process of data collection by using the screenshots of the pdf. Do not upload the entire pdf to an LLM. Instead extract individual pages, or use screenshots of portions of pages containing the information. Having uploading the relevant pages, prompt the LLM to extract the data for you as we did in class on 15 October. This portion of the exercise may require some trial and error. For Step 4, you can use any tools that you feel comfortable with: chatGPT, Gemini, Mistral, Claude, etc. None of the available tools will be perfect for historical materials. You can even compare the performance of the different AI tools and will give insight into the iterative way that you went about creating the data collaboratively. You could also use the same company but different versions.

One of my former students, R.K., who works for an AI company in Canada said to me “I find the new models are the best and then they tend to deteriorate or hallucinate more often as they are trained on new noisy data or the engineers stop paying attention to them while they focus on newer versions.” Feel free to comment on this point if it is relevant. In other words, you can use the age of model as a point of comparison.

Part of the exercise is about how you prompt and refine your prompts. Keep track of what you prompted and include them in your discussion and analysis below. Please provide the best prompts you use to extract the data. A callout (with the > sign in markdown) would be an excellent way to do this.

There will be some cleaning that is required to bring your dataset into shape so that it can be visualized. Please comment on what was required.

Step 5: There are many ways that you can arrive at a final dataset. In particular, it may be interesting to combine some of the elements with bespoke tags (e.g. if you are working with lists of export items, you may add on a column to the data that combines all foodstuffs together, all animal products together, or all minerals together). This possible enriching of the data will be different according to the source you have chosen. You could also use the technique we explored in Overpass Turbo to add a data layer from OSM that is indirectly related to the locations of your data (e.g. historical monuments in the region).

This might be the case if you have data over time. You can bring together different tables, by the addition of a date column.

Finally, you have two options for geocoding: you can use the draft version of the Gazetteer created by the OpenGulf group for Lorimer or by attempting to automating that process using Geocode by Awesome Table in Google Sheets so that you have latitude and longitude columns for the Zanzibar Gazette.

Step 6: Visualizing the data.

You can upload your CSV to the Notebook.

When you have created an self-contained HTML file create a folder in your assets folder called maps and push the html file to it. Instructions for embedding the map can be found here.

Step 7:

The last step is reflecting on the entire process, what you were able to create from Lorimer or the Zanzibar Gazette and the challenges of the process.

Guiding questions for this assignment:

  • What kind of source is Lorimer or the ZG?
  • What kinds of data does that source provide? Were you surprised by any of the data that colonial governments were collecting?
  • Was the data directly or indirectly communicated? In other words, did you have to assume anything if you created new columns?
  • How did you create prompt for ChatGPT, Gemini, Mistral or other tool to automate thse extraction of the information? What kind of “engineering” did you have to do to get better results with the prompts?
  • How good was the automation? What did you have to change in order to clean the data?
  • what issues mentioned in the “Data Organization in Sheets” article did you encounter?
  • When you mapped the data, did you see any interesting patterns? Were there areas of concentration of points? Did mapping lead to any visible clusters?
  • If you were to do this over many more pages then you did for this assignment (say, look at exports over the years or over all 800+ entries in Lorimer), what kind of results do you think the different scale would give?
  • If you were to do this assignment with a source not included here, what would your ideal source be?

Your assignment should be about 1500 words long, with relevant several images, embedded links. Please place your data table into your assets folder (or other location in the repository where you host your GitHub site) and link to it in your assignment. Use a markdown cheatsheet such as this one to stylize your post, adding different layout features and embedded links if needed. You can refer to other readings if you want to, but this is not necessary for this assignment.

You May Also Enjoy