Text+Plus, #01: From Treaty to FAIR Data: FriVer+

Text+ annually supports cooperation projects for a maximum duration of 12 months. Individuals, working groups, research consortia as well as research institutions are invited to apply. The cooperation projects of the first funding period started their work on January 1, 2023. Our blog series Text+Plus provides insights into their activities.

In 2023, the Leibniz Institute of European History (IEG) in Mainz is running the project Europäische Friedensverträge der Vormoderne in Daten (FriVer+). This initiative, funded by Text+ as a cooperation project, aims to provide (meta)data pertaining to more than 1800 early modern European peace treaties in XML format. The goal is to make this data more accessible and usable for the scholarly community. This blog post outlines the various stages of the project, reports on the progress made so far, and highlights upcoming tasks.

FriVer+ originated from a predecessor project called Europäische Friedensverträge der Vormoderne online, which was funded by the DFG and ran at the IEG from 2005 until 2010. This earlier project collected metadata from over 1800 European peace treaties dating from 1450 and 1789. Additionally, this dataset included 40 editions and 120 transliterations of the text of these treaties. All this data is stored in a SQL database accessible through a website, which not only provides the data pertaining to individual treaties, but in a number of cases digital images as well. The compilation of these images from various archives remains a unique feature of the IEG collection of European peace treaties. It should be noted, however, that in some cases the right of use is restricted.

Accessing and working with the data from this SQL database presented challenges. For example, if a user wanted to extract the text of a specific edition, they had to locate the edition on the website and manually copy and paste the text into a separate document. Furthermore, some editions only revealed their text when hovering over associated digital images. Additionally, information about individuals mentioned in the text was embedded in HTML tags as footnotes. Some database cells even contained entire editions structured in HTML. This situation fell short of the ideals encapsulated in the FAIR principles (Findable, Accessible, Interoperable, Reusable), which underlie the FriVer+ project’s mission.

To tackle these issues, FriVer+ began by examining the SQL database’s structure, although this was complicated by the lack of comprehensive technical documentation – a common issue in early digital projects. After surveying the database’s structure and its various types of information related to peace treaties, the project team reacquainted themselves with the latest version of the XML schema established by the Text Encoding Initiative (TEI). To align with the FAIR principles, FriVer+ adopted TEI P5, the most current guidelines offered by this consortium. TEI was a logical choice, given its wide use among scholars across various academic fields.

The XML schema used by FriVer+ is based on TEI P5 and mirrors the XML schema of a related project, Europäische Religionsfrieden Digital (EuReD). EuReD, a cooperation between various academic institutions in Mainz and Darmstadt, aims at creating digital editions of – as the project’s name indicates – European religious peace treaties. After the Friver+ XML schema was tested and finalized, the project moved on to export data from the SQL database and convert it into valid XML. This was a complex and complicated process, partly due to the fact that, as already mentioned, the database contains large texts marked up in HTML, which proved challenging to transform efficiently. Ultimately, it turned out to be quicker and more accurate to transform portions of the data in the SQL database separately and reassemble it later. Metadata was exported to a large CSV file and, after the data had been cleaned, enriched, and standardized, transformed into XML with the help of a simple but highly efficient online convertor. The edition and transliteration texts were exported and saved as large XML files. Subsequently, data from the various sources was manually reassembled and subjected to rigorous quality control.

It should be noted that the data in the resulting XML files was standardized and enriched, leading to discrepancies between the XML data and the SQL database. Here are some examples: The database grouped many signatories of peace treaties under “Holy Roman Emperor,” whereas the XML transcriptions differentiates between the different Holy Roman Emperors who signed a particular treaty. Moreover, additional research resulted in more individuals mentioned in the text being tagged, increasing the project’s data’s value for scholars. Lastly, people singled out in the XML transcriptions were linked to their corresponding records in online resources such as the GND Explorer or Wikidata. This process was also conducted for the locations where the treaties were signed. By integrating our data into a broader array of datasets pertaining to the early modern world, we enhance its discoverability and usability.

Currently, work on the editions is ongoing, followed by finalizing the XML transcriptions. In the coming months, in cooperation with the Centre for Digital Editions in Darmstadt (CEiD), a partner of the EuReD project, FriVer+ will upload its data to the servers of TU Darmstadt. Notably, not only will the raw XML data be made available, but it will also be displayed online. To ensure a seamless presentation of the data, relevant CSS files will be adjusted accordingly. Another important task is the creation of a comprehensive documentation, which proofed invaluable during the FriVer+ project. The documentation will detail the transformation process as well as the editorial decisions made while transforming and enriching the data, facilitating its future use by scholars. Time permitting, there are plans to update or refine the functionalities of the existing “Europäische Friedensverträge der Vormoderne online” website.

While it’s too early to draw firm conclusions about FriVer+, as a quarter of the project’s timeline remains, the project’s value is evident. Thanks to the generous funding provided by the NFDI consortium Text+, FriVer+ allows us to generate structured data in line with FAIR principles, serving the scholarly community with a valuable dataset. Furthermore, the FriVer+ project helps to ensure the continued relevance of the earlier project “Europäische Friedensverträge der Vormoderne online”. What is more, this project provides the IEG with several “lessons learned” for future legacy projects in. Lastly, FriVer+ enables us to enhance particular skills, such as data modelling and transformation, that will be invaluable in designing and executing future research projects. While the success of the project’s primary outcome – the XML data – will ultimately be assessed by our colleagues, at this stage, FriVer+ already proves highly beneficial for the IEG.

Zitiervorschlag
Jaap Geraerts: Text+Plus, #01: From treaty to FAIR data: FriVer+, in: Text+ Blog, 29.09.2023, https://textplus.hypotheses.org/7450.

Beitragsbild: ETH-Bibliothek Zürich, Bildarchiv / Fotograf: Metzger, Jack / Com_L16-0320-0002-0002 / CC BY-SA 4.0, http://doi.org/10.3932/ethz-a-000969258.

---

---