Libraries and research can be seen as separate islands with different goals, languages and cultures. Communication and collaboration between the two can be expected to encounter difficulties, for example in relation to authority files such as the GND (Gemeinsame Normdatei, or Integrated Authority File). The following blog post reflects on these differences, and does so in an assured way, as the author knows both islands: the librarian’s and the researcher’s perspective.

Moving from one island to another

Before I started working in the library in Göttingen (Niedersächsische Staats- und Universitätsbibliothek Göttingen, SUB), I worked with authority files in the context of several DH projects on digital corpora and editions of literary texts in Romance languages, especially Spanish. After a few years of working as a subject librarian, I now realise that many of my ideas were either partially or completely wrong. In this post, I will try to illustrate some of my own former misconceptions, because I think they are (to some extent) typical of what many researchers think or expect from libraries and institutions responsible for authority files. I think it is particularly important that these institutions better understand the expectations that people have of them, especially the false ones, and decide how to address them.

Starting point

I wanted to identify people and works of literature

That’s it. I didn’t want to identify other entities like places, publishers or organisations, just people and works. More specifically, authors and literary works. And I would say that this is something very common in literary studies. That was true… for a number of years. At some point, I also wanted to identify places, such as the author’s place of birth or death, or the place where the action of the story takes place. And of course the locations were much more difficult, because many of them did not exist, or were a fictional version of a real town, sometimes with a different name. I will come back to that later.

GND was not my first option for identifying entities

Even though I was working in Germany, at a German university, with German funding, and in the German field of Romance Studies, I thought that the fact that I was not analysing people and works from the German-speaking area was reason enough to discard GND (Gemeinsame Normdatei) as the main resource for identifying these entities. As I will explain later, this was only partly true. I decided to work rather with VIAF (Virtual International Authority File) and authority files of the Spanish National Library (BNE).

Less “important” entities are less likely to be identified in authority files

I expected that many less “important” authors or works would not be identified in the authority files. What do I mean by important? There are many ways to measure the importance of entities: number of reprints, influence on other authors, translations, editions, reception by scholars, number of libraries that have books about these works or by these authors… In general, many of these possible metrics correlate with each other, and so we can only remain in a certain vague and intuitive idea of importance. In my work as a researcher and subject librarian, I have observed that this intuition was not wrong. Furthermore, in a conference paper that I will describe further in the next section, we were able to quantify it. Yes, in general, “more important” entities have a better chance of being included in authority records.

Misconceptions about authority files

Now I present some wrong expectations I had.

Misconception #1: GND has (almost) only entities from German-speaking countries

As I said, I decided not to work with the GND not only because I thought that other resources would be better suited to my purposes. In fact, I thought that the GND would only have a very small number of entities from Romance literatures.

Was I wrong? I would say yes and no. Together with my colleagues Nanette Rißler-Pipka and Florian Barth, we analysed the coverage of authors and works in the ELTeC corpora. These corpora contain literary works from several European languages. In a conference paper presented at the DHd 2023, we have analysed the presence and the level of description in the GND, VIAF and Wikidata. Our results show that the majority of authors for all languages are identified in the three resources. In other words, I was wrong, there are many non-German-speaking authors identified in the GND. However, German authors are significantly more extensively described in the GND than authors from other countries.

What about works? Well, in this case, the GND has only acceptable coverage for German-speaking works, and both Wikidata and especially VIAF have better coverage. More details can be found in the abstract of the paper.

In other words, I could have actually used the GND to identify authors, while it was a good idea to use other resources (such as VIAF or the authority files of the Spanish National Library) to identify works.

Misconception #2: GND is assigned by the DNB

One of the reasons why I thought the GND was only interesting for German Studies was the misconception that the GND is only issued by the German National Library (DNB). Like other national libraries, the DNB collects all publications from or about the country, in this case, Germany. This means that you can expect to find in the DNB catalogue either literary works originally written in German or literary works translated into German. Besides, we also expect a rather small group of editions published in Germany but in a foreign language. For example, the publisher Reichenberger would be a good example of a German publisher publishing many literary works in foreign languages (in this case, Spanish). In general, the catalog of the DNB would not be a suitable resource for foreign philologies. Therefore, we expect similar characteristics in the authority files.

However, it happens that the GND is not only assigned by the DNB. I was wrong, but I am not the only one with this misconception, many people working in research, infrastructure projects and even libraries think that this is true. I have also heard this misconception, either explicitly or implicitly, at some Text+ meetings.

There are library networks in the German-speaking area (such as GBV, Hebis, SWB, hbz…) and in each of them there are people who can edit the GND. In other words, the people editing the GND do not have to be in Frankfurt and Leipzig (the two locations of the DNB), they can be in Vienna, Bern, Munich, Stuttgart, Berlin, Cologne or Göttingen, to name a few. And many of these libraries have historical, special collections and information projects (such as the FIDs) about other countries and languages, so we can expect many of the entities of other disciplines and languages to be covered in the GND. This explains why the people from ELTeC were pretty well covered in the GND.

Misconception #3: Different authority files and knowledge bases are mapped pretty well between them

I expected that if an entity is in the GND, I would also find it in VIAF and Wikidata. I expected that VIAF and Wikidata in particular would do a good job of harvesting entities from the National Libraries and the GND.

I haven’t looked at this statistically, but I’m often surprised how frequently this doesn’t happen. VIAF often has no link to Wikidata, the same seems to be true in the opposite direction. My current understanding is that if a person is in the GND, we can expect them to be in the VIAF. But we should not expect the same for other entities (works) or in other relationships between source and target resources. This should be improved in the future.

Misconception #4: Authority file systems provide ways (APIs) to retrieve information from them

I expected that each authority file system would provide an API to retrieve all information from it. Otherwise, many of the things I wanted to do with the authority files would not be possible.

I was surprised to learn that the most commonly used API for the GND is actually Lobid, which is supported by the hbz library network. In fact, the GND offers other alternatives, such as the MARC-XML and TURTLE-RDF APIs, or the Entity Facts API (here for a general overview about the APIs). However, these APIs are much less well known to potential users.

In any case, we shouldn’t expect to automatically find APIs for retrieving data from authority files.

Besides, it can happen that you cannot get all data or metadata about the records and that you need to retrieve them from other data bases or formats. Which leads us to the greatest secret of the libraries!

Misconception #5: The master format of authority files is some LOD format

I expected the native format of authority files to be some Linked Open Data format, such as RDF or Turtle.

Now I almost blush to think that I believed that. This is far, far from the truth. The native format of the GND is the same format that many libraries use for their catalogues. It is something called PICA, which comes in two flavours: PICA+ and PICA3, both expressed in SGML.

This PICA format even has its own internal identifier (called PPN: “PICA Production Number”). Each publication in a catalog has a PPN, each class in a classification system has also a PPN (for example, each class of the Basic Classification system). In the case of the GND, each entity has also a PPN and in many cases this PPN corresponds to what we call the GND-ID, although this is not always the case. For example, the GND-ID of William Shakespeare is https://d-nb.info/gnd/118613723. The last element of this URL (118613723) is the PPN. In the original PICA format, the data of this entity looks like that:

And that, my friends, is PICA, the original format of many library resources in some countries, also of the GND.

The data in PICA can be transformed into other formats. But we know what happens in transformations from one format to another: information is lost, either because the mapping is not complete or imperfect or because the target format simply doesn’t allow certain information to be expressed. This is why for a few fields it is not enough to access the GND or catalogue data from the RDF or MARC21 representation, we also need the original PICA files. Even if it’s itchy to work with it.

Having worked as a librarian for a number of years, I understand why libraries have their own old format for much of their data. Libraries have a historic role to play in structuring and preserving information. They need stability over several decades, including technological stability. Unlike research projects, they cannot change their technology every 5 years. So it should come as no surprise that libraries use formats and systems that were created in the 90s. Even for authority files. Also, the systems that work with PICA are incredibly fast: you can run relatively complex queries on tens of millions of records and get answers in a matter of seconds. Not every technology can say that.

Open questions

Now I want to present some open questions that I had at the time, and that I think many other researchers have.

Open question #1: Identifying entities makes sense, but why exactly?

In the various projects I have worked on with authority files, there have always been multiple authors and works, and at some point we wanted to make sure that the data we were collecting on the same people or works was consistent. Searching for authority file IDs helped us to know that we were homogenising our data. Also, there was always the implicit promise that we could use these links for further steps. So it made sense to do this work.

But what exactly were we hoping to achieve with these IDs? It was never clear. Moreover, the authority files do not actually promise that we will find specific data from the identified entities. Their primary purpose is to identify entities, not to describe them. They will tend to use only as many descriptors as there are necessary for the identification.

From today’s perspective, I can say that researchers can really absorb data from the authority file systems and enrich their own data… Mostly.

Basically, it depends on the prestige of the entities (language, period, importance), the type of entity (person, organisation, work), and the type of information the researcher wants to extract. If you have a list of canonized German or British authors and you want the date and place of birth, it is very likely that the GND or Wikidata will give you this data for all your authors. If you have a list of theater plays by Cuban authors and you want their subgenre, the probability of getting the information you want from these resources is very close to zero. At some point, it feels like a matter of luck whether you get what you want from the authority files. And, of course, this has a negative effect on whether you will do the work of identifying these entities in the next project.

One advantage of the authority files is that they always get richer. Perhaps you don’t get the data you need now, but who knows which data they are going to offer in two, five or ten years!

Open question #2: This instance is not in the authority file, now what?

People who work in libraries and similar institutions underestimate how remote these institutions are from the general public. If a researcher cannot find their author in the authority file system, they will very likely give up. They may try an alternative system, or they may simply leave the person unidentified in their dataset. The researcher will rarely think, “Well, I will suggest this to the people behind this authority system”. They probably don’t have the time, don’t know how, and probably think they’ll never get an answer. If they are using Wikidata, they probably know they could add it themselves, but they probably don’t know how and they might make a few mistakes and someone will come along and delete the whole thing, so why bother?

Now that I work in the library and know the right people, I know that they are actually interested in receiving these suggestions and that they will implement the change very quickly. But of course, I am part of the library now.

If we want to get those suggestions, we need to make it much easier for researchers. For example, in the different platforms where people search (DNB catalogue, Lobid, GND explorer), we should offer links to the suggestion forms every time someone searches for an item, specially if there is no match to their query.

Open question #3: An entity is duplicated, what now?

Sometimes we have the opposite problem: we are looking for an author and we find them in the authority system… twice… Same name, same birth year, same birth place, but two identifiers. We can be pretty sure that these two identifiers are actually trying to identify the same person which has been duplicated. What to do now? Which one should I take? Both? Neither? The one with the most data?

The correct answer is probably to report it to the colleagues behind the GND and wait until they merge both IDs, but as I said, the majority of researchers will not do this. In recent years, I have reported such cases to GND colleagues at my institutions and they have corrected the data within days. The FAQ of the GND explain how to do this (question 13). Again, people need to have a very easy way to report this, otherwise they will not do it.

Open question #4: There are an open number of authority file systems. I may be ignoring some of them that are important for my research

One thing that still bothers me is that I do not know which authority file systems and similar resources (Wikidata, DBpedia, VIAF) exist, how I can find them, what I can expect from them and what I should use them for. As mentioned above, in one corpus I annotated the place where the action took place. As you would expect in literature, some of these were fictional. There are specific authority file systems for real places, but what about fictional ones? Are there any? Can I expect to find a single resource for identifying fictional places related to Greek mythology, the Bible, Star Wars or Marvel? Should I try to find specific ones? I know that many European libraries have this data, can I expect similar resources for other regions of the world? Are there specific resources for subjects that do not conform to current national boundaries, such as Latin or religions?

Normally I would try to find a colleague who works in that area and ask them. But there should be better, more structured and consistent ways of finding this.

Two cultures, two islands, many bridges

Libraries and research are like two different cultures, two different islands. They work together, most of the time they like each other, sometimes they find the inhabitants of the other island strange or annoying.

It is very positive to see the cooperation that takes place within the NFDI consortia between libraries and research. These islands are connected by several bridges, and an important one are authority files. But we have to be aware that they are still different islands, which has several implications:

1. We cannot expect the other island to behave like our island.
2. My expectations of or knowledge about the other island may be biased or simply wrong. In my opinion, while people from research know that libraries are different and recognise that they do not understand many things, people from libraries tend to think that they know how research works, often wrongly. We need to talk to several people from the other island to make sure that our assumptions are valid.
3. We need time and patience to communicate about what our island wants, expects or can offer the other island.

And all this also applies too within Text+.

Cite this article as
José Calvo Tello: 5 Misconceptions and 4 open questions about authority files and the GND from a researcher’s perspective, in: Text+ Blog, 05/31/2023, https://textplus.hypotheses.org/5199.

---

---