Mapping Online Book Reception Across Cultures and Languages (25-27 January 2024), ZiF-Workshop (Bielefeld)

Authors: Tina Ternes (University of Basel), Xenia Bojarski (University of Zurich)

As cultural discourse has shifted increasingly to the internet, authentic data on book reception has become available in large volume and from readers all over the world. This offers a whole new perspective regarding questions on readers’ experience, literary quality, and even world literature. Therefore, it is essential to develop and evaluate new theoretical models as well as best practices regarding balanced approaches to building data sets, data privacy, and methodology for analysis, in order to account for online book reception as a global and multidimensional phenomenon.

To tackle this challenge, 23 researchers from Europe, North America, and Asia gathered for the workshop “Mapping Online Book Reception Across Cultures and Languages” (25 – 27 January 2024). At the core of the workshop stood the idea to examine online book reviews, conceived as a cultural practice of non-professional readers in the digital sphere, with a diverse and interdisciplinary set of methodological approaches.

The three-day workshop was hosted at the ZiF in Bielefeld. ZiF is the Center for Interdisciplinary Research at the University of Bielefeld (the acronym derives from its German name: “Zentrum für Interdisziplinäre Forschung”). It is an independent institution where researchers and research groups from all over the world can receive support and funding to do research and work on a project. An overview along with currently ongoing projects and more information on the ZiF can be found here.

One of the main goals of the workshop was the development of a parallel corpus of book reviews in seven languages (English, Korean, Chinese, Dutch, Italian, Danish, and German). To achieve this, various approaches from disciplines like cultural, media and literary studies, psychology, linguistics, art-philosophy, digital humanities, cultural analytics, and marketing were presented and discussed. In terms of practices, the presented works ranged from fully qualitative close readings of individual reviews to the analysis of millions of reviews via large-scale data mining.

Processing and Annotating Reviews

Berenike Herrmann (Bielefeld University) kicked off the workshop by stating that people have had a hunger for stories, as well as a need to exchange ideas about that experience, for thousands of years. Today, digital media seem to offer a fecund environment for both. At the same time, research on readers’ behavior is still a marginal arena of literary studies, while the review data produced by today’s readers offer a huge reservoir for pursuing a host of research questions. Herrmann proceeded with her presentation of a corpus of reviews from the LovelyBooks platform consisting of more than 1.2 million reviews and a wide range of different genres. Part of the corpus was initially manually annotated, which was the basis for an ensuing automatic annotation using BERT, a large transformer-based language representation model used for machine learning. This allowed forthe comparative analysis of evaluative sentences, revealing significant differences to non-evaluative sentences in their linguistic forms and discourse functions. Concluding, Herrmann summarized the next steps for the dataset such as mapping out different user types, analyzing sentiment, working with the different genres, and tackling the challenge of having an unbalanced dataset.This endeavor is also being continued by Simone Rebora (University of Verona), whose project is currently adapting a theoretical framework to a more detailed annotation scheme for evaluative language in reviews on LovelyBooks. The ultimate goal of the project is to see whether it is possible to automate this task with neural language models as well as validating the theoretical framework against review data. Rebora also identified a range of challenges, as reviews are often used, and in many cases also written, as advice for buying a certain book, but do not follow a strict pattern. One sentence can include multiple types of evaluation, while another evaluative statement can be spread over the entirety of the review or only be understood in the context of the entire review.

Experiencing Books

While it is possible to collect huge quantities of reader reviews, one has to keep in mind that reader reviews represent the response of only a small subset of readers. The majority of them read without leaving traces on the internet. Nonetheless, reader reviews are an excellent source for investigating reading experience and behavior.

Moreover, the findings presented during the course of the workshop point to the applicability of reviews for reading research. For instance, Marijn Koolen and other researchers at the Huygens Instituut (Amsterdam) found that Dutch and English readers use the same linguistic patterns in reviews as they use in in-depth interviews. While this is a promising finding for the project’s endeavor, which is to see whether reader types that have been identified in in-depth interviews are similar to those emerging from reviews on online platforms, online reviews come with their own set of challenges. For example, the motivations of online reviewers differ widely from those of interviewees and often remain opaque to the researcher.

Moniek Kuijpers (University of Basel) also aimed to test “real world” measures on online social reading data and found that reader reviews on the review platform Goodreads contained evidence of story world absorption (that is, the experience of being transported into a story, stepping into the shoes of protagonists etc.). Moreover, Kuijpers introduced an annotation scheme that could capture this phenomenon in reviews, which had previously only been investigated via a questionnaire used in studies that assessed people’s experience post-hoc. This work does not only improve the ecological validity of the concept as captured by the questionnaire but also helped to expand and adapt said instrument to match the language and experience generated naturally by readers. Furthermore, the annotation was done for English as well as German reviews, which highlighted cultural differences in talking about literature. Beside the humanities, reviews are also a much discussed topic in marketing research. This perspective was introduced by Anne Mareike Flaswinkel (Bielefeld University), whose research found that music recommendations are more likely to be accepted if they come from friends rather than from algorithms.

Different Platforms, Different Reviews

Hu, Yuerong (2024), Complexities and Nuances of Cross-Cultural Book Ratings, Mapping online book reception across cultures and languages -Towards interdisciplinary modeling & methodology, ZiF Bielefeld, p. 13.

One of the most important aspects of reviews is the context of their publication. This includes the platforms they are published on, as they come with different conventions, aims, technical affordances, as well as a cultural background. Some platforms even establish a direct connection between readers and writers or act as publishers, as is the case with Wattpad, a social storytelling platform. Claire Parnell (University of Melbourne) used a qualitative approach to explore this intersection between reading and writing. Assessing the structure of the platform itself as well as interviews with authors, Parnell shed light on targeted harassment and negative comments by readers towards the authors of still developing stories.

Federico Pianzola (University of Groningen) and colleagues found significant differences in book review writing style and average rating between retail and non-retail platforms. While in this study these differences are similar across Italian, Korean and English users, there are some local specificities related to genre. Yuerong Hu (University of Illinois Urbana-Champaign) compared different languages, too, and identified that star ratings for the same works differ significantly between the international platforms Goodreads and Douban (Book), which has a mostly Chinese user base. However, Hu also identified major challenges for the comparison, like a lack of books that are popular on both platforms, different rating systems (10 vs. 5 maximum stars) as well as the overall idiosyncrasy of rating behaviors and motivations among single users.

The Book behind the Review

Apart from the reviewers, their practices and experiences as well as the conventions and structures of the platforms, another important factor is the texts that are being reviewed.

Joris van Zundert and colleagues (Huygens Instituut, Amsterdam) mapped ‘features of impact as reported in online reviews to textual features of the novel’, i.e. the way in which stories affect readers, by looking at the books’ story arcs as well as different narratological features such as the perspective, characters, and dialogues, using among other things the method of word embeddings. Similarly, from a Computational Humanities perspective, Kristoffer Nielbo (University of Aarhus) aimed to find a ‘recipe for literary success’ by examining the story arcs of different works of literature. Using adaptive fractal analysis, Nielbo concluded that successful literature displays a tension or balance of ‘local uncertainty but global coherence’. Yuri Bizzoni (University of Aarhus) also used reviews to refer back to the book, extracting the perception of literary quality by investigating the respective online reviews. Bizzoni’s corpus consists of different sources that were compared by their average ratings. The results show that it is indeed possible to predict literary quality reception by examining different factors and their complex interplay. From a psychological perspective, Jana Lüdtke (Freie University Berlin) investigated the relation between annotated textual emotions and felt emotions in children’s fairy tales using a corpus consisting of 80 tales by the brothers Grimm.

Li, Lucy (2024), Reader Response to Characters of Color in Literature Taught in U.S. Schools, Mapping online book reception across cultures and languages -Towards interdisciplinary modeling & methodology, ZiF Bielefeld, p. 11.

Similarly, Federico Pianzola (University of Groningen) linked emotion arcs to reader response. More precisely, Pianzola and colleagues wanted to ascertain whether happier stories receive more positive reviews than sadder stories. This research shows that there is a significant association between book sentiment and review sentiment, however, different genres display different non-linear patterns for this association.

Lucy Li (University of Berkeley California) and Katja Tereshko (Huygens Instituut Amsterdam) each worked with a particular corpus for a specific purpose. Tereshko looked at reader response to present-tense fiction in Dutch. The goal was to evaluate whether present-tense literature is written in a more ‘spoken discourse-like way’ and whether there is a difference in ‘impact of literature’ that can be traced back to tense use. By assessing the amount of reviews per book, Tereshko concluded that present-tense fiction overall gets more reviews, a result that leads to interesting follow-up questions regarding style, quality, and absorption. 

Lucy Li’s research touches on the broad subject of canonical literature, investigating reader response to characters of color in literature currently taught in US schools. By looking at curricula, Li built a corpus of books used for teaching, finding that only a small proportion of books read in advanced Literature classes contain non-white main characters.

TL;DR

The three workshop days highlighted a vast variety of approaches as well as relevant dimensions to the overarching theme of mapping online book reception across cultures and languages. They showed the importance of connecting people from diverse disciplines and cultural backgrounds that conduct research within the same area of interest, which can yet be very diverse. The multilingual dataset also showed great potential for further research. This will indeed be possible, since the dataset will be available for further research thanks to the Text+ (https://text-plus.org/en/) consortium, which preserves text- and language-based research data for the long term, enabling their broad usage in the scientific community. Follow-up workshops on the same topic are in preparation for 2024 and 2025, and all interested scholars are invited to contact the coordinators of this initiative (Berenike Herrmann and Federico Pianzola).

---

---