The purpose of the COBRA network is to train the next generation of researchers to accurately characterize and model the linguistic, cognitive and brain mechanisms that allow conversation to unfold in both human-human and human-machine interactions.
The network includes world-level academic research centers on language, cognition and the human brain as well as 4 non-academic partners that include fast-developing SMEs and one world-level company. The partners’ both unique combined expertise and high complementarity will allow COBRA to offer early-stage researchers (ESRs) an excellent training programme as well as very strong exposure to the non-academic sector.
A new research paradigm
There is a longstanding tradition of research on the relationships between language, cognition and the human brain. However, work in this domain has long been limited to studying language production or comprehension in talkers individually exposed to highly-controlled linguistic material. Recently, major advances have been simultaneously accomplished in language sciences, cognitive sciences and neurosciences, which have brought us on the verge of a new research paradigm. Language sciences have entered a new phase as they move away from individually-administered protocols towards the characterization of how spoken language is jointly used by two or more talkers as a shared set of resources for interacting with each other. This has occurred in conjunction with the advent of increasingly large databases on conversational spoken language, together with that of powerful large-scale spoken-language processing tools and techniques.
Cognitive sciences and neurosciences have also undergone a paradigm shift, which has made them pass from a single-brain to a multi-brain frame of reference (Hasson et al. 2012; Schilbach et al. 2013) as a new challenge has arisen that consists in understanding how the brains of two people speaking with each other are temporarily coupled. These advances make it possible to explore language and the brain in the context in which they both primarily develop, i.e. social interactions. Our network will aim to train 15 Europe-based PhD students in this emerging highly interdisciplinary research field: conversational brains.
Focusing on interactive alignment and prediction
Research and training activities will focus on two major mechanisms employed by speakers in a conversational interaction, namely interactive alignment, and prediction. Interactive alignment refers to the process by which people align their representations at different linguistic levels at the same time, by making use of each others’ choices of sounds, words, grammatical forms, and meanings (Garrod & Pickering 2004). Alignment contributes to making conversation easy by setting a common ground between speakers, for these speakers to have a better joint understanding of what they are talking about. Prediction refers to the process by which one speaker predicts what the other speaker is likely to say next, as well as how and when it will be said (Pickering & Garrod 2013). It is a major factor in what makes conversation fluent, and in the dynamics of turn-taking, those key moments in the interaction where one speaker hands over the conversational floor to her/his interlocutor. New metrics will be devised for the quantitative assessment of inter-speaker alignment and prediction and the speakers’ underlying cognitive and brain activities. ESRs will be trained to explore these mechanisms across a large variety of languages and communicational settings, and will address two main challenges. The first challenge will be to determine how alignment and prediction may both rely on and contribute to setting up brain-to-brain coupling relationships. The second challenge will relate to the development of computational models of alignment and prediction for more effective and socially-acceptable text-to-speech synthesizers, human-machine dialogue systems, and social robots. This will open the way towards using neurobehavioral measures for both the on-line monitoring of artificial agents, and the offline evaluation of quality of communication in human-machine interactions.
Within the consortium, eight European languages (English, Dutch, Finnish, French, German, Italian, Slovak, Swedish) as well as Mandarin Chinese will be represented that will allow the ESRs to confront a wide set of linguistic phenomena spread across the European territory and Asia.
Technological and societal implications
Transfer-of-technology activities will include innovative experimental set-ups for the joint monitoring of brain and physiological activities in two or more people engaged in a spoken-language interactions, as well as the development of spoken-dialogue systems and social robots with high-level conversational skills. The training programme will also have major societal implications as it will address issues that relate to communication in people’s native and non-native languages. The focus on the dynamics of conversational exchanges between people and on the mechanisms employed for the establishment of a conversational common ground will make it possible to significantly further our understanding of what makes spoken language communication efficient and successful in a large variety of languages and interaction situations.