Cohort 5 (2023-2027)

This project will focus on how an artificial agent can analyse and explain data patterns in user-collected health data. This will investigate how to build a trustworthy relationship between a user and an artificial agent. The project will conduct a systematic review of health tracking and analysis apps and build an artificial agent that can analyse, explain and interpret user-collected health data (using wearables and apps) to find patterns. Health data will be collected from users via user studies that investigate how these patterns can be best communicated to users. The PhD programme will contribute technical approaches to build trustworthy artificial agents, a better understanding of principles underlying trustworthy social relationships between agents and humans, and the ability to engage in collaborative sense-making of health data. This will be able to improve current approaches to health apps to interconnect personalised healthcare and recommend behaviour changes. In year 1, the focus will be on the taught programme with a start in exploring the relevant literature and technologies. In Year 2, the focus will be on conducting a systematic literature review while making a start to collect sample data from users to develop a data analysis package. In years 3 and 4, an artificial agent will be built that can interact with users to allow for the study of what aspects of interactions result in successful and trustworthy relationships.

References
Richard Brown, Lynne Coventry, Elizabeth Sillence, John Blythe, Simone Stumpf, Jon Bird, and Abigail C. Durrant. 2022. Collecting and sharing self-generated health and lifestyle data: Understanding barriers for people living with long-term health conditions – a survey study. DIGITAL HEALTH 8: 20552076221084456 (link here).

Adrian Bussone, Simone Stumpf, and George Buchanan. 2016. It Feels Like I’m Managing Myself: HIV+ People Tracking Their Personal Health Information. In Proceedings of the 9th Nordic Conference on Human-Computer Interaction (NordiCHI ’16), 1–10 (link here).

Jessica Schroeder, Ravi Karkar, Natalia Murinova, James Fogarty, and Sean A. Munson. 2019. Examining Opportunities for Goal-Directed Self-Tracking to Support Chronic Condition Management. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 3, 4: 151:1-151:26 (link here).

Lena Mamykina, Daniel A. Epstein, Predrag Klasnja, Donna Sprujt-Metz, Jochen Meyer, Mary Czerwinski, Tim Althoff, Eun Kyoung Choe, Munmun De Choudhury, and Brian Lim. 2022. Grand Challenges for Personal Informatics and AI. In Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems (CHI EA ’22), 1–6 (link here).

Perepelkina, Olga, and Alessandro Vinciarelli. “Social and Emotion AI: The Potential for Industry Impact.” 2019 8th International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW). IEEE, 2019.

Aims and Objectives
This project will establish key features of just-in-time adaptive interventions (JITAIs; Nahum-Shani et al., 2018) that contribute to habit formation. While JITAIs have been shown to be more effective than statically controlled interventions (Wang & Miller, 2020), little is known about the precise interaction of adaptive intervention features and psychological processes that lead to lasting health habits. The research will be conducted in the domain of hydration, i.e., water drinking. Water drinking is an ideal domain to study the effect of JITAIs, given that water drinking is a relatively simple health behaviour, compared to, for example, eating or physical activity; water drinking needs to happen frequently each day, so that it is susceptible to habit formation; and indeed, healthy water drinkers seem to rely heavily on habits (Rodger et al., 2021). However, many people are underhydrated, with implications for cognitive functioning, mood, and physical health (e.g., risk of diabetes, overweight, kidney damage; see Muñoz et al., 2015; Perrier et al., 2020).

Working with the PUL smartcap and accompanying smartphone app, we will examine how habit formation occurs using an intervention that provides goal setting, monitoring, feedback, as well as situated and personalised reminders. We will address questions such as: Which intervention features predict habit formation, and how can the intervention be optimised to facilitate this process? Given that habits form in response to stable context cues, do reminders at specific, fixed times facilitate habit formation compared to “smart”, adaptive reminders? What is the role of rewarding feedback in habit formation and habit maintenance (cf. Papies et al., 2020)? Does the intervention lead to “specific” habit formation (i.e., drinking water with the PUL device) or to “generalised” habit formation (i.e., drinking water)? Which intervention features (e.g., dynamic goals, smart reminders, visual reward signals) or intervention effects (e.g., reduced dehydration symptoms) predict continued engagement with the app? In addressing these questions, we will conduct research that speaks to JITAI development, as well as to fundamental psychological questions about situated learning and habit change.

Methods, Outputs and Impact
We will work closely with the PUL Hydration team to conduct both qualitative, quantitative, and mixed-methods experimental studies to assess the questions outlined above in real-life settings. We will present and publish our findings as three empirical subprojects (averageing one per year) in both Computer Science and Psychology conferences and journals. In addition to academic and industry users, the findings on how to develop healthy water drinking habits will be of interest to the general public. The Healthy Cognition Lab regularly engages in knowledge exchange activities, which the student would participate in. We also regularly engage with industry and third-sector partners, such as Danone or the British Dietetic Association. Finally, the student on this project would work closely with other ECR lab members working on hydration, and on healthy and sustainable eating behaviours.

References
Muñoz, C. X., Johnson, E. C., McKenzie, A. L., Guelinckx, I., Graverholt, G., Casa, D. J., … Armstrong, L. E. (2015). Habitual total water intake and dimensions of mood in healthy young women. Appetite, 92, 81–86 (link here).

Nahum-Shani, I., Smith, S. N., Spring, B. J., Collins, L. M., Witkiewitz, K., Tewari, A., & Murphy, S. A. (2018). Just-in-Time Adaptive Interventions (JITAIs) in Mobile Health: Key Components and Design Principles for Ongoing Health Behavior Support. Annals of Behavioral Medicine: A Publication of the Society of Behavioral Medicine, 52(6), 446–462 (link here).

Papies, E. K., Barsalou, L. W., & Rusz, D. (2020). Understanding Desire for Food and Drink: A Grounded-Cognition Approach. Current Directions in Psychological Science, 29(2), 193–198 (link here).

Perrier, E. T., Armstrong, L. E., Bottin, J. H., Clark, W. F., Dolci, A., Guelinckx, I., Iroz, A., Kavouras, S. A., Lang, F., Lieberman, H. R., Melander, O., Morin, C., Seksek, I., Stookey, J. D., Tack, I., Vanhaecke, T., Vecchio, M., & Péronnet, F. (2020). Hydration for health hypothesis: A narrative review of supporting evidence. European Journal of Nutrition (link here).

Rodger, A., Wehbe, L. H., & Papies, E. K. (2021). “I know it’s just pouring it from the tap, but it’s not easy”: Motivational processes that underlie water drinking. Appetite, 164, 105249 (link here).

Wang, L., & Miller, L. C. (2020). Just-in-the-Moment Adaptive Interventions (JITAI): A Meta-Analytical Review. Health Communication, 35(12), 1531–1544 (link here).

This project explores how neuromodulation based on closed-loop interaction systems and brain stimulation can regulate brain activity responsible for locomotion [1] in human participants. This involves coupling an external stimulator (for transcranial alternating current stimulation, tACS) to specific dynamics of neural circuits, which has shown promising results for interacting with rhythmic brain and motor activity (see e.g. [2] for suppressing tremor in Parkinson; see [3] for an application in locomotion).

Brain stimulation through tACS is a powerful, non-invasive, versatile tool for studying the human brain in vivo. It can alter rhythmic brain activity and associated sensory, cognitive and motor processes, as well as cortical excitability and subsequently neuroplasticity as revealed by the lasting changes in evoked potentials. Beyond application in sensory, motor and cognitive neuroscience, it is under investigation as a non-invasive therapeutic tool in several neurological disorders. In this context, immediate and long lasting effects in synaptic efficacy are imposed to either enhance cortical excitability, or inhibit it, likely by entrainment of underlying brain oscillations [4].

Oscillations are considered to play a fundamental role in the communication of brain cells and hence in the organization of brain networks. In fact, recent findings support a complex communication system based on cross-frequency coupling. The promise of closed loop approaches lays in the fact that it is possible to modulate this oscillatory network activity of the brain reflecting brain states, translating in the modulation of motor and mental processes, including shift of attention, emotions and so on.

Devising a closed-loop system of brain stimulation is a powerful way to provide fine control modulations of brain activity. Human motion data can be acquired and processed online to extract information of repetitive patterns [5,6] and thus help to shape neuromodulation in order to enable brain network entrainment.

References
1. McNamara et al. ‘Stable, interactive modulation of neuronal oscillations produced through brain-machine equilibrium’, Cell Reports, 2021.

2. Brittain et al. ‘Tremor Suppression by Rhythmic Transcranial Current Stimulation’, Current Biology, 2013.

3. Koganemaru et al. ‘Cerebellar transcranial alternating current stimulation modulates human gait rhythm’, Neuroscience Research, 2020.

4. Lakatos et al. ‘A New Unifying Account of the Roles of Neuronal Entrainment’, Current Biology, 2019.

5. Domingos et al. ‘Intention Detection of Gait Adaptation in Natural Settings’, IEEE Symposium Series on Computational Intelligence, 2021.

Cultivating and maintaining mental health is a significant challenge for many residents in care homes. Depression and loneliness, self isolation and low levels of life satisfaction are common among the elderly, who are also more likely to suffer from other physical health problems compared to community-dwelling elderly citizens. Many care homes do not (and can not) provide one-on-one, person-centred care due to lack of resources. This project aims to build data-driven AI models using multimodal machine learning to create a comprehensive picture of care home residents’ mental health. The aim will be to use these models to predict the emergence of potential mental health problems as early as possible, based on analysing multimodal data (audio, video, wearables) collected from care homes.

Main Objectives and Novelty
There has been an increased interest in automatic detection of mental health problems over the past several years, mainly focussing on (1) signals collected from mobile phones and wearables; and (2) younger, tech-savvy populations. However, audio-visual signals provide a vast array of extra cues that can improve inference models (Lin et al. 2021, Zhang et al. 2020), but which are currently underused. The main aims of this project are to:

1. Build a dataset of structured interviews to be collected at care homes using multimodal sensors (audio/video/wearable sensors).
2. Devise novel machine learning models that extend state-of-the-art methods to analyse and use the multimodal signal collected to predict mental health conditions.
3. Validate and evaluate the accuracy of these models via quantitative and qualitative measures of loneliness, depression and anxiety among aged care residents, as well as build a clear picture of aged care residents’ feelings and personal experience with this technology through qualitative interview methods.

Methods
This project will use experimental methods on collecting, validating and evaluating multimodal data related to mental health (Lin et. al. 2021, Laban et. al. 2021,2022). Using the collected data, it will also build on and extend state-of-the-art approaches on multimodal data representation and feature selection to devise inference models to predict and correlate with mental health conditions and risks identified within care homes.

Likely Outcome and Impact
Nearly a half million UK residents currently live in care homes, representing 4% of the population older than 65, and 15% of those aged 85 and over. The onset of the global coronavirus pandemic, and resulting restrictions on face-to-face meetings with friends, family and loved ones has highlighted how fragile human mental health is when faced with even short-term restrictions to socialising, and these effects have been experienced even more acutely by older individuals living in live-in care settings (Cross and Henschel 2020). If we are able to achieve our goals with this project, we will be able to develop tools to assist residents as well as care home staff to identify when individuals are at risk of deteriorating mental health and/or in need of extra 1:1 care or companionship from staff.

References
[1] Lin, W., Orton, I.,Li, Q., Pavarini, G. and Mahmoud, M. (2021). Looking At The Body: Automatic Analysis of Body Gestures and Self-Adaptors in Psychological Distress. IEEE Transaction on Affective Computing.

[2] Zhang, Z., Lin, W., Liu, M. and Mahmoud, M. (2020). Multimodal Deep Learning Framework for Mental Disorder Recognition. IEEE International Conference on Automatic Face and Gesture Recognition.

[3] Laban, G., Ben-Zion, Z. & Cross, E. S. (2022). Social robots for supporting post-traumatic stress disorder diagnosis and treatment. Frontiers in Psychiatry (in press).

[4] Laban, G., George, J.-N., Morrison, V. & Cross, E. S. (2021). Tell me more! Assessing interactions with social robots from speech. Paladyn: Journal of Behavioral Robotics, 12(1), 136-159.

[5] Cross, E. and Henschel, A. (2020) The neuroscience of loneliness – and how technology is helping us (link here).

Remember the moments when you find yourself tweaking the same line of code over and over or find yourself reading the same paragraph again and again! Those are the moments when your brain was overtaxed or – clinically – you had an acute mental fatigue episode. While effective fatigue mitigation strategy is a subject of intense research, recent studies have shown that therapeutic music can help mitigate fatigue and the associated sleep disorder and memory decline. In this research, we ask – what contributes to mental fatigue and aspire to devise techniques to measure these factors using sensory earables accurately? Next, we ask what makes music therapeutic and aim to study therapeutic features of music towards the automatic transformation of songs and music to their therapeutic versions. Finally, we want to bring these two facets together, i.e., identifying and predicting fatigue episodes and contextually offering mindful intervention to help manage fatigue with therapeutic music. We will leverage sensory earables for modeling fatigue building upon observed biomarkers and for real-time generation and playback of therapeutic music using acoustic channels. We will evaluate the developed solution initially in controlled lab settings followed by ecologically valid in-the-wild settings assessing both efficacy and usability of the solution. We anticipate, our findings will uncover a set of unique physiological dynamics that explain why we feel what we feel and advocate the principles to design a practical fatigue management toolkit with earables.

References
Fahim Kawsar, Chulhong Min, Akhil Mathur, and Alessandro Montanari. “Earables for Personal-scale Behaviour Analytics”, IEEE Pervasive Computing, Volume: 17, Issue: 3, 2018

Andrea Ferlini, Alessandro Montanari, Chulhong Min, Hongwei Li, Ugo Sassi,and Fahim Kawsar. “In-EarPPG for Vital Signs”, IEEE Pervasive Computing, 2021

Greer et al. A Multimodal View into Music’s Effect on Human Neural, Physiological, and Emotional Experience. In Proceedings of the 27th ACM International Conference on Multimedia (MM ’19). 167–175 (link here).

Human social interaction is increasingly mediated by technology, with many of the signals present in traditional face-to-face interaction being replaced by digital representations (e.g., avatars, nameplates, and emojis). To communicate successfully, participants in a conversational interaction must keep track of the identities of their co-participants, as well as the “common ground” they share with each—the dynamically changing set of mutually held beliefs, knowledge, and suppositions. Perceptual representations of interlocutors may serve as important memory cues to shared information in communicative interaction (Horton & Gerrig, 2016; O’Shea, Martin, & Barr, in press). Our main question concerns how digital representations of users across different interaction modalities (text, voice, video chat) influence the development of and access to common ground during communication. To examine the impact of digital user representations on real-time language production and comprehension, the project will use a variety of behavioral methods including visual world eye-tracking (Tanenhaus, et al. 1995), latency measures, as well as analysis of speech/text content. In the first phase of the project, we will examine how well people can keep track of who said what during a discourse depending on the abstract versus rich nature of user representations (e.g., from abstract symbols to dynamic avatar-based user representations), and how these representations impact people’s ability to tailor messages to their interlocutors, as well as to correctly interpret a communicator’s intended meaning. For example, in one such study, we will test participants’ ability to track “conceptual pacts” (Brennan & Clark, 1996) with a pair of interlocutors during an interactive task where each partner appears (1) through a video stream; (2) as an animated avatar; or (3) as a static user icon. In the second phase, we will examine whether the nature of the user representation during encoding affects the long-term retention of common ground information. In support of the behavioural experiments, this project will also involve developing a range of conversational agents, both embodied and speech-only, and defining appropriate behaviour models to allow those agents to take part in the studies. The defined behaviour will incorporate both verbal interaction as well as non-verbal actions, to replicate the full richness of human face-to-face conversation (Foster, 2019; Bavelas et al., 1997). Insights and techniques developed during the project are intended to improve interfaces for computer-mediated human communication.

References
Bavelas, J. B., Hutchinson, S., Kenwood, C., & Matheson, D. H. (1997). Using Face-to-face Dialogue as a Standard for Other Communication Systems. Canadian Journal of Communication, 22(1).

Brennan, S. E., & Clark, H. H. (1996). Conceptual pacts and lexical choice in conversation. Journal of Experimental Psychology: Learning, Memory, and Cognition, 22, 1482.

Foster, M. E. (2019). Face-to-face conversation: why embodiment matters for conversational user interfaces. Proceedings of the 1st International Conference on Conversational User Interfaces – CUI ’19. the 1st International Conference.

Horton, W. S., & Gerrig, R. J. (2016). Revisiting the memory‐based processing approach to common ground. Topics in Cognitive Science, 8, 780-795.

O’Shea, K. J., Martin, C. R., & Barr, D. J. (2021). Ordinary memory processes in the design of referring expressions. Journal of Memory and Language, 117, 104186.

Tanenhaus, M. K., Spivey-Knowlton, M. J., Eberhard, K. M., & Sedivy, J. C. (1995). Integration of visual and linguistic information in spoken language comprehension. Science, 268, 1632-1634.

Aims and Objectives

This project will explore which features, and behaviour change techniques apps could employ, in order to effectively support individuals with reducing meat-consumption and increasing plant-consumption for health, sustainability, or animal welfare reasons. The project aims to add to the evidence base of which features can be helpful to include within mobile app interventions tailored towards people interested in eating more plant-based but that are facing barriers, to help guide them through the stages of change as modelled by the Transtheoretical Model (TTM) with a goal of long-term behaviour change and habit formation. The project will also consider the potential usefulness of artificial intelligence strategies to aid this goal, such as considering which user information can be used to personalize the intervention components.

The project aims to follow the Medical Research Council’s (MRC) guidelines for complex intervention development (Skivington et al., 2021) and draw on various theories of behaviour change as well as the Behaviour Change Wheel framework (Michie et al., 2014).

Background

There are many reasons why supporting individuals with eating more plant-based is important. For example, processed and red meats have been classified by the World Health Organization as being carcinogenic (Bouvard et al., 2015). Despite this, research estimates that 34% of the UK population exceeds the recommendation by the Scientific Advisory Committee on Nutrition (SACN) of consuming no more than 70 g per day of red or processed meats (Stewart et al., 2021). Additionally, less than 0.1% of UK adults adhere to all nine Eatwell Guide recommendations for optimal nutrition, with only 7.2% meeting the recommendation for dietary fibre intake and 26% meeting the recommendations for fruit and vegetable intake (Scheelbeek et al., 2020).

Another reason for adopting more plant-based diets is their effectiveness of lowering one’s individual contribution to emitted greenhouse gas (GHG) emissions, thus making it an empowering tool for individuals concerned about climate change. For example, Burke et al. (2023) found there’s a potential to reduce one’s individuals contributions to GHG emissions within high income countries by 69% by switching from a typical omnivorous diet to a vegan diet, and by 22% by switching to a lacto-ovo-vegetarian/pescatarian diet from an omnivore diet.

There are also a variety of animal welfare ethical arguments for adopting more plant-based diets which also interlinks with human health arguments. For example, the majority of meat, dairy, eggs, and fish are produced in factory farms or fish farms, in which animals and fish are confined to tight unhygienic spaces to the extent that it is a standard practice to give the fish and animals anti-biotics as a preventative measure for illness. For example, 73 % of all anti-microbials used in 2017 worldwide was used in animals, thus making the animal food industry a major contributor to increases in antimicrobial resistance world-wide, with implications for the sustainability of our food system and human health (Mulchandani et al., 2023).

Given the many important and pressing reasons for including more plants in one’s diet and decreasing consumption of animal foods, that are relevant to most people worldwide, it is important to develop tools that are scalable and accessible to individuals. Smartphone apps fall into this category, with studies consistently finding they can support with developing healthier eating habits, especially internet-based mobile apps such as nutrition specific and social media apps (Seid et al., 2024). However, little is known about what features to build into apps to specifically support people with both improving the nutritional quality of their diet by including more plants, and to reduce the consumption of animal foods. Despite data suggesting there’s an increased awareness of reasons for eating more plant-based, people face a variety of barriers to adopting more plant-based diets which would need to be addressed (e.g., Rickerby & Green, 2024).

References

Bouvard, V., Loomis, D., Guyton, K. Z., Grosse, Y., Ghissassi, F. E., Benbrahim-Tallaa, L., Guha, N., Mattock, H., & Straif, K. (2015). Carcinogenicity of Consumption of Red and Processed Meat. The Lancet Oncology, 16(16), 1599–1600. https://doi.org/10.1016/S1470-2045(15)00444-1

Burke, D. T., Hynds, P., & Priyadarshini, A. (2023). Quantifying Farm-to-Fork Greenhouse Gas Emissions for Five Dietary Patterns Across Europe and North America: A Pooled Analysis from 2009 to 2020. Resources, Environment and Sustainability, 12, 100108. https://doi.org/10.1016/j.resenv.2023.100108

Michie, S., Atkins, L., & West, R. (2014). The Behaviour Change Wheel: A Guide to Designing Interventions. Silverback Publishing.

Mulchandani, R., Wang, Y., Gilbert, M., & Boeckel, T. P. V. (2023). Global Trends in Antimicrobial Use in Food-Producing Animals: 2020 to 2030. PLOS Global Public Health, 3(2), e0001305. https://doi.org/10.1371/journal.pgph.0001305

Rickerby, A., & Green, R. (2024). Barriers to Adopting a Plant-Based Diet in High-Income Countries: A Systematic Review. Nutrients, 16(6), Article 6. https://doi.org/10.3390/nu16060823

Scheelbeek, P., Green, R., Papier, K., Knuppel, A., Alae-Carew, C., Balkwill, A., Key, T. J., Beral, V., & Dangour, A. D. (2020). Health Impacts and Environmental Footprints of Diets that Meet the Eatwell Guide Recommendations: Analyses of Multiple UK Studies. BMJ Open, 10(8), e037554. https://doi.org/10.1136/bmjopen-2020-037554

Seid, A., Fufa, D. D., & Bitew, Z. W. (2024). The Use of Internet-Based Smartphone Apps Consistently Improved Consumers’ Healthy Eating Behaviors: A Systematic Review of Randomized Controlled Trials. Frontiers in Digital Health, 6, 1282570. https://doi.org/10.3389/fdgth.2024.1282570

Skivington, K., Matthews, L., Simpson, S. A., Craig, P., Baird, J., Blazeby, J. M., Boyd, K. A., Craig, N., French, D. P., McIntosh, E., Petticrew, M., Rycroft-Malone, J., White, M., & Moore, L. (2021). A New Framework for Developing and Evaluating Complex Interventions: Update of Medical Research Council Guidance. BMJ, 374, n2061. https://doi.org/10.1136/bmj.n2061

Stewart, C., Piernas, C., Cook, B., & Jebb, S. A. (2021). Trends in UK Meat Consumption: Analysis of Data from Years 1–11 (2008–09 to 2018–19) of the National Diet and Nutrition Survey Rolling Programme. The Lancet Planetary Health, 5(10), e699–e708. https://doi.org/10.1016/S2542-5196(21)00228-X

Main Aims and Objectives
The main goal of this project is to develop approaches for automatic inference of social and psychological phenomena taking place in phone-based conversations. The focus will be on personality traits, conflict management style and interpersonal attraction. The key-assumption is that such phenomena leave physical, machine detectable traces in terms of social signals [Vin09], i.e., verbal and nonverbal behavioural cues such as laughter, tone of voice, lexical choices, turn-taking, pauses, etc. [Pog12]. If such cues can be detected automatically, it is possible to use them for inferring the underlying social and psychological phenomena. The project is highly interdisciplinary and it requires one to take into account both technological and psychological aspects of the problem.

Proposed Methods
The main experiments will be performed over an available corpus of 60 conversations (120 speakers in total) involved in a negotiation (the Winter Survival Task), for a total of roughly 12 hours of material [Vin15]. The data is annotated in terms of psychological constructs (personality traits, conflict management style, interpersonal attraction) and major nonverbal behavioural cues (laughter, hesitations such as “ehm” or “uhm”, pauses, etc.). This will allow the development of multimodal Deep Learning approaches [Bal19] capable to jointly process speech signals and automatic data transcriptions to output information about the social phenomena above. Such part of the project will involve the application of Computational Paralinguistics methodologies (extraction of Low Level Descriptions, etc.) [Sch13] and deep learning models for the analysis of text (word embeddings, BERT, etc.) and speech signals (Recurrent neural Networks, Long Short-term Memory Networks, Convolutional Neural Networks, etc.).

Likely Outputs
This project will lead to the development of multimodal methodologies for the automatic analysis of conversations. While being developed using the data described above, the approaches will be designed to generalize, i.e., to work over other types of data too.

Possible Impact
The project will contribute to endow machines with social intelligence, i.e., the ability to make sense of social interactions in the same way as people do. This is seen as one of the last steps towards the development of AI approaches capable to seamlessly integrate the life of people.

Alignment with Industrial Interests
The project is highly relevant to Conversational AI, one of the key-areas underlying the development of commercially important technologies such as personal assistants (Alexa, Siri, etc.), Human-Robot Interaction, interactive entertainment, etc. In this respect, the project is likely to be of major interest to all industrial sectors in which speech interfaces are of interest.

References
[Bal19] T. Baltrušaitis, C. Ahuja and L. -P. Morency, “Multimodal Machine Learning: A Survey and Taxonomy,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 41, no. 2, pp. 423-443, 2019.

[Pog12] I.Poggi and F.D’Errico, “Social signals: a framework in terms of goals and beliefs”, Cognitive Processing, Vol. 13, pp. 427-445, 2012.

[Sch13] B.Schuller, A.Batliner, “Computational paralinguistics: emotion, affect and personality in speech and language processing”, John Wiley & Sons, 2013.

[Vin09] A.Vinciarelli, M.Pantic, and H.Bourlard, “Social Signal Processing: Survey of an Emerging Domain“, Image and Vision Computing Journal, Vol. 27, no. 12, pp. 1743-1759, 2009.

[Vin15] A.Vinciarelli, E.Chatziioannou and A.Esposito, “When the Words are not Everything: The Use of Laughter, Fillers, Back-Channel, Silence and Overlapping Speech in Phone Calls“, Frontiers in Information and Communication Technology, 2:4, 2015.