Léa Pillette

General context:
Mental-Imagery based Brain-Computer Interfaces (MI-BCIs), which are neurotechnologies enabling their users to command external digital devices, such as neuroprostheses or videogames, only by performing mental imagery tasks, such as imagining an object rotating in space [Clerc et al., 2016]. While MI-BCI users perform mental-imagery tasks, their brain activity is recorded (often using electroencephalography), processed and translated into commands for the system. To control MI-BCI, the user first needs to train to produce a reliable brain activity recognizable by the system. During the training, a feedback, which is an information provided to a learner regarding aspects of the performance or understanding of the task/skills to learn, is provided by the MI-BCI system. The feedback is a fundamental component of the MI-BCI training [Lotte et al., 2013]. However, the literature and experimental results suggest that current MI-BCI training protocols, including the feedback, are inappropriate to acquire MI-BCI related skill [Jeunet et al., 2016]. Some of the users’ traits and states, i.e., respectively their stable and temporary personal characteristics, are correlated with the ability to acquire MI-BCI skills. Tensed and non-autonomous users, for example, tend to have lower control over MI-BCI than relaxed and autonomous users [Jeunet et al., 2016]. The current lack of reliability of MI-BCI limits the development of the technology outside of research laboratories.

Supportive feedback during MI-BCI training:
The current feedback used for MI-BCI user training contains little or no supportive content, i.e., a social presence and/or emotional feedback. Only simple forms of feedback with a supportive content had been tested for MI-BCI training, e.g., smileys [Kübler et al., 2001]. However, neurophysiological studies as well as theoretical MI-BCI studies demonstrate the importance of a supportive content [Mathiak et al., 2015 ; Sexton et al., 2015]. We hypothesized that the fact that non-autonomous and anxious people have poorer performance in controlling MI-BCI than others, could be related to this lack of social context and emotional feedback [Jeunet et al., 2015]. Therefore, we tested the influence of two complex forms of feedback with a supportive content.

Using learning companions to provide supportive MI-BCI feedback:
A learning companion is “a computer-simulated character, which has human-like characteristics and plays a non-authoritative role in a social learning environment” [Chou et al., 2003]. Learning companions provide a supportive feedback and have been shown to improve training in different disciplines, but they had never been been explored for MI-BCIs.
In a set of experiments, we designed, implemented and tested the first artificial learning companion dedicated to MI-BCI user training. Our learning companion was called PEANUT for Personalized Emotional Agent for Neurotechnology User Training (see Picture). PEANUT provided a social presence and an emotional feedback to the users in between trials through interventions that were composed of both spoken sentences and displayed facial expressions. The effect of PEANUT on the training outcomes, i.e., performances and user-experience, were evaluated in a study including 28 participants (10 participants trained with PEANUT, 18 control participants, 3 sessions of 2h per participant). This study revealed that PEANUT had a differential impact depending on the autonomy of the users. PEANUT had a beneficial influence on non-autonomous persons’ performances, who were shown to have lower performances than the others in previous studies [6]. Though, a detrimental effect seemed present for autonomous participants. Also, PEANUT had a beneficial impact on the perception that users had of their ability to learn and memorize how to use MI-BCI, i.e., a component of the user-experience. All the details regarding the implementation, design and test of PEANUT can be found in the article published in International Journal of Human-Computer Studies in 2020 .

Using learning companions to improve MI-BCI feedback in general:
These encouraging results encouraged us to investigate the other potential benefits that learning companions could bring to the feedback for MI-BCI user training. A learning companion could be used for much more than providing supportive feedback. For example, an example-based learning companion could be used to help develop more informative feedback on how users should perform mental imagery tasks. Several guidelines, open challenges and potential solutions were suggested to design and use learning companions for MI-BCI user training. All of which can be found in the article published in Workshop on Artificial Companion Affect Interaction Conference in 2018