The overall goal of BRAINSPACE project is to develop novel hybrid multimodal interfaces that use biopotentials-based methods which will allow the astronauts to use supplementary communication channels while performing simple or complex robotic tasks during missions in space. The project will be achieved by actually augmenting capabilities and virtually adding “peripherals” to the user’s body. By achieving the proposed goal, the hybrid interface will provide to the ESA astronauts the possibility to perform hands-free tasks while ensuring a higher degree of safety and increased control performances.
This project has the ambition to advance a solution that aims at the concept of “tracking the human thinking” in which the user effort is minimized to control robotic systems in space missions. To develop such a solution we will use the whole range of relevant tracking modalities like electrooculography (EOG), brain computer interfaces (BCIs), electromyography (EMG) and eye/gaze tracking, innovating the necessary sets of interaction metaphors for interaction tasks. The user should be able to exploit the natural interactive potential of the interfaces on one hand, and to increase the productivity in each case, on the other hand. The role of our approach is to define the right combination of modalities and their right proportions of use within the multimodal hybrid interfaces, in order to minimize user’s effort considering that users in microgravity perform all actions different than the users on Earth. Problem to be solved and initial situation Space activities represent complex, high effort and valuable research activities for mankind. People always tried to discover as much as possible from the Universe and among the exploration related activities there have been multiple Moon exploration mission and of course since 1998 the International Space Station (ISS). The purpose of this kind of missions is to explore the space or to perform different experiments in multiple domains like biology, human biology, physics, astronomy and many more. In space, compared with Earth, the astronauts are confronted with complications and requirements when it comes about interaction with computers or machines. In microgravity the performance of experiments or even object manipulation activities required the astronauts to use their hands and feet in order to keep a given posture.
Since there are multiple risks involved in some activities the crew has manuals written in details with instructions and they are always trained for procedures for weeks or even months on ground. Usually, during an experiment, the person in charge has to take out one hand from the test in order to use the laptop for further explanations regarding it and then to turn back to the actual task at hand. Frank de Winne, ESA astronaut, states in International Review of Neurobiology that “he would be highly interested in any technology that would allow to keep him concentrated on an actual manual task and to have his both hands freed to perform them“. He also gives an example that he is sometimes using even his nose on the laptop’s track-pad just to keep his hands on the performed task. Also, astronauts are sometimes required to go outside the space station in order to perform maintenance tasks. Extra-vehicular activities are dangerous for astronauts especially due to cosmic radiation. Such extravehicular activities may be performed using robotic systems teleoperated by astronauts by means of noninvasive brain-machime interfaces (BMIs). Also, the BMIs could be used to perform multitask operations. Planned goals The project combines a series of very mature state-of-the-art industrial technologies (robotics which can be classified as “TRL 9 – Systems for which the functionality has already been proved”) with some very new advances in the field of human biopotentials based interaction like BCI, EOG, EMG and eye tracking (these can be classified as “TRL 4 – technology validated in lab”). Overall, the project aims to demonstrate a technology of the level “TRL 5 – experimental model functionality validated” by means of experimental trials. Expected results and findings Usually the alternative communication channels have been used for disabled persons which in general manage to achieve medium to high accuracy values in controlling different devices. Related to space missions, and in particular to astronauts, they should manage to achieve high to maximum control using such interfaces since their motor and cognitive control pathways are in clinical healthy conditions.
We expect to integrate biopotentials in robotic space applications with maximum control speaking in terms of classification accuracies. Also, the expected impact is related to the advances in the capacity of human-machine interaction technologies which will enable the astronauts to control with a higher degree of freedom robotic applications and to navigate within computer applications by keeping their hands out of keyboard and mouse. ESA programs relationship Brain machine interfaces (BMIs) presents a high interest from the ESA since it is expected that BMI technology will experience a great progress in the near future. Since the space is a dangerous and hostile environment for astronauts, extra-vehicular activity is required to be limited as much as possible. Thus, robotic systems should replace as much as possible such activities, but the astronauts should achieve a higher degree of control over such robotic systems. It is thus required to optimize the interfaces between astronauts and such devices by using supplementary and augmented communication and control channels. The approach from ESA to BMIs in presented here. Also, NASA presented a high interest in such interfaces since the current findings presents very promising results.