This vision is for a seven-year time horizon: it is to be achieved by the end of the regular funding period of the HBP, i.e., by the time the HBP enters the status of a European Research Infrastructure. So, by 2023, we expect our current research in “Future Computing and Robotics” to have produced a number of unique, tangible results in the form of “products” and a number of ground breaking “base technologies and methods” that will significantly facilitate and accelerate future research in the European Infrastructure in a diverse range of fields.
In conjunction with future computing, HBP’s robotics research plays multiple, significant roles in the HBP:
(Closed Loop Studies): it links the real world with the “virtual world of simulation” by connecting physical sensors (e.g., cameras) in the real world to a simulated brain. This brain controls a body which, in turn, can impact and alter the real world environment. Robotics, therefore, provides the potential to perform realistic “closed-loop-studies”: perception – cognition – action. This will establish a whole new field of robot design: virtual prototyping of robots that can then be readily built as real machines and function like the simulated ones. This will not only speed up robot development by orders of magnitude, it will also dramatically improve the testing and verification of their behaviour within a wide variety of circumstances.
(Brain-Derived Products): it links brain research to information technology by using scientific results (e.g., data, and models of behaviour) obtained in brain research and refining it to a readiness level where it can be used by commercial companies and easily taken up and rapidly turned into new categories of products, e.g., using specialized neuromorphic hardware, also currently being developed by HBP. This will allow novel control technologies that achieve robustness and adaptivity far beyond todays algorithmic controls… and ones that actually rival biologic systems.
(Virtualised Brain Research): it links information technology to brain research by designing new tools for brain researchers, with which they can design experiments and then carry them out in simulation. For example, one can study a completely simulated animal’s navigation or sensorimotor skills as it operates in a completely simulated environment (e.g., a maze or a straight or sinusoidal vertical path), and the signals of the simulated brain will be recorded in real-time for immediate analysis. These same principles can be applied to humans and humanoid avatars, allowing bold and fruitful research on degenerative brain diseases, for example.
We envision that the unique integration of the above three paths will lead to widespread mutually beneficial fertilization and research acceleration through the two-way inspiration of the involved disciplines. The vehicle for bi-directional translation (brain science « robotics) is the HBP’s neurorobotics platform.
At this point, we can see the following vision taking shape: we have taken the first steps towards the design of a virtual mouse. This animal, which only exists in a computer, has eyes, whiskers, skin, a brain, and a body with bones and muscles that function exactly like its natural counterpart. Clearly, all of these elements are still far from being perfect, i.e., from exhibiting behaviour and function corresponding to the original creature. However, the more brain scientists learn about these functions and the more data become available, the more we can integrate said results into the virtual mouse, and the faster we can improve the “mouse fidelity”. In parallel, we will apply the same principles to the simulation of human embodiment. The possibilities are endless.
Using the virtual mouse (or humans, or any other animals) in the future, brain scientists can not only copy traditional design experiments into the computer and study the results immediately, they can also modify the mouse any way they want, e.g., introduce lesions into the brain or cut muscles and study the impact it has. Moreover, they can place as many electrodes or other sensors in the body as they want. But perhaps the most astounding benefits of these new possibilities are that scientists can perform experiments that are very, complex – if not impossible to perform in the real world. This includes very long-term studies with permanent recordings (and these can be done 10,000 times faster than in real-time!), animal swarms with parallel recordings, and plasticity and learning effects over many years.
On the technology side, we can envision a number of brain-derived base technologies that result from our work. One straightforward example is robot-based prostheses that have myo-electric interfaces and which can not only be developed in simulation, but which can be tailor-made or personalized to the properties of one specific person – because every single aspect can be simulated. This is a rather simple example; the disruptive products will most likely involve a complex artificial brain running on neuromorphic hardware and capable of super-fast learning, which, for the first time, would make highly intelligent household robots possible that can adapt their behaviour to various tasks.
Substantial progress towards both a comprehensive understanding of the brain and technologies that are derived from the brain’s working principles can only be made by advancing theory and methodology at the system level. While the fields of artificial intelligence and machine learning in particular have recently gained unprecedented momentum that is primarily driven by the success of big data and deep neural networks, the resulting tools, models, and methods are still highly domain-specific. With the ubiquitous availability of cheap storage, massive processing power, and large-scale datasets, the actual challenge no longer lies in the design of a system that performs a specific task, but in the integration of the wealth of different narrow-scoped models from machine learning and neuroscience channelled into a coherent cognitive model. The platform infrastructure of HBP enables the design and implementation of such a model by integrating different tools, methods and theories in a common research environment. For the very first time, different brain theories, neural network architectures and learning algorithms can be directly comparable to both each other and to experimental ground truth. In this context, neurorobotics serves as a central “workbench” for the study of integrated cognitive models in real-world tasks and as a prototyping tool that enables the seamless transfer of these models into new products and services.
To achieve these goals, we need to reinforce the “input side”, i.e., brain scientists need to talk to roboticists much more intensively than they have done up to now. Then, really new concepts can emerge. One particularly attractive concept could be the automatic generation of models from data: data driven model generation. This would make it possible to use every new data collection to improve the virtual models with a minimum of human intervention and hence keep the virtual robot permanently and synergetically coupled to developments in brain science. Of central importance is the permanent adjustment and calibration of these data models with the corresponding cognitive brain system, which in itself is a complex and long-term endeavour. This goal can only be achieved on the basis of a very close interaction between theorists, data/computer scientists and engineers – and as such, could be a perfect example of a synergistic transdisciplinary cooperation that can only be performed in a European Research Infrastructure.