The Moonshot Goal 1 aims for the "realization of a society in which human beings can be free from limitations of body, brain, space, and time by 2050." To achieve this, research and development spanning various fields of technology has begun with aims of social implementation of Cybernetic Avatars (CAs), which can be used as substitutes for the human body in a variety of social activities, whether in virtual or real space. These can range from highly detailed 3D images to use in the Metaverse, to remotely operated robot technology.

Among the many faces of this movement, leading the R&D team of Cybernetic Avatar using Brain Machine Interfaces (BMI-CA) — which directly conveys human brain activity — is project manager (PM) at ATR’s (Advanced Telecommunications Research Institute International) Business Development Department and CEO of Araya Inc., Ryota Kanai. As a milestone towards the ultimate BMI-CA that can be operated in accordance with human intentions by 2050, we had Kanai sit down with Wired Japan's editor-in-chief, Michiaki Matsushima to discuss the potential for BMI-CA to become a media service that surpasses the limits of the modern day internet.

The current state and challenges of BMI-enabled Cybernetic Avatars

─First, Kanai-san, could you introduce how you are researching BMI-CA as it pertains to the Moonshot Goal 1?

Kanai: Our team is particularly focused on BMI for this goal. BMIs are devices that extract signals from the brain such as thoughts and information to be used as output, or even to move objects around. They include both non-invasive methods such as EEG (electroencephalogram) and fMRI (functional magnetic resonance imaging) that can be measured from outside the head, and invasive methods such as surgically implanting devices. Our goal is to use BMI to operate a Cybernetic Avatar (CA) that can act as a substitute for oneself, in both physical and digital spaces. Ultimately, we aim to be able to recreate all human experiences, such as seeing and smelling things, by directly inputting signals into the brain, similar to an evolved version of virtual reality.

However, one basic hurdle of current BMI research, is that when attempting to do this safely with non-invasive means, the accuracy is far from sufficient. The brain information that can be obtained from EEG and other non-invasive methods is limited and there is a lot of noise. It has not yet reached a level that everyone can agree on its usefulness. On the other hand, as for invasive methods, I think it has the potential to expand considerably if we could really perform surgery on the surface of the brain and embed electrodes to retrieve signals. But in reality, casually conducting those kind of human experiments for trial and error purposes is still out of the question. Our current challenge is how to overcome the limitations of both approaches.

One of the distinctive features of our BMI research is that we aim to achieve direct brain-to-brain communication, or THINK-Communication, as our ultimate goal. In other words, we are trying to achieve something like telepathy. Of course, there are still many unexplored areas in terms of technology and theory, and there is confusion about where to set specific targets, but at the least we aim to have the concept established and be in a situation where it can be implemented in society by 2050.

Matsushima: In terms of invasive BMI, the "conventional approach" is for implanted devices, such as those developed by the folks at Neuralink, co-founded by Elon Musk, to support sensory and motor functions for people with disabilities. Do you see projects of this kind that use CA as having the potential to become a part of a broader societal infrastructure, rather than just for meeting medical needs?

Kanai: Yes, it could. In the past we’ve held an event called "Brain Pick" as an approach that is more like light entertainment. That could be one direction that simple non-invasive BMI could go. The event was an initiative where you measure brain waves and actually move avatars and play with them. I think it is difficult to assume that there is a practical need for BMI-CA that is essential to daily life, but having fun and trying to see how much you can move, I think that’s an important standpoint to have going forward. Our current goal is to explore ways to popularize it in society by using it as, say, a daily wearable tool like a smartwatch to monitor one's health status or to simply play with the technology that is currently available.

Kanai: This non-invasive project is mainly led by Professor Junichi Ushiba of Keio University. They’re planning various initiatives. Specifically, we conducted a simple race using the brainwave device "PLUG" developed in our Moonshot project, where people controlled characters in the popular game Fortnite with brainwaves and moved them along the course. Even from a technological standpoint, it’s a lot of fun to watch. This could one day even be developed into a social event for disabled sports, such as the e-Paralympics. Simply by continuing these efforts, even non-experts can imagine what BMI could do if it were to hit the world. Especially in 2050, the year that this Moonshot R&D Program has chosen as its goal line, it will be an era where today's children, and those who have not been born yet, will become the main players. It is important to create a prototype image, an example that makes these people think "oh, the future’s gonna be interesting!"

How will BMI-CA change the relationship between "media" and "body"?

─Matsushima-san, based on the overview so far, as someone who handles magazines and web media, what kind of potential do you see for BMI-CA as a media?

Matsushima: Well, going back to Marshall McLuhan (*1) in terms of media theory, media is fundamentally positioned as something like an extension of the human body. His most widely quoted statement was the phrase, "The medium is the message," which means that it’s not only the content conveyed through a medium, but also the way in which the medium itself operates that carries a certain unconscious message.

For example, when we ask why young people are so obsessed with tools like TikTok, it's said that the smartphone—the very medium in which TikTok is embedded—has become a message, and that that is because people perceive the smartphone as an extension of their own bodies. Even with visual content, for instance, the sense of the distance and relationship between the sender and receiver is greatly influenced by the medium they use. In the past, watching movies in a dark theater with strangers was a visual experience in which the actors in the world beyond the screen and the audience watching them were separated. Then, as families started to watch TV together, absorbing visual content became a daily and continuous experience. Furthermore, TikTok, accessed via the smartphone which has become an extension of the body, as well as of consciousness for many people today, has a 1-to-1 interface where only a single screen is displayed on each side, making it easier for users to build more personal relationships, unlike when using other social media platforms with cluttered timelines.

Thinking about the essence of such media, BMI-CA is not an extension of the body, but rather a direct connection between brain and machine without having to go through the body, which fundamentally changes the concept of the media. Moonshot Goal 1 aims for people to "be free from limitations of body," but beyond that, I believe that we will see the appearance of a completely new worldview accompanied by a demand for the things that emerge there. As a prerequisite for thinking about this, we should question how inseparable the brain and mind are from the body. I am curious about your thoughts on this, Kanai-san.

Kanai: That's an interesting topic. There is a theme that researchers love to discuss, which is, how necessary is it for the brain system to perceive and move a body. I personally believe that the brain can function completely on its own in principle. However, because the brain is not used to being in a state without a body, it may be difficult to adapt practically. Without interacting with the outside world through the body, I don't think the brain can create a model of the world. In that kind of state, it only receives information and does not understand how the world works. I think we learn through the frame of active inference (*2) as posed by researchers such as Karl Friston.

For example, our spatial recognition is formed through information processing from a so-called first-person, two-dimensional perspective, which is a constraint of the bodily nature of the visual organ, the retina, which receives optical images in a planar manner. However, what if a new organ were to be created that allows 3-dimensional information of space to enter the brain as-is? In principle, it should be learnable and usable by the brain's neural plasticity, but integrating it with existing brain models while adjusting to the new modality would be challenging. There is a phenomenon called VR sickness, where even a slight discrepancy between the visual image and the brain model can cause severe discomfort, so one can imagine how uncomfortable an even more drastic change could make you feel. So in that sense, it may be difficult for humans to become completely free from bodily constraints, and there may be some truth to that viewpoint.

Matsushima: In other words, if we were to gain a third arm through technology such as CA, and we tried to control it using BMI, many people would need to struggle to modify their image of their own body, which has been based on a lifetime of experiencing the physical world through a body with two arms, in order to "extend" it further, and thus acquire the sensation of having a third arm.

Kanai: Yes, that's right. So, in the initial stages, I think it's necessary to actually interact with the world and learn things like, "if I do this, this will happen." But on the other hand, once a world model can be constructed in the brain, simulation of the world can be done using only information input to the brain. For example, I think that when you are dreaming, you are generating a simulation of reality based solely on the world model constructed in the brain, without the use of bodily perception or movement.

Kanai: In that sense, dreams might actually act as a foothold for BMI-CA research. With current methods, it's hard to evoke clear sensations in the brain while a person is awake. But when you're dreaming, things feel very real, don't they? There's something called lucid dreaming, and some studies suggest that it's possible to stimulate the brain to make it easier to have lucid dreams. So my idea is to create a situation where people are artificially induced into a lucid dream state where external information is blocked, and then introduce input to the brain. With this method, even a small amount of stimulation could produce extremely realistic image transmission in the dream state.

Matsushima: I see. So dreams are the medium. A biologist who said that what we do when sleeping is our main activity as humans, and that being awake is merely an auxiliary activity to sleeping. So if we can create a medium using dreams, it could literally become exactly like that — where we are awake just to sleep, so to speak.

Kanai: The things we see in our waking state right now, and what we see in our dreams are essentially the same, as far as the brain's system is concerned. When we are awake, we perceive what the brain creates, as it matches our perception of the outside world in real-time, which is why we perceive it as reality. However, the nature of the brain's information processing that forms the perception of dreams and reality, is the same. In other words, reality is also a dream, but it becomes a dream that corresponds to the outside world, so it can be shared with others and serves a purpose in forming society. This process of forming a sense of reality, in and of itself, could be redefined as a kind of media.

Therefore, the target is to create a state similar to lucid dreaming, where you are conscious, but you are aware that you are in a world created by your own brain and the content seen there can be freely created by stimulating the brain from the outside via BMI. If that is achievable, we might be able to create the ultimate VR media. If communication between brains is possible while in a state of lucid dreaming, with the consent of everyone involved, it may be possible to create a media that allows individuals to enter dreams. However, it seems a bit difficult to achieve that by 2050.

This heated debate isn’t over.
In part 2, we will look to the future in 2050 and explore further discussions. We will delve into the differences in the brain's environmental world and the new potential of BMI-CA that could change the media society.

Head to Part 2

Interview/text: Daichi Nakagawa
Photos: Hiroshi Nakamura