This is the second half of the epic conversation between project manager Ryota Kanai, leader of Moonshot Goal 1, which aims for the "realization of a society in which human beings can be free from limitations of body, brain, space, and time by 2050," and Wired Japan Editor-in-Chief Michiaki Matsushima. In part one, they discussed the potential for Cybernetic Avatars using Brain Machine Interfaces (BMI-CA) to overcome the current limitations of the Internet and become a form of media service.

In this second part, they share their unique perspectives on the role BMI-CA can play in connecting diverse "environmental worlds" (or "Umwelt") (*1) of people and the potential for changing a media-centered society itself, including AI collaboration.

Link to Part 1

Connecting diverse "environmental worlds"

─In order for a device that shows content in dreams using the brain's world model to function as a form of media, it is necessary to align and connect the dreams of oneself with that of others. Do the world models constructed by the human brain have enough commonality that dreams of different individuals can be connected? Is it possible that the brain models that generate dreams are uniquely structured for each individual, making it difficult to share content as a single media? What are your thoughts on this?

Kanai: Actually, that is the focus of theoretical research, and while there may be some commonalities in the neurons between brains, there is a high possibility that they are speaking in different languages. For example, even if it is the same concept, we use different words to express its meaning between English and Japanese, right? Similarly, in the brain, there may be cases where different neurons represent "dog" or "cat" for different people, or where, say, a certain concept exists in one person's brain but does not exist in another's.

Matsushima: I see. So, it's like how different AIs will produce different outputs, even if the same prompt is used for input.

Kanai: Yes. As a theoretical approach to this, we believe that using AI for brain-to-brain translation is effective. Even with deep learning in AI, their content structure can be different even if we teach it the same thing, so we are working on research to translate between different AIs using neural networks used for language translation. To expand on that point, our current thinking is that AI can ultimately also be applied to translate concepts and world models between different brains with actual neural activity.

[In this future scenario], as different people can communicate in natural language, they should share similar semantic functions, and there should be some similar structure in the brain. We think that by finding out what kind of correspondence there is there, we might be able to establish an information theory to manipulate meaning. If there is a standardized brain protocol to connect people who were not connected before, we would like to find it.

Matsushima: The imagination behind directly connecting brains in a BrainNet-like manner, so to speak, has been repeatedly depicted in cyberpunk and other science fiction stories, and it may stem from a fundamental human desire to aspire for a oneness, where everything is connected. However, I feel that the hurdle is higher than we can imagine. For example, Wittgenstein (*2) said, "If a lion could speak, we could not understand him." In other words, the lion and humans exist in such different worlds that even if we understood their language in terms of its grammar and word definitions, we may not be able to comprehend what that language was referring to. In this regard, to what extent is it possible for different people to standardize one another's Umwelt when their brains are connected, or is it even possible at all? I find this question is absolutely fascinating.

Kanai: That's a really intriguing point, isn’t it? We've just started exploring this area, so we have no idea how far off the horizon is, but there may be some insights that can be gained as a tool to capture even a glimpse of the environmental worlds that can only be felt by animals, even if we cannot fully understand them. For example, in the field of consciousness research, there is often discussion about how we would feel if we were bats. When a bat emits ultrasound and the echo bounces back, does it feel like it "saw" something or "heard" something? This is still an ongoing, almost eternal mystery among consciousness researchers. However, when it comes to, "alright, let's actually connect the brains and find out," we can only imagine what will be understood and what kind of mystery will be solved, if any. My aim is to get to the point where we can actually try it and see for ourselves.

What possibilities can BMI-CA open up for media-centered societies?

—In the context of using BMI-CA as a communication medium, what possibilities do you think could be considered from the perspective of solving limits and issues that societies face today and building a better world? The ideal of the internet, which was once thought to be to connect people around the world and unleash individual creative potential, has recently been overshadowed by, say, the increase of negative aspects of information filtering and echo chambers by huge platforms, and the harmful effects of promoting divisions among people. What possibilities does this research have for overcoming these problems in media communication and society? Please share your views.

Matsushima: This is something we often write about in WIRED, but these days the paradigm surrounding media seems to be undergoing a major shift from 2D to 3D. Until now, whether it was in print media or on your PC or smartphone, or even on movie theater screens, the essence of media technology was to package the experience of our 3D world into a 2D form of expression. However, we are now able to handle and mold 3D information in its original 3D form. The Metaverse and BMI-CA are representative of the technologies that embody this shift, so I think this will be a major turning point in human history.

Returning to McLuhan, Gutenberg's invention of movable type printing made it possible for knowledge to be transmitted to ordinary people through books rather than orally, which was the traditional method, thereby liberating knowledge and religious authority from the exclusive control of the church. This led to the acquisition of individual "inner" knowledge, the birth of nation-states, and the development of modern societies that utiliized capitalism, science, and technology. Of course, there have also been negative aspects such as unprecedented world wars and environmental destruction, but the invention of media has played a significant role in changing human society.

The shift to media in 3D will bring about a change that is comparable to the changes of the past in all aspects of society, ranging from individual consciousness to the entire social structure. This is an exciting prospect, and we will spend the next 100 years exploring how to increase the positive aspects of this change while being mindful of its potential downsides.

Kanai: I think what Matsushima-san just said about the development of media can be rephrased as an increase in the modality of communication. Books mainly used text as a medium, but smartphones have both images and videos, making the texture of the messages conveyed richer for the human body. With the use of BMI, we might see the appearance of something even further beyond that. My hunch is there will be more sharing of intuitive senses, things that cannot be put into words or images.

It's interesting to imagine what will happen when we’re there, but alas, we can only imagine. First of all, it's not even clear if it really will be useful, and we’ll need to put concrete regulations on the ability to directly manipulate human emotions and motivations. In regards to Moonshot Goal 1, we are also conducting parallel discussions in a section led by Professor Keigo Komamura of Keio University Law School, one of the project's promoters, on how to address legal and societal issues.

Matsushima: I feel there is huge potential in sharing preconscious feelings that cannot be verbalized or visualized, and at the same time, I agree that ethical considerations are equally important. To speak to the kind of hope we might find there, if we look at visually based two-dimensional media as a type of boundary that separates us from the other side of the screen, we can improve the quality of our media communication by correcting communication biases and distortions through the addition of more physical or three-dimensional direct experiences.

For example, there is a study within Stanford University's Virtual People (*3) course in which students from majority groups used VR technology to walk virtually around the city in the “shoes” of people from minority groups in order to induce a sense of empathy. In line with this, as we touched on earlier regarding the topic of Umwelt, in a future post-anthropocene era (*4), it may be difficult to achieve global coexistence unless we can acquire a perspective that puts us in the “shoes” of animals and plants, rather than of humans alone. When that happens, BMI-CA, which can actually help us "embody" non-human entities, could help humanity to acquire a variety of cross-world and animistic (*5) perspectives.

Kanai: With the advent of humans connecting to the internet through BMI, there is a possibility that the entire Internet could become like a super-giant brain. In our Moonshot research, we are pursuing ambitions of connecting brains, but in fact, this is already happening within our own brains. For example, the right brain and the left brain are separate, but they are directly connected by a high-bandwidth wiring called the corpus callosum, so there is not a right and a left self in our head, but a unified consciousness. In the future, thanks to technology, if the connection between our brains and others' is also made possible, we might be able to remove the boundary between oneself and another.

In language-based communication, similar things are happening to a significant extent even now. In a world of information overload, individuality is lost among people who receive rapidly and collectively spread information. In the age of mass media, everyone had no choice but to watch the same TV, but with the introduction of the Internet, I think it instilled an expectation in us that we could be free from that. However, currently, information clustering occurs pretty easily. Even looking at Twitter, although there are supposed to be many people in the world, people pretty much tweet the same opinions about the same news.

I think a current weak point of humans is that communication is fundamentally language-based, so we can only understand things through very slow processes like reading and listening. However, with BMI-CA, if we can directly communicate with AI and other brains, we may be able to understand information at speeds ranging from tens of thousands to billions of times faster, which could enhance human intelligence and potentially give rise to a new future for humanity.

Interview/text: Daichi Nakagawa
Photos: Hiroshi Nakamura