Mini-screens and Big Screens: Aspect of Mixed Reality in Everyday Life
Mixed reality (MR) is important as the major concept in the age of digital technology. MR as a conecpt will become a model with which the relationship between the real space and the virtual or cognitive space will be understood. In anticipating what MR would bring to the society, the nature of mixed reality should be considered from a wider aspect. Telecommunication technology combined with large-format and mini-sized screens we see everywhere in today's urban life is already realizing a mixed reality in the way we see the world in our daily life.
The way we see the world goes through changes in relation to scientific discoveries and technical inventions. Digital technology has brought a series of major changes in the way we see and recognize the world, including our sense of space and body. While telecommunication technology has been influencing our notion of space and distance, digital imaging technology has blurred the boundary between the real, physical world and imaginary worlds. Images are seen everywhere, from regular TV screens to extra large screens on the street corner or to mini-screens on mobile phones, mixing the real and the virtual, the close and distant. In Japan, mobile phones with color displays are becoming the most common mobile tool for the Internet access, sending email or downloading images, sounds or even games, and exchanging photos with friends, etc. At the same time extra large screens on street corners display all sorts of images including life size elephants, whales, houses and airplane.(1) In such an environment filled with real and virtual images, our sense of reality cannot remain the same as the way it used to be.
Mixed reality(MR) has technically made it possible to blend the real and the virtual or distant worlds. However, the importance of MR is not limited in the technical issue. As virtual reality has become a popular idea in our society in understanding what a virtual world created with digital technology can be, MR will soon become a key concept in understanding the world today where the reality is so often mixed with virtuality. Analyzing the nature of mixed reality from everyday life aspects is needed in order to understand what we mean by living in mixed reality.
2. Space Recognition with VR and MR
In virtual reality, the user is supposed to be immersed in the virtual environment. Ideally it means that the real, physical world including the user's real physical body disappears from both his/her sight and consciousness. As a physical device, HMD provides such a situation where the user would see nothing but the virtual world, immersed in the virtual environment. If necessary, the user's hand would be shown as a 3D floating hand on the screen. It also means that there is a clear boundary between the real world and the virtual world. "Diving into the virtual world" is considered as the process a user would experience. (2)
On the other hand, in mixed reality, there is no such clear boundary. The real and virtual coexist and overlap in the environment around the user. By mixing the optical information from both on our retina and sending the mixture to our brain, the real and the virtual are theoretically treated equally in our cognitive systems. If the virtual world is created realistically enough, the boundary becomes invisible. Since images from the virtual world will be shown typically either on see-through glasses or as projection on real space, the user's body is visible and integrated into the mixed reality environment.
Therefore there is a critical difference between VR and MR in the way a user relates him/herself with the environment. While VR brings the user into an artificial environment, MR keeps the user in the real space. However, the boundary between the real and the virtual, and the distance between here and there, become blurred with mixed reality.
3. Mixing Unreal and Real in an Environment
We are used to environments where there are screens and speakers everywhere, blending images and sounds from TV, video, games, CD, DVD, etc. with the real landscape and sounds cape around us. We regard images on billboards, big screens on the facades of buildings, loud music from the car stereos on the street as a part of the real landscape/sounds cape. A huge screen at the corner of Shibuya, Tokyo, is even larger than a house, showing all kinds of images from movie, TV, music video, and commercials. Most of the images are computer manipulated or generated. We live in "real" environments that are flooded with "unreal" images.
If they belong to the real environment, what about personal, portable sources of images and sound?
It is known that Sony's Walkman has brought an epistemological change to its users as they walk on the streets. The everyday landscape becomes a different experience because of the music that accompanies. A walkman user carries his/her sound environment, which can be mixed with the real sounds cape (if the sound volume is set not too high), and combined with the changing landscape.
A similar situation has been taking place with mobile phones. Already with its original use as a telephone, it was observed that a user talking to a friend lives both in the real world and the virtual world which is shared by him/her and the friend. (30) Today, many Japanese walk on the streets or take trains while checking information, exchanging messages and playing games on their i-mode phones. They carry their small screens with them while living and walking in the real landscape.
Digital media and telecommunication technologies have changed our sense of space, as well as of reality. In our real life we live in environments surrounded by images and sounds that consist of bits, not atoms, which are sent from or generated somewhere.
The boundary between real and imaginary is no longer clear. Today we live in what is called a mediated society. Both real and imaginary information arrive through media technologies in the same manner, forming our knowledge of the world. In such a situation, maintaining a sound sense of one's own body is an important issue. We have been developing the way how to deal with the real and unreal at the same time.
Mixed reality can be considered as a technical answer to the above discussed situation. The way the real world and additional information from the virtual world are integrated in one's perception can be identified as a simulation of our nature; perception/recognition system. Actually that might be why mixed reality seems to be familiar to us, and will have a great possibility in visualizing our thought and the way we see the world.
(This paper was presented at Cast01 Conference in Bonn, September 2001)