Sutherlands vision of the Ultimate display and the Reality-Virtuality Continuum

Who knows whether we are not all the time already in some kind of ultimate display….

In 1994 Paul Milgram published with a work, where he introduced the idea of a Reality-Virtuality Continuum. You can think of it in the following way: With the rise of technology, like smartphones or in general some Silicon Valley Hardware device, the line between what is reality and what is virtual becomes more and more unclear.

Image Credits - Milgram et al., Augmented Reality: A class of displays on the reality-virtuality continuum, 1994

When you are wearing headphones, the sound you here is artificial, so it is further away from the real reality. If you even wear noise cancelling headphones, you cancel even more from the real reality, so you move further to the virtual reality.

Also imagine to wear a hololens, that has a glass display in front of your head, where you can project further content on it. Compare this to a “full” virtual reality glass. Then the hololens is in this spectrum closer to the augmented reality, whereas the VR is closer to the full virtual reality.

Long before the idea of this spectrum was published, Ivan Sutherland published in 1965 the idea of an ultimate display. With our devices we have, we try to generate some image on a display, or to generate some sound to listen to. The company feelreal tries to move on even more and build an extension for a VR-Glass to generate the impression of smell or taste.

On their website they even sell special smell kits for special films or games. So if you ever wondered what Skyrim smells like, there you go!

Example of a mask with smell kit In his theoretical work Sutherland extended this idea to an extreme and suggested a room, where a computer system is in control to create physical matter. In this room a computer can create sound, light and can make a chair appear.

The question is now, where we would put this creation on our Reality-Virtuality-Continuum? Maybe we start of by the idea, that we need for the functionning of this device no influence from the outside. So we could put it as far to the virtual reality on the right as we want. But as the device is in control to create physical matter, it can of course create all known mixed-reality devices as well, that we can use in the real world. Thus we could consider the device also as a mixed-reality device inbetween reality and virtuality.

But the most difficult question is probably, can we consider this object as reality. The matrix film creators gave us already a lot of material to think about this question. It is an open and very deep question, whether we have some higher power, that is in control about everything that is happening around us. To decide whether the ultimate display is to be put to the leftmost point, we would need to give an answer, whether our entire existance is controlled by someone else and could therefore be recreated by an ultimate display. I cannot give an response to that, so depending, whether there is a higher power responsable for our existance, the Ultimate display would be the entire interval, if you do not believe in it, it would just be the half open interval, with the point of reality missing.