Realistic expectations

Jurassic Park's dinosaurs showed the dramatic potential of virtual reality but set unattainable standards for the technology's more everyday uses. John Vince explains.

Virtual reality systems should offer quality computer graphics, real-time operation, realistic simulation and efficient modes of interaction. These features do not present a problem individually, but collectively, their implementation is daunting.

Take the issue of graphics. We know from films such as Terminator 2 and Jurassic Park that animated synthetic imagery has surpassed anyone's expectations of what can be achieved with a computer. What is not always appreciated is the time and effort that goes into creating these sequences.

Fifteen years ago the rendering time for a single frame was 15 minutes; today, even with our incredibly fast workstations, rendering times are still of this order. It is as if we had come up against a fundamental constant of nature. Fortunately this is not the case, for while faster processors and parallel architectures have multiplied computer performance, rendering algorithms have become equally sophisticated and hungry for machine instructions.

If an animated sequence is destined for film projection, 24 frames are required for every second. A rendering time of 15 minutes per frame means six hours of rendering for each second of projection time. This is not very efficient. For the same computers to create these images in real time would require a speed increase of approximately 20,000 times. No matter what anyone expects, we will not be able to buy such systems in Dixons this century.

The flight simulation industry was aware of this problem 20 years ago, and companies like Evans & Sutherland and Rediffusion Simulation collaborated to develop real-time image generators. These systems produce images and nothing else. In fact, they generate animated images at frame rates between 25 and 60 Hz.

You are probably wondering why the animation industry is still struggling to produce a single image in less than 15 minutes, when the simulation world only requires 20 milliseconds. The answer lies in the level of realism required by the two communities. Whereas a dinosaur may require in excess of 100,000 polygons to model its surface, one can display a very credible view of Heathrow airport with approximately 1000 polygons. For a dinosaur to look real it must be decorated with texture, bumps and shadows. Although an airport must also appear realistic, it does not have to appear as believable as Jurassic Park.

These are real issues for the VR community, for many VR applications are demanding too much from today's technology. Databases containing several hundred thousand polygons are typical for engineering projects, but to display them at 25 Hz or more is currently impossible.

Tremendous strides are being made in this area, in workstations from Silicon Graphics and products like Pixel-Planes from Division. However, if VR systems are to be used commercially in such areas as medical simulation, engineering product design and scientific visualisation, this progress will have to continue for some years. It will take time to develop this technology, and even more time to manufacture it at a price that can be borne by industry.

Perhaps the central issue for the future of real-time image generation is the amount of dedicated graphics hardware that systems will contain. Rendering a high-resolution, anti-aliased image with texture mapping, environment mapping and transparency is possible in less than 20 ms - but at a price. Such systems have a short lifetime as they are quickly overtaken by new technology.

The virtual environments - the places and objects displayed - have to be hand crafted and tuned to work within the system's operational envelope. Furthermore, the system architecture is so rigid that there is little room for manoeuvre to introduce new effects. When the system is eventually replaced by something with twice the performance and half the cost, old databases have to be abandoned and new ones built. This is a non-trivial, costly exercise.

An attractive alternative is to keep the geometry and rendering algorithms in software. Then, as hardware platforms come and go, the integrity of the software is not compromised.

But wait a moment: this is what the computer animation industry has been doing for years. Their product continuity comes from keeping software, as far as possible, independent of the hardware platform. The drawback is that general-purpose hardware is currently 20,000 times too slow for real-time animation.

If we are willing to trade realism for speed, then perhaps there is a solution between these two extremes that is applicable to the world of VR. This trade-off is already being exploited in so-called desktop VR systems. Superscape's VRT software is an excellent example, showing that a 486 or Pentium PC provides adequate power for certain types of interactive visualisation. Division's dVS and dVISE products perform a similar role on workstations.

By today's standards, the first VR systems were appalling. The image resolution of head-mounted displays was intolerably poor, and the slow response to head movements disappointed those who immersed themselves in this technology. What was confusing was the contradictory claims coming from California, where the systems were being used. However, five years of exposure to VR systems have washed away most of the hype, and industry now knows how to distinguish between what is possible today, and what will be possible tomorrow.

It is difficult to estimate how important head-mounted displays will be to the future success of VR. Very little is known about our ability to work with this form of display over long periods of time. It is difficult to imagine engineers, who currently spend 37 hours a week at a workstation, undertaking similar duties with head-mounted displays. Even if an engineer only dons a headset at critical points in a design, the interruption in the design process will be significant.

An headset provides an intuitive "first-person" view of the virtual domain, and transforms the traditional 2D image on a monitor into a 3D immersive experience. The potential for this form of display is significant, but for various reasons it has been difficult to realise.

Currently, display resolution varies from 250 lines with iquid crystal display panels, to the 1,000 lines achieved with miniature cathode ray tubes. Fabricating special LCDs for the relatively small VR market is still expensive, which means that components have to be employed from the portable TV and camcorder markets.

Fakespace's Binocular Omni-Orientation Monitor (BOOM) display, mounted on a movable arm, avoids some of the problems associated with headsets. Fakespace abandoned LCDs because of their slow response times, which are still in the order of 40 milliseconds, compared with the 16 ms available from a cathode ray tude at 60 Hz. CRTs remain far superior to LCDs in resolution, contrast ratio, brightness, colour gamut and grey scales.

The ergonomics of a BOOM are totally different from those of a headset. When required, the user simply grabs hold of the display and while looking through two lenses explores different views of a virtual world. The electro-mechanical tracking system provides fast and accurate monitoring of the display's position in space.

Most optical systems are notoriously passive and include the minimum of adjustment - whereas the eye is the most interactive organ of the human body. It has an angle of view in the order of 150 degrees. The central foveal region of the retina can resolve fine detail, while peripheral vision has evolved to measure motion. The eye's lens can accommodate objects from a few centimetres away to infinity.

This flexibility cannot be ignored lightly. When an object in the real world approaches us, our eyes automatically adjust focus and converge. Feedback from these motor systems helps the brain estimate the object's distance and speed. headsets, on the other hand, are fixed focus and oblige the wearer to focus at a distance of two or three metres. Although the displayed view of the virtual world may imply depth, the brain has to rely upon parallax differences in the two images for depth information. If stereoscopic displays are to play a significant role in future VR systems, and if users are expected to wear them for long periods of time, there is plenty of room for improvement in display technology.

Desktop VR may only require a spaceball or similar pointing device with six degrees of freedom, but an immersive configuration requires accurate head and hand tracking. Current technologies have limited range and accuracy, and those which depend on magnetism can be sensitive to local metallic structures. The problem of slow response times is no longer so serious, but every millisecond counts.

In a flight simulator the virtual environment is based upon a specific airport. A very large percentage of the database is static. This includes runways, terminal buildings and surrounding terrain. The only active objects are airport service vehicles, and other planes landing and taking off. Realistic animation of these objects is achieved by computing their behaviour in advance. This form of subterfuge has to be used in some VR applications.

The simulation of physical processes in real time pushes VR to the very limits of technology. Take for example a training simulator for "key-hole" surgery. The virtual environment now takes the form of bones, veins, arteries and flesh. To simulate the behaviour of such virtual structures with any realism requires massive processing power. In the real world, atomic forces prevent one object passing through another. In a virtual world, geometric databases are not so disciplined, and are kept on a tight rein to prevent self-intersection. Such monitoring has a price, often in the form of extra processors whose sole duty is collision detection.

It is worth restating the challenge VR faces as it embraces some of the more demanding applications for virtual environments. Some industrial visualisation projects are daunting and have to be supported by sophisticated computer-aided design systems and physical mock-ups. It is unreasonable to expect any VR system to solve these problems overnight. VR's message today is not about some vague promise that might or might not be kept, but a realistic prediction of the way computer technology is advancing.

John Vince is chief scientist at Thomson Training and Simulation, visiting professor of computer graphics at Brighton University, and president of the Virtual Reality Society.

The society's World Wide Web pages can be found at: http://www.enc.hull.ac.uk/cs/vrs/