Natural lighting on the model of an alien (above); Dr Wong in his laboratory with two of his PhD students (right)

  Currently, lighting in games involves a lot of “tricks,” says Principal Investigator Dr Tien-tsin Wong.
  Unlike in real life, the lighting in games, from Ace Combat to Zapper, does not change as the game player’s viewpoint changes.
  Dr Wong’s award-winning “plenoptic illumination” technique, however, provides both a theoretical and practical solution.
  Using a triangular mesh grid, which is standard in the graphics and games industry, he records the actual lighting effect at the vertex of each triangle. The number of triangles per scene depends on the geometry of an object and its distance from the viewpoint. The data for all possible scene lighting configurations is then stored.
  “But that’s the easy part,” said Dr Wong. A simple scene in 24-bit colour may require 900 MB (megabytes) of storage space, he says; about one-fifth of the capacity of a DVD.
  Dr Wong says his contribution to the problem is in developing and applying a “rendering-friendly compression algorithm” which compresses the data and decompresses it in real-time when the game scene is being played.
  Compression is at a ratio of about 200:1 so the 900 MB of data is reduced to between 3 and 4 MB. This makes practical applications feasible. The technique compresses the data in two ways; by “squeezing” it, which leads to a reduction of between 15 and 20 percent, and by taking out data that the human senses cannot perceive.
  The compression algorithm is based on signal processing theory and is currently being tested by game developers such as Sony.
  Said Dr Wong: “It’s not useful if you can only compress the data. There are some very successful compression techniques like JPEG and MPEG2 for DVD-Video.
  “The problem is that they cannot be applied to games applications because in computer graphics we randomly access data unlike a movie which is played frame by frame. So our algorithm is very different from the traditional compression algorithm.”
  He added: “We had to tailor make a very specialised algorithm. Decompression is coupled with the compression. When we designed the compression, we had to bear in mind how to decompress the data. The decompression takes place during the rendering and it’s important not to use a lot of computing power.”
  Dr Wong came up with the pre-computing approach because he said even today’s computers lack the power to process data for complex lighting effects, like caustics, in real-time.
  “But retrieving data takes only a little time so we decided to pre-compute everything,” he said.
  The research has been awarded the prestigious IEEE Transactions on Multimedia Prize Paper Award 2005 and the CUHK Young Researcher Award 2004.

Principal Investigator
Dr Tien-tsin Wong : ttwong@cse.cuhk.edu.hk