A method to automatically create a 3D model and camera geometry images from a series of photographs, which are two-dimensional, has been developed in Hong Kong, the first method of its kind in the research field of computer vision.
  Applications are many, said Principal Investigator Prof Long Quan from the Department of Computer Science at The Hong Kong University of Science and Technology; from video games and 3D graphics modelling to the post-production of TV films and movies.

An image (left), with feature points (centre) and the 3D model

  “Taking a photograph of an object reduces it to 2D,” said Prof Quan. “In our process, we take a series of photographs of an object and then recreate it digitally in 3D.”
  A handheld digital or video camera can be used to take the photographs, moving around the object in a natural way. “That’s very important, as it can be done by laymen,” said Prof Quan. “We don’t need to put cameras in certain positions and then calibrate them, or measure anything in space from the object.”
  With the images fed into a computer, a complex algorithm takes over to recover the object in 3D. This is done by extracting features such as interesting corner points from the images and finding “correspondences” or the same points in the different images. The algorithm then automatically creates a 3D image, storing the data in a 3D format such as VRML.
  “Doing this by hand is tedious,” said Prof Quan. “In our method, 40 two-dimensional images can be processed automatically in 10 to 20 minutes.”
  Another novel approach used in the method is the “quasi-dense” reconstruction process.
  Prof Quan explained: “The dense approach uses every pixel as a feature point but that’s too many and theoretically is almost impossible. For example, a 1,000 by 1,000 pixel picture would have one million feature points.
  “On the other hand, if you use the sparse approach with just a few feature points, it would not be enough to reconstruct a 3D image.
  “Our quasi-dense approach uses a few remarkable feature points that may be sparse in places and dense in others. It’s just enough and very efficient.”
  Having developed the tool, which involves complex computer coding, Prof Quan and his researchers are now looking at specific applications.
  One is in the post-production processes of film making. For example, after a film sequence is shot by a camera it may be decided to take a different camera path, said Prof Quan.
  This can easily be done using our technique without having to re-shoot the scene. It also enables camera positions that may be difficult or impossible “like jumping to the sky,” he said.

Principal Investigator
Prof Long Quan : quan@cs.ust.hk