Freeviewpoint

What's next beyond 3D? Free viewpoint video is the ultimate 3DTV.

This will be the next hottest technology.

Free viewpoint video offers the same functionality that is known from 3D computer graphics. The user can choose an own viewpoint and viewing direction within a live scene. They are free to move around and interactive navigate as if they could control the camera.

What's different from your 3D video-game Virtual environment is, Free viewpoint video targets real world scenes as captured by real cameras. This could be live, streaming or recorded.

This is interesting for consumers where you could now where the viewer could freely chose the viewpoint as well as for professional production and post.

Tools for post-production are already being used (e.g. for sports, movies, EyeVision, Matrix-effects).

I have seen this described as Virtual Cameras, and Synthetic views.

The secret is setting up a many cameras around the "event" and taking video from all angles, then compiling them together to form the virtual space.

With sufficient video coverage off all surfaces and enough data to construct accurate 3D models this data can then be used to drive a standard PC graphics card to renders it in full 3D from any position and angle.

This would allow a user to zoom around like it were a virtual room in a video game.

One things about this technology is it could render better quality images then the source cameras by fulling pixels from one camera with pixels from other cameras.

Not all cameras would need to be identical, so some could be SD and other HD and even low frame rate very high def like digital still cameras. All of this image data can be integrated together and use to refine the texture maps in the 3D Model.

The applications of this live sports could change everything. We could see the game from the perspective of any player on the field, from the referees or even the the ball.

New cameras would not be needed to produce high quality 3D content, the saving to studios would be incredible. In addition post production techniques could be used to correct for eye separation and perspectives making 3D content easier on the eyes and helping to eliminate headaches and fatigue.

In CCTV and surveillance also it would allow an operate to do far more then just zoom in. Cameras could be strung like Christmas lights and placement and reliability of a single camera wouldn't be as critical.

In Military, AUV and fixed cameras could be integrated to provide total battle field awareness.

By tracking an objects position with several cameras and determining it's 3D position with sub pixel accuracy it becomes possible to integrate video from several cameras across many video frames to produce super resolution images and 3D models of faces and objects.

When this technology is applied to Head Mounted Displays (HMD) in a VR / AR type headgear you would literally be able to walk around in a live action scene like the
Star Trek Holodeck or Matrix construct as a ghost. You could see them, but they will not see you unless they are wearing AR Goggles too. A fly on the wall.

VR / AR = Virtual Reality / Augmented Reality

Free viewpoint television - Wikipedia

Viewpoint - Technologies - Viewpoint Media Player

3D VIDEO AND FREE VIEWPOINT VIDEO – TECHNOLOGIES, APPLICATIONS AND

FIFA World Cup in Full Court 3-D Vision

Free-Viewpoint TV (FTV) System