Synthetic views create virtual cameras.

This is something I have been thinking about and wanting to do for the past 15 years. Well someone finally did it.

The idea is to place a number of cameras around a stadium. These camera are sent to an image processing system that constructs a real time 3D model of the live action with texture maps.

This is then fed to a high end graphics card that can generate a 3D view of live action from any view point.

These KDDI's solution is taking some shortcut and the images look like cardboard cutouts playing on the field. Still I think the main idea comes across.

I want to be able to do with with enough quality and resolution that we could do this for the Superbowl. Football would never been the same. A high end professional studio solution, not just a run on your PC end user solution.




3D Free Viewpoint Video in Sports Stadium

KDDI R&D Laboratories has developed the world's first technology for synthesizing and displaying 3D video of stadium sports in real time, from any angle chosen by the viewer. This technology was exhibited at CEATEC JAPAN 2010.

"Until now, the viewing angle for live soccer has only been changeable by switching between camera positions. This technology provides a new video experience, by offering views from angles where cameras can't be installed, such as the middle of the pitch or in the air."

"This screen shows the stadium as seen by eight HD cameras, and these are synthesized pictures from places where cameras aren't installed, including aerial views, and shots from in among the players. On the other hand, this screen shows pictures from a single high-resolution camera, of the sort used for 4K rather than HD. This system creates pictures from any angle chosen by the viewer. Because the camera has such a high resolution, there's relatively little loss of picture quality even if it zooms right in among the action."

This technology uses a unique method to enable high-speed estimation of players' positions on the field in 3D. It utilizes the fact that when soccer is broadcast, the image consists simply of players and a background.

"What viewers mainly want to see is the players. So we thought of technology for synthesizing pictures that look natural, by cutting out just the parts with players, and putting them against a plane. Then, the plane is transformed, in line with the direction or angle the viewer has chosen. The pictures you're seeing now are being synthesized in real time using a single PC. So if you have a PC at home, this kind of 3D processing can be viewed immediately on a 3D TV."

KDDI has also developed server technology that stores stadium 4K video content taken from many angles, then extracts and delivers pictures at high speed with low processing load. This achieves a real-time delivery system for pictures from any angle chosen by the viewer.