The Technology
As any lover of live sports events knows, user control of a video camera’s viewpoint can be a highly prized feature. Unfortunately, it is not feasible to allow each viewer to actively control his own robotic video camera from home. Some partial solutions to this problem include the simultaneous streaming of multiple video feeds to the viewer, which can then pick one or more. To obtain a clear picture of an event, the viewer may watch the same scene from different angles using these streams. This feature exists on some video DVDs. Though video cameras which also provide depth information (‘z-cams’) exist, one such camera is insufficient for reconstructing the full 3D scene due to a singular viewpoint and inevitable concealments. Using more cameras usually results in unwanted artifacts such as apparent ‘holes’ in objects in the scene. To overcome this problem, additional z-cams are usually used, which dramatically increases the cost of the system. Our technology enables cost-effective reconstruction of a live 3D scene using a small number of z-cams and minimal computing power. Ordinary video cameras are used in conjunction with the z-cams in order to complete the scene while reducing costs. The technology is also able to merge 2D video streams and textureless 3D geometry to provide a textured 3D model in real time. The reconstruction and rendering method is able to greatly reduce visual artifacts in occluded areas by estimating the missing geometry. By analyzing the 2D silhouettes of a certain object from various angles, it is possible to estimate its confining hull. Extending this idea to 3D, we re-define this hull using the umbra of each 3D shell captured by the z-cam. Our algorithm makes use of GPU (Graphical Processing Unit) in order to accelerate processing and enable real-time applications. The resulting textured 3D model can be further manipulated and incorporated into various 3D environments
Advantages
- Real-time textured 3D model from real-life scenes ; Allows the user to pick his own viewpoint in 3D space ; Real-time manipulation of the 3D geometry is possible, i.e. body weight or height ; Cost savings through fusion of data from ordinary and 3D cameras ; Minimal use of capture hardware and processing power
Applications and Opportunities
- Sports events and other live broadcasts, movies, online shopping and auctions, electronic tourism, online dating, live-participation virtual environments, online multi-player gaming, interactive media, video conferencing
Business Development Contacts