Thursday, July 30, 2020

Seurat : system for image-based scene simplification for VR


Seurat is a scene simplification technology designed to process very complex 3D scenes into a representation that renders efficiently on mobile 6DoF VR systems.
Seurat works by taking advantage of the fact that VR scenes are typically viewed from within a limited viewing region (the box on the left below), and leverages this to optimize the geometry and textures in your scene.
It takes RGBD images (color and depth) as input and generates a textured mesh, targeting a configurable number of triangles, texture size, and fill rate, to simplify scenes beyond what traditional methods can achieve.
Seurat is available as an open source project on GitHub, and includes plugin support for generating depth images for scenes in both Unity and Unreal.

Seurat - Documentation

What is Seurat?

Seurat is a system for image-based scene simplification for VR. It converts complex 3D scenes with millions of triangles, including complex lighting and shading effects, into just tens of thousands of triangles that can be rendered very efficiently on 6DOF devices with little loss in visual quality. It delivers high fidelity graphics on mobile VR devices. (One way to think of it is as a serving the same role as stereo panoramas on 3DoF VR devices, on 6DoF devices.)
The processing pipeline for static environments generates data for a single headbox (e.g. 1 m³ of space). Input data can be generated with any rendering system, e.g. a real-time game engine or an offline ray tracer. We have plugins for Unity, Unreal and Maya. Seurat outputs a mesh with an RGBA texture atlas, which can be rendered in any real-time engine. Dynamic content can be composited on top of the static Seurat environments.

Optimizing for 6DOF mobile VR with Google's Seurat

I lifted this from a 3D photo off facebook.  Still trying to make sense of it...
You can see this image have been broken in to subimages, I suspect these are mapped on to various planes with different Z buffer levels, something that renders quickly in webgl.

No comments: