Kandao Obsidian: A New Generation of Cameras Focuses on Affordable 360-Degree Virtual Reality Video
By Lori Cameron
Published 08/08/2018
Share this on:
The next generation of 360-degree cameras will revolutionize multimedia—allowing you to create your own high-resolution panoramic videos and VR gaming machinery like a pro.
The problem is, many current models suffer from bulkiness, poor video quality, low battery life, and painfully slow render times.
To begin solving these problems, developers created the award-winning Kandao Obsidion camera.
“A good-quality 3D, 360-degree camera is still too expensive for mass adoption, costing as much as $20,000,” say the authors of “360-Degree Virtual-Reality Cameras for the Masses” in the January-March 2018 issue of IEEE MultiMedia. (Login may be required for full text.)
“To make the technology more accessible to the public, a device must be affordable, portable, reliable, high quality, and user friendly. We describe the challenges in meeting these goals and the techniques that Kandao—a VR startup company based in China—used to conquer them when designing its Obsidian cameras.”
Newer model: Kandao QooCam, a switchable 4K 360° & 3D camera
Jack Tan, co-founder of Kandao, and Rui Ma, a Kandao researcher fellow, teamed up with Gene Cheung of the National Institute of Informatics in Tokyo, an inter-university research institute corporation and a research organization of information and systems, to design the Kandao Obsidian, which won the German Design Award 2018, one of the most prestigious design competitions in the world, for the Excellent Product Design category.
Capturing better stereoscopic panoramas
In 1992, omni-directional stereo (ODS) technology relied on a slit camera that rotated around an inter-pupil-distance (IPD) circle to scan the real world.
“The image slices are stacked to form a rectangular panoramic image. An inherent drawback of this and other early methods is that they do not work on dynamic scenes with moving objects,” the authors say.
The original ODS uses rotating slit cameras for the left and right eyes.
An updated version of ODS takes a computational photography approach.
Instead of actual scanning, the rotating slit camera is synthesized using computational imaging and vision techniques.
A modern ODS illustration. Point A is captured by two cameras centered at O and O’ as a and a’. Conversely, given a and a’, the corresponding point A can be reconstructed. Thus, the images in the left eye and right eye can be reconstructed, as well.
Building a user-friendly camera
The Kandao Obsidian camera also has a rich, compact interface and shape, making it easy to rotate and hold in your hand or mount on a stand.
Kandao Obsidian cameras are easy to hold, carry, and interact with.
With six fisheye lenses, the Kandao Obsidian camera can see everything in a seamless panoramic view.
Finding an easy way to create a synthetic dataset
The authors say that building a dataset for computational ODS is very difficult to do, because “one would need to accurately map every point densely in 3D and generate two slit cameras scanning the scene by rotating around an IPD circle.”
Instead, they took a computational approach by constructing a synthetic dataset with rich information: 3D models (mesh and texture), simulated virtual cameras, pixel correspondences, depth data, object labels, and segmentation boundaries.
Images of the synthetic dataset built for simulation and training.
Finding a new optical flow algorithm for six fisheye lenses
To build a high-quality ODS with six lenses, a highly accurate optical flow algorithm is necessary. The figure below shows how difficult this is to do.
Illustration of ODS projection error caused by error in optical flow. The error Δθ in optical flow causes Q to drift to Q ‘. (right) The error (arc length) of QQ ‘ due to a unit error in optical flow at different ray positions (angles in degree) with different numbers of imaging modules.
The authors have proposed a neural network that is first trained on popular open-source datasets and further fine-tuned on their own synthetic dataset.
The deep-learning-based method proves to be a full 100 times faster than traditional optimization-based optical flow. The Kandao team compared their neural network with FlowNet 2.0, a state-of-the-art deep learning method.
Flow method comparison: (a) input image, (b) FlowNet 2.0 results, (c) results using Kandao’s CNN, and (d) a quantitative comparison on various criteria.
Kandao’s CNN method is more accurate, efficient, and compact than FlowNet 2.0, the authors contend.
The challenges ahead
Several challenges remain before 360-degree VR cameras become more widely used. One challenge involves improving optimal flow for more temporal-consistent dense pixel correspondence. Another involves finding a way to encode and package VR video into sub-streams for faster transmission over bandwidth-limited networks.
“Many research groups are currently addressing these and other challenges, and the day will soon arrive when VR video becomes truly ubiquitous—easily produced, consumed, and enjoyed by the masses,” the authors say.
Related research on photography in the Computer Society Digital Library
Lori Cameron is a Senior Writer for the IEEE Computer Society and currently writes regular features for Computer magazine, Computing Edge, and the Computing Now and Magazine Roundup websites. Contact her at l.cameron@computer.org. Follow her on LinkedIn.
