The next generation of 360-degree cameras will revolutionize multimedia—allowing you to create your own high-resolution panoramic videos and VR gaming machinery like a pro.
The problem is, many current models suffer from bulkiness, poor video quality, low battery life, and painfully slow render times.
To begin solving these problems, developers created the award-winning Kandao Obsidion camera.
"A good-quality 3D, 360-degree camera is still too expensive for mass adoption, costing as much as $20,000," say the authors of "360-Degree Virtual-Reality Cameras for the Masses" in the January-March 2018 issue of IEEE MultiMedia. (Login may be required for full text.)
"To make the technology more accessible to the public, a device must be affordable, portable, reliable, high quality, and user friendly. We describe the challenges in meeting these goals and the techniques that Kandao—a VR startup company based in China—used to conquer them when designing its Obsidian cameras."
Newer model: Kandao QooCam, a switchable 4K 360° & 3D camera
Jack Tan, co-founder of Kandao, and Rui Ma, a Kandao researcher fellow, teamed up with Gene Cheung of the National Institute of Informatics in Tokyo, an inter-university research institute corporation and a research organization of information and systems, to design the Kandao Obsidian, which won the German Design Award 2018, one of the most prestigious design competitions in the world, for the Excellent Product Design category.
In 1992, omni-directional stereo (ODS) technology relied on a slit camera that rotated around an inter-pupil-distance (IPD) circle to scan the real world.
"The image slices are stacked to form a rectangular panoramic image. An inherent drawback of this and other early methods is that they do not work on dynamic scenes with moving objects," the authors say.
An updated version of ODS takes a computational photography approach.
Instead of actual scanning, the rotating slit camera is synthesized using computational imaging and vision techniques.
The Kandao Obsidian camera also has a rich, compact interface and shape, making it easy to rotate and hold in your hand or mount on a stand.
With six fisheye lenses, the Kandao Obsidian camera can see everything in a seamless panoramic view.
The authors say that building a dataset for computational ODS is very difficult to do, because "one would need to accurately map every point densely in 3D and generate two slit cameras scanning the scene by rotating around an IPD circle."
Instead, they took a computational approach by constructing a synthetic dataset with rich information: 3D models (mesh and texture), simulated virtual cameras, pixel correspondences, depth data, object labels, and segmentation boundaries.
To build a high-quality ODS with six lenses, a highly accurate optical flow algorithm is necessary. The figure below shows how difficult this is to do.
The authors have proposed a neural network that is first trained on popular open-source datasets and further fine-tuned on their own synthetic dataset.
The deep-learning-based method proves to be a full 100 times faster than traditional optimization-based optical flow. The Kandao team compared their neural network with FlowNet 2.0, a state-of-the-art deep learning method.
Kandao's CNN method is more accurate, efficient, and compact than FlowNet 2.0, the authors contend.
Several challenges remain before 360-degree VR cameras become more widely used. One challenge involves improving optimal flow for more temporal-consistent dense pixel correspondence. Another involves finding a way to encode and package VR video into sub-streams for faster transmission over bandwidth-limited networks.
"Many research groups are currently addressing these and other challenges, and the day will soon arrive when VR video becomes truly ubiquitous—easily produced, consumed, and enjoyed by the masses," the authors say.
Login may be required for full text.
About Lori Cameron
Lori Cameron is a Senior Writer for the IEEE Computer Society and currently writes regular features for Computer magazine, Computing Edge, and the Computing Now and Magazine Roundup websites. Contact her at l.cameron@computer.org. Follow her on LinkedIn.
