Publicerad: 2017-11-27
ISBN: 978-91-7685-384-9
ISSN: 1650-3686 (tryckt), 1650-3740 (online)
With depth sensors becoming more and more common, and applications with varying viewpoints (like e.g. virtual reality) becoming more and more popular, there is a growing demand for real-time depth-image-based-rendering algorithms that reach a high quality. Starting from a quality-wise top performing depth-image-based-renderer, we develop a real-time version. Despite reaching a high quality as well, the new OpenGL-based renderer decreases runtime by (at least) 2 magnitudes. This was made possible by discovering similarities between forward-based and mesh-based rendering, which enable us to remove the common parallelization bottleneck of competing memory access, and facilitated by the implementation of accurate yet fast algorithms for the different parts of the rendering pipeline. We evaluated the proposed renderer using a publicly available dataset with ground-truth depth and camera data, that contains both rapid camera movements and rotations as well as complex scenes and is therefore challenging to project accurately.
[BWSB12] BUTLER D., WULFF J., STANLEY G., BLACK M.: A naturalistic open source movie for optical flow evaluation. In Proceedings of European Conference on Computer Vision (2012). 1, 5
[Feh04] FEHN C.: Depth-image-based rendering, compression and transmission for a new approach on 3D-TV. In Stereoscopic Displays and Virtual Reality Systems (2004), SPIE. 1
[KPL05] KIM S.-M., PARK J.-C., LEE K. H.: Depth-image based full 3d modeling using trilinear interpolation and distance transform. In Proceedings of the 2005 International Conference on Augmented Tele-existence (2005), ICAT ’05. 3
[MLS14] MALL R., LANGONE R., SUYKENS J.: Agglomerative hierarchical kernel spectral data clustering. In IEEE Symposium on Computational Intelligence and Data Mining (2014). 2
[MMB97] MARK W. R., MCMILLAN L., BISHOP G.: Postrendering 3d warping. In Proceedings of 1997 Symposium on Interactive 3D Graphics (1997). 1
[NIH_11] NEWCOMBE R. A., IZADI S., HILLIGES O., MOLYNEAUX D., KIM D., DAVISON A. J., KOHI P., SHOTTON J., HODGES S., FITZGIBBON A.: Kinectfusion: Real-time dense surface mapping and tracking. In 2011 10th IEEE International Symposium on Mixed and Augmented Reality (2011). 3
[OF17] OGNIEWSKI J., FORSSÉN P.-E.: Pushing the limits for view prediction in video coding. In Proceedings of the 12th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 4: VISAPP, (VISIGRAPP 2017) (2017). 1
[PG10] PALOMO C., GATTASS M.: An efficient algorithm for depth image rendering. In Proceedings of the 9th ACM SIGGRAPH Conference on Virtual-Reality Continuum and Its Applications in Industry (2010). 1
[SR10] SOLH M., REGIB G. A.: Hierarchical hole-filling(HHF): Depth image based rendering without depth map filtering for 3DTV. In IEEE International Workshop on Multimedia and Signal Processing (2010). 3
[SV14] SCALZO M., VELIPASALAR S.: Agglomerative clustering for feature point grouping. In IEEE International Conference on Image Processing (ICIP) (2014). 2
[Sze11] SZELISKI R.: Computer Vision: Algorithms and Applications. Springer Verlag London, 2011. 1
[TLLG09] TIAN D., LAI P.-L., LOPEZ P., GOMILA C.: View synthesis techniques for 3D video. In Proceedings of SPIE Applications of Digital Image Processing (2009), SPIE. 1
[WLC15] WANG C., LIN Z., CHAN S.: Depth map restoration and upsampling for kinect v2 based on ir-depth consistency and joint adaptive kernel regression. In IEEE International Symposium onCircuits and Systems (ISCAS) (2015). 3
[WSB03] WANG Z., SIMONCELLI E. P., BOVIK A. C.: Multiscale structural similarity for image quality assessment. In 37th IEEE Asilomar Conference on Signals, Systems and Computers (2003). 6
[YHL16] YAO L., HAN Y., LI X.: Virtual viewpoint synthesis using cuda acceleration. In Proceedings of the 22nd ACM Conference on Virtual Reality Software and Technology (2016). 1
