Institut für Kognitionswissenschaft

Institute of Cognitive Science


Osnabrück University navigation and search


Main content

Top content

Bio-Inspired Architecture for Deriving 3D Models from Video Sequences

Supplementary Material

Abstract The reconstruction of complex 3D objects from video sequences captured by surveillance, smartphone, and other cameras is a common technique in Hollywood blockbusters and TV series. Unfortunately, the automatic or interactive 3D object reconstruction from this kind of videos is not yet possible in the real world. Enabling computers to recognize the actual 3D shape of objects from complex video sequences, we developed a bio-inspired processing architecture, motivated by findings in the area of human object recognition. By utilizing viewpoint-specific object recognition, changes in position and size of the object of interest in video sequences can be eliminated to a great extent. The result is a representation, comprised of multiple pictures showing 2D projections of the object of interest (OOI) from different viewpoints. Based on this representation, a 3D point cloud (PC) from the object can be obtained. After a detailed description of our architecture and its similarities to the human view-combination scheme, we demonstrate its potency by reconstructing several OOI from complex video sequences. Because some processing modules of the architecture cannot yet be fully automatized, we introduced interactive modules instead. Thus the prototypical implementation of our approach could be realized. Based on the resulting PC, we evaluate our architecture and consider more analogies between human and computer vision, which improve image-based 3D reconstruction.

Video Demonstration showing our Bio-Inspired Architecture for Deriving 3D Models from
Video Sequences

Resulting 3D Models

Result: Red Car Result: Red Car
(a)

Result: Red Car Result: Red Car
(b)

Result: Red Car Result: Red Car
(c)

Resulting point clouds of the (a) red car, (b) table, and (c) desk.

Download

Resulting Point Clouds

(a) red car
(b) table
(c) desk

Used Ground Truth

(b) table
(c) desk

Reference

[1] J. Schöning & G. Heidemann.
Ventral Stream-Inspired Process for Deriving 3D Models from Video Sequences.
In New Trends in Image Analysis and Processing --- ICIAP 2017, pages: 72-83, 2017. Springer International Publishing.
| PDF | DOI | URL | BibTeX
[2] J. Schöning & G. Heidemann.
Bio-Inspired Architecture for Deriving 3D Models from Video Sequences.
Computer Vision -- ACCV 2016 Workshop, pages: 62-76, ISBN: 978-3-319-54427-4, 2016. Springer Nature.
| PDF | DOI | URL | BibTeX