Page Areas:



Current Submenu:

Additional Information:

Student Projects

Hier den Alternativtext zum Bild eingeben!

Displays Book (free image material)

Hier den Alternativtext zum Bild eingeben!

Spatial Augmented Reality Book (free ebook)

Hier den Alternativtext zum Bild eingeben!

VIOSO smartprojecting

Hier den Alternativtext zum Bild eingeben!

The Flagship Publication of the IEEE Computer Society

Hier den Alternativtext zum Bild eingeben!

The Caleydo Project

Hier den Alternativtext zum Bild eingeben!

Location - Computer Science Building (S3)

Location of the Institute for Computer Graphics: Computer Science Building (Science Park 3)

Position Indication:

Content

Research

The Institute of Computer Graphics carries out research in a modern field that has been coined "visual computing". Our core disciplines cover the imaging, processing, visualization and displaying of visual data. These are enabled by new fields and technologies, such as light fields, projector-camera systems, responsive optics, mobile computing, and visual analytics.

Please select one of the following topics or years for detailed information.















Mobile Computing

PhoneGuide: Adaptive Image Classification for Mobile Museum Guidance


With the increasing computational capacity of camera-equipped mobile phones, object recognition on such devices is shifting away from centralized client-server approaches, in which the phones act only as input/output front-ends, to local on-device classification systems. The advantages of such a decentralization are shorter response times, scalability with respect to a large number of simultaneous users, and reduced network traffic costs. Mobile image classification can support applications that rely on device localization, such as museum or city guidance, by supplementing existing positional information retrieved, for instance, from GPS or
GSM cells. The challenge for mobile image classification, however, is to become as robust as possible, even when applied in large, highly dynamic, and uncontrollable public environments: Hundreds to thousands of objects must be recognized from different perspectives, from varying distances, and under changing lighting conditions, while recognition rates must remain usable. The key to solving this problem may be automatic adaptation to dynamic changes in the environment and to the most common user behavior.
This paper summarizes the various components of our mobile museum guidance system PhoneGuide.

Bimber, O and Bruns, E., PhoneGuide: Adaptive Image Classification for Mobile Museum Guidance, invited paper, In proceedings of 5th International Symposium on Ubiquitous Virtual Reality (ISUVR'11), 2011

  • application/pdfManuscript (4.2 MB)
  • Mobile Museum Guidance using Relational Multi-Image Classification


    We present a multi-image classification technique for mobile phones that is supported by relational reasoning. Users capture a sequence of images employing a simple near-far camera movement. After classifying distinct keyframes using a nearest-neighbor approach the corresponding database images are only considered for a majority voting if they exhibit similar near-far inter-image relations to the captured keyframes. In the context of PhoneGuide, our adaptive mobile museum guidance system, a user study revealed that our multi-image classification technique leads to significantly higher classification rates than single image classification. Furthermore, when using near-far image relations, less keyframes are sufficient for classification. This increases the overall classification speed of our approach by up to 35%.

    Bruns, E. and Bimber, O., Mobile Museum Guidance through Relational Multi-Image Classification, International Conference on Multimedia and Ubiquitous Engineering (MUE’10), 2010, Best Paper Award

    • application/pdfManuscript (17.3 MB)
    • Localization and Classification through Adaptive Pathway Analysis


      We present an unobtrusive technique for supporting and improving object recognition approaches on mobile phones. To accomplish this we determine the present and future locations of museum visitors by evaluating user-generated spatio-temporal pathway data. In the context of our adaptive mobile museum guidance system called PhoneGuide we show that this improves the classification performance significantly and can achieve recognition rates comparable to those of traditional location-based image classification approaches. Over a period of four months, we collected the pathway data of 132 regular museum visitors at the Natural History Museum of Erfurt, Germany.

      Bruns, E. and Bimber, O., Localization and Classification through Adaptive Pathway Analysis, IEEE Pervasive Computing, 2010 (submitted: July 2009, accepted: August 2010), April-June issue, 2012

      • application/pdfManuscript (1.6 MB)
      • Mobile Museum Guidance using Relational Multi-Image Classification


        We present a multi-image classification technique for mobile phones that is supported by relational reasoning. Users capture a sequence of images employing a simple near-far camera movement. After classifying distinct keyframes using a nearest-neighbor approach the corresponding database images are only considered for a majority voting if they exhibit similar near-far inter-image relations to the captured keyframes. In the context of PhoneGuide, our adaptive mobile museum guidance system, a user study revealed that our multi-image classification technique leads to significantly higher classification rates than single image classification. Furthermore, when using near-far image relations, less keyframes are sufficient for classification. This increases the overall classification speed of our approach by up to 35%.

        Bruns, E. and Bimber, O., Mobile Museum Guidance using Relational Multi-Image Classification, In proceedings of International Conference on Multimedia and Ubiquitous Engineering (MUE), 2010

        • application/pdfManuscript (17.3 MB)
        • Subobject Detection through Spatial Relationships on Mobile Phone


          We present a novel image classification technique for detecting multiple objects (called subobjects) in a single image. In addition to image classifiers, we apply spatial relationships among the subobjects to verify and to predict locations of detected and undetected subobjects, respectively. By continuously refining the spatial relationships throughout the detection process, even locations of completely occluded exhibits can be determined. Finally, all detected subobjects are labeled and the user can select the object of interest for retrieving corresponding multimedia information. This approach is applied in the context of PhoneGuide, an adaptive museum guidance system for camera-equipped mobile phones.
          We show that the recognition of subobjects using spatial relationships is up to 68% faster than related approaches without spatial relationships. Results of a field experiment in a local museum illustrate that unexperienced users reach an average recognition rate for subobjects of 85.6% under realistic conditions.

          Brombach, B., Bruns, E. and Bimber, O., Subobject Detection through Spatial Relationships on Mobile Phones, In proceedings of International Conference of Intelligent User Interfaces (IUI2009), pp. 267-276, 2009

          • application/pdfManuscript (9.3 MB)
          • Phone-to-Phone Communication for Adaptive Image Classification


            We present a novel technique for adapting local image classifiers that are applied for object recognition on mobile phones through ad-hoc network communication between the devices. By continuously accumulating and exchanging collected user feedback among devices that are located within signal range, we show that our approach improves the overall classification rate and adapts to dynamic changes quickly. This technique is applied in the context of our PhoneGuide system -- a mobile phone based museum guidance framework that combines pervasive tracking and local object recognition for identifying a large number of objects in uncontrolled museum environments. We explain a technique that distributes the user feedback information during runtime through ad-hoc network connections between local devices. By doing so, we enforce cooperative classification improvements during the actual stay of the visitors. The general functionality of our technique has been tested with a small number of real devices in a museum. For proving its scalability, however, we have developed a simulator that evaluates our method for many hundred devices under several conditions. The simulation parameters have all been gathered in a museum, and are therefore realistic. We will show that ad-hoc phone-to-phone synchronization not only leads to higher overall classification rates, but also to quicker adaptations to dynamic changes during runtime.

            Bruns, E. and Bimber, O., Phone-to-Phone Communication for Adaptive Image Classification, In proceedings of International Conference on Advances in Mobile Computing & Multimedia (MoMM2008), pp. 276-281, 2008

            • application/pdfManuscript (3.3 MB)
            • Adaptive Training of Video Sets for Image Recognition on Mobile Phones


              We present an enhancement towards adaptive video training for PhoneGuide, a digital museum guidance system for ordinary camera–equipped mobile phones. It enables museum visitors to identify exhibits by capturing photos of them. In this article, a combined solution of object recognition and pervasive tracking is extended to a client–server–system for improving data acquisition and for supporting scale–invariant object recognition. A static as well as a dynamic training technique are presented that preprocess the collected object data differently and apply two types of neural networks for classification. Furthermore, the system enables a temporal adaptation for ensuring a continuous data acquisition to improve the recognition rate over time. A formal field experiment reveals current recognition rates and indicates the practicability of both methods under realistic conditions in a museum.

              Bruns, E. and Bimber, O., Adaptive Training of Video Sets for Image Recognition on Mobile Phones. In Journal of Pervasive and Ubiquitous Computing, vol. 13, no. 2, pp. 165-178, 2009 (submitted: June 2007, accepted: February 2008)

              • application/pdfManuscript (5.3 MB)
              • Mobile Phone Enabled Museum Guidance with Adaptive Classification


                We present a new adaptive classification system for museum guidance tasks. It uses camera-equipped mobile phones for on-device object recognition in ad-hoc sensor networks and provides location and object aware multimedia content to museum visitors. Our approach is invariant against perspective, distance and illumination. It supports the scalable identification of single objects and multiple sub-objects, pervasive tracking, phone-to-sensor and phone-to-phone communication. It adapts to user behaviour and environmental conditions over time and achieves high recognition rates under realistic conditions. Our decentralized classification approach makes the system highly scalable to an arbitrarily large number of users since the heavy-weight training process is carried out off-line on the server while the lower-weight classification task is performed individually and in parallel by each mobile phone.

                Bruns, E., Brombach, B. and Bimber, O., Mobile Phone Enabled Museum Guidance with Adaptive Classification. In IEEE Computer Graphics and Applications, vol. 28, no. 4, pp. 98-102, 2008 (submitted: November 2007, accepted: March 2008)

                • application/pdfManuscript (1.0 MB)
                • Unsynchronized 4D Barcodes


                  We present a novel technique for optical data transfer between public displays and mobile devices based on unsynchronized 4D barcodes. We assume that no direct (electromagnetic or other) connection between the devices can exist. Time-multiplexed, 2D color barcodes are displayed on screens and recorded with camera equipped mobile phones. This allows to transmit information optically between both devices. Our approach maximizes the data throughput and the robustness of the barcode recognition, while no immediate synchronization exists. Although the transfer rate is much smaller than it can be achieved with electromagnetic techniques (e.g., Bluetooth or WiFi), we envision to apply such a technique wherever no direct connection is available. 4D barcodes can, for instance, be integrated into public web-pages, movie sequences, advertisement presentations or information displays, and they encode and transmit more information than possible with single 2D or 3D barcodes.

                  Langlotz, T. and Bimber, O., Unsynchronized 4D Barcodes. In proceedings of International Symposium on Visual Computing (ISVC'07), pp. 363-374, 2007

                  • application/pdfManuscript (7.1 MB)
                  • Enabling Mobile Phones To Support Large-Scale Museum Guidance


                    Mobile phones have the potential of becoming a future platform for personal museum guidance. They enable full multimedia presentations and –assuming that the visitors are using their own devices– will significantly reduce acquisition and maintenance cost for museum operators. However, several technological challenges have to be mastered before this concept can be successful. One of them is the question of how individual museum objects can be intuitively identified before presenting corresponding information. We have developed an enhanced museum guidance system called PhoneGuide that uses widespread camera equipped mobile phones for on-device object recognition in combination with pervasive tracking. It provides additional location- and object-aware multimedia content to museum visitors, and is scalable to cover a large number of museum objects. In a field survey our system was able to identify 155 real museum exhibits from multiple perspectives with a recognition rate of 95% and a classification speed of less than one second per object. A coarse grid of only eight low-cost Bluetooth emitters distributed over two museum floors was used to achieve these results. Once an object has been recognized, related multimedia presentations such as videos, audio , text, computer graphics and images are displayed on the phone.
                    Special thanks to the City Museum of Weimar and to CellIQ for their support.

                    Bruns, E., Brombach, B., and Zeidler, T., Bimber, O., Enabling Mobile Phones To Support Large-Scale Museum Guidance. In IEEE MultiMedia, vol. 14, no. 2, pp. 16-25, 2007 (submitted: October 2005, accepted: July 2006)

                    • application/pdfManuscript (4.2 MB)
                    • PhoneGuide: Museum Guidance Supported by On-Device Object Recognition on Mobile Phones


                      We present PhoneGuide – an enhanced museum guidance approach that uses camera-equipped mobile phones and on-device object recognition. Our main technical achievement is a simple and light-weight object recognition approach that is realized with single-layer perceptron neuronal networks. In contrast to related systems which perform computational intensive image processing tasks on remote servers, our intention is to carry out all computations directly on the phone. This ensures little or even no network connectivity and consequently decreases cost for online times. Our laboratory experiments and field surveys have shown that photographed museum exhibits can be recognized with a probability of over 90%. We have evaluated different feature sets to optimize the recognition rate and performance. Our experiments reviled that normalized color features are most effective for our method. Choosing such a feature set allows recognizing an object below one second on up-to-date phones. The amount of data that is required for differentiating 50 objects from multiple perspectives is les than 6KBytes.
                      Special thanks to the Senckenberg Museum of Natural History, Frankfurt, to the Museum für Ur- und Frühgeschichte, Weimar, and to CellIQ for their support.

                      Foeckler, P., Zeidler, T., Brombach, B., Bruns, E., and Bimber, O. PhoneGuide: Museum Guidance Supported by On-Device Object Recognition on Mobile Phones. In proceedings of International Conference on Mobile and Ubiquitous Computing (MUM'05), pp. 3-10, 2005

                      • application/pdfManuscript (654KB)
                        • video/mpegSenckenberg Museum - Content Provider (6.9 MB)
                        • video/mpegSenckenberg Museum - User (2.7 MB)
                        • Video See-Through AR and Optical Tracking with Consumer Cell Phones


                          To enable mobile devices, such as head-mounted displays and PDAs, to support video see-through augmented reality is a popular research topic. However, such technology is not widely-spread outside the research community today. It has been estimated that by the end of the year 2005 approximately 50% of all cell phones will be equipped with digital cameras. Consequently, using cell phones as platform for video see-through AR has the potential of addressing a brought group of end users and applications. Compared to high-end PDAs and HMDs together with personal computers, the implementation of video see-through AR on the current fun- and smart-phone generations is a challenging task: Ultra-low video-stream resolutions, little graphics and memory capabilities, as well as slow processors set technological limitations. We have realized a prototype solution for video see-through AR on consumer cell phones. It supports optical tracking of passive paper markers and the correct integration of 2D/3D graphics into the live video-stream at interactive rates. We aim at applications, such as interactive tour guiding for museums and tourism, as well as at mobile games.
                          Special thanks to Nokia Research for their support.

                          Moehring, M., Lessig, C. and Bimber, O. Video See-Through AR on Consumer Cell Phones. In proceedings of IEEE/ACM International Symposium on Augmented and Mixed Reality (ISMAR'04), pp. 252-253, 2004

                          Moehring, M., Lessig, C. and Bimber, O. Optical Tracking and Video See-Through AR on Consumer Cell Phones. In proceedings of Workshop on Virtual and Augmented Reality of the GI-Fachgruppe AR/VR, pp. 193-204, 2004


                    English Version English Version