The combination of advanced software algorithms and optics opens up new possibilities for display, imaging, and lighting. It makes possible responsive optical systems that adapt to particular situations automatically and dynamically. Visual computing is a relatively young research field that provides a foundation for many of these approaches. It represents a tight coupling between image synthesis, image analysis, and visual perception. While optics is all about image formation, visual computing deals with the general processing of images. This paper summarizes several examples that illustrate how graphics, vision, perception, and optics are combined to realize smart projectors, smart cameras, and smart light sources.

We show that optical inverse tone-mapping (OITM) in light microscopy can improve the visibility of specimens, both when observed directly through the oculars and when imaged with a camera. In contrast to previous microscopy techniques, we pre-modulate the illumination based on the local modulation properties of the specimen itself. We explain how the modulation of uniform white light by a specimen can be estimated in real-time, even though the specimen is continuously but not uniformly illuminated. This information is processed and back-projected constantly, allowing the illumination to be adjusted on the fly if the specimen is moved or the focus or magnification of the microscope is changed. The contrast of the specimen's optical image can be enhanced, and high-intensity highlights can be suppressed. A formal pilot study with users indicates that this optimizes the visibility of spatial structures when observed through the oculars. We also demonstrate that the signal-to-noise (S/N) ratio in digital images of the specimen is higher if captured under an optimized rather than a uniform illumination. In contrast to advanced scanning techniques that maximize the S/N ratio using multiple measurements, our approach is fast because it requires only two images. This can be beneficial for image analysis in digital microscopy applications with real-time capturing demands.

Coding a projector's aperture plane with adaptive patterns together with inverse filtering allow the depth-of-field of projected imagery to be increased. We present two prototypes and corresponding algorithms for static and programmable apertures. We also explain how these patterns can be computed at interactive rates, by taking into account the image content and limitations of the human visual system. Applications such as projector defocus compensation, high quality projector de-pixelation, and increased temporal contrast of projected video sequences can be supported. Coded apertures are a step towards next-generation auto-iris projector lenses.

We show how temporal backdrops that alternately change their color rapidly at recording rate can aid chroma keying by transforming color spill into a neutral background illumination. Since the chosen colors sum up to white, the chromatic (color) spill component is neutralized when integrating over both backdrop states. Being able to separate both states, however, additionally allows to compute high quality alpha mattes. Besides neutralizing color spill, our method is invariant to foreground colors and supports applications with real-time demands. In this article, we explain different realizations of temporal backdrops and describe how keying and color spill neutralization are carried out, how artifacts resulting from rapid motion can be reduced, and how our approach can be implemented to be compatible with common real-time post-production pipelines.

We synchronize film cameras and LED lighting with off-the-shelf video projectors. Radiometric compensation allows displaying keying patterns and other spatial codes on arbitrary real world surfaces. A fast temporal multiplexing of coded projection and flash illumination enables professional keying, environment matting, displaying moderator information, scene reconstruction, and camera tracking for non-studio film sets without being limited to the constraints of a virtual studio. The reconstruction of the scene geometry allows special composition effects, such as shadow casts, occlusions and reflections. This makes digital video composition more flexible, since static studio equipment, such as blue screens, teleprompters, or tracking devices, is not required. Authentic film locations can be supported with our portable system without causing a lot of installation effort. We propose a concept that combines all of these techniques into one single compact system that is fully compatible with common digital video composition pipelines, and offers an immediate plug-and-play applicability.

We present a simple and cost-efficient way of extending contrast, perceived tonal resolution, and color space of reflective media, such as paper prints, hardcopy photographs, or electronic paper displays. A calibrated projector-camera system is applied for automatic registration, radiometric scanning and superimposition. A second modulation of the projected light on the surface of such media results in a high dynamic range visualization. This holds application potential for a variety of domains, such as radiology, astronomy, optical microscopy, conservation and restoration of historic art, modern art and entertainment installations. In our experiments, we achieved contrast ratios of up of 45,000-60,000:1 with a peak luminance of more than 2,750 cd/m^2, could technically re-produce more than 620 perceptually distinguishable tonal values. Furthermore, we attained color space extensions of up to a factor of 1.4 (compared to a regular projection on white screens) or factor of 3.3 (compared to regular paper prints under environment light). Thereby, the hardcopy resolution can be several thousand DPI or several hundred LPI, while luminance and chrominance are modulated with a registration error of less than 0.3 mm. Thus, compared with most existing interactive HDR displays, we support near distance viewing at a contrast frequency of up to 7 cpd (given our current registration precision and assuming a viewing distance of 50 cm).

Radiometric compensation techniques allow seamless projections onto complex everyday surfaces. Implemented with projector-camera systems they support the presentation of visual content in situations where projection-optimized screens are not available or not desired - as in museums, historic sites, air-plane cabins, or stage performances. We propose a novel approach that employs the full light transport between projectors and a camera to account for many illumination aspects, such as interreflections, refractions, shadows, and defocus. Pre-computing the inverse light transport in combination with an efficient implementation on the GPU makes the real-time compensation of captured local and global light modulations possible.

We present a novel multi-step technique for imperceptible geometry and radiometry calibration of projector-camera systems. Our approach can be used to display geometry and color corrected images on non-optimized surfaces at interactive rates while simultaneously performing a series of invisible structured light projections during runtime. It supports disjoint projector-camera configurations, fast and progressive improvements, as well as real-time correction rates of arbitrary graphical content. The calibration is automatically triggered when mis-registrations between camera, projector and surface are detected.

We present a novel adaptive imperceptible pattern projection technique that considers parameters of human visual perception. A coded image that is invisible for human observers is temporally integrated into the projected image, but can be reconstructed by a synchronized camera. The embedded code is dynamically adjusted on the fly to guarantee its non-perceivability and to adapt it to the current camera pose. Linked with real-time flash keying, for instance, this enables in-shot optical tracking using a dynamic multi-resolution marker technique. A sample prototype is realized that demonstrates the application of our method in the context of augmentations in television studios.

We present a system that applies a custom-built pan-tilt-zoom camera for laser-pointer tracking in arbitrary real environments. Once placed in a building environment, it carries out a fully automatic self-registration, registrations of projectors, and sampling of surface parameters, such as geometry and reflectivity. After these steps, it can be used for tracking a laser spot on the surface as well as an LED marker in 3D space, using inter-playing fisheye context and controllable detail cameras. The captured surface information can be used for masking out areas that are critical to laser-pointer tracking, and for guiding geometric and radiometric image correction techniques that enable a projector-based augmentation on arbitrary surfaces. We described a distributed software framework that couples laser-pointer tracking for interaction, projector-based AR as well as video see-through AR for visualizations with the domain specific functionality of existing desktop tools for architectural planning, simulation and building surveying.

Projecting images onto surfaces that are not optimized for projections becomes more and more popular. Such approaches will enable the presentation of graphical, image or video content on arbitrary surfaces. Virtual reality visualizations may become possible in everyday environments - without specialized screen material or static screen configurations. Upcoming pocket projectors will enable truly mobile presentations on all available surfaces of furniture or papered walls. The playback of multimedia content will be supported on natural stonewalls of historic sites without destroying their ambience through the installations of artificial projection screens. We present a hybrid technique for correcting distortions that appear when projecting images onto geometrically complex, colored and textured surfaces. It analyzes the optical flow that results from perspective distortions during motions of the observer and tries to use this information for computing the correct image warping. If this fails due to an unreliable optical flow, an accurate –but slower and visible– structured light projection is automatically triggered. Together with an appropriate radiometric compensation, view-dependent content can be projected onto arbitrary everyday surfaces. An implementation mainly on the GPU ensures fast frame rates.

Our new radiometric compensation algorithm considers the human visual perception properties to reduce visible artefacts resulting from the limited dynamic range and brightness of projectors. It preserves a maximum of luminance and contrast and is implemented entirely on the GPU. Real-time frame rates are achieved for supporting animated and interactive content. Initially our algorithm performs an off-line analysis of the projection surface’s geomety and reflectance. The image content is then analyzed to determine the average luminance values, the amount of high spatial frequencies, and a luminance threshold map. The threshold map stores information about the maximum non-perceivable luminance differences for each pixel. The radiometric compensation is carried out in two passes: In the first pass the intensity values are translated and scaled globally depending on the surface reflectance and the image content itself. The result is analyzed for clipping errors. These errors are then blurred with a Gaussian kernel. The applied sigma is inverse proportional to the amount of high spatial frequencies in the local image areas. In the final pass the image intensities are translated and scaled globally, but the luminance values are also adjusted locally depending on the defocused clipping errors. Time dependent adaptation factors are used for global and local transformations to avoid popping artifacts in animated and interactive content.

Large scale optical holograms require large scale display technology for combining them with interactive graphical elements. Shuttered projection screens (SPS) can be used to sequentially display stereoscopic graphics (in the diffuse mode) and to reconstruct the holographic content (in the transparent mode). While the SPS is shuttered with 50Hz, the stereo pairs are synchronized and time-modulated at approximately 100Hz. Depth information of the holographic content are required to create consistent occlusion and illumination effects with the graphical content. A two-lens stereo camera system can be used for scanning the hologram partially. The different point clouds have to merge into a common coordinate system to form the whole surface. Due to the limited resolution of the range sensor, small gaps appear between the actual surface points. Instead of triangulating the points into a mesh of triangle primitives, the points remain unconnected. They are rendered as point primitives (splatted) with appropriate radii to fill the gaps of missing surface information. The splat size and resolution is adapted dynamically with respect to the observer’s position to ensure interactive frame rates.

Concavely shaped projection screens, such as CAVEs, two-sided workbenches, domes, or cylinders scatter a fraction of light to other screen portions. The amount of indirect illumination adds to the directly projected image and causes the displayed content to appear partially inconsistent and washed out. We have developed a reverse radiosity method that compensates first-level and higher-level secondary scattering effects in real-time. The images appear more brilliant and uniform when reducing the scattering contribution. A numerical solution is approximated with Jacobi iteration for a sparse-matrix linear equation system on the GPU. Efficient data structures allow packing the required data into textures which are processed by pixel shaders. Frame-buffer objects are used for a fast exchange of intermediate iteration results, and enable computations with floating point precision. Our algorithm’s result can be optimized for quality or performance.

Many multi-projector rendering techniques exist that aim at creating a high consistency of image geometry, intensity and color. We proposed a concept and a solution for considering and optimizing a fourth image property – its focus. We describe a novel multi-focal projection concept that applies conventional video projectors and camera feedback. Multiple projectors with differently adjusted focal planes, but overlapping image areas are used. They can be either arbitrarily positioned in the environment, or can be integrated into a single projection unit. During an automatic one-time calibration process, structured light projection together with camera feedback allows to measure the relative focus value of every projector pixel on an arbitrary diffuse surface. Thereby, the focus values are geometrically and radiometrically corrected. If this is known, a final image with minimal defocus can be composed in real-time from individual pixel contributions of all projectors. Our technique is independent of the surfaces’ geometry, color and texture, of the environment light, as well as of the projectors’ parameters (i.e., position, orientation, luminance and chrominance).

With this work we take a first step towards an ad-hoc stereoscopic projection within real environments. We show how view-dependent image-based and geometric warping, radiometric compensation, and multi-focal projection enable a view-dependent visualization on ordinary (geometric complex, colored and textured) surfaces within everyday environments. All these techniques are accomplished at interactive rates and on a per-pixel basis for multiple interplaying projectors. Special display configurations for immersive or semi-immersive VR/AR applications that require permanent and artificial projection canvases might become unnecessary. Such an approach does not only offer new possibilities for augmented reality and virtual reality, but also allows merging both technologies. This potentially gives some application domains – like architecture – the possibility to benefit from the conceptual overlaps of AR and VR.
Special thanks to the Faculty of Architecture, Bauhaus-University Weimar for their support.

Holography and computer graphics are being used as tools to solve individual research, engineering, and presentation problems within several domains. Up until today, however, these tools have been applied separately. Our intention is to combine both technologies to create a powerful tool for science, industry and education. We are currently investigating the possibility of integrating computer generated graphics and holograms. We presents several applications of interaction techniques to graphically enhanced holograms and give a first glance on a novel method that reconstructs depth from optical holograms.
Special thanks to Deutsche Forschungsgemeinschaft (DFG) for their support.

Video projectors will play a major role in future home entertainment and edutainment applications – ranging from movies and television, over computer games, to multimedia presentations. With video projectors images can be displayed that are larger than the devices themselves. However, we have to give up living space and ambience to set up artificial canvases that have to be as large as the desired image. Smart projectors are able to display correct images onto arbitrary existing screen surfaces, like wallpapered walls or window curtains. Thus it can function without an artificial canvas and consequently leaves a bit more freedom to us in the decision on how to arrange our living space. Our smart projectors combine camera feedback with structured light projection to gain information about the screen surface and the environment. The calibration of such a device is fast, fully automatic and robust, and the correction of video signals can be achieved in real-time. Neither geometry information nor projector or camera parameters need to be known. Instead, the entire calibration and correction (geometry and color) is done on a per-pixel level – supported by modern pixel shader hardware. Such devices might make it possible to convert your bookshelf into a TV screen, or your kid’s closet into an interactive virtual playground.
Special thanks to the Bennert Group for their support.

Pictorial artwork, such as paintings and drawings, can tell interesting stories. The capabilities of museums to communicate such and other information, however, are clearly limited. Text legends and audio guides can mediate facts, but offer little potential for presenting visual content, such as embedded illustrations, pictures, animations, and interactive elements. Also due to the difficult economic situation, edutainment is becoming an important factor for museums. By applying new media technologies – such as computer graphics, virtual reality and augmented reality– exhibit oriented information might be communicated more effectively, but certainly in a more exciting way. We describe a novel technological approach, a mathematical model, a real-time rendering algorithm and examples of presentation techniques for integrating almost any kind of visual information directly into pictorial artwork. It allows displaying such information while keeping the observers’ attention on the original artifact, and does not require additional screens.
Special thanks to the Museum het Rembrandthuis, Amsterdam for their support.

The Holographic Workstation (short: HoloStation) is a desktop display that allows the integration of interactive stereoscopic or auto-stereoscopic graphical elements into high-quality optical holograms.
Archaeologists, for instance, use optical holograms to archive and investigate ancient artefacts. The distribution of copy holograms enables scientists to perform their research without having access to the original artefacts or inaccurate replicas. Combining these static holograms with interactive computer graphics will allow them to integrate real-time simulation data or to perform experiments that require a direct user interaction, such as packing reconstructed soft-tissue into a hologram of a fossilized dinosaur skull. In addition, specialized interaction devices can simulate haptic feedback of the holographic and the graphical content while performing such interactive tasks. An entire collection of artefacts will fit into a single album of holographic recordings, while a light-box-like display can be used for visualization and interaction.
The current HoloStation prototype can be used in combination with many different holograms, such as transmission and reflection types. It is a compact and flexible display that provides several forms of interaction via integrated touch-screen and real-time range scanner.
Special thanks to Deutsche Forschungsgemeinschaft (DFG) for their support.

A hologram is a photometric emulsion that records interference patterns of coherent light. The recording itself stores amplitude, wavelength and phase information of light waves. In contrast to simple photographs (which can record only amplitude and wavelength information), holograms have the ability to reconstruct complete optical wavefronts. This results in a three-dimensional appearance of the captured scenery which is observable from different perspectives.
Today, many applications for optical holograms exist. Examples include interferometry, copy protections, data storage and holographic optical elements. Optical holograms, however, are static and lack in interactivity. Electro-holography aims at the computer-based generation and display of holograms in real time. Since a massive amount of data has to be processed, transmitted and stored to create holograms, today’s computer technology still sets the limits of electro-holography. To overcome some of the performance issues, advanced reduction and compression methods have been developed. This results in electro-holograms that are interactive, but small, low-resolution and pure in color.
A novel approach has been proposed that combines holography, autostereoscopy and projection technology to integrate interactive graphical elements into high-quality optical holograms. Beside the conceptual idea and the optical solutions, efficient rendering techniques and a proof-of-concept prototype are presented. Furthermore, potential application areas, such as museums and scientific visualization are outlined.
Special thanks to Deutsche Forschungsgemeinschaft (DFG) for their support.

To achieve a consistent lighting situation between real and virtual environments is important for convincing augmented reality (AR) applications.
A rich pallet of algorithms and techniques have been developed that match illumination for video- or image-based augmented reality. However, very little work has been done in this area for optical see-through AR. We believe that the optical see-through concept is currently the most advanced technological approach to provide an acceptable level of realism and interactivity.
Methods have been developed which create a consistent illumination between real and virtual components within an optical see-through environment – such as the Virtual Showcase. Combinations of video projectors and cameras are applied to capture reflectance information from diffuse real objects and to illuminate them under new synthetic lighting conditions. For diffuse objects, the capturing process can also benefit from hardware acceleration – supporting dynamic update rates. To handle indirect lighting effects (like color bleeding) an off-line radiosity procedure is outlined that consists of multiple rendering passes. For direct lighting effects (such as simple shading, shadows and reflections) hardware accelerated techniques are described which allow to achieve interactive frame rates. The reflectance information is used in addition to solve a main problem of a previously introduced technique which creates consistent occlusion effects for multiple users within such environments.

Interactive digital storytelling techniques are recently being applied in combination with new media forms, such as virtual reality (VR) and augmented reality (AR). Thereby the technological progress that is being made within these areas allows shifting interactive digital storytelling more and more into the third dimension and into the physical world. One of the main advantages of this transition is the possibility to communicate information more effectively with digital means by telling stories that can be experienced directly within a real environment or in combination with physical objects. The user experience is thus transformed from relating different pieces of information to one another to ‘living through’ the narrative. The perceptual quality and the unique aura of a real environment (e.g., a historical site) or object (e.g., an ancient artifact) cannot be simulated by today’s technology. Thus it is not possible to substitute them with virtual or electronic copies without them losing their flair of originality. This circumstance can be a crucial reason for using augmented reality as a technological basis for interactive digital storytelling. Several research groups apply personal displays, such as head-mounted displays (HMDs) to realize AR-based new media experiences. These all-purpose display types, however, have to face technological problems that up until today have not been sufficiently solved1. These shortcomings cause a substantial credibility gap if a certain level of realism is required.
We want to discuss how an application-customized type of augmented reality display –the Virtual Showcase– overcomes most of these technological shortcomings, and how it can be used as a new platform for digital storytelling.
To underline our statements we discuss and evaluate a case study that focuses on using AR digital storytelling to communicate scientific information to a novice audience in a museum context. We have introduced the application that led to our case study in a previous publication. In this paper, we want to describe the technical components that have been developed to realize this application.
In addition, we present a first user feedback and illustrate our current efforts of improvement.

Displays which require a non-linear pre-distortion to present undistorted gfx on non-planar surfaces, or to neutralize optical distortion usually apply multi-pass rendering techniques. Projective textures or uniform grids are used to deform the image generated during the first pass before it is displayed as a texture map during the final pass. However, these approaches do not consider the error that is generated from a piecewise linear texture interpolation to adapt the underlying geometry.
Curved mirror displays, for instance, that stereoscopically produce three-dimensional gfx generally don't pre-distort the graphics before they are displayed. Yet, some systems apply additional optics (such as lenses) to stretch or undistort the reflected image. But these devices constrain the observer to a single point of view or to very restricted viewing zones. For the Virtual Showcase mirror display, however, a view-dependent rendering is required to support freely moving observers. The existing image warping algorithms for curved Virtual Showcases predistort a uniform image grid and consequently do not consider the local error that is generated from a piecewise linear texture interpolation. These algorithms can only produce an acceptable image quality within a significantly large amount of rendering time.
The algorithm that is presented improves the uniform image warping techniques for curved Virtual Showcases by defining an appropriate error metric and implementing a quadtree-based selective refinement method that generates adapted local levels of detail. It approaches to produce a predefined image quality within a minimum amount of rendering time. While display specific details of our algorithm are explained based on the Virtual Showcase display, its general functionality is valid for other non-linear displays -such as for curved projection-based systems.

Paleontology is filled with mysteries about the plants and animals that lived thousands, millions, even billions of years before the first humans walked the earth. To answer questions about these organisms, paleontologists rely on the excavation, analysis, and interpretation of fossils. Embedded and preserved in the earth’s crust, fossils are the remains or traces of ancient life forms, including bones, teeth, shells, leaf imprints, nests, and footprints.Fossils can disclose how organisms evolved over time and their relationship to one another. While they reveal much, such as the general shape and size of ancient living things, fossils keep us guessing about these organisms’ color, sound, and—most significantly—their behavior. For several years, modern paleontologists have used 3D computer graphics to help reconstruct these pieces of the past. State-of-the-art scanning technology produces 3D fossil replicas that scientists can process and study without physical constraints. Paleontologists typically generate volumetric data sets for analysis, such as magnetic resonance imaging or computed axial tomography scans, and they use surface models for digital preservation and reproduction. To study ontogeny—an organism’s growth and form—paleontologists apply mathematical models for simulation and visualization. Likewise, computer animations help study dinosaur locomotion. Beyond building knowledge of our world, the results of this work influence how dinosaurs appear in museums, illustrations, and movies, and as toys. In the past 40 years, technological advances have continued to blur the boundary between real and computer-generated worlds. Augmented reality leverages this technology to provide an interface that enhances the real world with synthetic supplements. Paleontologists can use AR to present virtual data, such as 3D computer graphics, directly within a real environment rather than on a flat monitor. We coined the term augmented paleontology to refer to the application of AR to paleontology. AP seeks to support paleontologists in their research, and communicate the results of paleontology to museum visitors in an exciting and effective way. An interdisciplinary team of paleontologists, graphics designers, and computer scientists has already applied the AP interface to soft-tissue reconstruction and the study of dinosaur locomotion.

Projection-based augmented reality systems, such as the Virtual Showcase, share many positive properties of projection-based virtual environments. These displays provide high resolution, improved consistency of eye accommodation and convergence, little motion sickness potential, and the possibility of an integration into common working environments. One of the main challenges for projection-based AR systems as well as for head-mounted optical see-through displays is the generation of correct occlusion effects between virtual and real objects. Additionally shadows of virtual objects cast onto real ones and consistent illumination of the real and virtual scenery are often difficult to achieve. We introduce projector-based illumination techniques for view-dependent optical seethrough AR displays. This approach has the potential to solve all of the above mentioned problems. Here, we focus on using projector-based illumination for creating correct occlusion effects for mixed reality configurations.
We have implemented and tested such a system in the context of the Virtual Showcase, which consists of a horizontal projection screen and a convex half-silvered mirror assembly. Virtual and real objects can be displayed in the same space inside the showcase. The original Virtual Showcase used a standard light bulb to illuminate real objects. This setup does not provide very much control over the lighting situation. By using a computer-controlled video-projector as a replacement for the simple light bulb, we are able to fully control the lighting situation inside the showcase on a perpixel basis. Our main contribution is a solution to the problem of correct occlusion for mixed reality scenarios with viewdependent optical see-through displays. Our method produces correct occlusion effects between virtual and real objects by projecting shadows onto real objects located behind virtual ones using projector-based illumination.

Intuitive access to information in habitual realworld environments is a challenge for information technology. An important question is how can we enhance established and well-functioning everyday environments rather than replace them by virtual environments (VEs)? Augmented reality (AR) technology has a lot of potential in this respect because it augments realworld environments with computer-generated imagery.
Today, most AR systems use see-through head-mounted displays, which share most of the disadvantages of other head-attached display devices. We introduce a new projection-based AR display system—the Virtual Showcase. The Virtual Showcase has the same form factor as a real showcase, making it compatible with traditional museum displays. Real scientific and cultural artifacts are placed inside the Virtual Showcase allowing their 3D graphical augmentation. Inside the Virtual Showcase, virtual representations and real artifacts share the same space providing new ways of merging and exploring real and virtual content. Solely virtual exhibits may also be displayed. The virtual part of the showcase can react in various ways to a visitor, which provides the possibility for intuitive interaction with the displayed content. Another interesting aspect of our system is its support for two to four simultaneously tracked users looking at the Virtual Showcase from different sides. This feature allows the collaborative exploration of artifacts shown in the Virtual Showcase. These interactive showcases contribute to ambient intelligent landscapes, where the computer acts as an intelligent server in the background and visitors can focus on exploring the exhibited content rather than on operating computers. The Virtual Showcase consists of two main parts: a convex assembly of halfsilvered mirrors and a graphics display. So far, we’ve built Virtual Showcases with two different mirror configurations. Our first prototype consists of four halfsilvered mirrors assembled as a truncated pyramid. Our second prototype uses a single mirror sheet to form a truncated cone. We placed these mirror assemblies on top of a projection screen. Users can see real objects inside the showcase through the half-silvered mirrors merged with the graphics displayed on the projection screen. We illuminated the showcases’ contents with a controllable light source while presenting view-dependent stereoscopic graphics to the observer. For our current prototypes, we use standard shutter glasses controlled by infrared emitters. Head tracking is accomplished using an electromagnetic tracking device. Our pyramid shaped prototype supports up to four viewers simultaneously looking at the showcase from four different sides. Our cone-shaped prototype provides a seamless surround view onto the displayed artifact.

We describe a prototype of an optical extension for table-like rear-projection systems -the Extended Virtual Table. A large half-silvered mirror is used as the optical combiner to unify a virtual and a real workbench, whereby the shortcomings that are related to the Reflective Pad and Transflective Pad (i.e., indirect line-of-sight, limited field of view, and tracking distortion) are reduced. The virtual workbench has been enabled to display computer graphics beyond its projection boundaries and to combine virtual environments with the adjacent real world. A variety of techniques is described that allow indirect interaction with virtual objects through the mirror.
Systems, such as the Extended Virtual Table, approach a conceptual and technical extension of traditional Virtual Reality by means of Augmented Reality (xVR), and a seamless integration of such technology into habitual work environments. Furthermore, optical distortions that are caused by the half-silvered mirror combiner and the projector, as well as non-optical distortions caused by the tracking device are analyzed and appropriate compensation methods are described.

Traditional Augmented Reality (AR) displays, such as head-mounted devices entail a number of technological and ergonomic drawbacks. Their limited resolution, the restricted field-of-view, the visual perception issues that are due to the fixed focal length caused by a constant and head-attached image plane, the increased incidence of discomfort provoked by simulation sickness and their cumbersomeness prevent their usage in a number of application areas. To overcome some of these drawbacks, but also to open new application areas for AR, we propose a projectionbased AR concept that combines spatially aligned optical see-through elements (essentially half-silvered mirror beam-splitters) with off-the-shelf projection-based Virtual Reality displays. This concept offers possibilities to combine the advantages of both technologies: the well established projection-based Virtual Reality with the potentials of Augmented Reality. We describe the early stages of a new proof of-concept prototype that has been developed to extend the scope of projection-based AR towards the engineering domain.

Employing virtual reality (VR) technology, computer-based work environments are clearly evolving into a major component of the next-generation workplace. Over the past few years, VR has become a practical reality for many applications, thanks to a number of technology inventions. Among these has been the development of large-screen display systems that make use of stereoscopic projection. Systems such as the Virtual Table and Responsive Workbench both employ a horizontal projection philosophy, whereas other devices, such as the Powerwall and surround-screen projection systems (SSPS) like the CAVE , also employ vertical back-projection. Due to their effective application, these devices are gaining greater acceptance by the research and industrial user communities. Such devices offer a large field of view, brilliant image quality, high resolution and (especially the table-like devices) a promising compatibility with traditional workspaces. To support this integration, however, ergonomic and human-centered interaction techniques must be developed.
We introduce ideas, proposed technologies, and initial results for a seamless integration of virtual reality in habitual workspaces, thus successively transforming them into efficient, high-tech work environments.

We describe the use of a hand-held semi-transparent mirror to support augmented reality tasks with back-projection systems. This setup overcomes the problem of occlusion of virtual objects by real ones linked with such display systems. The presented approach allows an intuitive and effective application of immersive or semi-immersive virtual reality tasks and interaction techniques to an augmented surrounding space. Thereby, we use the tracked mirror as an interactive image-plane that merges the reflected graphics, which are displayed on the projection plane, with the transmitted image of the real environment. In our implementation, we also address traditional augmented reality problems, such as real-object registration and virtual-object occlusion. The presentation is complemented by a hypothesis of conceivable further setups that apply transflective surfaces to support a mixed reality (i.e., combined AR and VR) environment.

We introduce the idea of using real mirrors in combination with rear-projection systems for the purpose of interacting with and navigating through the displayed information. Subsequently a derived application is described. For this, we use a hand-held planar mirror and address two fundamental problems of applying head tracking with rear-projection planes: the limited viewing volume of these environments and their incapability of simultaneously supporting multiple observers. Furthermore, we describe the possibility of combining a reflective pad with a transparent one, thus introducing a complementary tool for interaction and navigation.

Leonardo da Vinci's drawings of machines and other objects illustrate one of the most fundamental purposes of sketches: the ability to communicate design and functionality to others. Nowadays, it is widely accepted that sketching is a form of critical, reflective dialog that handles communication on one or more different levels of abstraction simultaneously. Various approaches have been taken to support this kind of dialog between humans and computers, and to build human-computer interfaces that are able to interpret such freehand sketches for different purposes.
In this context, the creation or reconstruction of 3D objects from 2D sketches is of major concern in many application areas. This so-called 'pencil-and-paper' approach is used for rapidly designing approximate threedimensional scenes. While some systems analyze the orthographic or perspective projections to reconstruct 3D shapes that, based on psychological assumptions, are most plausible to the human observer, others interpret 2D gestures while the objects are sketched. Within the last decade, the conceptual design phase has been increasingly supported by sketch systems that allow the expression of ideas on a computer-aided, but still human-centered basis. However, putting an emphasis on sketching, most of these systems are sealed off from real-world applications rather than being generally applicable as components.
We introduce a framework for sketch-based interaction within three-dimensional virtual environments. We describe each layer of the framework using illustrative examples depicting its realization. Furthermore, we want to present a variety of domain-specific applications of sketching within 3D virtual environments based on our framework instead of implementing yet another application for sketching.