Airborne Optical Sectioning: Visualization and Flight-Planning

Topics: drones, image-based rendering, user interfaces, light fields
Supervision: David Schedl, Indrajit Kurmi, Oliver Bimber
Type: BSc Thesis, MSc Thesis (two projects)

Drones are becoming increasingly popular for remote sensing of landscapes in archeology, cultural heritage, forestry, and other disciplines. They are more efficient than airplanes for capturing small areas, of up to several hundred square meters. LiDAR (light detection and ranging) and photogrammetry have been applied together with drones to achieve 3D reconstruction. With airborne optical sectioning (AOS), we have present a radically different approach that is based on an old idea: synthetic aperture imaging. Rather than measuring, computing, and rendering 3D point clouds or triangulated 3D meshes, we apply image-based rendering for 3D visualization. In contrast to photogrammetry, AOS does not suffer from inaccurate correspondence matches and long processing times. It is cheaper than LiDAR, delivers surface color information, and has the potential to achieve high sampling resolutions. AOS samples the optical signal of wide synthetic apertures (30–100 m diameter) with unstructured video images recorded from a low-cost camera drone to support optical sectioning by image integration. The wide aperture signal results in a shallow depth of field and consequently in a strong blur of out-of-focus occluders, while images of points in focus remain clearly visible. Shifting focus computationally towards the ground allows optical slicing through dense occluder structures (such as leaves, tree branches, and coniferous trees), and discovery and inspection of concealed artifacts on the surface.

In this project, students will develop an image-based visualization and flying path-planning framework for AOS. The goal is to support users with an intuitive navigation through recorded image data, and easy flight-planing to capture images with the drone.

Novel Computational Imaging Concepts

Topics: imaging, compressed sensing, image reconstruction, SLM
Needs: Matlab, C++, CUDA
Supervision: Oliver Bimber, Bettina Heise
Type: BSc Thesis, MSc Thesis, MSc Practicum, BSc Practicum

Imaging and reconstructing objects by a single pixel camera concept became an emerging novel method introduced and demonstrated recently. This approach has the advantage that only a point detector is needed, which can be beneficial with respect to speed or the simple fact that area cameras are not available or to expensive (relevant e.g. for area camera sensitive in mid infrared or THz wavelength range or for non-linear optics applications). But also for the visible wave length range such a concept shows it challenges with respect to an effective reconstruction: There a sequence of recordings obtained as a (temporal) series of (spatial) projections of the object wave field modulated by an intensity or phase pattern are captured by the point detector. Meanwhile advanced methods are derived from this original technique applying e.g. multiple apertures or using the medium itself as tool for imaging. This leads to fully novel imaging methods and will enable further perspectives for imaging, which will provide insights where imaging as technique itself can be used: e.g. for material sciences, medical diagnostics, or in the depth of astronomy.
Here we have the possibility to implement and test in some principal optical settings such novel imaging concepts containing different types of spatial light modulator (SLM), such as a digital light processor (DLP) or a liquid crystal SLM-types.

In contact and supported by the imaging group at CDL MS-MACH/ZONA such configurations should be further develop, evaluated and optimized for different illumination schemes (coherent and incoherent), modulation types (spatial and temporal modulation) or material types, and in particular computationally effective reconstructions should be performed, where approaches of sparse signals and compressive sensing might be suitably included.

Rubik’s Cube Solver Visualization

Topics: visualization, machine learning
Supervision: Marc Streit
Type: BSc Practicum, MSc Practicum

Everyone who has tried to solve Rubik’s cube, the 3D combination puzzle invented in 1974, without instructions knows that this is a challenging task. Today, computers are better and faster in solving the cube automatically. But how is the solving strategy different to the strategy a human applies?

To answer this question, we developed an interactive visualization for comparing different solving strategies by projecting them in 2D space. The goal of this follow-up project is to create a demonstrator that combines a LEGO® MINDSTORMS® robot, which is able to solve the cube without the help of a human, with the interactive visualization that shows the solving process on the fly. The demonstrator should be showcased to the general public and future students at science fairs and similar events.


Visualizing the Evolution of Instance Classification in Deep Learning

Topics: visualization, machine learning, D3.js, Javascript
Supervision: Peter Ruch, Marc Streit
Type: BSc Practicum, MSc Practicum, BSc Thesis, MSc Thesis

In machine learning, classification is a supervised learning method in which the computer program learns from given observations (instances)  and then uses this learning to classify new data. During training of deep neural networks, the learning algorithm optimizes the model by feeding the training instances multiple times (multiple epochs) into the network in order to increase the performance of the model steadily.
Existing approaches [1] visualize the final performance of the model after training by visualizing correctly and incorrectly classified instances and their class association (see figure). However, they lack the ability to show the evolution of instance classification across the epochs during training.
The goal of this project is to design and implement a novel visualization that allows users to observe the learning behavior during training in deep neural networks. The solutions should be implemented using web-based technologies such as D3 and JavaScript/TypeScript.


Novel Github Repository Visualization

Topics: visualization, source code management, D3.js, Javascript
Supervision: Holger Stitz, Marc Streit
Type: BSc Practicum, MSc Practicum, BSc Thesis, MSc Thesis

As the leading web platform for source code management and versioning control Github processes millions of commits every day. Especially for large repositories with many developers it is difficult for users to keep track of changes across different branches. The current Github user interface and state-of-the-art visualizations are limited in terms of scalability and not able to represent changes for different file types, such as source code, tabular data, and images, over time. The aim of this project is to design and implement an integrated representation for visualization changes of files associated with Github repositories. File changes must be quantified according to file type and considered over time. The visualization should allow users to get an overview over all changes and also drill-down to investigate changes of interest in more detail. The project should be implemented using web-based techniques such as D3.js.

Discovering Usage Patterns in Exploratory Data Analysis

Topics: visualization, big data, guidance, D3.js
Supervision: Holger Stitz, Marc Streit
Type: BSc Practicum, BSc Thesis, MSc Practicum, MSc Thesis

Exploratory data analysis allows domain experts to find new insights in large and complex data. Together with partners from Harvard University and University of Utah we developed visual data analysis tools for drug discovery, cancer genomics and other applications. In such exploratory tools we are able to track full history of the exploratory data analysis process including all loaded datasets, the visual and analytical tools applied to the data, and all interactions done by the user. Collecting this information results in a large and fast growing provenance graph. In order to make the information accessible to the user, an existing project at the institute deals with the interactive visualization of this graph and finding of similar states in the graph. The goal of this project is to run meta-analysis on the provenance graph to derive recurring chains of actions and usage patterns that can then be used to speed up the exploratory analysis and also to visually guide the user through the analysis by making suggestions.
The project will be implemented as part of the Caleydo project, which uses D3.js, JavaScript and Python as base technology.

Copyright Information