Go to JKU Homepage
Institute of Computer Graphics
What's that?

Institutes, schools, other departments, and programs create their own web content and menus.

To help you better navigate the site, see here where you are at the moment.

Application of Visualization and Machine Learning Techniques for Knowledge Graphs

Topics:            Knowledge Graphs, Information Visualization, Machine Learning
Supervision:   Christian Steinparz, Marc Streit
Contact:          christian.steinparz@jku.at
Type:               BSc Practicum, BSc Thesis, MSc Practicum, MSc Thesis

Knowledge graphs are widely used to explicitly model and represent knowledge in various domains including semantic web, natural language, and biomedical data. However, due to the potentially large number of nodes and edges, the visualization and navigation of knowledge graphs prove to be challenging. They require the use of advanced techniques such as user interactivity, aggregation, automated analysis, machine learning, and the combination thereof.

Projects in this area can address tasks such as:

  • prototyping a visualization workflow for the exploration of KGs
  • exploring the use of machine learning techniques to identify and extract relevant information from KGs
  • experimenting with advanced ways of positioning nodes and edges such as KG embedding and down-projection
  • visualizing constraint violation information in KGs
  • applying interpretable ML techniques to predict information in the KG and utilizing the model explanations
  • comparison of KGs by developing visual fingerprints or by applying ML-based similarity prediction
  • visualizing provenance data of how KGs have changed over time (due to update/insert/delete of triples)

Students are encouraged to propose their own ideas in relation to the topic.

Application of Visualization and Machine Learning Techniques for Knowledge Graphs

Experiments with and Development of Neural Additive Models (NAMs) as an Interpretable Machine Learning Method

Topics: Machine Learning, Interpretable ML / Explainable AI, Visualization
Supervision: Christian Steinparz, Marc Streit
Contact: christian.steinparz@jku.at
Type:  BSc Practicum, BSc Thesis, MSc Practicum, MSc Thesis

NAMs, opens an external URL in a new window (Agarwal et al. 2021, paper, opens an external URL in a new window) are models which are “as interpretable as logistic regression while achieving similar accuracy as standard deep neural networks”. Building on the idea of Generalized Additive Models and Explainable Boosting Machines, a NAM learns separate DNN subnets for each input feature. These are learned in parallel and then additively combined. As a result, the model is fully interpretable since each learned subnet can be visualized. Moreover, the subnets can be replaced with feature graphs.

Possible tasks of the project include:

  • the explanation of your favorite ML problems using this architecture
  • developing a tool for better (interactive) visualization of these interpretable models, potentially also for the less clear multitask learning scenario
  • perhaps in combination with the above, creating an editor to easily allow users to alter the learned feature graphs
  • creating an ablation of the performance of this architecture on relevant datasets
  • experimenting with ways of adapting this architecture for CNNs / sequence data / unsupervised learning and projection methods / reinforcement learning

Students are encouraged to propose their own ideas in relation to the topic.

Experiments with and Development of Neural Additive Models (NAMs) as an Interpretable Machine Learning Method

Real-time Storyline Visualization for Spectating League of Legends Matches

Topics:             information visualization, esports, spectatorship
Supervision:     Günter Wallner
Contact:            guenter.wallner@jku.at
Type:                MSc Practicum, MSc Thesis

Storyline visualizations depict the relationships between entities over time and have thus become a useful means to summarize stories. With games being narrative constructs, such visualizations could be helpful to summarize matches to help spectators develop a sense of the ‘story’ of a match. The goal of this project is to develop a storyline visualization for League of Legends (LoL) which is updated in real-time during a match. This involves capturing data from the real-time API provided by LoL and developing a storyline algorithm which is capable of constructing the storyline on the fly based on movement and event information.

Storyline Visualization ©Image Source: https://xkcd.com/657/

Developing a Testbed Game for Studying Map Visualizations

Topics:             information visualization, esports, spectatorship
Supervision:     Claire Dormann, Guenter Wallner
Contact:            guenter.wallner@jku.at
Type:                BSc Practicum, MSc Practicum, MSc Thesis

Maps are an important part of many games to help players orient themselves within the game’s environment. However, maps are currently understudied in the context of gaming and their effects on player behavior. The goal of this project is to develop a game, which can be gradually extended and serves as a testbed environment for studying different map visualizations for games. The game should be developed in Unity.

Map ©Image Source: Photo by mauRCIO SANTOS on Unsplas

Developing Effective Map-based Analytics for Videogames

Topics: Game Analytics, player data, visualisation

Supervision: Claire Dormann, Guenter Wallner
Contactclaire.dormann@jku.at, guenter.wallner@jku.at
Type:   BSc Practicum, BSc Thesis, MSc Practicum, MSc thesis

In videogames, the use of maps to visualize game analytics is very limited, mostly centred on heat maps. However, elsewhere, map-based analytics has been successfully used, revealing patterns that may not have been discovered otherwise. Limitations with map-based analytics for videogames include formats, choice of data, design parameters, etc. Furthermore, map-based analytics are seldom layered, annotated, animated thereby diminishing their meaningfulness and effectiveness.

The goal of the project is to advance knowledge about map-based analytics for videogames, thus producing guidelines that will help game developers. Based on existing data sets, map-based analytics will be developed and implemented.  The design of user interface and mechanisms that would enhance the efficiency and meaningfulness of map-based analytics will also be investigated. Finally, to support the development work, the usability of the new map-based analytics will be evaluated.

Visualizations ©https://www.mdpi.com/2076-3417/11/21/10078/htm ; https://www.mdpi.com/2076-3417/11/21/10078/htm

Developing an innovative Augmented Board Game

Topics: Augmented Board Game, gameplay, technology
Supervision: Claire Dormann, Guenter Wallner
Contact: claire.dormann@jku.atguenter.wallner@jku.at
Type: BSc Practicum, BSc Thesis, MSc Practicum

Playing board games is fun. Indeed, they are an important source of entertainment, engagement and education. The motivation for augmented board games stems from keeping social participation associated with board games and the tactile manipulation of physical pieces. However, they are also designed to overcome some of the limitations of traditional board games: simplification of game logistics, or with the used rich media (sound, animation, etc.). Finally, augmented board games enable the emergence of new gameplay experiences.

Technology used to develop augmented board games, vary from the simple addition of smart phone components, combining 3D features with board games, or using augmented game pieces. There is thus much scope for innovation both, in term of board game design and of the augmented component.

Working from an existing cooperative board game, you will first develop an augmented board game protype. Then you will experiment with different ways the games can be augmented, looking at roles of the digital component can play, and how augmentation affects gameplay and players experiences. Finally, you will implement the final version to be tested by players.

The project steps consist in:

  • Reviewing and analysis of augmented features of augmented board games
  • Develop innovative augmented components
  • Design an augmented board game prototype
  • Evaluate the prototype.
Augmented Map ©©© Smartech group, Barcelona https://www.youtube.com/watch?v=9vF_T6FBivU; Stop thief https://www.engadget.com/

FPV AOS

Topics: drones, first-person view flying, user interface, HCI
Supervision: Oliver Bimber
Contactoliver.bimber@jku.at
Type:  BSc Practicum, MSc Practicum

Airborne Optical Sectioning (AOS, https://github.com/JKU-ICG/AOS/, opens an external URL in a new window) is a wide synthetic-aperture imaging technique that employs manned or unmanned aircraft, to sample images within large (synthetic aperture) areas from above occluded volumes, such as forests. Based on the poses of the aircraft during capturing, these images are computationally combined to integral images by light-field technology. These integral images suppress strong occlusion and reveal targets that remain hidden in single recordings.

We have ported AOS to DJI platforms, as they are mainly applied by bluelight organizations, such as police, red cross, mountain rescue, or fire brigades. The latest release can be found here, opens an external URL in a new window.

An initial survey, however, revealed that the current user-interface is too complex for quick rescue missions, being operated by technically inexperienced drone pilots that have to focus on the rescue task rather than on operating the AOS interface. The goal of this project is to redesign and reimplement the DJI-AOS user-interface for practical usage. This includes, for instance, the integration of a first-person view (FPV) mode being supported by googles, and an intuitive operation of GUI elements and free camera control while wearing the googles.

(c) DJI ©DJI

Motion Anomaly Detection

Topics: drones, image processing, object detection, signal processing, machine learning
Supervision: Oliver Bimber
Contactoliver.bimber@jku.at
Type:  BSc Practicum, MSc Practicum, BSc Thesis, MSc Thesis

Airborne Optical Sectioning (AOS) is a wide synthetic-aperture imaging technique that employs manned or unmanned aircraft, to sample images within large (synthetic aperture) areas from above occluded volumes, such as forests. Based on the poses of the aircraft during capturing, these images are computationally combined to integral images by light-field technology. These integral images suppress strong occlusion and reveal targets that remain hidden in single recordings.

Recently, we have shown that not only classification benefits from image integration, but also color anomaly detection. In contrast to classification where, individual classes have to be known and often must be trained, color anomaly detection indefinites pixels that have an unnormal color with respect to the background’s color statistics. We also have presented AOS approaches that are able to detect and track moving targets (such as walking people) through densely occluding foliage.  

In this project, we want to investigate motion anomaly detection (i.e., identify pixels whose sources move over time) for identifying and tracking targets, such as people, animals, or vehicles. Obviously, larger motion over time is an anomaly that can be used to differentiate moving targets from (manly static) vegetation.  We are interested in exploring and evaluating existing (model-based and machine-learning-based) motion anomaly detectors in the scope of AOS. 

Many researchers and students have contributed to JKU’s innovative AOS framework to explore novel solutions as search and rescue, wildlife observation, and wildfire detection. Several high-impact publications in Nature and Science have covered our scientific findings. See https://github.com/JKU-ICG/AOS/, opens an external URL in a new window, opens an external URL in a new window for details.

We would like to point out that when playing the video, data may be transmitted to external parties. Learn more by reading our data privacy policy
Data protection information

Visual Analysis of Model Compression and Its Effects

Topics: Explainable AI, Visual Analytics, Machine Learning
Supervision: Christian Steinparz, Christina Humer, Andreas Hinterreiter, Marc Streit
Contact: andreas.hinterreiter@jku.at
Type:  BSc Practicum, BSc Thesis, MSc Practicum, MSc Thesis

In machine learning research, model compression is the act of reducing a neural network’s footprint, usually in an attempt to make its deployment more memory- and energy-efficient. A typical way of achieving smaller networks is to introduce sparsity by setting specific network weights to zero. Apart from making networks smaller, model compression can come with beneficial or detrimental side effects with respect to the model’s accuracy.

We are collaborating with machine learning experts who want to better understand the outcome of different compression algorithms and their side effects. The goal of this project is to develop a prototype that lets our collaborators analyze their compressed and uncompressed models visually. The visualization could be based on an interactive view of the network architecture, with functionality to filter for interesting parts, such as individual convolution kernels. Users should be able to find interesting data instances and inspect how the compression affects the activations.

Schematic of nework sparsification Image source: Hoefler, T. et al. Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks. J. Mach. Learn. Res., 22(241), 1-124.

Real-time Training Visualization for League of Legends

Topics: games user research, visualization, analytics
Supervision: , opens an external URL in a new window Günter Wallner
Contact: guenter.wallner@jku.at
Type: MSc Thesis

Training visualizations for players are getting increasingly important in the competitive gaming scene to learn and improve in-game skills. League of Legends (LoL) has emerged as a popular but at the same time difficult to learn game. The goal of this topic is to develop and evaluate solutions (e.g., visualizations, tools) that help players learn and reflect upon in-game activity in real-time by utilizing the Live Client API offered by LoL. The topic can be approached from different perspectives and interested students are encouraged to propose their own ideas in relation to it. Other games than LoL could be focused on as well.

Tasks

The overall goal is to develop visual solutions that can be used in a live setting for training purposes. Overall the steps include 1) requirement analysis with players, 2) develop a solution, and 3) evaluate the solution with players.

Requirements

  • Knowledge analytics or related domains
  • Knowledge of information visualization
  • Good programming skills
  • Interest in exploration of novel solutions
  • Knowledge of League of Legends is of advantage
  • Knowledge of English language (source code comments/documentation should be in English)

Encounter Visualization for DOTA 2

Topics: games user research, visualization, analytics
Supervision: , opens an external URL in a new window Günter Wallner
Contact: guenter.wallner@jku.at
Type: MSc Thesis

Dota2, a multiplayer online battle arena game, is enjoying wide popularity in the competitive gaming scene. The goal of this thesis is to develop a visualization based on the concept of battle maps to convey information of encounters/events happening within Dota2 matches. Battle maps have long been used by historians to concisely convey the course of a battle. This includes 1) detecting interesting events and encounters and 2) condensing movement patterns and contextual information into battle maps. The work can draw upon previous work on encounter detection and battle maps.

Tasks

Work will include developing or including existing encounter detection algorithms to extract events of interest and develop a visualization front-end which conveys information surrounding these events. This can initially happen post-play but extension to real-time detection and visualization should be accounted for. Developed solutions need to be evaluated with the target audience to assess its usefulness.

Requirements

  • Background in analytics or related domains
  • Knowledge of information visualization
  • Good programming skills
  • Willingness to crack problems and to show self-initiative
  • Knowledge of Dota2 is of advantage
  • Knowledge of English language (source code comments/documentation should be in English)