Page Areas:



Additional Information:

Location

Science Park 3

We are located on the second floor of the Computer Science Building (Science Park 3) ...  more of Location (Titel)


Position Indication:

Content

Report on the scientific work

This project made significant impact in terms of scientific results achieved, publications, awards, visibility in the community, industrial adoption, inter-disciplinary collaborations, and other criteria.

Information on the development of the research work

Overall scientific work
Documenting and maintaining the traceability among requirements, design/architecture models, and code is one of the foremost challenges of model-based software development. Yet, traceability is rarely captured immediately while models and code co-evolve but usually recovered later. By then key people may have moved on or their recollection of facts may be blurred or inconsistent. The goal of the project was to develop an approach that helps developers during trace generation and trace validation. In total, we identified five main objectives:

  • Generating Trace Links through Model Contents and Code
  • Generating Trace Links through Trace Automation
  • Detecting Trace Errors
  • Generating Trace Links through Transitive Trace Implications
  • Case Studies/Evaluation

Was there a change of direction in the field between the start and the end of the project?
All objectives identified in the proposal were investigated. However, one objective was partially solved only. This is discussed below. Moreover, an additional activity was performed that we did not propose but the opportunity was too good to pass on. The timings of the various activities varied slightly from the proposal or were adjusted to coordinate them with collaborations and visits.

The partially satisfied objective was “Generating Trace Links through Transitive Trace Implications.” The goal of this objective was to demonstrate that it was possible to derive transitive relationships among traces. We partially satisfied this goal because our Trazer approach (see below) does use transitive relationships involving code/trace patterns for trace generation and trace validation. However, we believe we did not fully satisfy this goal because we also intended to try transitive traces among different kinds of models. The basic foundation of our TraceAnalyzer approach (see below) and its language is able to account for transitive input but we failed to complete this tasks because of severe budget cuts from the proposed vs. accepted funding (€383,911 requested vs. €221,508 approved). We tried our best to meet all objectives but the funding cut was too severe and it essentially deprived the project from another PhD student over the entire lengths of the project. In turn this did not allow us to obtain relevant case studies, perform the necessary algorithmic adaptations, and conduct the necessary empirical evaluations, including publications to support multi-dimensions traceability.

It should be noted that we also engaged in an additional activity that was not proposed (see below) but the opportunity arose and it would have been a great loss not do this (particularly because it has the potential of becoming a well cited paper). It should be noted that this additional activity did not prevent us from fulfilling the partially satisfied objective discussed above. It was a low-hanging fruit that took a few months of effort only – a small part in comparison to the larger effort it would have taken to complete the above objective.

Results and their significance

Two Complementary Approaches
In this project, we developed two complementary approaches for capturing and maintaining traceability. The first approach, called TraceAnalyzer allows developer to capture traceability at arbitrary levels of correctness, uncertainty, and completeness. For this purpose, the approach provides a language that characterizes requirements-to-model-to-code traceability. The approach also provides a reasoning engine that takes into account ambiguous, incomplete, and possibly incorrect facts about the traceability that developers provide as input. It validates the correctness (error detection) of these assumptions and completes the input by inserting their logical consequences (trace generation). The second approach, called Trazer, allows for trace prediction and subsequent validation (error detection) through calling relationships within the code. As input, the approach requires an executable system with test scenarios and the corresponding requirements/model elements. It then generates a prediction on the requirements/model-to-code traces based on patterns of traces and calling/data relationships. These prediction can be used to either auto-complete traceability input (trace generation) or to validate existing traceability (error detection).

Bonus: trace benefit experiment
During Dr. Mäder’s stay at our institute, we performed a study that used data/results from this project to gauge the value and usefulness of traceability. Despite its growing popularity, there exists no published evaluation about the usefulness of requirements traceability. It is important, if not crucial, to investigate whether the use of requirements traceability can significantly support development tasks to eventually justify its costs. A two controlled experiments with 71 subjects performed real maintenance tasks on two third-party development projects: half of the tasks with and the other half without traceability. The studies showed that subjects with traceability performed on average 24% faster on a task and created on average 50% more correct solutions -- suggesting that traceability not only saves downstream cost but can profoundly improve software maintenance quality. Clearly this is an important finding and it is not limited to our approaches. These findings apply to traceability in general (at least in context of code evolution) and our hope is that this paper will be cited as the first solid evidence. Note that this study was recognized as one of the top papers of ICSM (now ICSME).

Validations and Case Studies
A critical problem was how to obtain case studies to validate our approaches. Plenty of open source systems are available; however, for validating our work we required access to traceability information in addition to requirements, model, and code. The required this traceability information to gauge the effectiveness of our approach – for example, to measure our approaches’ precision and recall in correctly generating or evaluating traces. We were able to identify only one system that provided this information: iTrust. While the iTrust is a good case study system, it alone would have been too little for a broad evaluation. We thus choose an unusual route in obtaining additional traceability information by paying key developers from several, larger open-source projects to generate the RTM for us. This way, we were able to obtain useful traceability information for GanttProject, jHotDraw and other systems. Together they provided a good foundation for evaluation.
 
Adoption
Even though the tools were completed recently only, we had several successful adoptions of our approaches. We had an academic collaboration with Catia Trubiani and Vittorio Cortellessa from the University of L'Aquila, L'Aquila, Italy who found our approach useful for connecting performance analysis results with architectural models. A paper was written and is currently under submission at TACAS. The industrial collaboration was in context of the CD Lab and the company KEBA (http://www.keba.com/en/). KEBA develops and produces hardware, software, and tools for industrial automation solutions including injection molding machines, robotics, and heating system control. Keba develops and maintains several heterogeneous platforms that exist in numerous variants. Hence there is not a single product but many similar products. However, software engineering methods for analysing and managing variable software systems rely on accurate feature-to-code mappings (traceability to relate high-level variability abstractions, such as features or decisions, to locations in the code where variability occurs. Due to the continuous and long-term evolution of many systems such mappings need to be extracted and updated automatically. We successfully developed a technology, that in part relied on the outcome of this proposed work, for capturing feature to code dependencies (outcome under submission at a ICSME NIER paper). Furthermore, we had another successful academic collaboration involving HongYu Kuang, Hao Hu, and Jian Lv from the Nanjing University, China and LiGuo Huang from the SMU University, USA. This group built on our Trazer technology and investigated with data dependencies within source code could complement call dependencies in understanding traceability. This work was published in ICSME and a journal paper is currently under submission.

Proof of Concept Tool
Both Trazer and TraceAnalyzer were implemented in form of proof-of-concept tools. These tools are publically available on our web site: www.isse.jku.at/tools. We used the tools as the foundation for the large-scale, empirical validation of our approaches. Without the tools, we would have been unable to publish these works as successfully and extensively as we did. The tools are research tools and not of industrial quality; however, they are quite useable, having good user interfaces and documentation. The tool is already used at the university and in context of the aforementioned collaborations.

PhD Dissertations and Master’s Thesis
Altogether one PhD dissertation, a Master’s thesis, and a part time Post Doc were funded with this project. Achraf Ghabi, the PhD student, has essentially finished his thesis but still needs to defend. Lukas Linsbauer, the Master’s student, was critical in his support of the industrial case study. And Dr. Nöhrer was instrumental in providing reliable SAT-based reasoning for traceability and also the mechatronics case study. Note that we also supported another PhD student for over a year but, unfortunately, she did not finish her studies.

Relevance for other (related) areas of science
In form of two cooperations we demonstrated that the approach is applicable beyond the narrow focus of the original proposal and even beyond software engineering (the area of expertise of the PI). Traceability also plays a vital role among different engineering disciplines and is evident in the different engineering disciplines relying on different tools and technologies that are not connected. There, trace generation and validation was needed in order to establish links among engineering artefacts. For example, to link the computation of a robot’s arm length with the robot’s drawing in a CAD tool. For this purpose, we integrated a part of our approach into the DesignSpace cloud solution that support the linking of arbitrary engineering artifacts and applied it to two application domains: the construction of a robotic system with the Austrian Competence Center for Mechatronics (ACCM), Austria and the construction of a motor controller with the Flanders' Mechatronics Technology Centre (FMTC), Belgium. The first cooperation applied traceability to artefacts involved CAD drawings, Excel computations, and code. The second cooperation applied traceability to electrical engineering artefacts and code. Together these two cooperations demonstrated a wider usefulness of traceability in form of artefact linking.