FWF P 25289-N15
Consistent Change Propagation in Software Design Models
Design models describe different viewpoints of a software system – separating functionality, from structure, behavior, or usage. While these models are meant to be separate in their description, they are nonetheless related by manifold dependencies. After all, they describe the same system. It is necessary to understand this network of dependencies to understand how the various models are connected. Yet, this network of dependencies is also the most significant reason for failure during software modeling. It is the root cause for failure to propagate changes correctly and completely. Although change propagation as a whole is a daunting challenge to tackle, this proposed work suggests an approach for solving this problem in context of software models. In context of such models, incorrect or incomplete changes are detectable if these changes cause inconsistencies among the models involved. This proposed work thus builds on technology we pioneered under a predecessor project (FWF project P21321-N15 titled “Inconsistencies and their Impact on Software Design Models”). That project developed a methodology for understanding how inconsistencies are caused and what changes can repair them. That predecessor project is now in its final stages and this proposed work will build on its results to develop a methodology for change propagation in software design models. Models change continuously and if a change is not correctly applied to all models of a software system then inconsistencies are caused. Since there are often near infinite choices for repairing inconsistencies, this project will tailor the repairing of inconsistencies by not seeing inconsistencies in isolation but by understanding the cumulative effect of changes over time (during propagation). The basic observation is that change propagation may cause further inconsistencies and their repair should not contradict earlier changes. This simple idea forms the basis for understanding relationships among inconsistencies which in turn helps us explore the computationally expensive space of change combinations based on a novel idea that is analogous to triangulation. This work will 1) research the methodology for change propagation across arbitrary design models governed by constraints, 2) develop a proof-of-concept tool to demonstrate the approach, and 3) evaluate the methodology and tool on industrial models available to us. As we did in the predecessor project, we will continue to publish the research results in the best peer-reviewed journals and conferences; and pursuit its transition to industrial use. This proposal requests funding for two PhD students and one Master’s student only plus their travel needs.
Software models let engineers describe systems from different points of views. This has the benefit that engineers can separate the modeling of different parts of systems, different domains, or even different functionality. However, these different models do have to fit together and today many consistency rules exist that let engineers define relationships among models. Consistent change propagation is then simply about the ability to propagate changes among these software models without causing (further) inconsistencies. If an engineer changes a model then this change may cause inconsistencies with other models if the change was not carried through correctly and completely. An inconsistency in this case is not an error but rather a symptom for incomplete change propagation. Resolving such inconsistencies (by repairing them) is then equivalent to continuing the change propagation. The problem of incomplete change propagation has been a significant contributor to many high profile engineering failures. This project showed that consistent change propagation is feasible and is applicable to a wide range of engineering domains. We demonstrated this on model/meta model co-evolution, the consistent merging of models, and software/systems evolution in other domains.
The research project
The basic idea of consistent change propagation is to find the synergy between the repairing of errors in design models with the natural evolution of design models. That is, engineers do change design models on a regular basis – for example, to change features or add new functionality. In previous work, we developed a technology that quickly identifies errors that engineers inadvertently introduce during such changes. There are referred to as inconsistencies and we demonstrated that we can derive meaningful repairs for them. However, not all inconsistencies are errors. Inconsistencies may also be the result of incomplete changes. This is where consistent change propagation comes into play. Consistent change propagation makes repair suggestions under the assumptions that changes where correctly albeit incompletely carried out. In short, consistent change propagation utilized repair strategies to propagate changes.
Consider, for example, two engineers that are working on (and hence change) two different sets of engineering artifacts – say, a modeler working on an UML model and a programmer working on the corresponding source code. Each engineer is responsible for his or her set of engineering artifacts. Obviously, model and code need to adhere to basic consistency rules – say, every UML design class needs to have some implementation in the code or every sequence diagram needs to be implemented as code behavior. Let us assume that one engineer changes one of the sets of engineering artifacts. For example, the modeler changes the UML design model due to requirements changes. Even if the changes made by the modeler on the UML design model are complete and correct, inconsistencies may likely result with the source code because the programmer has not yet made the corresponding changes in the code. Consistent change propagation assumes that the model changes are correct and it would guide the repair of the resulting model inconsistencies onto what to do with the code to resolve the inconsistencies. This is essentially equivalent to change propagation that is guided by the repairing of inconsistencies.
To realize this idea, this projected funded the equivalent of two PhD students (7 years full time equivalent) who where mostly supervised by the PI (Alexander Egyed, unpaid) and partially supervised by two post docs (mostly unpaid except for roughly 0.5 years full time equivalent funding). This project also supported several Masters theses and at times employed these students to provide more direct programming support for activities that related to the proof of concept implementation (prototype development)
Roughly 0.5 Years Full Time Equivalent
Roughly 7 Years Full Time Equivalent (30 hrs per week per FWF rules)
Roughly 1 Year Half-Time Equivalent (we do not employ students full time to ensure that their studies continue)
As part of this work, we published over a dozen papers in premier, peer reviewed conferences and journals. Two of these papers also received best paper awards.
1 IEEE Computer
1 IEEE Transactions of Software Engineering
2 Journal of Systems and Software
1 Automated Software Engineering Conference (ASE)
1 Symposium On Applied Computing (SAC)
1 International Conference on Software Maintenance and Evolution (ICSME)
1 Computer Software and Applications Conference (COMPSAC)
1 Conference on Software Architecture (WICSA)
1 International Software Product Line Conference (SPLC)
1 Genetic and Evolutionary Computation Conference (GECCO)
1 Congress on Evolutionary Computation (CEC)
1 International Symposium on Search-Based Software Engineering (SSBSE)
1 Computational Intelligence and Quantitative Software Engineering
1 Advanced Information Systems Engineering Workshops – CaiSE
We originally intended to investigate consistent change propagation in context of UML models only and then move beyond software models in the third year. We decided that this was too risky because it might have been hard to extend this work onto other modeling languages later .We thus teamed up with the Austrian Center for Mechatronics early on and started to investigate in what ways a wider range of engineering models might be affected by change propagation. This exercise was valuable because it made us realize that change propagation often has to go across tool boundaries. The difficulty of change propagation is thus to keep the artifacts of different tools in sync – even though these tools are separate from one another and there is often no platform for exchanging the artifacts captured within them. This observation (while obvious) strongly shaped this project. While the proposal ignored tool bounaried and mostly only focused on UML as a multi-diagram language (i.e., a multi sub-model language), we found it essential to consider cross-tool, distributed, diverse engineering knowledge from the get-go. Our methodology and prototype tool support was built around this concept. This led us to the development of the DesignSpace infrastructure that supports the integration of modeling artifacts across multiple tools. While this infrastructure is more ambitious than what we initially proposed, we found it essential for the project and also successfully obtained additional funding (university funding and a project with the Austrian Center for Mechatronics) to support its development which were not considered in this project. These additional funding opportunities also led to collaborations and publications with Klaus Zeman (Mechatronics, Johannes Kepler University, Austria), Peter Hehenberger (Systems Design, FH OÖ, Austria), and Davy Maes (Mechatronics, Flanders MAKE, Belgium).
Another deviation was caused by Markus Riedl-Ehrenleitner leaving the PhD program after two years without completing it. This is always undesirable but sometimes unavoidable. We did hire Michael Tröls and Roland Kretscher to compensate this loss, however, it is not straightforward to transition know-how easily in such situations. Hence, we had to redo several tasks that Mr. Riedl-Ehrenleitner hat started doing. To avoid the project having to bear the extra cost of this, we paid them in-part through university-funded research associate positions. This is the main reason why there are still several papers under submission/unpublished as these students are still paid (even though this project has ended) and they are expected to graduate within the next one year.
This project strongly benefited from two international groups of researchers whose collaborations and/or visits were not foreseen in the initial proposal but led to interesting research directions.
First, there were the sabbatical of Hoa Khanh Dam (University of Wollongong, Australia) and the research stay by Michael Winikoff (University of Otago, New Zealand). Both were interested in the repair of design models but found the domain to be computations infeasible – which we concurred. Consistent change propagation provided the means of limiting search spaces and in this context, our collaboration demonstrated the usefulness of consistent change propagation on model merging. Model merging was not an area we initially intended to investigate because we did not see a strong connection to consistent change propagation during proposal writing. However, the basic principles that guide model merging where solvable by our approach and this led to fruitful collaborations.
Second, there were the visits and interactions with Javier Ferrer, Francisco Chicano, and Enrique Alba (University of Malaga, Spain). These were triggered because of synergies between our work (constraint-based reasoning and evolution) and theirs (testing). Again, on the surface, these two topics appear independent but the connection is in the research on search spaces. In context of this collaboration we learned considerably about evolutionary computation and how it can be used to explore vast search spaces. This was something we did not consider as part of this project but found strongly relevant to explore because consistent change propagation is foremost an exploration of search spaces.
Finally, this project also benefited from the hiring of Dr. Roberto E. Lopez Herrejon. His research interest was is product line development and hence about variants of systems. During the course of his stay here at JKU, we also found a great connection to this project because the typical clone-and-own approach to dealing with variations of software is ill suited for change propagation. Yet, a change made in one variant (say a bug fix) does have to be propagated to other variants. This is another aspect of consistent change propagation we did not foresee but found invaluable to explore in the opportunity presented.
Most important results and brief description of their significance
This research project led a large number of research papers. Most of them have been published or accepted for publication at high ranking journals and conferences (A, A+ ranked). Our publications target primarily researchers but also people from industry as potential candidates for applications of the results and further collaborations. Since FWF does not fund industrial applications, these were funded with separate grants but we summarize some experiences in this report as well.
For consistent change propagating we required a profound understanding of what causes inconsistencies (how and why). Only in understanding the causes of inconsistencies, we were able to understand how to repair them consistently. And the consistent repairing of inconsistencies is the very foundation of consistent change propagation. We developed a sound theory for understanding the causes of inconsistencies and published it in the premiere journal for software engineering
- Alexander Reder and Alexander Egyed: “Determining the Cause of a Design Model Inconsistency,” IEEE Transactions on Software Engineering (TSE), Volume 39, Issue 11, 2013, pp 1531-1548.
Based on this foundation, we were able to publish key papers that explored change propagation. The most significant publication here was also published in one of the best journals in software engineering:
- Andreas Demuth, Markus Riedl-Ehrenleitner, Roberto E. Lopez-Herrejon, Alexander Egyed: Co-evolution of metamodels and models through consistent change propagation. Journal of Systems and Software 111: 281-297 (2016)
This paper describes the evolution of models and meta models when changes occur. Metamodels are always evolving requiring models created from them to be co-evolved. This paper demonstrates how consistent change propagation helps solve the co-evolution between metamodels and models. It relies on incremental management of metamodel-based constraints that are used to detect co-evolution failures (i.e., inconsistencies between metamodel and model). After failure detection, the approach automatically generates suggestions for correction.
In addition to this core paper, we managed to publish another paper on an aspect of scalability of our technology at a premier software engineering conference:
- Andreas Demuth, Markus Riedl-Ehrenleitner, and Alexander Egyed: “Efficient Detection of Inconsistencies in a Multi-Developer Engineering Environment”, International Conference on Automated Software Engineering (ASE), Singapore, Singapore, 2016
While consistency checking was a part of the previous FWF project, this new work was needed and useful here because change propagation affects multiple engineers and hence for consistent change propagation a mechanism was needed that could handle multi-user consistency implications concurrently in a scalable manner.
Consistent change propagation also requires the ability to look across engineering tool boundaries. The focus of this project is particularly on propagating changes across different kinds of models and, hence, across the tools where these models are persisted, with three additional papers – one of which published at the IEEE Computer magazine as a cover feature.
- Alexander Egyed, Klaus Zeman, Peter Hehenberger, Andreas Demuth: „Maintaining Consistency across Engineering Artifacts.” IEEE Computer 51(2): 28-35, 2018
- Andreas Demuth, Markus Riedl-Ehrenleitner, Alexander Nöhrer, Peter Hehenberger, Klaus Zeman, and Alexander Egyed: „DesignSpace -- An Infrastructure for Multi-User/Multi-Tool Engineering,” 30th Symposium On Applied Computing (SAC), Salamanca, Spain, pp. 1486-1491, 2015
- Markus Riedl-Ehrenleitner, Andreas Demuth, and Alexander Egyed: Towards Model-and-Code Consistency Checking”, 38th Annual Computer Software and Applications Conference (COMPSAC), Vasteras, Sweden, July 2014, pp. 85-90
These papers were particularly relevant here because they also introduced the general infrastructure that we used to allow for the identification of inconsistencies across different kinds of engineering artifacts (models and tools). This is a prerequisite for consistent change propagation.
Finally, we published an application of our findings on consistent change propagation to the domain of software processes:
- Demuth A., Riedl-Ehrenleitner M., Kretschmer R., Hehenberger P., Zeman K., Egyed A.: Towards Flexible and Efficient Process and Workflow Support in Enterprise Modeling, in: Advanced Information Systems Engineering Workshops – CaiSE 2015 International Workshops, Stockholm, Sweden, June 8–9, 2015, Proceedings, Page(s) 270-281, Springer International Publishing, 2015.
This paper presents how the technologies developed in the project can be applied to automatically evolve not only inconsistencies between those artifacts, but also to identify those areas that are most crucial for the project. We are confident that this information will blend in with the work previously done to be able to identify those changes to design models that are most useful in practice (e.g., we can identify those changes that re-establish consistency and that change artifacts that are of relatively low importance to other artifacts -- reducing the probability of follow-on inconsistencies to arise that need to be propagated again over different artifacts).
We managed to explore several synergetic research angles that we did not (and could not have) foresee in the proposal. These works lead to excellent publications and two best paper awards. We are thankful that the FWF allows (even encourages) researchers to engage in such collaborations.
We were able to apply our technology on engineering artifacts beyond software engineering. The most significant application was with Flanders MAKE (a Belgium Mechatronics Competence Center) where consistent change propagation was explored in the collaborating of an electrical engineer and a software engineer. Once again, we managed to publish our findings in a high impact venues conference which offers the first convincing evidence that our technology on consistency checking and impact is indeed useful and beneficial:
- Andreas Demuth, Roland Kretschmer, Alexander Egyed, and Davy Maes "Introducing Traceability and Consistency Checking for Change Impact Analysis across Engineering Tools in an Automation Solution Company: An Experience Report," International Conference on Software Maintenance and Evolution (ICSME), Raleigh, NC, USA, October 2016, pp. 529-538.
This paper received the IEEE Software Best Paper Award. We do want to point out that the study was being funded by Flanders MAKE and not FWF (in accordance to FWF rules) but the technology we applied was the direct result of this project. Also note that the paper merely talks about the detection of inconsistencies. The repair is only alluded to because this aspect of the work has been published yet. We are still planning on writing another paper that will publish this research direction.
As often, there are unforeseen opportunities to be explored. The most interesting such opportunity arose with the research stays (sabbaticals) from Hoa Khanh Dam (University of Wollongong) and Michael Winikoff (University of Otago, New Zealand) at JKU. During their stays, we discovered the strong relevance of consistent change propagation to the merging of model changes if multiple engineering work on the same model:
- Hoa Khanh Dam, Alexander Egyed, Michael Winikoff, Alexander Reder, and Roberto E. Lopez-Herrejon,: “Consistent Merging of Model Versions”, Journal of Systems and Software (JSS), Volume 112, pp. 137-155, 2016
- Hoa Khanh Dam, Alexander Reder, and Alexander Egyed: “Inconsistency Resolution in Merging Versions of Architectural Models”, 11th Working IEEE/IFIP Conference on Software Architecture (WICSA), Sydney, Australia
The second paper in this list received the Best Paper Award by Conference on Software Architecture (WICSA). While many engineering tasks can, and should be, manageable independently, it does place a great burden on explicit collaboration needs – including the need for frequent and incremental merging of artifacts that software engineers manipulate using these tools. These papers focused on the merging of models where we consider not only conflicts but also arbitrary syntactic and semantic inconsistency issues. Consistent artifacts are merged fully automatically and only inconsistent/conflicting artifacts are brought to the users’ attention, together with a systematic proposal of how to resolve them.
A change in a system can be caused by evolution in time where the previous revision of the system is replaced by the new one (e.g. when applying bug fixes). Another kind of change is caused by evolution in space where both variants of the system (the one before and the one after the change) co-exist in parallel and need to be maintained concurrently (e.g. when applying customer-specific changes due to deviating requirements). Consistent change propagation in the latter scenario requires an understanding of how changes are related and in which concurrently existing variants of a system a change is replicated (i.e. cloned). We investigated this part of the project and he published several papers:
- Lukas Linsbauer, Roberto Erick Lopez-Herrejon, and Alexander Egyed: “Feature Model Synthesis with Genetic Programming”, 6th International Symposium on Search-Based Software Engineering (SSBSE), Fortaleza, Brazil, August 2014, pp. 153-167
- Lukas Linsbauer, Roberto E. Lopez-Herrejon, and Alexander Egyed: “Recovering Traceability between Features and Code in Product Variants,” Proceedings of the 17th International Software Product Line Conference (SPLC), Tokyo, Japan, 2013, pp. 131-140
Search-Based Software Engineering
Finally, we hired Dr. Roberto E. Lopez Herrejon part time to help supervise a critical aspect of this work (we found that there are strong parallels between search-based software engineering and change propagation). The repair of inconsistencies caused by incomplete change propagation is a search space problem and we spend considerable time understanding evolutionary algorithms (leading to a comparative analysis). While the testing papers are not directly relevant to this project, our combined findings then led to this new research direction.
- Roberto Erick Lopez-Herrejon, Javier Ferrer, Francisco Chicano, Alexander Egyed and Enrique Alba: “Comparative Analysis of Classical Multi-Objective Evolutionary Algorithms and Seeding Strategies for Pairwise Testing of Software Product Lines” 2014 IEEE Congress on Evolutionary Computation (CEC), Beijing, China, July 2014, pp. 387-396.
- Roberto E. Lopez-Herrejon, Javier Ferrer, Francisco Chicano, Alexander Egyed, and Enrique Alba: “Evolutionary Computation for Software Product Line Testing: An Overview and Open Challenges”, in Computational Intelligence and Quantitative Software Engineering, Volume 617, 2016 pp. 59-87
- Roberto E. Lopez Herrejon, Javier Ferrer, Francisco Chicano, Evelyn N. Haslinger, Alexander Egyed, and Enrique Alba: “A Parallel Evolutionary Algorithm for Prioritized Pairwise Testing of Software Product Lines,” Genetic and Evolutionary Computation Conference (GECCO), Vancover, Canada, July 2014, pp. 1255-1262.
Roberto E. Lopez Herrejon: The years as a post doc were highly beneficial for his career (see citation impact). He is now an Associate Professor at the École de Technologie Supérieure, Montreal, Canada. https://scholar.google.at/citations?user=SsACdxUAAAAJ&hl=de&oi=ao
Andreas Demuth: He successfully completed the last year of his PhD with this project. This helped the project considerably since he previously worked on UML, consistency, and repair and thus was able to help transition previously developed technologies. The interest in this project convinced him to continue on as a Post Doc for two more years, paid in-kind by a university position, supervising various PhD students. He is now in industry.
Roland Kretschmer: He joined us as a PhD student after his Masters and developed the core of the consistent change propagation solution. He is expected to finish his PhD in 2018, thanks to this grant.
Markus Riedl Ehrenleitner: He joined us as a PhD student after his Masters and was part of the core research team. Unfortunately, he chose to leave for industry and did not graduate. As he was a core researcher for the project, this was a significant setback. We hired Michael Tröls to replace him.
Michael Tröls: He was hired to replace Markus as a PhD student and has since been part of the core research team. With Markus leaving, we were forced to restart several tasks. To avoid the project having to bear the extra cost of this, we paid him through a university-funded research associate position. He is expected to finish his PhD in 2019, thanks to this grant.
Lukas Linsbauer: He did most of his PhD work under this project and pursued an interesting and complementary research angle in dealing with the propagation of changes through multiple variants of a system. He successfully completed his PhD with a strong citation impact and is now a post doc with interest in pursuing an academic career.
Several students also built their Masters theses around tasks of this grant: Sebastian Wilms, Markus Riedl Ehrenleitner, Ahmad Al Ktifan, Roland Kretschmer, or Michael Jahn. They have since graduated.
Effects beyond the scientific field
The PI recently co-funded the Pro2Future competence center with a budget of €18M in the next four years. This competence center engages in direct collaborations with industry – notably Engel, Siemens, Wacker Neuson, Fabasoft, AVL, and others. The co-evolution of models investigated in this work is directly relevant to the “Cognitive Robotics and Shop Floors” area of Pro2Future with its focus on modular and interoperable production systems. This may not be obvious at first. This area combines the engineering of production machines and their runtime behavior in production system-of-systems (PSoS). In contrast to other systems, the focus is on the production process spanning multiple machines (systems) under the assumption that these machines are from different manufacturers as is commonly the case. Enterprise Interoperability in general and process interoperability in particular plays a fundamental role. Machines are autonomous, developed by different manufacturers, and form a larger systems on demand – the assembly lines and other shop floors. Co-evolution is therefore highly relevant because of the need to adapt production processes to changing production situations – for example, when altering the hardware of the machines (i.e., with physical modules) or when altering the software of the machines (i.e., same, existing hardware but changed behavior characteristics) to better interact with other machines.
Other important aspects
The IEEE Computer Magazine chose our work on consistency for a cover feature
- Alexander Egyed, Klaus Zeman, Peter Hehenberger, Andreas Demuth: „Maintaining Consistency across Engineering Artifacts.” IEEE Computer 51(2): 28-35, 2018
Moreover, the PI gave two keynotes that were strongly based on the results of this project
- “Effectively Using and Reusing Engineering Knowledge in a Collaborative Engineering Environment,” at KKIO Software Engineering Conference, Rzeszów, Poland, September 2017.
- “Collaborative Engineering across Disciplines and Tools”, at Flanders MAKE Seminar on Improving systems engineering of mechatronic systems, Leuven, Belgium, March 2016.
Finally, the PI gave a number of invited tasks that in part or full reflected on the work of this project
- “On Consistency and the Impact of Changes in Collaborative Engineering.” At the Technical University of Munich, Germany, March 2018
- „Wiederverwendung von Softwareprodukt- und Produktionswissen in variantenreichen Systemen,“ at the Software Research Day, May 2017
- “Error Detection and Change Propagation in a Collaborative, Multi-Tool Engineering Environment,” at the University of Lisbon, April 2017
- “Effective Reuse across Many Variants of Software Systems”, at the University of Luxemburg, Luxemburg, November 2015.
- “Enhancing Clone-and-Own with Systematic Reuse for Developing Software Variants”, at the Pierre and Marie Curie University, Paris, France, February 2015.
- “Extraction and Composition for Clone-and-Own and Evolving Product Lines”, at the University of Paderborn, November 2014.
- “Extraction and Composition for Clone-and-Own”, at the Copenhagen Meeting on Variability, May 2014
- “Linking and Constraining Engineering Artifacts”, at the University of Tübingen, Germany, February 2014.
- “Consistency and the Impact of Changes during Model-Driven Engineering ,” at the University of Southern California, USA, July 2013.