Traceability between an EMF Java model and the corresponding source code

MoDisco 0.9M4 has been released this week with Eclipse Indigo M4.

One of the new interesting features is the traceability between an EMF model created from the reverse-engineering of a Java application and the corresponding source code.

It allows to directly navigate from elements of the Java model in the MoDisco Model Browser to the corresponding code fragment in a Java editor.

This feature is provided by a new Java discoverer which can be executed with the MoDisco menu "Discover Java and Inventory model from Java source code" available on a Java project.

This discoverer automatically creates three EMF models:

• a Java model describing the content of the source code: it is the model created by the basic Java discoverer.
• a KDM Inventory model describing the structure of the Java project (all the files and folders).
• a traceability model containing relations between the Java model elements and source code fragments

If you need to know more about these models, you can read the documentation of the discoverer.

When you launch the discoverer on a java project, the traceability model is automatically opened in the MoDisco model browser. 

To activate the synchronization with the Java source code editor, you need to activate the "Synchronize with source code option". 

Now, when you click on a model element in the model browser, the corresponding source code is automatically highlighted in the Java source code editor (otherwise, the double-click on an element opens the property view).

Spy On Your Models - Part 4

In this sequence of posts, I explain how to use the Model Browser provided by MoDisco to inspect an EMF model.

In the first three posts, I have described how to directly access to instances of a given type, how to navigate through the model elements and how to customize the browser.

In this last post, I will now describe how to create and execute queries on model elements directly from the browser.

The features presented in the previous posts, allow you to navigate through the elements of your model by traversing the links. You can also highlight some characteristics of specific elements by defining a customization which dynamically change their rendering (icon, label, font, color, etc).

But in some situations, you would like to calculate values from model elements. These values could be of basic types (numbers or strings) or could be other model elements.

MoDisco provides a mechanism to execute queries interactively on model elements selected in the Model Browser. First, I will explain how to execute queries, and we will see later in this post how to declare and implement them.

There are two possibilities to execute a query.

The first one consists in selecting model elements in the browser and clicking on the menu "Execute Query ...". It opens the "Query Execution" view which contains two main parts:

• The elements on which to execute the query. It is filled with the elements you have selected, but you can also remove elements from this list or drag&drop new elements  from the browser.
• The registered queries which can be executed on the selected elements

When you click on "Execute" the selected query is executed on each model element and the result is displayed in a table (more possibilities will be provided with future versions).

The table displays each returned value on a line. If the result is a basic type, the table contains only one column. But if the result is a model element, the table contains several columns:

• A colum "Label" which contains the calculated label of the object.
• A column "Metaclass" which contains the type of the object
• A column "/eContainer" which contains the parent of the object
• A column "Query Context" which contains the object for which the query has returned the object
• A column for each attribute of the corresponding metaclass
• A column for each reference of the corresponding metaclass (for nary references, the tablel displays the number of related objects).

On each line, it is possible to re-execute a query or to browse the selected element with the model browser.

To execute a query from the browser, the second possibility consists in adding the query to a model element in the browser by clicking on the menu "Dispaly Queries on Selected Element":

A dialog is opened with all the applicable queries:

The selected query is added to the model element. When you click on this query, it is executed and the result is displayed under the query:

Now, to create your own query, the first step consists in creating a QuerySet in a MoDisco project: right-click on the project, select the menu "New>Other" and select "MoDisco / QuerySet model":

Give a name to your new QuerySet and MoDisco creates it in the project:

A QuerySet is a model which contains the declaration of the queries. The new QuerySet is empty. So, to start declaring the queries, just double-click on the QuerySet to open it with the dedicated editor:

Now, you need to associate the QuerySet to a metamodel: all the queries will apply on model elements of the corresponding models. 

Let's say we want to declare queries for UML model elements. This association is done in two steps: 

• Load the ECore definition of the metamodel: right click on the ModelQuerySet instance, select the menu "Load meta-model ressource" and select the right ECore definition.

• Associate this metamodel to the QuerySet: double-click on the ModelQuerySet instance to open its property view, click on the button in the "Associated Metamodels" property field, and select the metamodel.

Now, you can start declaring the queries: right-click on the ModelQuerySet instance and click on the "New Child" menu. You can declare queries of three kinds: Java, OCL or XPath.

For each query, you have to declare its name, its scope (on which elements the query is applicable), its return type and its multiplicity (lower and upper bounds). Implementing the body of the query depends on the kind of query:

• OCL: enter an OCL expression in the "Query" property
• XPath: enter an XPath expression in the "Query" property
• Java: enter the name of a Java class which implements the query

To know more about how to declare a Query, you can read the reference documentation of the MoDisco QueryManager.

MDD Case-Study: RCP Code Generation in an Airline Company

I was recently asked to translate into english  the case-study presented by BritAir during MD Day 2010. So, here is a summary.

BritAir, a french airline company, is a subsidiary of Air France which transported about 3,5 millions of passengers last year.

Another activity of BritAir is the training of pilots, commercial staff and maintenance technicians of other companies.

Despite BritAir has no developer, the company wanted to keep the control and the quality of its applications:

• Normalize the user interface of the applications
• Normalize the way the applications are designed
• Improve the development productivity
• Document the applications

To achive these goals, Olivier Corbel, the IS Manager, decided to put in place a new development process based on a Model-Driven approach. This project was conducted with Mia-Software and Sodifrance.

The principle of the BritAir approach is based on two main ideas:

• A model describing the business data (entities, identifiers, attributes, child/parent relationships)
• An automatic generation of graphical interfaces (Eclipse RCP) for classic functions (cread, read, update, delete, search and export to Excel)

In addition, the business logic is also described in the model: a concrete syntax has been defined to declare some rules in french. It allows to declare variables, to set values, to test, to loop, etc.

The model is created with MagicDraw, an UML modeling tool, and translated into code (Eclipse RCP and Hibernate) with Mia-Studio, the model transformation engine developped by Mia-Software.

The generated applications are complete and don't require any additional code (the business rules are also transleted into Java). They are deployed on servers with Tomcat, Xfire and Oracle.

It took 300 days (total effort between march 2008 and july 2009) to setup the process, define the target architecture, adapt the tools and train BritAir's staff. The team was composed of three people of Mia-Software (a project manager, a MDA enginer and a JEE architect) and one architect of BritAir (requirements and validation).

Several applications have been developped in parallel, in order to challenge and test the new process and tools.

Now, each time a specific development starts, it uses this approach. Until now, six applications have been developped:

• small maintenance of aircrafts
• management of aircraft events
• cost accounting
• financial forecasts
• management of passengers
• preparation of flights. 

Olivier Corbel has identified several benefits of this MDD process:

• Industrialization of the development activity: normalization of modeling and coding, and better documentation.
• Independance of the models from the target architecture (neither RCP nor Hibernate concepts in the models).
• Design is mandatory, in order to generate the application.
• Better quality of code, since it has been qualified first and then converted into generation rules
• Better productivity: during its talk, Olivier Corbel said that the development managers have evaluated to 10 the productivity ratio on these projects!
• User experience: the applications are homogeneous and always provide the same ergonomy to the users
• Prototyping: it is easier to provide early versions to users and take their feedback into account to improve the application

For BritAir, the next step now is to bring evolutions to the platform:

• Support authorization management
• Extend the expressiveness of the business rules language
• Improve the integration of new applications with existing ones (generated or COTS)
• Add a new generation target (web client)

MD Day 2010

Like every year, the french modeling community has met during the MD Day last week in Paris.

It was the fourth edition, with a format which makes the success of this event: each partner (11 this year) must come with one of its customers who presents a real-life project where model-driven engineering has been used. Once again, despite optimistic forecasts, we still ran out of place, and we have been forced to close the registrations three weeks before the event ! Finally there have been 275 attendees (234 last year).

This year, the conference started with Steve Cook's keynote "UML: Past, Present and Future". 

Stevecook uml mdday2010

I already knew the past: I've worked a while with Booch notation in the 90' (remember these little clouds...) ;-) 

What I found really interesting was the part about the UML dilemnas:

• What is the real value proposition of UML ? There is often a misandurstanding about why to create UML models.
• Should DSLs be based on UML ? UML should be more reusable.
• Is UML appropriate to describe non software domains ? It was not the initial intention of this metamodel.
• Can UML be a pivot to model semantics ? New versions of OO languages integrate concepts which are not supported by UML.

Regarding the future of UML, Steve presented the main initiatives:

• UML is too complex: it is going to be simplified, in order to facilitate its modularization and isolate the parts which are really used.
• MOF is going to disapear: UML will be defined by itself, in order to facilitate the manipulation of metamodels with any UML compliant tool.
• UML Diagrams are going to be formally defined: a Diagram Definition initiative has started

These problems are real, but as Eclipse Modeling Project already proposes solutions to solve this kind of problems, I hope that UML team is not going to reinvent the wheel.

After the keynote, the conference continued in two tracks, with partners and users talks.

At Mia-Software we had invited Olivier Corbel, the IS manager of BritAir, an airline company, who presented  its model-driven development process.

Britair mdday2010

BritAir uses MagicDraw to create very simple UML class diagrams which represent the domain model of each new application (entities and relations).

This UML model is used to fully generate RCP applications which allow to manipulate the data (CRUD). The generation rules are defined and executed with Mia-Studio.

When some business logic is required, it is defined with a textual DSL (business rules in french) and then automatically translated into Java.

This approach and the corresponding tools are now used each time BritAir has to develop a specific application. The code is 100% generated. For now, six applications have been developped (small maintenance of aircrafts, management of aircraft events, cost accounting, financial forecasts, management of passengers, preparation of flights). Olivier Corbel said that the development managers have evaluated to 10 the productivity ratio on these projects!

After this presentation, I had decided to talk (in french again) about Model-Driven Development and how to reconcile developers with Model-Driven engineering.

Mia-Software at MD Day 2010

In the first part of my presentation I constated that, in 2010, Model-Driven Engineering adoption continues to increase:

• A lot of successful projects have already proven the benefit of Model-Driven Development. In many domains (bank, insurance, defence, aeronautics, transportation, etc) and for various technologies (Java, C#, Flex, Smalltalk, Ada, C, Cobol, etc).
• Model-Driven Engineering is not only dedicated to the development of new applications. It is also used to maintain the existing ones and facilitate their modernization. The MoDisco project is an example of such initiative.
• The offer is very rich! Just have a look at the Eclipse Modeling Project: during the last nine months, there have been 14M lines of code committed on 45 active projects, by 113 committers of 20 different companies ! In addition, standards such as UML2 (even if the specification could be simplified) is now supported by a lot of mature tools.
• The MD Day reflects this adoption with a continuous augmentation (since 2007) of both partners and attendees.

Despite this fact, we must also admit that Model-Driven Engineering still faces strong resistance. 

It is easy to challenge the most current arguments:

• Complexity of UML? UML is complex (even OMG admitted it by launching the UML Specification Simplification RFP), but 20% of UML (class diagram) is sufficient for 80% of needs. Do you really think that a class diagram is more complex than all the web technologies (WSDL/SOAP, Hibernate, Spring, etc) that most developers are familiar with ? And if it's still too complex, you can use a DSL with only your vocabulary: Model-Driven Engineering is not only UML!
• Too much abstractness? MDD implies that the model should be more abstract than the code to develop (this is what makes the model more sustainable than only the source code). But the level of abstraction is free: it's up to you to define the good level of abstraction depending on the maturity of the team.
• Complicated Process? MDD implies two new activities (the production of the model and the generation). But these activities can be integrated to any kind of process (from agile to waterfall). And modeling usually corresponds to the design of the application. For those who still think that design is not compatible with agility, just read principle #9 of Agile Manifesto: "Continuous attention to technical excellence and good design enhances agility".

Nevertheless, one argument againts MDD is a real one: the difficulty to keep in sync the model and the code. If you loose the synchronization, you loose the benefit of MDD. But if you put all the energy of your team in maintaining this synchronization, you loose in agility.

Hopefully there are solutions to avoid this pitfall: 

• Model at runtime: the model is not transformed into code, it is directlty deployed and interpreted by a runtime engine. This approach is similar to declarative languages based on XML, for example to describe GUIs (XUL, XAML, etc). The drawback is that you need to implement a runtime engine.
• Model compilation: the model is transformed into a complete source code, where no additional coding is required (this is the solution adopted by BritAir). This approach is similar to the compilation of a 3GL language (Java, C++, C#, etc) into C or byte code. The drawback is that you need to create comprehensive models and develop complex generators where all the cases must be supported.

But in most cases, you can't describe all your system with a model. Then the generated code has to be completed manually. In this situation, to avoid the desynchronization between the model and the code, a solution consists in applying the Agile principles to modeling and code generation (we can call this approach Agile MDD).

Agile Modeling consists in:

• Useful modeling: only model what is really helpful for the understanding of the system, and what will be exploited by the generator. Avoid modeling any information which is redundant with the code (it would rapidely be out of sync).
• Simple modeling: if the system is complex, the model can be complex, but it should always be more simple than the code.
• Viewpoints modeling: decompose the model into submodels describing viewpoints on the system
• Incremental modeling: the model should be continuously challenged by other people on the project and/or by the generator.

Scott Ambler has defined several other core principles that you can read on http://www.agilemodeling.com/principles.htm

To produce code, these principles should be combined with other Agile principles related to generation. The objective of these Agile Generation is to reduce the gap between the model and the code:

• Targeted generation: you must be able to select only what you need to (re)generate, depending on what has changed (in the model, or in the generator). If each re-generation is global and takes too much time, you may want to delay the modification in the model and directly modify the code.
• Incremental generation: if the generated code has to be completed manually, these modifications must be protected by the next generations (now, all the generators have this functionality with protected areas).
• Traceability: the generator should keep the trace between the model and the code to allow to easily navigate from a file to the corresponding model element.
• Impact analysis: each time you click on the "generate" button, you must be informed of what exactly happened. Which are the new generated files ? Which parts of the existing files have been modified ?
• Assisted coding: after a generation, you should have a direct access to the parts of the code which have to be manually completed.
• Adaptability: the generated code reflects architectural and coding choices which can evolve during the project. Templates engine already allow to easily modify the generator. But to facilitate the modification of the right template, traceability mechanisms should give a direct access from the generated code to the corresponding template definition.

Of course, these Agile MDD principles are not exclusive: they should be combined, when it is possible, with Model at Runtime and Model Compilation approaches.

To illustrate Agile MDD, I ended my talk with a presentation of the Mia-Studio tool suite which supports these Agile Generation principles. 

Then Tony Rostren shown a short demonstration of the tool:

• Modeling of the cinematic of an application in UML with MagicDraw
• Generation of an executable application based on Struts
• Edition of the generated code in Eclipse
• Navigation from Java source code editor to corresponding element in MagicDraw
• Modification of the model and regeneration of the corresponding code
• Access to a template from a selected fragment of generated code

The full demonstration is available here:

http://www.mia-software.com/html/miaStudio/download/demo/mdday2010.htm

I spent the rest of the conference talking with customers and partners and demonstrating our tools. I could only attend one other presentation. It was the Sodius one, with Yann Lebeaupin talking with EADS about tools interoperability to support system engineering. 

You can find all the other presentations on slideshare: http://www.slideshare.net/MDDAY/presentations