Traceability between an EMF Java model and the corresponding source code

MoDisco 0.9M4 has been released this week with Eclipse Indigo M4.

One of the new interesting features is the traceability between an EMF model created from the reverse-engineering of a Java application and the corresponding source code.

It allows to directly navigate from elements of the Java model in the MoDisco Model Browser to the corresponding code fragment in a Java editor.

This feature is provided by a new Java discoverer which can be executed with the MoDisco menu "Discover Java and Inventory model from Java source code" available on a Java project.

This discoverer automatically creates three EMF models:

• a Java model describing the content of the source code: it is the model created by the basic Java discoverer.
• a KDM Inventory model describing the structure of the Java project (all the files and folders).
• a traceability model containing relations between the Java model elements and source code fragments

If you need to know more about these models, you can read the documentation of the discoverer.

When you launch the discoverer on a java project, the traceability model is automatically opened in the MoDisco model browser. 

To activate the synchronization with the Java source code editor, you need to activate the "Synchronize with source code option". 

Now, when you click on a model element in the model browser, the corresponding source code is automatically highlighted in the Java source code editor (otherwise, the double-click on an element opens the property view).

Spy On Your Models - Part 4

In this sequence of posts, I explain how to use the Model Browser provided by MoDisco to inspect an EMF model.

In the first three posts, I have described how to directly access to instances of a given type, how to navigate through the model elements and how to customize the browser.

In this last post, I will now describe how to create and execute queries on model elements directly from the browser.

The features presented in the previous posts, allow you to navigate through the elements of your model by traversing the links. You can also highlight some characteristics of specific elements by defining a customization which dynamically change their rendering (icon, label, font, color, etc).

But in some situations, you would like to calculate values from model elements. These values could be of basic types (numbers or strings) or could be other model elements.

MoDisco provides a mechanism to execute queries interactively on model elements selected in the Model Browser. First, I will explain how to execute queries, and we will see later in this post how to declare and implement them.

There are two possibilities to execute a query.

The first one consists in selecting model elements in the browser and clicking on the menu "Execute Query ...". It opens the "Query Execution" view which contains two main parts:

• The elements on which to execute the query. It is filled with the elements you have selected, but you can also remove elements from this list or drag&drop new elements  from the browser.
• The registered queries which can be executed on the selected elements

When you click on "Execute" the selected query is executed on each model element and the result is displayed in a table (more possibilities will be provided with future versions).

The table displays each returned value on a line. If the result is a basic type, the table contains only one column. But if the result is a model element, the table contains several columns:

• A colum "Label" which contains the calculated label of the object.
• A column "Metaclass" which contains the type of the object
• A column "/eContainer" which contains the parent of the object
• A column "Query Context" which contains the object for which the query has returned the object
• A column for each attribute of the corresponding metaclass
• A column for each reference of the corresponding metaclass (for nary references, the tablel displays the number of related objects).

On each line, it is possible to re-execute a query or to browse the selected element with the model browser.

To execute a query from the browser, the second possibility consists in adding the query to a model element in the browser by clicking on the menu "Dispaly Queries on Selected Element":

A dialog is opened with all the applicable queries:

The selected query is added to the model element. When you click on this query, it is executed and the result is displayed under the query:

Now, to create your own query, the first step consists in creating a QuerySet in a MoDisco project: right-click on the project, select the menu "New>Other" and select "MoDisco / QuerySet model":

Give a name to your new QuerySet and MoDisco creates it in the project:

A QuerySet is a model which contains the declaration of the queries. The new QuerySet is empty. So, to start declaring the queries, just double-click on the QuerySet to open it with the dedicated editor:

Now, you need to associate the QuerySet to a metamodel: all the queries will apply on model elements of the corresponding models. 

Let's say we want to declare queries for UML model elements. This association is done in two steps: 

• Load the ECore definition of the metamodel: right click on the ModelQuerySet instance, select the menu "Load meta-model ressource" and select the right ECore definition.

• Associate this metamodel to the QuerySet: double-click on the ModelQuerySet instance to open its property view, click on the button in the "Associated Metamodels" property field, and select the metamodel.

Now, you can start declaring the queries: right-click on the ModelQuerySet instance and click on the "New Child" menu. You can declare queries of three kinds: Java, OCL or XPath.

For each query, you have to declare its name, its scope (on which elements the query is applicable), its return type and its multiplicity (lower and upper bounds). Implementing the body of the query depends on the kind of query:

• OCL: enter an OCL expression in the "Query" property
• XPath: enter an XPath expression in the "Query" property
• Java: enter the name of a Java class which implements the query

To know more about how to declare a Query, you can read the reference documentation of the MoDisco QueryManager.

MDD Case-Study: RCP Code Generation in an Airline Company

I was recently asked to translate into english  the case-study presented by BritAir during MD Day 2010. So, here is a summary.

BritAir, a french airline company, is a subsidiary of Air France which transported about 3,5 millions of passengers last year.

Another activity of BritAir is the training of pilots, commercial staff and maintenance technicians of other companies.

Despite BritAir has no developer, the company wanted to keep the control and the quality of its applications:

• Normalize the user interface of the applications
• Normalize the way the applications are designed
• Improve the development productivity
• Document the applications

To achive these goals, Olivier Corbel, the IS Manager, decided to put in place a new development process based on a Model-Driven approach. This project was conducted with Mia-Software and Sodifrance.

The principle of the BritAir approach is based on two main ideas:

• A model describing the business data (entities, identifiers, attributes, child/parent relationships)
• An automatic generation of graphical interfaces (Eclipse RCP) for classic functions (cread, read, update, delete, search and export to Excel)

In addition, the business logic is also described in the model: a concrete syntax has been defined to declare some rules in french. It allows to declare variables, to set values, to test, to loop, etc.

The model is created with MagicDraw, an UML modeling tool, and translated into code (Eclipse RCP and Hibernate) with Mia-Studio, the model transformation engine developped by Mia-Software.

The generated applications are complete and don't require any additional code (the business rules are also transleted into Java). They are deployed on servers with Tomcat, Xfire and Oracle.

It took 300 days (total effort between march 2008 and july 2009) to setup the process, define the target architecture, adapt the tools and train BritAir's staff. The team was composed of three people of Mia-Software (a project manager, a MDA enginer and a JEE architect) and one architect of BritAir (requirements and validation).

Several applications have been developped in parallel, in order to challenge and test the new process and tools.

Now, each time a specific development starts, it uses this approach. Until now, six applications have been developped:

• small maintenance of aircrafts
• management of aircraft events
• cost accounting
• financial forecasts
• management of passengers
• preparation of flights. 

Olivier Corbel has identified several benefits of this MDD process:

• Industrialization of the development activity: normalization of modeling and coding, and better documentation.
• Independance of the models from the target architecture (neither RCP nor Hibernate concepts in the models).
• Design is mandatory, in order to generate the application.
• Better quality of code, since it has been qualified first and then converted into generation rules
• Better productivity: during its talk, Olivier Corbel said that the development managers have evaluated to 10 the productivity ratio on these projects!
• User experience: the applications are homogeneous and always provide the same ergonomy to the users
• Prototyping: it is easier to provide early versions to users and take their feedback into account to improve the application

For BritAir, the next step now is to bring evolutions to the platform:

• Support authorization management
• Extend the expressiveness of the business rules language
• Improve the integration of new applications with existing ones (generated or COTS)
• Add a new generation target (web client)

MD Day 2010

Like every year, the french modeling community has met during the MD Day last week in Paris.

It was the fourth edition, with a format which makes the success of this event: each partner (11 this year) must come with one of its customers who presents a real-life project where model-driven engineering has been used. Once again, despite optimistic forecasts, we still ran out of place, and we have been forced to close the registrations three weeks before the event ! Finally there have been 275 attendees (234 last year).

This year, the conference started with Steve Cook's keynote "UML: Past, Present and Future". 

Stevecook uml mdday2010

I already knew the past: I've worked a while with Booch notation in the 90' (remember these little clouds...) ;-) 

What I found really interesting was the part about the UML dilemnas:

• What is the real value proposition of UML ? There is often a misandurstanding about why to create UML models.
• Should DSLs be based on UML ? UML should be more reusable.
• Is UML appropriate to describe non software domains ? It was not the initial intention of this metamodel.
• Can UML be a pivot to model semantics ? New versions of OO languages integrate concepts which are not supported by UML.

Regarding the future of UML, Steve presented the main initiatives:

• UML is too complex: it is going to be simplified, in order to facilitate its modularization and isolate the parts which are really used.
• MOF is going to disapear: UML will be defined by itself, in order to facilitate the manipulation of metamodels with any UML compliant tool.
• UML Diagrams are going to be formally defined: a Diagram Definition initiative has started

These problems are real, but as Eclipse Modeling Project already proposes solutions to solve this kind of problems, I hope that UML team is not going to reinvent the wheel.

After the keynote, the conference continued in two tracks, with partners and users talks.

At Mia-Software we had invited Olivier Corbel, the IS manager of BritAir, an airline company, who presented  its model-driven development process.

Britair mdday2010

BritAir uses MagicDraw to create very simple UML class diagrams which represent the domain model of each new application (entities and relations).

This UML model is used to fully generate RCP applications which allow to manipulate the data (CRUD). The generation rules are defined and executed with Mia-Studio.

When some business logic is required, it is defined with a textual DSL (business rules in french) and then automatically translated into Java.

This approach and the corresponding tools are now used each time BritAir has to develop a specific application. The code is 100% generated. For now, six applications have been developped (small maintenance of aircrafts, management of aircraft events, cost accounting, financial forecasts, management of passengers, preparation of flights). Olivier Corbel said that the development managers have evaluated to 10 the productivity ratio on these projects!

After this presentation, I had decided to talk (in french again) about Model-Driven Development and how to reconcile developers with Model-Driven engineering.

Mia-Software at MD Day 2010

In the first part of my presentation I constated that, in 2010, Model-Driven Engineering adoption continues to increase:

• A lot of successful projects have already proven the benefit of Model-Driven Development. In many domains (bank, insurance, defence, aeronautics, transportation, etc) and for various technologies (Java, C#, Flex, Smalltalk, Ada, C, Cobol, etc).
• Model-Driven Engineering is not only dedicated to the development of new applications. It is also used to maintain the existing ones and facilitate their modernization. The MoDisco project is an example of such initiative.
• The offer is very rich! Just have a look at the Eclipse Modeling Project: during the last nine months, there have been 14M lines of code committed on 45 active projects, by 113 committers of 20 different companies ! In addition, standards such as UML2 (even if the specification could be simplified) is now supported by a lot of mature tools.
• The MD Day reflects this adoption with a continuous augmentation (since 2007) of both partners and attendees.

Despite this fact, we must also admit that Model-Driven Engineering still faces strong resistance. 

It is easy to challenge the most current arguments:

• Complexity of UML? UML is complex (even OMG admitted it by launching the UML Specification Simplification RFP), but 20% of UML (class diagram) is sufficient for 80% of needs. Do you really think that a class diagram is more complex than all the web technologies (WSDL/SOAP, Hibernate, Spring, etc) that most developers are familiar with ? And if it's still too complex, you can use a DSL with only your vocabulary: Model-Driven Engineering is not only UML!
• Too much abstractness? MDD implies that the model should be more abstract than the code to develop (this is what makes the model more sustainable than only the source code). But the level of abstraction is free: it's up to you to define the good level of abstraction depending on the maturity of the team.
• Complicated Process? MDD implies two new activities (the production of the model and the generation). But these activities can be integrated to any kind of process (from agile to waterfall). And modeling usually corresponds to the design of the application. For those who still think that design is not compatible with agility, just read principle #9 of Agile Manifesto: "Continuous attention to technical excellence and good design enhances agility".

Nevertheless, one argument againts MDD is a real one: the difficulty to keep in sync the model and the code. If you loose the synchronization, you loose the benefit of MDD. But if you put all the energy of your team in maintaining this synchronization, you loose in agility.

Hopefully there are solutions to avoid this pitfall: 

• Model at runtime: the model is not transformed into code, it is directlty deployed and interpreted by a runtime engine. This approach is similar to declarative languages based on XML, for example to describe GUIs (XUL, XAML, etc). The drawback is that you need to implement a runtime engine.
• Model compilation: the model is transformed into a complete source code, where no additional coding is required (this is the solution adopted by BritAir). This approach is similar to the compilation of a 3GL language (Java, C++, C#, etc) into C or byte code. The drawback is that you need to create comprehensive models and develop complex generators where all the cases must be supported.

But in most cases, you can't describe all your system with a model. Then the generated code has to be completed manually. In this situation, to avoid the desynchronization between the model and the code, a solution consists in applying the Agile principles to modeling and code generation (we can call this approach Agile MDD).

Agile Modeling consists in:

• Useful modeling: only model what is really helpful for the understanding of the system, and what will be exploited by the generator. Avoid modeling any information which is redundant with the code (it would rapidely be out of sync).
• Simple modeling: if the system is complex, the model can be complex, but it should always be more simple than the code.
• Viewpoints modeling: decompose the model into submodels describing viewpoints on the system
• Incremental modeling: the model should be continuously challenged by other people on the project and/or by the generator.

Scott Ambler has defined several other core principles that you can read on http://www.agilemodeling.com/principles.htm

To produce code, these principles should be combined with other Agile principles related to generation. The objective of these Agile Generation is to reduce the gap between the model and the code:

• Targeted generation: you must be able to select only what you need to (re)generate, depending on what has changed (in the model, or in the generator). If each re-generation is global and takes too much time, you may want to delay the modification in the model and directly modify the code.
• Incremental generation: if the generated code has to be completed manually, these modifications must be protected by the next generations (now, all the generators have this functionality with protected areas).
• Traceability: the generator should keep the trace between the model and the code to allow to easily navigate from a file to the corresponding model element.
• Impact analysis: each time you click on the "generate" button, you must be informed of what exactly happened. Which are the new generated files ? Which parts of the existing files have been modified ?
• Assisted coding: after a generation, you should have a direct access to the parts of the code which have to be manually completed.
• Adaptability: the generated code reflects architectural and coding choices which can evolve during the project. Templates engine already allow to easily modify the generator. But to facilitate the modification of the right template, traceability mechanisms should give a direct access from the generated code to the corresponding template definition.

Of course, these Agile MDD principles are not exclusive: they should be combined, when it is possible, with Model at Runtime and Model Compilation approaches.

To illustrate Agile MDD, I ended my talk with a presentation of the Mia-Studio tool suite which supports these Agile Generation principles. 

Then Tony Rostren shown a short demonstration of the tool:

• Modeling of the cinematic of an application in UML with MagicDraw
• Generation of an executable application based on Struts
• Edition of the generated code in Eclipse
• Navigation from Java source code editor to corresponding element in MagicDraw
• Modification of the model and regeneration of the corresponding code
• Access to a template from a selected fragment of generated code

The full demonstration is available here:

http://www.mia-software.com/html/miaStudio/download/demo/mdday2010.htm

I spent the rest of the conference talking with customers and partners and demonstrating our tools. I could only attend one other presentation. It was the Sodius one, with Yann Lebeaupin talking with EADS about tools interoperability to support system engineering. 

You can find all the other presentations on slideshare: http://www.slideshare.net/MDDAY/presentations

An Eclipse Modeling Week

Last week, I participated to Eclipse Summit Europe 2010 in Germany.

I started tuesday with a talk at the Eclipse Modeling Symposium (you can read my previous post: Using EMF to represent Eclipse 3.x plug-ins , and a complete summary of the symposium on Ed Merks' blog).

Then thursday, with Grégoire Dupé, I presented EMF Facet, the new Eclipse/Modeling project we have created to allow dynamic extensions of existing EMF models.

EMF Facet at ESE 2010

We have finished the week friday, at Eclipse Day Paris, by a presentation of MoDisco, the Eclipse/Modeling project dedicated to Model-Driven Software Modernization.

MoDisco at Eclipse Day Paris 2010

Now, we can come back to a normal activity ... until MD Day 2010, the 25th of November in Paris.

Using EMF to represent Eclipse 3.x plug-ins

Yesterday was the first day of Eclipse Europe Summit 2010 in Ludwigsburg. The afternoon, I participated to the Eclipse Modeling Symposium organized by Ed Merks and Sven Efftinge. 

The room was crowded: I evaluated more than 150 attendees to this 4 hours session. It has been a very good opportunity to get an overview of how dynamic is the modeling world in Eclipse. There have been 15 presentations about models migration, graphical modelers (even in 3D !), models testing, code generation, documentation generation, new features for textual DSLs, and many other topics… (you can find a summary on the Mäd Meiers' blog).

I really appreciated the Papyrus presentation, especially when Remi Schnekenburger of CEA explained how they have integrated MoDisco to provide customizable navigation through the models ;-)


I also got the chance to give a presentation during this session. This presentation was about using EMF to describe existing plug-ins developed for Eclipse 3.x. The idea is to create models from the main artifacts which compose an existing plug-in: 

• An inventory model (using KDM Source) for the organization of the plugin (folders and files)
• A comprehensive Java model (using MoDisco) from the Java source code
• A specific model to describe the MANIFEST.MF file
• Several XML models (using MoDisco) for the .classpath, .project and plugin.xml files
• Two KDM models for the content of the plugin.properties and build.properties

These models are referenced by a last model which describes the whole plug-in. 

Here, the main benefit of the approach is not to raise the level of abstraction (the classical benefit of Model-Driven Engineering), but to go from an heterogeneous world (with various file formats) to an homogeneous world (EMF models) where you can manipulate all the information by using EMF APIs (and then any EMF compliant tool).


Then, I presented three examples of how this kind of model can be used:

• Quality Analysis: by introspecting the model of the plug-in to check development rules, we can create a model of violations and inject this model into the Eclipse Problem View.

• Refactoring: by transforming the model of the plug-in and then regenerate it, we can adapt an existing Eclipse 3.x plug-in to fit with the E4 compatibility layer.

• Release Engineering: by extracting additional models from the content of update sites, we are experimenting constraints solver to calculate and evaluate possible build configurations and initialize B3 models.

The slides of my presentation are available on Slideshare.

Mia-Software at Eclipse Modeling Symposium 2010

View more presentations from fmadiot.

Today I will try to combine official sessions (to discover what is coming from other projects) and side discussions with persons who are interested in our work (several meetings are already planned about MoDisco and EMF Facet). 


ESE is definitely too short :-(

Spy On Your Models - Part 3

In this sequence of posts, I explain how to use the Model Browser provided by MoDisco to inspect an EMF model.


In the first two posts, I have described how to directly access to instances of a given type, and how to navigate through the model elements. In this third post, I will now describe how to customize the browser.



When your model contains a large number of model elements, you need help to identify the nature of the objects your are looking for. A good idea is to define a graphical extension to associate a specific graphical rendering (icon, label, color, etc). It can be achieved with the EMF.Edit framework by developing content and label provider classes. You can develop and deploy a new plug-in, and once this plug-in is installed, the customization is applied systematically. 


But in some situations, you need a more flexible customization. A customization which presents the model according to a given viewpoint. A customizations that you apply temporarily, depending on a working situation. 
This is what you can do by creating a UICustom model. 


A UICustom model defines graphical properties for elements of a metamodel (a ECore model). These properties can be loaded dynamically by the Model Browser and applied to the model elements.
First, you have to create a new MoDisco project:

Then, create a UICustom model in this project:

The UICustom creation wizard asks which is the metamodel to be customized. It proposes all the ECore models registered in your Eclipse:

Once created, you can edit the UICustom model with the editor provided by MoDisco:

Now you can select a metaclass and edit its graphical properties (label, bold, italic, font, color, icon, visible, etc). You can also customize the attributes and the references of the metaclass.


Each graphical property can be set with a static value (for example, define a new icon for all the instances of Class) or a dynamic value, computed by a Query (for example, the label of an instance of Class can be italic if its "isAbstract" value is true).


To understand how to define a customization, you can read the user manual in the MoDisco wiki.


Once saved, a UICustom is applicable. Click on the "Load Customization" button in the toolbar :

Your UICustom model appears with all the other registered UICustom models. You can select it to apply it on the current model loaded in the Model Browser.

In the next post, I will describe how to create and execute queries on model elements directly from the Model Browser.

Spy On Your Models - Part 2

In this sequence of posts, I explain how to use the Model Browser provided by MoDisco to inspect an EMF model.

In the first post, I have described the Types panel which allows to directly access to all the EMF instances of a given EClass. In this second post, I will now describe the Instances panel which allows to inspect an EMF object and navigate to all the other connected EMF objects.

This panel is a tree containing all the instances of the type selected in the Type panel. If you open a model element, you can see all its properties: the attributes and the references declared by the type (the metaclass) in the corresponding ECore model (the metamodel).

The attributes are displayed with a green bullet and the value of the selected instance. The references are displayed with an icon inspired by UML associations: containment references have a diamond (like UML compositions), references with no opposite are terminated by an arrow (like UML navigable roles). Derived properties are prefixed by "/".

On each reference, you can right-click and select the "Add child" menu to add new EMF objects.

If you want to hide empty and derived references, unselect the buttons "Show Empty Links" and "Show Derived Links". Then the tree will only display setable references which have at least one connected object.

When you open a reference, you can see all the objects connected through this reference. This is very helpful to know how the objects are connected to each other. Especially when the current type declares several references of the same type: in this case, you see precisely the connected objects under each reference.

This instance tree proposes several other interesting options.

Let's see some of these options:

• Show Multiplicity: This option displays the upper and lower bounds (between square brackets) of each structural feature as they are defined in the corresponding ECore model.

• Show Ordering: This option displays the index (between square brackets) of each element under a reference which has the "ordered" ECore property set to true. It is useful if you have already selected the "Sort instances" option which displays the elements in an alphabetical order. See below the instances of Property sorted by name under the ownedAttribute reference: they are suffixed by the index within the list of attributes of the current class.

• Show Opposite Links: this option indicates the name of the opposite links (if there is one defined in the ECore model). This option helps to know which reference allows to go back to a given object. Below, you can see that the owner reference can be used to go from an instance of Comment back to the class CollectInformation.

• Show Container: When there is no opposite link defined, this option helps you to go back to the container (the objects which holds the current object with a "containment" reference). The eContainer reference is part of the EMF framework.

• Show Types of Links: This option indicates the Type declared for each reference. It is useful to know what kind of objects you can find under a reference. Below, you can see that the packagedElement reference accepts instances of PackageableElement. It explains why you find instances of Class and Association which both inherit from PackageableElement.

In the next post, I will explain how this browser can be dynamically extended and customized.

Spy On Your Models - Part 1

Have you ever been stuck with an EMF model, trying to find what was going wrong? Which relation is broken or missing ? Where is that model element which should be there, related to that other one ? How many instances this EClass has in the model ?

Of course, you can use the EMF reflexive editor. It is suitable for small models, but when your model contains a lot of model elements it is hard to find your way (you must start from the roots elements, you don't see the name of the relations between the objects, the tree doesn't show the objects pointed by non-containment relations, etc).

So, if you really need this kind of information (and if you are not an expert in reading XMI files in plain text), the model browser provided by MoDisco is a solution for you.

To allow inspecting the details of large and complex EMF models, MoDisco (the Eclipse/modeling project dedicated to Software Modernization) provides an editor with lots of interesting features.

After MoDisco installation, you can right-click on any EMF resource and open the browser by selecting the menu "Open with > MoDisco Browser".

In this suite of posts, I will describe the main features of this tool.

The first feature is a panel, on the left-side of the browser, displaying the classes (metaclasses) of the corresponding ECore definition (the metamodel). The selection of a type displays all its instances in the right-side panel. Each type is suffixed by the number of its instances. By default, only the types with at least one instance are listed, but if you click on the "(0)" button, the empty metaclasses are also listed.

Sometimes, it can be helpful to see also the instances of the subtypes too. For example, in a UML model, if you consider the DataType metaclass you may need to access also to the Enumeration and PrimitiveType instances. It is possible by clicking on the button "Display Instances of Submetaclasses".

By clicking on the button "Show Derivation Tree", you can also display these types in a tree, organized according to the "eSuperTypes" relation.

Four other options are available: to sort the types alphabetically, to sort them by the number of their instances, to group them by package and to show their full qualified name.

Of course, standard features provided by EMF on EClass are still available. For example to get the references to a metaclass, the menu "Show References" opens the view "EClass References".

Another important feature provided by the browser is the management of large models. If a type as more than 1000 instances, they are displayed by groups of 1000 instances.

In a next post, I will describe the features related to the Instances panel which allows to view the properties of each instance and to navigate through its references to other instances.