Tagarchief: aggregate

Maven aggregate reports for multi-module projects

December 2013

Maven aggregate reports are useful. They combine the reporting information of a number of maven projects, for example it shows the total amount of code. This article describes how to generate these reports.

If you are looking for a quick copy and paste solution, here is the maven root POM providing all reporting configuration for the aggregate-reports and the child-modules-reports. That’s it, one file, there is nothing to see in the child-modules POM files.

To start with, one of the best tools for quality reports is Sonar. Use it if you can. It allows you to detect changes in code quality over time and it detects hot-spots in your codebase. If it is not possible to use Sonar, or maybe in addition to sonar, the static maven reports can still be very useful.

Reporting for a single project is easy, but aggregate reports are often the most useful because they combine the results of all project-reports in a multi-module project. Configuring the aggregate reports takes some extra effort. The information about maven reporting is scattered and I decided to summarize it in this article.

There are several strategies that reporting-plugins use to create aggregate reports. After an introduction about reporting this article summarizes the aggregate configuration strategies for several widely used plugins using the maven recommended format for configuring reports.

Maven Reporting

Reporting goals and plugins

A reporting goal is a plugin goal that was designed to generate some report as a result of an invocation by the Maven Site Plugin. Reporting goals are marked in the plugin documentation ‘Report?’ column. See for example the Maven Javadoc Plugin goals. Project reporting plugins are maven plugins supporting one or more reporting goals. Maven Project reporting plugins are marked with an ‘R’ in the ‘Type’ column in the plugins overview.

The Site Plugin

Maven uses the Maven Site Plugin to generate the reports with the invocation of the site goal in the site lifecycle (mvn site:site). It executes the reporting plugins configured in the reporting section of the POM. By default, the Maven Site Plugin invokes every reporting goal of the plugins configured in the reporting section. Configuring a plugin without reporting goals in the reporting section has no effect. Even if you point a reportSet to some goal of the plugin. The Site Plugin just ignores the plugin or reportSet if it does not support reporting.

Configuring

If you need to customize the reporting behaviour then you can tune the plugin using reportSets specifications. A reportSet points to one or more reporting goal with the report element. If the plugin is configured without reportSets then all reporting goals are executed once.
If there are no special reporting requirements for some modules, it is a best practice to put all configuration in the parent POM. In this example the root and parent POM are the same project.

Maven aggregate support

The default behaviour of the Maven Site Plugin is to execute all reporting goals of the plugins mentioned in the reporting secion of the POM. This is why reporting plugins with an aggregate goal require no further configuration. Other plugins use an aggregate configuration parameter in the reporting goals. Some plugins support an aggregate goal and and configuration parameter to produce the aggregate reports. In this case using the goal is the way to go. The parameter is often deprecated in favour of the aggregate goal.

There is one problem with maven generating aggregate reports. It creates the reports of the root project before the child projects. Aggregate reports often need the child projects reporting results. That is why we need to run the site goal twice.

Source code & build

A simple multi-module project used for this article is available in bitbucket: https://bitbucket.org/bartswen/aggregate-report-demo

Download it and execute the build commands in the root project directory:

  1. mvn verify site
  2. mvn site:site site:stage

You will find the reports in ./target/staging/index.html

There are two ways to configure reports in maven 3. This article is based on the recommended ‘classic’ configuration format using the <reporting> section of the POM. Note that many online examples use the <reportPlugins> element in the site plugin configuration. This is not recommended and the element will be removed.

Reporting plugins configuration

I divide the reporting plugins into three categories.

1. Plugins with automatic aggregate support

Some reporting plugins are smart. They automatically detect an ‘aggregate scenario’. The JavaNCSS Plugin uses this strategy. If the project has no source and it has child modules then the plugins report goal tries to generate an aggregate. Unfortunately the NCSS Plugin does need two cycles to generate the aggregate reports. The declaration below is enough to produce the modules NCSS reports and the aggregate report showing the total amount of code of all modules:

2. Plugins supporting an aggregate goal

Other reporting plugins have a separate aggregate reporting goal. According to the site plugin documentation we should configure reportSets for these reports: read the ‘Notice’ below the code fragment on the Site Plugins reportSet documentation. But, although I am unable to find any documentation about this subject, these plugins seem to be smart enough to skip the aggregate goal on a non-aggregate project. The reports look good, even if we do not configure any reportSets to map the right plugin goals on the projects.

To validate a report, take for example the JXR Plugin. It produces a cross-reference of the project’s sources. Note how the aggregate report references the Mod1 from the Mod2 class. The non-aggregate report of the mod2 module has no reference to Mod1.

3. Plugins using an aggregate parameter

These plugins have no specific aggregate goals, but they require special configuration to produce aggregate reports. The trick here is to define two reportSets under the plugin configuration in the reporting section. One reportSet is to be inherited by the modules to build their reports. The other reportSet produces the aggregate report using <aggregate>true</aggregate> as a configuration parameter. In this case we must configure the reportSet to apply only to the root project and not be inherited by the modules, using <inherited>false<inherited>. The <report> element refers to a plugin reporting goal.

Taglist

Taglist Maven Plugin

Cobertura

Although there have been problems to create Cobertura aggregate reports, all seems to be working now with the current version of the Cobertura Maven Plugin. The <aggregate>true</aggregate> parameter does not seem to bother the modules projects report generation. So with Cobertura we only need one reportSet to generate both the project reports as well as the aggregate report.

And by the way, while you are there, for fun check the Cobertura report of the Cobertura Plugin. Ironically it’s pretty red itself (at the time of writing).

PMD and CPD reports

The PMD Plugin has two reporting goals. One for the PMD, and one for the CPD report. For each we need the two reportSets, so we have four reportSets here:

Unit and integration test reports

There exists a special plugin for generating the test-reports: the Surefire Report Plugin. Provided that you have configured the Failsafe Plugin to run the integration tests, it produces reports for unit-tests as well as for the integration-tests. So again we need to specify four reportSets: two for the unit-test reports, and two for the integration-test reports.

Final note

Unfortunately there exists a fourth category of reporting plugins: those that do not support aggregate reports. For example the FindBugs Plugin states that it is unable to generate these reports.

Resources

Maven

Reporting plugins not supported by the maven project

Managing multiple transactions in one request

November 2013

After reading Vaughn Vernon’s Implementing Domain-Driven-Design I decided to start the fourth redesign of an application that I developed. Just to get some experience with command and event based architectures. This article is about the transactional application-service layer. First I describe how I use multiple Spring transactions in one service call. Then I share how I discovered how to use the proper transaction propagation mode.

Why do I need this in the first place, why not use just one transaction? According to the Domain Driven Design principles transactions should not cross aggregate boundaries. Updating an aggregate could emit an event that needs to be processed by another aggregate in the same bounded context. If we keep things simple we let the same thread handle the emitted events. So we need a second transaction for the event processing. To be clear, I am not talking about a nested transaction, it’s one transaction followed by another.

In this example I use two aggregates. FirstAggregate produces a domain event. SecondAggregate is interested in this event to update its state. This diagram shows the basic idea:

diagram

I’ll start with the event processing to update SecondAggregate. Then I work my way back to the application service that updates FirstAggregate.

How to start a second transaction

It is important to explicitly start a new transaction here. Most conventional systems handle a request in one transaction. Then you could use the default transaction propagation mode, which is REQUIRED. In our case we need REQUIRES_NEW to force a new transaction for updating SecondAggregate.

This is the DomainEventProcessor updating SecondAggregate. In all fragments below, the code is kept simple and I left some code out to keep the focus on the event handling mechanism.

How to postpone event handling

We could let the DomainEventProcessor handle the events right after the moment that FirstAggregate emits them. This would lead us into problems. The DomainEventProcessor would have Spring suspend the first transaction and then start a second one. The transactionmanager will commit the second transaction and then possibly rollback the first. One solution for this is to buffer the events. Then, after the commit, ‘flush’ the events to have them processed:

How to trigger logic after tx commit

How to get Spring to flush a BufferedDomainEventSubscriber after the first transaction has successfully committed? It does not require much googling to find out that this is done using Spring’s [TransactionSynchronizationManager](http://docs.spring.io/spring/docs/3.2.4.RELEASE/javadoc-api/org/springframework/transaction/support/TransactionSynchronizationManager.html). This class enables you to register some transaction-related callbacks. Here is the class that implements the ‘after transaction commit callback’:

And here how to use TransactionSynchronizationManager to flush all buffered events to its subscribers. This, finally, is the application service receiving a command from a client:

So now what happens is that some client invokes FirstAggregateApplicationService.doActionOnFirstAggreate() and Spring starts transaction #1. FirstAggregate.changeState() gets invoked and will send out a domain event. The BufferedDomainEventSubscriber accepts the domain event and stores it. If a rollback happens to transaction #1 then the event is ignored. If instead a commit takes place, Spring will invoke the afterCommit() which sends the events to the DomainEventProcessor.process(). Spring will wrap transaction #2 around this invocation.

What happens if we do not use REQUIRES_NEW

Which of course is what I did in my first attempt. What I found what happens using default transaction propagation mode REQUIRES was that the changes to SecondAggregate were not persisted. So my first thought was:

Is the logic executing outside a transaction?

In other words, is Spring ignoring the second @transaction on DomainEventProcessor.process()? This seemed to be the case at first sight. There exist some conditions for Spring to recognize the @transactional. For example only public methods, proxied classes, … But the class executing the second update is proxied, and the conditions seem to be met. So I needed to:

Investigate what is going on

I have tried several ways of inspecting the transaction behaviour. For example logging the result of TransactionSynchronizationManager.isActualTransactionActive(). This may help to detect a situation where a transaction is never started, but it did not help me much with my problem. It told me that a transaction was always active whenever I updated my aggregates. So this could not explain why my changes were not persisted.

What did help me was to log the transactional behaviour and then inspect the log files. I configured debug level logging and on org.springframework.orm.jpa on my own packages to see what is going on in JpaTransactionManager:

What I saw was:

I noticed two important things:

  1. After the ‘Start flush 1 event’, Spring TM understands that we want to execute the process() logic inside a transaction. Spring recognizes the @transactional.
  2. But it seems to use the existing, already committed, transaction. It says: ‘Participating in existing transaction’, and no commit appears afterwards

Then I found an important note in the API doc of the TransactionSynchronization.afterCommit() that explains it all. The transaction is still active after the commit. Transactional code is executed, but never committed. The solution here is to explicitly use transaction propagation mode PROPAGATION_REQUIRES_NEW in the Processor.

Then the logs show:

Notice the Suspending current transaction, creating new transaction.... This line shows that the event processing code is actually running in a second transaction.

Summary

  1. The need for REQUIRES_NEW transaction propagation is not very common in conventional architectures. When applying DDD tactical patterns, we can use it to separate transactions for multiple aggregate-updates.
  2. The transactionmanager does not care if it executes transactional code in a transaction that it has already committed. It just runs the code while is does not actually participate in the transaction.
  3. Configure your logging. The Spring logging is very good and it tells you exactly what is happening. Use it to learn.

Resources and thanks to