Maven, 2 years after

I dislike maven, it is quite weird. Try to use ant creates macro, such as compile, package etc first, then call these macro in your main build.xml. This is a kind of customisation build tool...

see more http://www.geniuswiki.com/t/688e04f3-967a-43dc-8fac-75c9c38db369

I am a huge fan of gradle. I've used for some simple things, and the simple things really are simple. It's very extensible as well (when I'm able to fit it into my schedule I'll be writing my own plugin for it to announce releases via email/twitter/etc). There are many things I like about gradle, some of them:

• Uses Ivy under the hood. This means you'll have complete Maven compatibility, create POMs, read POMs, etc
• All of your builds are Groovy scripts. You have a complete scripting language at your disposal, because builds are complicated, plain and simple. Ant contrib or using if and unless on the targets just doesn't cut it some times.
• It's fast. Initial tests done by the gradle team (okay, yeah may be a little biased) put it just as fast or slightly faster than Maven on multi-project builds
• Mult-project builds are handled much nicer than in Maven, you can build part of the tree, the whole tree, or use cached versions of dependencies

Dan Allen I believe has some experience with it (he actually got me interested in it), if not then he's spoken with Gradle's founder Hans Dockter. Gradle is very active, the lists are very helpful.

I can't say much about Buildr. It looked interesting, but I was a little surprised they didn't try to integrate Ivy into it, maybe because it's purely ruby. That part actually worried me a little bit, it's one more way of parsing POMs and dealing with dependencies. Other than that it really looks similar to Gradle.

If you're building osgi bundles (aka eclipse plugins) then I recommend going the PDE route, or if that's too complex, the Athena route (which is a wrapper for PDE to make it simpler to use. I recommend Athena because I develop that project @ Eclipse, and continue to add new functionality and features to it.

If you're not planning to build for Eclipse or with Eclipse, or to use osgi in any way, then Ant with Ant-Contrib or else Gradle are viable alternatives. There's also EasyAnt, another Ivy-based solution.

(I too loathe Maven.)

Try Gradle http://gradle.org/. It is a huge step forward.

You have mentioned that you have two problems:

1. So anyway, what are the issues with this assumption? Well the first is that I cannot simply 'change directory' into a sub-project directory and build it and expect Maven to build the other projects upon which this sub-project depends for me automatically. For example, I cannot 'cd' into the hibernate-c3p0 project directory and run 'mvn compile' and expect Maven to first compile the hibernate-core project for me even though hibernate-c3p0 depends upon it. This may not sound like a big deal, but Hibernate is a big project with 14 jar-producing projects (not to mention numerous support projects like documentation and distribution-builders). Having to 'cd' around all those different projects just to compile a dependency graph that Maven already know about seems silly. But again, Maven must do this because of this assumption of project independence

It is possible. If you want to build some specific module that is part of big multi-module release unit without building all other modules, you can tell Maven to build this module and only modules on which it depends omitting others. For example to build module located in test-module subdirectory of release unit, call following command in root of the release unit:

mvn -am -pl test-test install

You can also build module and other modules that depend on it:

mvn -amd -pl test-module install

2. The other issue has to do with the notoriously bad support for performing releases. This has bitten me every time I have tried to do a Hibernate release with Maven using it's release plugin. In fact it just caused a 3 day delay in the Hibernate 3.5.0.Beta-1 release. I know the release plugin is just a wrapper around a few mvn commands you can run yourself (in fact that's how I finally ended the 3 day 3.5.0.Beta-1 release marathon); the issue is again back to Maven's support for multi-project builds. The release plugin is the only current plugin which offers support for managing the numerous version declarations that are required throughout all the sub-projects. So when releasing a project with multiple sub-projects you have 2 options: (1) use the release plugin or (2) (a) update the pom versions manually (b) commit (c) r-tag (d) update the pom versions manually (e) commit (f) checkout the tag (g) perform the mvn deploy. Sure would be nice if the version management stuff from the release plugin were usable separately.

You haven't wrote what problem you have with releasing multi-module builds. I am releasing Maven builds with hundreds of submodules without problem using release plugin. What doesn't work for you?

At the risk of getting into a me too reply stream, me, too, is growing weary of Maven. Multi-module builds always seem to run in to trouble when you try to do anything more complicated than a compile/install.

You haven't wrote what problem you have with releasing multi-module builds. I am releasing Maven builds with hundreds of submodules without problem using release plugin. What doesn't work for you?

I think one of the main problems Maven suffers from in multi-module builds is resolving artifacts from the reactor. Some plugins seem to be able to do it, others don't. Since dep resolution is supposed to be part of the core, I've never understood why this is so. Two open issues against Maven that demonstrate this:

• http://jira.codehaus.org/browse/MNG-4233
• http://jira.codehaus.org/browse/MNG-2979

http://jira.codehaus.org/browse/MNG-4233

Strange, I use dependency plugin in similar scenario and it works fine. I will have to take a look at your test case.

http://jira.codehaus.org/browse/MNG-2979

This is problem with site generation and not releasing. Site generation IMHO is generally broken in Maven and I hesitate to use it. Nevertheless it is hardly a problem, you just call install before site.

This is problem with site generation and not releasing. Site generation IMHO is generally broken in Maven and I hesitate to use it.

Yes, the second report is to do with site generation versus releasing but what I find interesting in these two bug reports is that the problem is almost identical in the two. An artifact that is part of the current build cannot be resolved unless it is in the local repository. Shorten the command to mvn clean install, and all artifacts in the current build can be resolved from the reactor.

Why? I accept this may very well be my misunderstanding of how Maven is supposed to work. In that case, I'll happily put my hands up, accept I've selected the wrong tool for the job and go find the right one for my build. Does it not seem, though, that this should just work?

Nevertheless it is hardly a problem, you just call install before site.

Sure I can, but when you want your continuous build system to execute mvn clean install site, it won't work. Can I work around this in my continuous build system? Yes, if I'm willing to make certain sacrifices. I'll admit to being an awful picky fecker in expecting things to work properly.

It does seem to me that Maven is still struggling to get one of its most basic (and attractive) pieces of functionality, dependency management, right after around 13 releases and it seems to be multi-module projects that suffer the most from it. Hopefully I can help to remedy that situation but in the meantime, I shall be grumpy about it. :)

For example, I cannot 'cd' into the hibernate-c3p0 project directory and run 'mvn compile' and expect Maven to first compile the hibernate-core project for me even though hibernate-c3p0 depends upon it.

You should be able to do this with no problem, as long as c3P0 lists core as a dependency. Problem could arise if you are not using a 'SNAPSHOT' version. Without the snapshot tag, mvn will only check the local repository for the version. 'SNAPSHOT' will force the parent to be built.

This is absolutely not true in my experience. Please point me to the documentation where this is outlined. The way maven works is that when you run compile in hibernate-c3p0 it will look for hibernate-core (its dependency) in repo (local, then remote).

If it is not there you are screwed. Maven will not compile hibernate-core for you.

If hibernate-core is out of date, well again you are screwed. Maven will not compile hibernate-core for you.

Rafał Hajdacki wrote on Aug 20, 2009 09:21:
For example to build module located in test-module subdirectory of release unit, call following command in root of the release unit: mvn -am -pl test-test install You can also build module and other modules that depend on it: mvn -amd -pl test-module install

Neither 'a' nor 'm' nor 'd' are valid options as of 2.0.9. Which is what I have installed here. Are you talking about a later version? Also, I regularly look in the http://www.sonatype.com/documentation/books/maven-defguide to see what might have changed. That documentation says it is up to date as of 2.2.0 and does not mentioned any of this (at least not in its A Multi-module Project which is where it would seem the most apropos. Perhaps you can point me to the documentation of this new feature?

Neither 'a' nor 'm' nor 'd' are valid options as of 2.0.9. Which is what I have installed here. Are you talking about a later version? Also, I regularly look in the http://www.sonatype.com/documentation/books/maven-defguide to see what might have changed. That documentation says it is up to date as of 2.2.0 and does not mentioned any of this (at least not in its 'A Multi-module Project' which is where it would seem the most apropos. Perhaps you can point me to the documentation of this new feature?

But still it's there since 2.1, for 2.0.9 you can use the maven-reactor-plugin to achieve approximately the same..

Here are some links.

With maven 2.0.X you have to use the reactor plugin : http://maven.apache.org/plugins/maven-reactor-plugin/

With maven 2.1+ you can use native command line options : http://www.sonatype.com/people/2009/03/maven-210-released/

I recommend you to use the latest 2.2.1. You also have performance improvements if you have many modules.

I think one major key is to have a real scripting language be the basis for your build environment. Gradle and Buildr are both good candidates. I've used Gant, but neither Gradle or Buildr. I will say this: ruby is a better scripting language. It's truly interpreted and has much better file APIs than groovy. Groovy is very Ant drivn for its file APIs and Ant notation is just awkward in Groovy ex: Ant.mkdir(dir: opt/blah) vs FileUtils.mkdir(opt/blah) in ruby. Also, JRuby has native extensions for all the major platforms, so you can script things that aren't possible in Groovy (like change current dir).

Aranud Héritier wrote on Aug 20, 2009 16:00:
Here are some links. With maven 2.0.X you have to use the reactor plugin : ... With maven 2.1+ you can use native command line options : ... I recommend you to use the latest 2.2.1. You also have performance improvements if you have many modules.

I have downloaded 2.2.1 and am completely unable to get -amd working.

First I tried the capability I am specifically talking about here, so I changed directory into the hibernate-c3p0 directory which defines the hibernate-c3p0 project which has a dependency on hibernate-core. So also make dependencies implies to me that mvn -amd compile would compile hibernate-c3p0 (pwd) as well as hibernate-core (dependency). But alas, that is not the case.

Then I read about this also new -pl option, so I started to look into this. Now I must say that the JIRA tracking these new features says that you should be able to use either groupId:artifactId or the relative path to the project my experience was that I could not at all use the groupId:artifactId syntax (it tried resolving that as a path component). But I did not have much luck with specifying the project directory either.

First I tried from the sub-project: mvn -amd -pl ../core compile which led to :: Couldn't find specified project in module list: /home/steve/projects/hibernate/core/trunk/connection-c3p0/../core

Then I tried from the root project: mvn -amd -pl connection-c3p0 :: This worked in that it found the c3p0 integration module, but it never tries to do anything with hibernate-core

openmind wrote on Aug 20, 2009 11:01:
doesn't matter, people just don't like Maven (anymore) and therefore they are unwilling to search the internet and find out how to do stuff. in 5 years they will start by saying that gradle is rubbish ... being narrow-minded, black/white is the trend nowadays.

This is completely inaccurate. I am regularly on the maven irc channel and regularly look at the definitive guide to see what improvements or changes have occured.

The problem is that Maven has a basic assumption that modules are independent. The reactor plugin and these make like options are simply putting lipstick on that pig...

I think I see the problem... -amd deals with dependants, not dependencies... That's very different then what I describe here.

To me Maven is a total mess and not something I would want to use either for my work or for my hobby projects. I recommend taking a good look at Schmant (no relation to or resemblance to horrible Ant ...) It lets you use any java 6 compatible scripting language to write build scripts. Also unexpected stuff like great documentation.

/ET

I like Maven2! It does what I want. But this is maybe the major problem. Maven2 is convention over configuration which means if you are satisfied with the provided functionality then use it otherwise don't.

Use -am instead of -amd

rm core/src/java/*
cd connection-c3p0
mvn -am compile

This still compiles even though I just nuked all .java source files in core which connection-c3p0 is dependent on.

What is the usecase for this -am and -amd if it doesn't build when classes changes in dependent modules ?

aah ;)

cd / 
mvn -am -pl hibernate-connection compile

does the trick.

Why oh why does I have to do that from the root ? and not just have -pl be implied from where i'm located ?

Why oh why does I have to do that from the root ? and not just have -pl be implied from where i'm located ?

Maven needs to construct and analyze all modules in the reactor to find what it have to build. If you are in a submodule, Maven cannot know that you have others modules in upper directories (perhaps it is a design issuewith actual versions). It cannot construct the reactor from the submodule.

something is just weird here. it doesn't work either.

ok, so that is just a funny limitation. But now that i've cleaned the project mvn -am -pl module goal in the root seem to work pretty nicely.

Thanks! :)

Arnaud Héritier wrote on Aug 21, 2009 13:00:
Why oh why does I have to do that from the root ? and not just have -pl be implied from where i'm located ? Maven needs to construct and analyze all modules in the reactor to find what it have to build. If you are in a submodule, Maven cannot know that you have others modules in upper directories (perhaps it is a design issuewith actual versions). It cannot construct the reactor from the submodule.

Right because underneath the covers all projects in Maven are independent...

Right because underneath the covers all projects in Maven are independent...

Yes because maven relies on the fact that dependencies that aren't in the reactor have to be find in repositories.

If you have better ideas to improve this behavior, you can submit it to maven 3.

We'll be pleased to study it.

Arnaud Héritier wrote on Aug 21, 2009 14:15:
If you have better ideas to improve this behavior, you can submit it to maven 3. We'll be pleased to study it.

I'd submit Gradle as an example of improved behavior in this particular regard. So I'd personally say to look at what they do. I believe that they, just like maven, have the convention that sub-projects (modules) exist one directory down; so conversely, the project aggregating a project as a module (if one) would exist one directory up. I have not studied Gradle innards in depth, so this is just an edumacated guess.

You are right, Steve. This is the behavior of Gradle, or at least the default behavior. If you like, you can map any physical sub-project layout to a logical Gradle project tree. In any case, the subproject is aware of the whole multi-project build. This allows us to provide partial builds. Which solves your use case mentioned above. There is a lot more Gradle has to offer. And Gradle is more and more used in the wild. We have big enterprise projects as users (one gt 3 million LOC with 100 subprojects). Probably there will be some interesting migration announcement soon :)

If you have any further questions, we are more than happy to answer them on our mailing list or by private mail.

--
Hans Dockter
Gradle Project Manager
http://www.gradle.org

The solution to your first issue (hibernate-c3p0) looks pretty obvious to me:

• Add the code of hibernate-c3p0 to hibernate-core
• Put an optional dependency in hibernate-core's pom.xml to c3p0: <dependency>...<optional>true
• Throw about the hibernate-c3p0 directory.

Spring does this trick all the time: spring-remoting has optional dependencies on jms, rmi, hessian, burlap. There are no submodules spring-remoting-jms, spring-remoting-rmi, ... If we want to use the hibernate c3p0 classes, we have no problem adding a c3p0 dependency ourselfes (instead of adding the hibernate-c3p0 dependency).

As for your second issue (release plugin): you're right: its time they clean that up.

Put an optional dependency

NOOOO

Never use optionnal dependencies if possible. I wrote an article few weeks ago in my blog (in french sorry) to explain why it is a really bad practice. To be short, you don't have to ask to your users to know which additionnal dependencies they need to use a piece of your library. If they choose to use hibernate-c3p0 or spring-jms they just have to add this module and it has the responsability to retreive all others required dependencies.

For the issue mentioned by Steve, the best solution today is to launch the build from the reactor with options -am -pl hibernate-c3p0. It will rebuild all projects necessary to build hibernate-c3p0. You can also build with options -amd -pl hibernate-c3p0 to check that your changes don't break modules which are using it.

I opened an issue to be able in the future to launch the build directly from the submodule with -am or -amd options : http://jira.codehaus.org/browse/MNG-4324

Any contribution is welcome :-)

I have to agree with Arnaud here. We actually moved from such a setup you suggest to this because this is a better option in terms of isolating the dependencies.

W.r.t. other tools to help updating versions in the pom....

have you looked at versions-maven-plugin (@ mojo.codehaus.org)

W.r.t. optional dependencies. I also agree with Arnaud

Now that we already put so much time into the maven migration, wouldn't it be possible to customize the maven release plugin to suite our needs? Given the size and complexity of the Hibernate code base we will have problems with any build system. I am sure if we use any of the new fancy tools, it will have its own issues.

As one who has to take a part of hibernate, hibernate-validator, and integrate it into Glassfish, I benefit hugely from your use of Maven. It makes my task easier. Personal convenience aside, however, I agree with Hr. Ferentschik here. Please stick with what you have and fix the problems as they arise.

Stephen Connolly wrote on Aug 26, 2009 20:43:
have you looked at versions-maven-plugin (@ mojo.codehaus.org)

Yes, yes I did. Unfortunately it makes assumptions about the setup of your project that are simply not true of Hibernate. The one that screwed me was that Hibernate splits the notion of parent and aggregator (the one with modules declared).

I often see this practice for Jboss projects to separate the rector from the parent, having a flat old structure a la Maven 1.

It's a good thing to have a separated parent when it has it's own lifecycle. In that case each child has it's own version and often the reactor pom is useless because this child is rarely dependending on the current snapshot of an other child. If the parent and its modules have different versions each of them should have its own trunk/tags/branches in SVN.

If the parent has the same version as all modules, I think it is a really bad practice with maven 2 to have a flat directory structure with the parent which isn't the reactor. Why ?

- Because the reactor cannot extends the parent : this the problem of egg and chicken. The reactor need the parent to be launched. The parent is built by the reactor. - Many plugins (release, site, ...) will be easier to setup if you use the maven convention : The parent is in the directory just upper the module. For example you'll just have to define SCM infos in the parent and not in all modules to use the release plugin.

Some samples taken on Jboss projects I studied recently :

- GateIn MOP (http://anonsvn.jboss.org/repos/gatein/components/mop/trunk/) : GOOD : 1 project with submodules, all have the same version thus the parent is the reactor

- GateIn PC (http://anonsvn.jboss.org/repos/gatein/components/pc/trunk/) : BAD : 1 project with submodules. all have the same version but the parent isn't the reactor. You see many duplicated things in the reactor compared to the parent. The release plugin mustn't work (theyshould define SCM for each module)

- JBoss Portal (http://anonsvn.jboss.org/repos/portal/maven/jboss-portal-parent/trunk/) : GOOD : They have one parent with it's own lifecycle. All modules have their own independent builds and versions (http://anonsvn.jboss.org/repos/portal/modules/)

Arnaud, this was discussed with Maven developers on IRC and in fact was the recommendation of quite a number of them. This illustrates another frustration with Maven; one expert tells you one thing and the next expert chastises you for doing it.

The reason to split is that I can in fact run an install (well several really, which is the pain point) without having to install everything. You need to remember that Hibernate's migration to Maven was over 2 years ago; there were no -am/-amd options back then. And even still with these new options, I argue that the split is natural; aggregation and inheritance are completely different concepts so why is it that Maven sorta/kinda forces you to combine those two concepts into the same file? And actually even worse, if its not going to work correctly why does it allow you to split them?!?

Because we come back to the same point... Maven assumes that these things are all separately version-able. It has no notion of such a prevalent project structure.

I agree and I understand your frustation. There are several design errors in maven 2. The first one I think from my point of view is to mix in the same document and without a clear separation of informations about the project (identification, dependencies, ...) and about its build. Another problem is the difference between what maven's core proposes (reactor and inheritence separation for example) and what plugins support.

Have you seen these blog posts?

http://www.sonatype.com/people/2009/11/maven-3x-paving-the-desire-lines-part-one-2/ http://www.sonatype.com/people/2009/11/maven-3x-paving-the-desire-lines-part-two/

Looks like there's hope :)

Ed Burns wrote on Sep 03, 2009 18:16:
As one who has to take a part of hibernate, hibernate-validator, and integrate it into Glassfish, I benefit hugely from your use of Maven. It makes my task easier. Personal convenience aside, however, I agree with Hr. Ferentschik here. Please stick with what you have and fix the problems as they arise.

See this is another very prevelant misunderstanding. You do not benefit from Hibernate using Maven to build, you benefit from Hibernate being available in a Maven repo. HUUUUGE difference. Gradle, for example, also publishes to Maven repos (after they generate the pom for you, in fact I have to laugh because they actuall upload better poms then Maven itself does).