What do you get from a 10+ year old open source framework, thousands of users within a wide range of roles, and complexity? A JIRA instance with over 3,000 unresolved tickets ;). Is that number indicative of low software quality? Definitely not. But therein lies the problem. A vast majority of the tickets are no longer issues, no longer relevant, or duplicates. Due to the quantity, it became nearly impossible to weed through them all.

So, recently, I've given some effort towards attempting to clean things up. This has been a combination of manual reviews of tickets, scripts identifying possible duplicates [1], and various forms of JIRA queries that attempt to show stale issues. So far, I've reduced the number of un-assigned, un-resolved tickets by almost 1k. To date, only a few of those have turned up being issues that were still legitimate in ORM 4+.

In addition, I've enacted a few policies that have helped (and will continue to help) tremendously, both for contributors and the community as a whole. Most notably, any new BUG ticket that does not have a provided test case is immediately moved into an Awaiting Test Case state. If nothing is received within around 3 months, we've begun to automatically reject them. Although most contributors generally appreciate reproducers that are either standalone or extend one of ORM's pre-existing unit tests (preferably), at least providing enough detail (entities, mappings, code snippets, etc.) is the bare minimum.

Along the same lines, we've been discussing a possible tactic on the mailing list [2]. We'd like to push all ORM 3 tickets to the Awaiting Test Case state and request a reproducer on ORM 4 or 5. This would affect only BUGs with: 1.) affectsVersions set to ORM 3 only (not ORM 4/5) 2.) unassigned and 3.) not updated in the last 90 days. Those would then fall under our policy of automatically rejecting if the ticket does not receive a test case within 3 months or so. Please see the mailing list for the discussion, as well as the query that would be used. If there are any opinions against giving that a shot, please say so.

It's also worth re-iterating that JIRA should not be used instead of the user forums [3] or Q&A sites like StackOverflow. Please create tickets only for concise issues w/ reproducing test cases (or, of course, new features and possible improvements).

If you're interested in more info about the steps I took during the actual cleanup, a somewhat detailed write-up is here:

If you have any questions or ideas, please contact me! brmeyer on Freenode IRC is your best bet, as well as comments on this post. Thanks!


02. Apr 2014, 04:39 CET | Link

IMHO forcing test cases is a huge mistake and a little arrogant. We're all slammed and Hibernate is just one of the many tools we use. Taken the time to write up an issue is a contribution in itself. I don't have time to write up test cases every time I run into an issue. I've seen this policy in place at YUI and all it lead to is real bugs disappearing. Automatically removing JIRA issues will surely lead to a cleaner slate but not necessarily a higher quality project. I would favour a yearly spring cleaning where doubtful issues and duplicates get cleaned out.

02. Apr 2014, 12:38 CET | Link

Hi Marc, thanks I appreciate your reaction, there certainly is a risk in appearing arrogant, and that worries me too.

Still while we all try hard and spend massive overtime to analyse these requests there are still thousands of JIRAs out there which didn't get proper attention, and to get higher quality we need to make sure we focus on the high priority ones first. Sadly to define priority you have to first have an accurate understanding of the described problem, so the details on a JIRA are essential.

Sometimes people are vague, sometimes a reporter is very precise but it's a complex matter for which a practical use case would help me better understand, the culprit is that very often people could add some more information but if asked a week later they'll have probably moved on by having applied some workaround which makes it good enough for them and not worth it to reply any more, or maybe they no longer have the ability to reproduce it.

So having some automation in JIRA to encourage adding a test or a solid description could provide some concrete improvements - provided the request doesn't sound arrogant indeed. Also I agree this should not come as an excuse for us to not keeping to monitor all JIRAs carefully, we will certainly still need to watch for issues not having a test but could still be valid or otherwise need our attention; worst case if nobody has shown interest in an issue in 3 months it's either not worth a high priority or it will show up again, that's the good thing from having millions of downloads. Hopefully this process could save us all some time and that time will be spent on actually fixing the issues :-)

Let's also remember that most of our time spent in this area is volunteers based, for sake of passion for project and love of the community around it, and it's not an exclusive club: anybody could go through JIRA to add tests to any issue, or suggest closing stuff if he knows it's no longer an issue, or even take ownership of something and fix it. That's not exactly our job, still we do it as any volunteer could too.

02. Apr 2014, 12:55 CET | Link

So, yeah, I agree that minimum standards of what constitutes a proper bug report make sense. And if those standards are not met, the bug case should be eliminated. But IMHO that minimum standard should not include the requirement of a test case. I see your point but I've also seen what it can result in. The majority of the bug reports I've submitted are without test cases. And I would venture to say that most of them had some value (some of them certainly were worthy of elimination for the reasons you outlined). If my reports without test case I don't have the time to write will be eliminated, why bother writing them at all? Most likely, this new policy will also lead to you guys ignoring reports without test cases. Anyway, I understand the reasoning behind this idea. I just think it's a bad call.

02. Apr 2014, 14:15 CET | Link

I think it's ok. I miss only one thing: show users good tutorial how to write those test cases. Usually problem arises in complex environment full of EJB/CDI beans which is impossible to recreate in test case without very complex method like Arquillian. Or sometimes it isn't possible to write test. How for example write test for not created index/column, missing Dialect feature etc?

03. Apr 2014, 18:47 CET | Link

> We're all slammed

For what it's worth, I'd argue the same thing, but in favor of requiring a test case. Our individual contributors tend to be extremely busy as well. Reducing the amount of time it takes to analyze an issue, create a test case from scratch, etc. allows us all to be more productive, therefore increasing the amount of bug fixes going into each release and the overall quality of the project.

Also note that when I say we require a test case, that does not necessarily mean attaching a full-blown, standalone, runnable reproducer (although that's certainly preferred). Typically, if enough detail is provided (entities, mappings, business logic code snippets, etc.), I'm a happy camper.

Sanne raises a fantastic point. Often, individuals want to get involved in open source, but aren't ready to dig into the framework itself. Contributing test cases is of tremendous help and a great way to be a part of the community. This is the entire reason behind the Awaiting Test Case state and leaving it there for 3+ months -- allowing anyone to contribute the reproducer. We're not saying we immediately reject a ticket without enough detail.

03. Apr 2014, 18:49 CET | Link

> show users good tutorial how to write those test cases

That's definitely a valid point. By far, the easiest thing to do has always been adding a method to one of our existing unit tests. We try hard to name them appropriately and descriptively, but admittedly they aren't as intuitive as they could be. Adding some info to our CONTRIBUTING file would be great idea. Also, feel free to reach out and ask questions on how to best represent the test. I'm always more than happy to discuss it!

03. Apr 2014, 19:06 CET | Link
  • Typically, if enough detail is provided (entities, mappings, business logic code snippets, etc.),I'm a happy camper.

Ok,that's what I would consider a minimum requirement as well.

03. Apr 2014, 19:10 CET | Link

What has been a showstopper for me even until today is that checking out Hibernate from Github doesn't lead to something that runs in Eclipse out of the box. I don't remember the exact reason but something is always missing (maven-docbook-plugin if memory serves). You try to fix, doesn't work. And before you know it, you're wasting an hour before you even start writing a test case. Anyway, if you can make sure you can just check out and compile in Eclipse AND provide a bootstrap test case (some entities, onetomany, manytoone, ready to run config, etc), with some entities, that would make it a lot easier.

08. Apr 2014, 17:48 CET | Link

I agree with Marc. A while back I attempted to contribute a minor fix (less than ten lines) and I really struggled. It took about thirty minutes to devise the fix, but I ended up spending almost a day trying getting git to work with eclipse to generate the pull request and I totally failed to work out how to build a test case.

My fix got in but I was scared off by the effort and, where I would normally be delighted to contribute, I have since found myself more comfortable working around issues than fixing them.

I suspect that, by making it easier to contribute, a wider community would roll their sleeves up and help out.

The ticket that highlighted this clean up to me (HHH-2796) was reported by someone else but the description makes sense to me and looks like it has more than enough information in it for someone, who has a working Hibernate build environment, to devise a test case relatively simply. Unfortunately I am not that person and I fear that it is likely that a valid ticket will be closed out.

There must be a better way of achieving ticket hygiene, without closing valid tickets.

27. Sep 2014, 09:49 CET | Link


15. Oct 2014, 10:25 CET | Link

Thanks for the great info. Friv <a href="" title="Friv 3">Friv 3</a> 2PG

24. Nov 2014, 03:50 CET | Link
18. Dec 2014, 12:57 CET | Link

18. Dec 2014, 12:58 CET | Link