Red Hat

In Relation To Java EE

In Relation To Java EE

Free support

Posted by    |       |    Tagged as Java EE

Recently, we've started to hear complaints that we don't put enough effort into free support in the Hibernate forums. This really kinda hurts, since everyone /used/ to comment that we gave such /great/ support, and since I still spend hours most days reading and responding to forum posts. I don't get paid for this, and I rarely get thanked for it either (even by the people I do get time to respond to). I've been doing this for almost three years now.

Now, these complaints, I suppose, are mostly from lazy people who can't be bothered solving their own problems, and come to us before actually reading the documentation and FAQ thoroughly, searching the forum for previous posts, and/or trying to isolate the problem and step through their code with a debugger. I highly doubt that any of the complainers have ever been on the other side of the fence: actually /giving/ support for free. Personally, I don't think we have much responsibility to help these people, but I guess it's sometimes hard to tell who is who...

I would like to believe that the nice people who actually /deserve/ free support do still get it. But I freely admit that support is not up to the standard it was at two years ago. Why?

Well, the problem here is that free support is fundamentally unscalable, and a lot of people don't seem to realize this. So let me give you some idea of the actual numbers. When Hibernate had a couple of hundred users, and only a few posts per day in the forum, it was possible to give extended responses to each question. However, the Hibernate forum got an average of 120 posts per day over the past year, including weekends (that means over 200 on many days). And that doesn't even include JIRA issues. I estimate (conservatively) that it takes about 15 minutes on average to understand and respond to any post. That means 210 hours per week of work responding to forum posts. There are maybe 5 or 6 members of the team who are active on the forums. Do the division yourself (35-42 hours per person per week, something like that). That means, for every one of us, answering questions is a /full time job/ that we don't get paid for. We would have to spend as much time in the forum as /you spend at work/. And then, somehow, we have to do our actual jobs. Now, that's ok for me, I'm used to working 12 - 16 hour days. But I'm certainly not going to expect that of the guys who help out /purely as volunteers/!

So, free support was a casualty of the success of the project. Is that a reason to not use Hibernate? I guess it's a reason to not use any successful open source project. But we want our project to scale even further! So, we've been trying to think outside the box.

OK, I confess: I am an opensource zealot. I want OSS to gradually replace commercial software in most fields, starting with middleware. (I have both moral and practical reasons for wanting this.) And I'm serious about seeing how that can actually become /real/. Now, one of the selling features of OSS is the rapid, free support that supposedly exists in free software communities. I've often been sceptical of this particular item, and my own personal experience is that it's been oversold. Certainly, free support is a wonderful thing, but making it work is Hard. We certainly /want/ to make it work!

First, we realized that a book was needed. We're really putting a lot of hope into this taking some of the pressure off. It was a really draining effort that took a lot of energy away from other areas of the Hibernate project, but it's done now and available at Manning's website. Having a userbase that has read the book, knows the basics, and knows our private language will make giving free support just so much easier!

Second, we've been (successfully, so far) building a business around Hibernate. Our dev support customers can be guaranteed responses to their questions. So, if you can't be bothered to RTFM, that's fine - just buy support, and we'll be at your service, for the most basic of questions, and the most arcane.

But now comes the interesting thing: the more paying customers we have, the more people we can get working full time on Hibernate, and so the more forum posts we can handle. Conversely, the better we handle free support in the forums, the bigger our user base grows, and the more paying customers we see. This demonstrates how free support and commercial support are /complementary/ - it's not zero sum, by any means.

And I think this is going to be common to any open source project that really wants to win against it's commercial competitors: it simply must have some commercial aspect to it, to level the playing field. We're continuing to win only because we have JBoss behind us now. A year ago, I had to take annual leave and beg or pay my own way if I wanted to speak at conferences. Meanwhile, our users were on their own if they had a problem too complex to be addressed online. Now, we can actually get out there, in the field, in front of people!

Finally, we realized that the only way that free support can scale is if the Hibernate community really starts to pitch in and help answer questions. I guess we got off on the wrong foot here, when I used to answer all questions personally. Christian told me to stop answering as many questions, to get people used to the idea of helping each other out.

Our honeymoon as a cult project is long over. We've moved into the space where other very successful open source projects like Struts or JBoss found themselves long ago: our user base no longer feels a personal connection to the developers of the project, and is much less likely to be forgiving of our wrinkles. We now start to get many developers who use Hibernate not by their own choice, but because someone else made the choice for them. We also start to see developers forced to use Hibernate where ORM is /not/ appropriate. All these things mean we start to get more negative feedback than before. We get lots of people who expect Hibernate to be perfect - especially less experienced developers who have no real appreciation of just how hard Java object/relational persistence was before solutions like Hibernate came along.

That's all very distressing for those of us who are putting our life into the project, but we need to take it for what it is: a measure of our success. We'll keep innovating regardless...

The Scope of Hibernate Three

Posted by    |       |    Tagged as Java EE

After more than a year of activity, development of the Hibernate2 branch has finally been wound up; Hibernate 2.1.3 will be one of the last releases and represents a rock-solid POJO persistence solution with essentially all the functionality needed by a typical Java application. Any future release of Hibernate 2.1 will contain only bugfixes. The branch that we have been calling 2.2, will actually be released as version 3.

The Hibernate project has previously had quite circumscribed goals - we have limited ourselves to thinking about just the very next release, and just what our users are asking for right now. That approach has finally reached the end of its usefulness. Hibernate3 is concieved in hubris, with the goal of innovating beyond what our users are asking for, or have even thought of. Certainly we will be adding features that go well beyond the functionality of the best commercial ORM solutions such as TopLink. (Some of these features will be most interesting to people working with certain specialized kinds of problems.)

So, in this spirit of hubris, I'll drop my normal policy of not boasting about things we have not yet implemented, and give a sketch of some of the things that are planned for Hibernate3 alpha (as usual, we do not commit to dates for future releases).

Virtualization

A number of important and interesting problems may be solved by presenting the user with a certain, filtered, subset of the data. For example, a user might want to see data valid at a particular point in time, or may have permissions to view data only in a particular region. Rather than forcing application code to specify this filter criteria in tedious query conditions, Hibernate 3 will allow the (parametrized) filters to be applied at the session level.

(This feature has not yet been implemented, though some of the needed refactorings are currently sitting on my laptop.)

More Mapping Flexibility

For the overwhelming majority of cases, Hibernate2 provides all the O/R mapping options you will need. However, there are now a few new features that are powerful, if used judiciously.

  • single-class-to-multiple-table mappings using <join>
  • table-per-concrete-class-mappings using <union-subclass>
  • flexible discriminators using SQL formula mappings

We have long argued favor of extremely fine grained classes, and have viewed multi-table mappings as a huge misfeature (except for the very important special case of table-per-subclass inheritance mappings). Unfortunately, several commercial vendors insist upon trying to spin this misfeature as an actual competitive advantage of their products, causing us regular irritation. So we have provided the <join> mapping mainly to shut them up.

Well, I do conceed that there is one great use-case for <join>. We can now mix together table-per-hierarchy and table-per-subclass mappings in the same hierarchy. For example:

<class name="Document" table="DOCUMENTS">
    <id name="id">...</id>
    <discriminator column="TYPE" type="string"/>
    ...
    <subclass name="Book" discriminator-value="BOOK"/>
    <subclass name="Newspaper" discriminator-value="NEWSPAPER"/>
    <subclass name="XML"  discriminator-value="XML">
        <join table="XML_DOCUMENTS">
            ....
        </join>
    </subclass>
</class>

Hibernate's implicit polymorphism is a nice way to achieve most of the functionality of table-per-concrete-class without placing unnatural restrictions upon column types of different tables. It also nicely allows table-per-concrete-class to be mixed with other inheritance mapping strategies in the same hierarchy. Unfortunately, it has two limitations:

  • we cannot have a polymorphic collection of the superclass type
  • queries against the superclass resolve to muktiple SQL queries, which could be slower than using an SQL UNION.

The new <union-subclass> construct provides an explicit way to map classes to a table-per-concrete-class model and is implemented using SQL UNIONs.

If your table-per-class-hierarchy mapping does not feature a nice simple discriminator column, where values map one-to-one to the different classes, formula discriminators are for you. You can now discriminate between subclasses using multiple columns, multiple values of the same column, arbitrary SQL expressions or functions, even subqueries! An example:

<class name="Document" table="DOCUMENTS">
    <id name="id">...</id>
    <discriminator type="string"
        formula="case when TYPE='MONOGRAPH' then 'BOOK' when type='TABLOID'\
        or type='BROADSHEET' then 'NEWSPAPER' else TYPE end"/>
    <property name="type" column="TYPE"/>
    ....
    <subclass name="Book" discriminator-value="BOOK"/>
    <subclass name="Newspaper" discriminator-value="NEWSPAPER"/>
</class>

All of these features have been implemented in the Hibernate3 branch.

Representation Independence

Hibernate was conceived as a persistence solution for POJO domain models, and that remains the focus. Occasionally, we run into people who would like to represent their persistent entities in some more dynamic way, as a map, essentially. We used to point them to OFBiz Entity Engine. Well, Hibernate3 lets you represent your domain model as trees of HashMaps, or, with a little bit of user-written code, as just about anything. Here are three immediate applications of this new feature:

  • reimplement JBoss CMP 2.1 engine on top of Hibernate
  • reimplement OFBiz EntityEngine using Hibernate
  • natively persist SDOs

Indeed, the end goal is to allow the application to switch between whichever representation is appropriate to the task at hand; the same entity might be represented as a typesafe POJO, a Map, or an SDO, all with just a single Hibernate mapping.

This feature has been implemented in the Hibernate3 branch.

JDK 1.5 Support

JSR 175 annotations are a perfect fit for Hibernate metadata and we will embrace them aggressively. Emmanuel Bernard is working on this.

We will also need to support Java generics, which basically boils down to allowing typesafe collections (which is very trivial).

Stored Procedure Support

We are not enormous fans of using stored procedures for CRUD operations (of course, there are some other use cases where SPs make wonderful sense) but people working with legacy databases often need Hibernate to call a SP instead of generating its own SQL statement. In Hibernate 2.1, it is possible to achieve this with a custom persister. Hibernate3 will allow arbitrary SQL statements to be specified in the mapping document for create, update and delete. Max Andersen is currently implementing this feature.

Full Event Driven Design

In the early days of the project, I applied YAGNI aggressively. Hibernate was occasionally criticized for supposed architectural deficiencies, due to the absence of a clear upfront design. (In my view, this lack of upfront design was not actually an architectural problem - at least by my understanding of the word architecture - but that debate is for another day...) Anyway, it was my firm belief that an elegant and flexible design would eventually grow naturally, and that our scarce attention and development resources were better spent solving actual user-visible problems. I now feel quite vindicated in this decision; Hibernate has flourished, and the design that has emerged is both powerful and reasonably elegant. Take that, YAGNI-skeptics!

There is one final step of this process, that is now in the hands of Steve Ebersole (and already partially implemented). Hibernate3 will feature an event-oriented design, with event objects representing any interesting thing that occurs, and listener classes that implement standard Hibernate behaviours, or customized user-defined behaviors. This user extension capability has applications ranging for auditing to implementing strange cascade semantics. (Our previous attempt to solve these problems - the Interceptor interface - proved to be insufficient.) Importantly, the redesign simplifies the current monolithic SessionImpl class.

New AST-driven Query Parser

Well, actually /two/ final steps. (No-one expected the Spanish inquisition, did they?)

When I wrote the HQL parser, I knew zip about parsers. Hibernate ended up with a wierd-ass handwritten pure-look-behind parser which, to my great and enduring surprise, has actually served us very well and been the source of very few bugs. (YAGNI, once again.) But it is now well past time we had a proper ANTLR-grammar-based AST, and Joshua Davis is currently writing one. This probably won't mean much in the way of user-visible changes, but it should allow us to more easily support other query languages such as EJBQL.

Declarative Session Management

This is a JBoss-specific feature for now. New users often find session management to be tricky at first. It can take a bit of thought to get your head around the schizophrenic role played by the Hibernate session as both a kind of cache, and a kind of connection. A JDBC Connection is stateless; a Hibernate Session, on the other hand, is a stateful connection! We further confuse the issue by telling people that, actually, the session is really representing a kind of transaction!

Rather than force people to implement their own session handling, messing with threadlocals and exception handling, it would be nice to have some way to specify the session model (session-per-database-transaction, or session-per-application-transaction) declaratively, in the metadata for a session bean. The Spring Framework already provides some support for this kind of approach. Michael Gloegl is working on implementing this functionality using a JBoss Interceptor. It would be wonderful to have this facility in appservers other than JBoss. J2EE needs portable Interceptors for EJBs (they already exist for servlets).

Well, I have some more things on my list, but that will do for now!

Books for free...

Posted by    |       |    Tagged as Java EE

This is a reply on the Hibernate forum, read the full thread for the context.

As some noticed, I'm a bit impulsive about that kind of thing right now. We get one question a day, sometimes friendly, sometimes not so friendly, if we could just set up the book (Hibernate in Action) for free download or simply send it over.

Don't take it personal if I'm getting annoyed about that. It's sometimes just embarrassing when someone gives you the feeling that you owe them something, just because they use Hibernate. I'd like to explain why its not about the money for the book.

First, and that is really my own personal opinion, a professional Java developer should have enough money for whatever book in any case. Books are important, read at least one each month, better two. I don't buy any of that offshore jobs talk, but knowledge is the only thing that keeps you in business. I don't want to discuss $5 Amazon shipping costs about that.

Now I'd like to start a little rant here, so you better stop reading if you are not in the mood :)

Our software is free, our documentation is of course free, and we will always provide free support on a personal level in this forum, and whenever we think it is needed.

I also think that our software and the whole project shows that we are serious about what we are doing. We worked many nights and weekends last year. Gavin and me quit our jobs to make this possible. Max, David, Steve, Emmanuel spend hours a day after work, writing interesting things and answering questions (now up to 120/day) on the forum.

We really believe in professional open source as a business model in the software sector. Actually I'm now also employed by JBoss Group. We have two people working fulltime on Hibernate.

How do we earn our salary?

With open source, there are no license fees. This is great for developers, as they can sell the software they'd like to use to their project managers and accountants easily. Well, of course only if all the other qualities are right:

  • good software
  • good documentation and free support
  • commercial support (risk management)
  • nice extras

A successful vendor of closed software can provide the second for free, but you have to pay for the other three most of the time. Sometimes its hidden, sometimes not. The situation is quite different with an open source software project. First, there are no hidden costs, because you actually can't hide anything. Second, we as software developers don't like hiding any costs somewhere, we are not good with that kind of thing. It's much easier to tell people what they need and what it costs upfront. I also like the look on peoples faces if you admit that Hibernate solves only 95% of their problems, not 100! Shocking! The truth!

It's hard to break with the traditions and I remember the early Linux days when it was first used used commercially. You actually had to tell people that free software is good because they don't pay money for licenses. They wouldn't believe you, just don't ask for a (real) reason. Usually they expected some hidden costs in the other three items and simply denied that it was any good at all.

Sometimes, I think it was too much propaganda back then, now everyone expects everything for free!

To support a business, some revenue must be made from at least one of the four elements I listed.

I'm not talking about the book, because that really doesn't pay. I think most people know that books don't pay in the end, at least if your name is not Fowler or Bloch. Why do we write it then? It helps us to get more people (and therefore ideas and opinions) into Hibernate, and we can finally write down some thoughts we can't express in any other medium.

And yes, we also hope that it will help us to grow the Hibernate business. Thats our job. There is no professional quality without that background, or at least not for a long period of time.

Hibernate is naturally a very open project driven by the users. This will not change, no matter what our business model is. We think that listening to users and balancing requests is the heart of the project, it is the reason why people like it and why it works(tm).

back to top