The first beta of Hibernate Search 3.3 is out. We had several goals in mind.

One of them is to morph the project into a more independent piece of software for Infinispan. We have a lot of exciting developments around Infinispan, search and persistence. But that's the subject of another post. On to the meat now.

Hibernate Search query DSL

Probably the most visible feature is the new Hibernate Search query DSL.

Writing Lucene queries is not easy, either you use the query parser limiting yourself to fairly simple queries and manipulating strings or you use the Lucene programmatic query API which is quite verbose and contains a myriad of settings and alternatives.

On top of that, you need to make sure you apply the same magic at query and indexing time: if you don't the index key you look for will not match and you will return no result. This is particularly true in an object world where two transformations occur:

• the object is transformed in a string via the Hibernate Search FieldBridge
• the string is transformed into terms via the analyzer

Each property can have different combinations of field bridge and analyzer.

Hibernate Search solves these problems by transparently applying the appropriate FieldBridge and analyzer of a given researched property. It's also built around a fluent API to make queries very easy to write and more importantly easier to read.

QueryBuilder mythQB = searchFactory.buildQueryBuilder().forEntity( Myth.class ).get();

//look for popular modern myths that are not urban
Date twentiethCentury = ...;
Query luceneQuery = mythQB
    .bool()
      .must( mythQB.keyword().onField("description_stem").matching("urban").createQuery() )
        .not()
      .must( mythQB
        .range()
        .onField("starred")
        .from(3).excludeLimit()
        .to(5)
        .createQuery() )
      .must( mythQB
        .range()
        .onField("creationDate")
        .above(twentiethCentury)
        .createQuery() )
    .createQuery();

This example shows many things:

• the fluent API in action (you've got to admit that it's more readable than a raw Lucene query)
• you pass objects and not string representations (Date and number in this case)
• description_stem uses a stemming analyzer (eg. transforming loving in its root word love): no need to apply it yourself before passing the matching string, the query DSL does that for you.

I will blog in more details about Hibernate Seach query DSL shortly.

Hibernate Core 3.6

This release is compatible with Hibernate Core 3.6 (in Beta3 at the time of writing). A side effect is that manual configuration of the event listeners is no longer necessary even when only using hbm.xml files.

Statistics

Hardy has been busy designing a statistics API (available from the SearchFactory). It gives you a lot of information about Hibernate Search:

• average and max time for a Lucene query execution
• average and max time for the object loading process following a Lucene query execution
• slowest query
• number of entities indexed of a given type
• and many more

Again, a more detailed blog post should come soon.

Integration tests for JTA and Spring

We have added integration tests for both Spring Framework and JTA. On the JTA side, we are testing against Bitronix and JBoss Transactions standalone.

Mutable SearchFactory

While not a public feature yet, Hibernate Search now has the ability to add new entity types on the fly.

From the ground up, we have made sure that Hibernate Search is extremely fast and efficient at runtime. To achieve that we have been using an immutable design for the SearchFactory: we pre-compute and store metadata to make indexing and querying efficient. and that forced us to know the list of indexed entity types ahead of time.

Infinispan, however, does not know necessarily knows the list of entities ahead of time. The new design uses a copy-on-change approach to keep the benefits of the immutable model while offering the ability to add new entities. As a user, you won't notice it but as a framework using Hibernate Search, you will :)

Bug fixes

Of course we also fix bugs :)

Check out the new release on JBoss.org's Maven repository or download the distribution. You can also read the documentation here. Be aware that this version breaks a couple of SPIs, make sure to check the migration guide.

Gunnar asked me an interesting question on Bean Validation. Instead of keeping the knowledge private, I thought it would be useful to share more wildly.

Is it possible to determine whether a given constraint is specified at field or property level using the constraint metadata API?

First off, in many case you don't need to do the distinction. Had Java support properties from the ground up, we would not have this problem (sigh).

Anyways, the answer is yes. You can fine tune what is returned from the metadata API.

PropertyDescriptor property = 
    validator.getConstraintsForClass(Address.class)
               .getConstraintsForProperty("street1");

Set<ConstraintDescriptor<?>> fieldConstraints =
    property
        .findConstraints()
            .lookingAt(Scope.LOCAL_ELEMENT)
            .declaredOn(ElementType.FIELD)
            .getConstraintDescriptors();

Set<ConstraintDescriptor<?>> propertyConstraints =
    property
        .findConstraints()
            .lookingAt(Scope.LOCAL_ELEMENT)
            .declaredOn(ElementType.METHOD)
            .getConstraintDescriptors();

The key here is the use of the findConstraints() fluent API. You have three ways to restrict the metadata retrieved:

• declaredOn(ElementType... types): defines where to look the constraints (METHOD, FIELD etc)
• lookingAt(Scope scope): defines whether to look for constraints hosted on superclass/interfaces (default) or not
• unorderedAndMatchingGroups(Class<?>... groups): restrict to the constraints matching a given set of groups for this element (note that the ordering of group sequences is not respected)

That's all.

While working on Hibernate Search 3.3, we have discovered a critical issue with Hibernate 3.2. If you use Hibernate Core 3.5 in a JTA environment (recommended), the way Hibernate Search 3.2 registers itself can lead to inconsistent indexing and generate assertion failures. All this is fixed in Hibernate 3.2.1 which you can get here and ported to trunk as well. We have also added tests to cover the JTA area.

We highly recommend people using Hibernate Search 3.2.0 to migrate to 3.2.1.

For more information on this bug and others fixed in 3.2.1, check out the changelog.

Many thanks to Tom Waterhouse for helping us all along the discovery, fix and testing process.

Hibernate Search 3.2 has been in development close to a year and now we are releasing it :) Instead of giving you a list of new features, let me highlight a couple of use cases we now cover:

Defining index mappings depending on customer / deployment

The primary way to express index mappings in Hibernate Search is via annotations. This works 95% of the time, but in some cases you want to adjust what gets indexed and how in a more fine grained way. For example:

• you deliver the same application to different customers and want to give them the opportunity to configure some of the available indexed properties
• you deploy the same domain model in different apps where each needs specific search capabilities

To achieve this, we have introduced an easy to use, easy to read fluent programmatic API to express index mappings. Check out the programmatic API in the reference documentation or this blog entry.

Index/reindex my data easy and fast

In Hibernate Search, there has been a couple of best practices to initially index your data. You needed to read your data from the database in batch, call the index operation, flush and clear the session and move on to the next batch.

Forget that now. We have a super easy API to index or reindex your data (as simple as two method calls). You can also configure how indexing is done via an intuitive fluent API (yes we've catched the fluent API virus and you haven't seen the end of it). Not only is the new approach easy to use but it's also massively faster than the previously recommended best practices. We highly recommend people to move to this new approach.

Check out the reference documentation or this blog entry for more info.

I can't use JMS, but I need index clustering

Let me first state that setting up a JMS queue is super simple and trivial in any of the modern application servers ( esp JBoss AS :) ) and you get tons of benefits from it (reliability etc). Of course, if you like to waste time and build your own stack on top of Tomcat or equivalent, too bad for you.

Anyways, you can now use an alternative approach to JMS for clustering. We now support raw JGroups communications between cluster members.

My sysadmin needs a way to see what indexing operation have failed and restart them

Luckily that does not happen often but when indexing failures happen, we need to do something about them.

You now have access to an API to listen to indexing errors and process them as you please. The default implementation logs the error but you can easily decide to push the errors to some queue for display or replay, send a message via SNMP etc etc. The actual error is provided as well as the list of entities that should have been processed (quite handy for replay).

I have a single instance updating the index, can I make it faster?

Yes, if a single instance of Hibernate Search is responsible to update the index, we can speed up things. Simply add

hibernate.search.[default|index name].exclusive_index_use true

What else?

Hibernate Search 3.2 runs on Hibernate Core 3.5 and JPA 2. And as always we did many more things for this release including various optimizations, bug fixes, simplifying the Hibernate Search settings (especially for dependencies), adding a simpler API for bridges.

Check out on the web site, download Hibernate Search or browse the reference documentation. We also have a migration guide from earlier versions of Hibernate Search.

PS: For the Maven users, JBoss has migrated to a new maven repository. Read this user guide to know more.

PPS: We are already on Hibernate Search 3.3. Stay tuned.

I am happy to announce the release of Hibernate Search 3.2 CR1. Crossing fingers, this is the latest release before the final version targeted in a few days.

A good 75% of our time has been spent on bug fixes fresh and old (some even fossilized). But we have also added a few interesting new features:

• we have polished the one we introduced in 3.2 Beta1
• we moved to Lucene 2.9 APIs as a first step towards Lucene 3.0's migration and we also have upgraded to Solr 1.4 for the declarative analyzer framework
• Amin, Sanne and I have been working on a new API to catch and process indexing errors: the default implementation logs the errors but you can write your own custom callback. You could log the failing indexing process to a DB, send a message to a sysadmin, queue the issues for automatic reprocessing etc).
• Hibernate Search 3.2 targets Hibernate Core 3.5 and use some of the new APIs
• a simpler API is at your disposal to add fields to a Lucene document in your custom bridges (thanks Sanne!)

You can download the release from sourceforce or our maven repository and read the documentation. Try it out!

Many thanks for the bug reports / feature requests you have send us: they helped polish this release. Atop the usual suspects, I would like to thank Gustavo Nalle Fernandez and Amin Mohammed-Coleman for their contribution.