Help

I'm working in the Hibernate and Infinispan teams at JBoss, caring about Lucene integration in products we support, striving to make it easier to use and to integrate in well known APIs and patterns, and finally to make it scale better; I love clean and well performing code.

I've been an early adopter of cloud deployments scaling Lucene to a huge number of requests on EC2 using Hibernate Search, and after that I worked with Sourcesense to make JIRA clusterable via Infinispan. Have been trainer on Seam and Hibernate courses.

Location: Newcastle, UK
Occupation: Doing stuff at JBoss, a Division of Red Hat Inc
Archive

You can now upgrade to Version 5.0.0.Beta3 of Hibernate Search, and benefit from the following improvements:

Indexing Performance

We did some further polishing of the shiny new backend improvements introduced by last week. I would be really happy to get some feedback on this, as you should be able to get a very significant performance boost on index writing - whatever the storage technology you're using. We're preparing some large scale tests, but the environments we can test on are limited so I'd be happy if you could send us a note on what your experience with it looks like.

The new design should have a significant improvement in throughput, but also requires less locking, needs less threads and will result into less pressure on GC as it has a lower allocation rate.

JDK9 compatibility

We now have continuous integration running for Java 9 (preview builds) running as well. Except the OSGi integration tests running in Apache Karaf, everything else seems to work fine.

API changes

We're now polishing the API, and it's possible that this might be the last Beta. Two very frequently used interfaces were renamed; please don't miss the Migration Guide.

As always, looking forward for your experience with it! ideas and suggestions on the mailing list or via IRC.

Sanne

Version 5.0.0.Beta1 of Hibernate Search is now available for download!

This summer we've been slower than expected, and based on your feedback we decided that some features that we wished to have in the 5.0 release are going to be moved to future 5.x releases: the current codebase supports latest Apache Lucene 4 since a while and we believe it's in excellent shape already, so we're aiming now at a fast feedback loop to release a 5.0.0.Final in 4-6 weeks.

So in the following weeks we're going to aim at:

  • last needed future-proofing API changes to make it possible to introduce the postponed features without (hopefully) affecting public API down the road of the 5.x cycle,
  • polish documentation
  • react on your feedback!

Noteworthy changes

Those looking at our detailed changelogs might have seen a log of small changes, apparently not too interesting.. that's because we've been working closely with the Infinispan team and some great improvements are being introduced disguised as a simple micro version update of our dependencies.

Significant performance improvements: Async backend

The new async backend is various orders of magnitude more efficient than the previous one! If you are an async user, you will love the new design. Also, while the previous implementation was simply async and wouldn't give any timing guarantees, now we expose a new configuration property index_flush_interval which could make it possible for many more systems to use the async backend: you can decide to have it flush for example every 5 minutes, so to not have a big performance penalty on writes and guarantee that new items are visible in approximately 5 minutes.

The performance benefit of this backend is extremely high, and we're looking forward to apply the same principles for the sync backend too; in case you're interested, the design is described at HSEARCH-1699.

Infinispan integration improvements

With the Infinispan team we've worked on several areas:

Infinispan Directory

Was updated to latest Apache Lucene 4.10 and it is now significantly more reliable: we could finally resolve all known issues and added lots of new tests, and a lot of work on performance was done as well.

Infinispan Query

Infinispan includes a Query module which exposes Hibernate Search 5 capabilities to all Infinispan users: if you know how to use Hibernate Search with a regular Hibernate ORM application (JPA), it will be very easy for you to learn how to use the full-text capabilities of Infinispan Query. Of course this last release 5.0.0.Beta1 is included in Infinispan 7.

WildFly modules for Infinispan integration

The Infinispan project now also provides a nice to use pre-packaged zip file with all modules you might need to copy into WildFly 8, and we've been working together to make sure that Hibernate Search can use these latest modules rather than the ones included by the container: that would allow you to immediately benefit from all new features, and generally speaking not be constrained by the versions of Infinispan that the container might be having at any point in time. This subject is still work in progress, in particular better documentation is needed but essentially you just need to download and unzip the modules package from both projects.

Debugging capabilities

Apache Lucene has the notion of an InfoStream, which outputs lots of detailed information about index writing to the console. Via HSEARCH-1658 you can now configure Hibernate Search to capture all those details and forward it to your logging stream: see also Troubleshooting: enable Lucene’s Infostream in the Lucene configuration chapter of the documentation.

Critical issue resolved in the Avro based serializer

Early alpha versions of Hibernate Search 5 were affected by a critical threadsafety issue in the code managing the Serializer service based on Apache Avro, making it essentially unusable except in trivial unit tests. This is now resolved, and does not affect any (older) stable release.

Avro protocol: version bumped

The serializer backend based on Avro is now able to transmit Flush commands as well; this is mostly to allow transparent improvements but as a user you need to keep in mind that you'd need to upgrade a master node first, as older master nodes will not be able to decode messages sent by this new version of Hibernate Search.

Evolution of the @ContainedIn annotation

This annotation used to be simply a counter-part of @IndexedEmbedded, but as discussed on HSEARCH-1580 it is useful as well to define any other form of indexing dependency. This did work in the past, but was unintentional and therefore broken during recent development, so now the functionality is restored and we're making it an intentionally supported feature.

Thanks Guillaume Smet, for both bringing it up and providing the patches!

Component upgrades

This version was built and tested against Hibernate ORM 4.3.7.Final, JGroups 3.6.0.Final, Infinispan 7.0.0.CR2, WildFly 8.1.0.Final, Apache Lucene 4.10.1.

Feedback needed!

Please try it out and let us know if you have any trouble with it, or if any part of the documentation is not clear enough. As always, we'll be monitoring the Forums, the Mailing Lists, and StackOverflow if you use tag hibernate-search.

While the team is busy on significant internal refactoring, we also accumulated 30 minor fixes and improvements which have been merged in the master branch for Hibernate Search 5.

Since it was a while since the last published tag, today we're publishing this 5.0.0.Alpha5 release to make all these minor improvements available already, while the cooler features will need some more work.

New attribute: includeEmbeddedObjectId

This new attribute of the IndexedEmbedded annotation allows you to control if you want to store all ids for related objects which are embedded in the parent's Lucene Document. Very useful to save space in your index, and so improve performance, when you don't need these identifiers.

Other changes

  • As documented in the Migration guide the API to implement a JMS master node was simplified (but changed!).
  • A possible loss of index update events was fixed, but not a critical issue as this could happen only if you had a significant load and were not using transactions.
  • Better interaction with second level caching.
  • Performance fixes: when the DocumentId doesn't match the JPA id we now still execute an optimal database query.

Full details available in the Release notes.

Today we released yet another milestone for the Hibernate Search 5 train. We worked in parallel on multiple fronts; the most notable changes are:

OSGi support, API changes

All our modules now provide OSGi metadata to make life easier for users running in OSGi containers. We also included an example features file and integration tests using Karaf.

Keep in mind that Apache Lucene is not providing this same metadata, so it might be worth looking into the features file we use to learn how the Lucene modules need to be wrapped.

Class Relocations

A consequence of being OSGi compliant is that we had to move some packages of well-known APIs; please see the migration guide for all details.

Apache Lucene 4.8.1, Java7 now required

Apache Lucene requires Java7 since version 4.8 and we don't want you to miss out some of the great improvements it provides, or potential bugfixes in the near future so we now require Java7 too.

Apache Lucene 4.8.1 was released today, so we could include it in this release too.

Bridge Providers loaded by auto-discovery

We always had a strong differentiation between FieldBridge(s) included in Hibernate Search, and custom (application provided) FieldBridges. From this release the discovery of built-in bridges uses the Service Loader pattern, so that we can move some bridge implementations to optional modules, and also eventually provide support for the new date/time types defined in Java8 but also by Joda Time, and potentially your own custom types but this will need some further refinement work.

Several other improvements

These won't make the headlines as a Java requirements change, still we have some more relevant news:

  • Infinispan upgraded to 7.0.0.Alpha4: now also requires Java7 and supports the distributed Lucene Directory for Apache Lucene 4.8
  • the needed Infinispan update implies using latest JGroups 3.5.0.Beta5
  • all our documentation was migrated to AsciiDoc , it's now much easier to contribute to documentation!
<dependency>
 <groupId>org.hibernate</groupId>
 <artifactId>hibernate-search-orm</artifactId>
 <version>5.0.0.Alpha4</version>
</dependency>

Last night I uploaded two bugfix releases of Hibernate Search stable branches:

  • 4.4.3.Final (Hibernate ORM 4.2 and JPA 2.0 users, JBoss 7.2 and EAP6)
  • 4.5.1.Final (Hibernate ORM 4.3 and JPA 2.1 users, WildFly 8)

They both contain several backported fixes, thanks to the excellent testing efforst of Guillaume Smet and Yoann Rodiere, who found very sophisticated issues and also helped with patches. I now added Yoann as committer too, congratulations!

Details of fixes can be found in the 4.4.3.Final changelog.txt and 4.5.1.Final changelog.txt.

Happy searching!

Showing 1 to 5 of 44 blog entries