Red Hat



In Relation To Sanne Grinovero

In Relation To Sanne Grinovero

Version 5.5.2.Final is now available, our latest stable version sporting integration with Hibernate ORM 5 and Apache Lucene 5.3 - the state of the art.

Creating this version to be compatible with these two great OSS projects kept us busy for a good deal of this past year; I remember discussing this option with superstar OSS contributors Uwe Schindler (Apache Lucene developer) and Gustavo Nalle (Infinispan developer) at FOSDEM in January 2015! I am grateful to both for their guidance and suggestions, as driving progress forward is sometimes challenging when we strive to keep backwards compatibility as best as we can.

On top of that, our same small but amazing team as been working hard on Hibernate OGM 5, a bit of Hibernate Validator, incredible performance improvements on Hibernate ORM "classic" and is still tinkering on the internals of Hibernate Search to make an Elasticsearch backend an alternative to plain Lucene.

Feel like helping with the Elasticsearch integration?

Version 5.6 will feature an experimental integration with Elasticsearch. Early feedback would be very welcome! If you feel like helping, quite some code is integrated in the master branch already and there are clear TODO and FIXME comments to be found in the code.

Feel free to join the hacking! But best to let us know which area you feel like working on to avoid duplication of efforts and conflicting patches.

The 5.5.2.Final release and our great community

Technically I pushed the release button the 24th, but I couldn’t publish the blog until now because of travels.

On top of the improvements from previous 5.5.1.Final release which sported significant performance improvements, this version now incorporates several important corrections by Yoann Rodière and Guillaume Smet around metadata which was producing incorrect range queries for embedded numeric types.

Yoann also contributed another great performance improving patch, by limiting the recursion triggered by @ContainedIn more strictly, by considering the indexed paths and not just the recursion depth. You might not notice this if you have very simple flat models, but this can provide a very significant boost for all those of you using Hibernate Search to index rich models with extensive usage of @ContainedIn annotations.

Many thanks to Yoann and Guillaume, they are "power users" who do the right thing: their hands-on regular feedback is invaluable, not to mention they send such great patches for everybody else to benefit from. Not least, this is very encouraging for us: let us know which great things you’re building with these libraries!

<dependency>
   <groupId>org.hibernate</groupId>
   <artifactId>hibernate-search-orm</artifactId>
   <version>5.5.2.Final</version>
</dependency>
<dependency>
   <groupId>org.hibernate</groupId>
   <artifactId>hibernate-core</artifactId>
   <version>5.0.6.Final</version>
</dependency>
<dependency>
   <groupId>org.apache.lucene</groupId>
   <artifactId>lucene-core</artifactId>
   <version>5.3.1</version>
</dependency>

How to get this release

Everything you need is available on Hibernate Search’s web site. Download the full distribution from here, or get it from Maven Central using the above coordinates, and don’t hesitate to reach us in our forums or mailing lists.

Hibernate Search 5.5.1.Final is now available!

Feedback about our recent 5.5.0.Final release has been great, and while the good news is that nobody reported significant issues, some people also pointed out that the new sorting system was a bit limited.

So we’ve been working on enhancing the FieldBridge API to make sure that those more expert users who implement their own bridging would have a better control on how sorting works as well.

We’ve also started some work to push the performance higher, and overall I’m proud to state that this 5.5.1.Final release is including some small internal polish, but results in measurable improvements.

Sorting & FieldBridge improvements

If you create custom FieldBridge implementations, you can now declare field metadata to benefit from the improved sorting performance.

Since Hibernate Search 5.5, we recommend you explicitly mark which fields will be used for sorting via the @SortableField annotation.

This has an effect on indexing and triggers improved query performance as long as Hibernate Search understands it can use the new more effective sorting strategy.

As kindly reported by Ashot Golovenko, when implementing a custom FieldBridge there was no way to let the Hibernate Search Engine component know which custom created fields are valid to sort on, so even if your implementation was clever enough to index fields appropriately to take advantage of the new sorting capabilities, this wouldn’t be used at query time.

This has been fixed by introducing an extension to FieldBridge: org.hibernate.search.bridge.MetadataProvidingFieldBridge which allows you to configure the metadata correctly. Beware though: this approach is meant for expert users, and the Query engine is going to trust that the metadata you define is actually reflected by what your bridge implementation writes into the index. In case of mismatches, you’ll have runtime exceptions during query time when sorting on a field which wasn’t indexed as declared.

Spot inefficient sorting operations easily in test suites

While Hibernate Search already would log a warning when forced to perform a query using a sub-optimal sorting strategy, that wasn’t making it very easy to spot mapping or usage mistakes.

You can now set this property:

    hibernate.search.index_uninverting_allowed = false

and you’ll have your tests fail with a reasonable exception rather than log the warning.

Performance improvements

There are many internal improvements related to performance.

The most interesting one is that now we’ll be able to automatically skip scoring in various index housekeeping operations. This implies you’ll see lower CPU usage on some index write and update operations, and also improved query performance when certain automatic filtering needs to be applied on your queries, such as for narrowing down the entity types, apply sharding related filtering.

The sorting operations have been improved as well: we can now skip the index uninverting process when sorting on distances during spatial queries, or when sorting by scores.

Memory usage has been reduced as well! Special thanks to Andrej Golovnin to have diagnosed and reported HSEARCH-2029, that fix alone will reduce our permanent memory usage.

We also reduced allocation of several short lived but heavy objects being used during indexing and query execution, overall this should improve the efficiency of the JVM.

The performance work is an on-going challenge: I’m quite happy to see very respectable figures, but we’re planning even more improvements, and if you have profiling data or other useful data to share don’t be selfish and share it! Always happy to improve it further.

Components upgrades

Several components were upgraded; most notably we’re now using the latest Apache Lucene version 5.3.1.

  • Upgrade Narayana to 5.2.5.Final

  • Upgrade JGroups to 3.6.6.Final

  • Upgrade Hibernate ORM to 5.0.4.Final

  • Upgrade Apache Lucene to 5.3.1

  • Upgrade to Hibernate Commons Annotations 5.0.1.Final

<dependency>
   <groupId>org.hibernate</groupId>
   <artifactId>hibernate-search-orm</artifactId>
   <version>5.5.1.Final</version>
</dependency>
<dependency>
   <groupId>org.hibernate</groupId>
   <artifactId>hibernate-core</artifactId>
   <version>5.0.4.Final</version>
</dependency>
<dependency>
   <groupId>org.apache.lucene</groupId>
   <artifactId>lucene-core</artifactId>
   <version>5.3.1</version>
</dependency>

Next?

We’re working on version 5.6, as previously announced it’s going to sport an experimental integration with Elasticsearch.

How to get this release

Everything you need is available on Hibernate Search’s web site. Download the full distribution from here, or get it from Maven Central using the above coordinates, and don’t hesitate to reach us in our forums or mailing lists.

Releasing Hibernate Search 5.5.0.CR1

Posted by Sanne Grinovero    |       |    Tagged as Hibernate Search Releases

The first Candidate Release for Hibernate Search 5.5 is now available, introducing integration with Apache Lucene 5.3 and several nice improvements.

As a reminder on versions: Hibernate Search now requires Hibernate ORM 5.0.0.Final at least, and at the time of writing only Infinispan 8.0.1.Final supports real time replication of an Apache Lucene 5.3 index.

<dependency>
   <groupId>org.hibernate</groupId>
   <artifactId>hibernate-search-orm</artifactId>
   <version>5.5.0.CR1</version>
</dependency>
<dependency>
   <groupId>org.hibernate</groupId>
   <artifactId>hibernate-core</artifactId>
   <version>5.0.1.Final</version>
</dependency>
<dependency>
   <groupId>org.infinispan</groupId>
   <artifactId>infinispan-directory-provider</artifactId>
   <version>8.0.1.Final</version>
</dependency>
<dependency>
   <groupId>org.apache.lucene</groupId>
   <artifactId>lucene-core</artifactId>
   <version>5.3.0</version>
</dependency>

Out of the box indexing of java.time types

Hibernate Search is now able to automatically provide a sensible mapping for java.time.Year, java.time.Duration java.time.Instant, java.time.LocalDate, java.time.LocalTime, java.time.LocalDateTime, java.time.LocalTime, java.time.MonthDay, java.time.OffsetDateTime, java.time.OffsetTime, java.time.Period, java.time.YearMonth, java.time.ZoneDateTime, java.time.ZoneId, java.time.ZoneOffset.

That means that it includes an out of the box FieldBridge for each of these types, and allows transparent indexing and querying of properties of these types. You can of course customize the indexing by providing your own FieldBridge, as usual.

This feature is only available if you are running on a Java 8 runtime, although all other features of Hibernate Search are still backwards compatible with Java 7.

Sorting improvements

Since Apache Lucene 5.0 (and including 5.3 as we currently require) we can provide a significant performance improvement if you let us know in advance which fields you intend to use for sorting.

For this purpose a new annotation org.hibernate.search.annotations.SortableField is available. If you start using this annotation in your projects you’ll benefit from improved performance, but for those who don’t want to update their mapping yet we will fallback to the older strategy.

This subject is discussed in detail in this follow-up post.

Encoding null tokens in your index

When using @Field(indexNullAs=) to encode some marker value in the index, the type of the marker is now required to be of a compatible field type as all other values which are indexed in that same field.

This was problematic for `NumericField`s, the ones optimised for range queries on numbers, as we would previously allow you to encode a string keyword like 'null': this is no longer allowed, you will have to pick a number to be used to represent the null value.

As an example for an "age" property you might want to use:

@Field(indexNullAs = "-1")
Integer nullableAge;

How to get this release

Everything you need is available on Hibernate Search’s web site. Download the full distribution from here, or get it from Maven Central using the above coordinates, and don’t hesitate to reach us in our forums or mailing lists.

Feedback always welcome! Please let us know of any problem before we release the Final version.

After two candidate releases and a bit of quiet time in waiting for Hibernate ORM 5.0.0.Final, we now published Hibernate Search 5.4.0.Final: the first stable release compatible with ORM 5.

Minimal API changes

This version is designed to minimize your work to upgrade the Hibernate stack, so that you can focus on the changes needed to get running with Hibernate ORM 5 without having to deal with changes in the Search API.

Plans for WildFly 10, Lucene 5

WildFly 10 is coming fast and will feature both Hibernate ORM 5 and Hibernate Search, so please take advantage of this Hibernate Search 5.4.0.Final release as a milestone to get your applications up to date.

In WildFly 10 however we plan to include a further evolved version of Hibernate Search, to also update Apache Lucene to the latest (currently Lucene 5.3).

So the plan is that we’ll include some evolution of Hibernate Search 5.5.x in WildFly 10, but upgrading both ORM and Lucene could be a lot of changes for you to face in one sprint so I highly recommend using this version 5.4.0.Final as an intermediate, reliable refactoring step.

Other improvements

Since the main mantra for this version was "don’t change anything" (except the Hibernate ORM version), the new and cool stuff will be coming in the next minor release.

Of course we also fixed some minor bugs; the most relevant ones are in the area of the Query DSL, improving the handling of null-tokens during query generation and better recognition of Numeric Fields.

For details, see:

  • HSEARCH-1981 QueryBuilder should not tokenize the null-token when searching for null

  • HSEARCH-1973 Make sure the metadata recognizes a field as Numeric even if it’s wrapped into a NullEncodingTwoWayFieldBridge

  • HSEARCH-1968 Avoid the NPE on empty results during a Faceting Query

Special thanks to Marcel Barbosa Pinto for identifying and fixing the NPE on Faceting queries!

How to get this release

Everything you need is available on Hibernate Search’s web site. Download the full distribution from here, or get it from Maven Central, and don’t hesitate to reach us in our forums or mailing lists.

In the last couple of months we’ve been working to upgrade Hibernate Search to use Apache Lucene 5, to keep up with the greatest releases from the Lucene community.

Today Hibernate Search 5.5.0.Alpha1 is available!

As the version suggests this is the first cut, but we’ll also need your feedback and suggestions to better assess the needed steps to evolve this into a great, efficient and stable Final release.

A stable API

As Uwe Schindler - a great developer from the Lucene community - had suggested me to try, it turns out it was possible to update from Apache Lucene version 4 to 5 (a major upgrade) in a minor upgrade of Hibernate Search: many things improved, some things are slightly different, but overall the API was not affected by any significant change.

You might be affected by some minor changes of the Lucene API if you were writing power-use extensions to Hibernate Search, in that case:

Please let us know if there are difficulties in upgrading: we can try to improve on that, or at least improve the migration guide. If you don’t tell us, we might not know and many other developers will have a hard time!

Improved Performance

Several changes in the Apache Lucene code relate to improved efficiency, in particular of memory usage.

However our aim for this release was functionality and correctness; we didn’t have time to run performance tests, and to be fair even if we had the time nothing beats feedback from the community of users.

We would highly appreciate your help in trying it out and run some performance tests, especially if you can compare with previous versions of Hibernate Search (based on earlier versions of Apache Lucene).

If you think you can try this, it’s a unique opportunity to both contribute and to get our insight and support to analyse and understand your performance reports!

Sorting and Faceting: use the right types!

It appears that when using Apache Lucene 4, if you were running a Numeric Faceting Query but the target field was not Numeric, it would still work - or appear to work.

Similarly, if you Sort a query using the wrong SortType, Apache Lucene 4 appears to be forgiving.

When using Apache Lucene 5 now, and also because of some internal changes we had to do to keep your migration cost easier, the validation on the right types got much stricter.

Please be aware that you’ll get runtime exceptions like the following when mistmaching the types:

Type mismatch: ageForIntSorting was indexed with multiple values per document, use SORTED_SET instead

I’m aware those are confusing, so we’ll working on improving the errors: HSEARCH-1951; still even with those error messages, the meaning is you have to fix your mapping. No shame in that, we figured this problem out as several unit tests were too relaxed ;-)

How to get it

Everything you need is available on Hibernate Search’s web site. Download the full distribution from here, or get it from Maven Central, and don’t hesitate to reach us in our forums or mailing lists.

While Hibernate ORM applies some more polish before releasing the final version, so did the Hibernate Search team.

Hibernate Search version 5.4.0.CR2 is now available! It was built and tested with Hibernate ORM 5.0.0.CR3, again we’re just waiting for that to be final but decided to release another Candidate Release as some fixes and improvements were recently applied.

The "Meridian crossing issue"

The Spatial query feature was affected by a math sign mistake which would reveal itself on distance based queries which would cross the 180' meridian. Thanks to Alan Field for the test and Nicholas Helleringer for debugging and fixing the issue!

Serialization issue on enabling Term Vectors

Kudos to Benoit Guillon, who debugged and fixed a mistake in the Avro based serializer, for which values of Term Vectors would not be indexed when enabling Term Vectors in combination with a remote backend. See also HSEARCH-1936.

Various small improvements

You can now have the MassIndexer set a different timeout for the internal transactions it will start, so if you’re running Hibernate Search in a container like WildFly you no longer have to make a choice between having a deadline of 5 minutes or changing the default timeout of the whole container.

  • Improved validation when choosing an invalid Sort definition

  • The Infinispan module will now get you a warning if your Maven project is using the old artifact ids

  • The performance integration tests have been re-enabled, and fixed to run on latest WildFly and Java8 and Java9

  • Fixed a potential issue with timezone encoding for indexed dates

  • Improved the code generating the path for index storage to handle "." elements in the configured paths

  • Upgraded to Hibernate ORM 5.0.0.CR3 and WildFly 10.0.0.Alpha6 (WildFly only used for testing)

Where to download it from

Everything you need is available on Hibernate Search’s web site. Download the full distribution from here, or get it from Maven Central, and don’t hesitate to reach us in our forums

What’s next?

Soon we’ll be releasing a preview of 5.5 to have you upgrade to Apache Lucene 5.2. The Hibernate Search version 5.4.0.Final will be released when Hibernate ORM 5 is released as stable.

Updated Roadmap for Hibernate Search

Posted by Sanne Grinovero    |       |    Tagged as Hibernate Search

The Hibernate Search project roadmap was quite outdated, so after some team chats on IRC and our developer’s mailing list I’ve summarized our plans on the project website.

What’s the plan?

Current 80% progress

Upgrade to Hibernate ORM 5

Coming in Hibernate Search 5.4, which is almost ready, and you could try it already.

Transactional improvements for the JMS backend

Also included in version 5.4, with improved documentation and configuration examples to follow probably in 5.4.

MassIndexer Improvements

The long standing limitation of transaction timeouts was finally resolved.

Apache Lucene 5 support

We’ve been working on this on separate branches, but we won’t merge it in 5.4 as this branch is stable now. The Lucene 5 update code doesn’t look too scary but we plan that to be 5.5 's highlight so that you can focus on one thing at a time: update to Hibernate ORM 5 now, and just after to Apache Lucene 5.

Neither upgrade is particularly complex, but I hope you’ll appreciate the effort to let you update step by step and take a break in between in case there is some surprise which needs extra attention.

Consider that the upgrade to Hibernate ORM 5 implies an upgrade to WildFly 10.

Near Future Intentions

Clustering, backends and reliability

We want to make it much simpler for you to use Hibernate Search in a clustered / cloud deployment. This has been possible since years, but the amount of requests for help are soaring and we have to admit that we can make several things easier. This is going to be an ongoing effort, with small improvements like the usage of JMSXGroupID already in 5.4, improved JMS examples in 5.5, and the driver for several of the improvements I’ve listed in the roadmap for 5.6…​ however depending on your help and feedback you could read that as "5.6 and beyond".

Java 8, Java 9, …​

With Java 8 very popular and Java 9 getting closer I hope that roadmap items such as out of the box indexing for the new Date/Time types don’t need any explanation. They could use some help though! We made it easy to register new bridges for any type so you can define your own custom types and define how they should best be indexed; this implies you can make an independent plug-in package for any type we don’t support yet, and if it’s a popular type like the ones from JSR 310 or JSR 354 we’d love to integrate it.

ElasticSearch and Apache Solr

There has been interest in such integrations for a while, but we failed to make any progress as the problem needs to be broken down in smaller steps. Several points on JIRA and the roadmap might seem only tangential but resolve roadblocks and pave the road to integrate Hibernate with such services. While we’re intimately familiar with Apache Lucene and these Lucene-as-a-service alternatives provide similar features, we don’t have direct experience with these, so some help will be appreciated. The plan is to prefer merging small iterative improvements over stalling development for months, so we’ll see several steps in a 5.6+ series and use that as the foundation to assess how we can deal with API inconsistencies across these slightly different backends.

Next!

After having all the above-listed new features nicely chiseled in our current stable API, I’m confident that to make the most of features like the ElasticSearch integration we’ll need to make changes to the API. This last refinement step will define Hibernate Search 6.0.

As always, these plans might need to change and we’ll always look forward to your suggestions. I’m unable to commit on dates or seasons; I would love to see this all happen before this winter but we’ll need your contribution for that to be realistic.

Hibernate Search is ready for Hibernate ORM 5

Posted by Sanne Grinovero    |       |    Tagged as Hibernate Search Releases

Hibernate Search version 5.4.0.CR1 is now available! It was built and tested with Hibernate ORM 5.0.0.CR2, essentially it’s all ready for ORM 5 and we’ll just be waiting for this to be marked Final.

<dependency>
    <groupId>org.hibernate</groupId>
    <artifactId>hibernate-search-orm</artifactId>
    <version>5.4.0.CR1</version>
</dependency>
<dependency>
    <groupId>org.hibernate</groupId>
    <artifactId>hibernate-core</artifactId>
    <version>5.0.0.CR2</version>
</dependency>
<dependency>
    <groupId>org.hibernate</groupId>
    <artifactId>hibernate-entitymanager</artifactId>
    <version>5.0.0.CR2</version>
</dependency>

No longer timeouts when using the MassIndexer in a container

You can now have the MassIndexer set a different timeout for the internal transactions it will start, so if you’re running Hibernate Search in a container like WildFly you no longer have to make a choice between having a deadline of 5 minutes or changing the default timeout of the whole container.

fullTextSession
   .createIndexer( User.class )
   .transactionTimeout( 1800 ) //seconds
   .startAndWait();

Great improvements in the JMS backend

Transactional JMS backend

As explained in more detail in last week’s post, Hibernate Search now provides an option to include its indexing operations within the same transaction as the RDBMS.

In short, it’s enabled by setting this property:

hibernate.search.worker.enlist_in_transaction=true

But keep in mind: it’s a global setting! If you want to use it, all your backends shall be set to use an XA enabled, transactional JMS queue.

Please let us know if you have a great use case which would require us to allow some form of mixed mode.

The JMS message header JMSXGroupID

Hibernate Search will now use the specific JMSXGroupID message header and set it to the index name. That will allow users of the JMS backend to take advantage of message grouping.

Modules versions and running in WildFly

The first WildFly version to use Hibernate ORM 5 is version 10.0.0.Alpha5, which was released the past weekend. So the JBoss Modules we create for this appserver are targeting now WildFly 10, but at least version 10.0.0.Alpha5.

Where to download it from

Everything you need is available on Hibernate Search’s web site. Download the full distribution from here, or get it from Maven Central, and don’t hesitate to reach us in our forums

Hibernate Search 5.3.0.Final now available!

Posted by Sanne Grinovero    |       |    Tagged as Hibernate Search

As suggested last week, today we released Hibernate Search version 5.3.0.Final.

Compared to the previous candidate release, the only changes are some minor clarifications in the documentation.

   <dependency>
      <groupId>org.hibernate</groupId>
      <artifactId>hibernate-search-orm</artifactId>
      <version>5.3.0.Final</version>
   </dependency>

Faceting API changes

The new great faceting integration comes at a small migration cost: remember you now need to use the new @Facet annotation as explained in the the example of the previous post.

What's next?

Barring some maintenance needs on this branch 5.3, we have no plans of other Hibernate Search releases to target Hibernate ORM 4.3.x. The focus is now on Hibernate 5 compatibility.

  • Artefact jars are available on Maven Central under the GAV org.hibernate:hibernate-search-orm:5.3.0.Final
  • Tarballs and zip bundles can be downloaded from our website
  • Feedback is welcome on the forums and emails, IRC

For those of you using Hibernate ORM version 5.0.0.CR1, you can now use the freshly released Hibernate Search 5.4 version 5.4.0.Alpha1.

What's new

Absolutely nothing! This Hibernate Search version is identical in terms of features and API to version 5.3.0.CR1: this should make it easier for you all to upgrade the Hibernate ORM libraries (hibernate-core, hibernate-entitymanager,..) without the distraction of changes because of Hibernate Search: focus on the changes you'll need to apply because of the major version upgrade of Hibernate (if any, as it's not too complex at all).

WildFly compatibility and JBoss Modules

With every release of Hibernate Search we normally also release a set of modules to run the latest version of it on WildFly, but in this case since the updated Hibernate ORM 5 integrations for WildFly have yet to be released, we skipped this step. Fear not, the WildFly integration will be finished soon and we'll then resume releasing such module packs as usual. Not least, this very same version of Hibernate Search will soon be available in WildFly 10, so the modules missing today won't actually be needed at all.

This is a great time to try Hibernate 5

While the latest polish is performed on Hibernate 5, we're all looking forward for feedback from you. It is likely that some more changes will be done, but we consider it good enough already to not expect any regression so please try it and let us know! We're at that sweet spot in which you can still propose changes without the chains of strong API compatibility requirements, but good enough for you to not be wasting time on a quickly changing target.

Versions reminder

This version of Hibernate Search requires:

  • Hibernate ORM 5.0.0.CR1
  • Apache Lucene 4.10.x
  • Java SE 7

Our rules and conventions for versions and compatibility are documented here on the GitHub Wiki.

  • Artefact jars are available on Maven Central under the GAV org.hibernate:hibernate-search-orm:5.4.0.Alpha1
  • Zip and tar bundles are available via our website
  • Feedback is welcome on the forums and emails, IRC
back to top