Red Hat

In Relation To Yoann Rodière

In Relation To Yoann Rodière

Just a quick heads-up to French-speaking developers: I will be presenting the Elasticsearch integration in Hibernate Search at the Strasbourg Java User Group (ElsassJUG) meetup, at 7 PM on Wednesday 26th of April.

I will briefly introduce full-text search (why and how it’s done), then present how to use Hibernate Search to keep Lucene indexes in sync with your Hibernate ORM entities, and I will show you how easy it is to target Elasticsearch instead of local Lucene indexes since Hibernate Search 5.6.0.

For more information about the location or to register, please refer to the Meetup page.

You asked for it, we deliver: Hibernate Search 5.7.0.Final, with support for Hibernate ORM 5.2.8, is out! And of course, like previous versions, it features experimental support for Elasticsearch integration (even with some improvements).

Stuck on an older ORM version? Don’t worry, we also released a bugfix version for 5.6, namely 5.6.1.Final.

Hibernate Search 5.7.0.Final is only compatible with Hibernate ORM 5.2.3 and later. There isn’t any version of Hibernate Search compatible Hibernate ORM 5.2.0 to 5.2.2.

If you need to use Hibernate ORM 5.0.x or 5.1.x and cannot upgrade, please use Hibernate Search 5.6.1.Final.

What’s new on 5.7 since the candidate release?

Below are the main changes since the candidate release.

If you are looking for advice on how to migrate, please refer to the migration guide.

For a full list of changes since 5.7.0.CR1, please see the release notes. For a full list of changes since 5.6.0.Final, please see this list of tickets on our JIRA instance.

General improvements

  • HSEARCH-2574: when it makes sense, you can now repeat Hibernate Search annotations on a single attribute/method. For example, if you want to add two index fields on the same title attribute, you can write:

        @Field(analyzer = @Analyzer(definition = "myAnalyzer")
        @Field(name = "title_sort", analyzer = @Analyzer(definition = "myAnalyzerForSort")
        private String title;

    Thus you don’t have to use the composite annotations (@Fields) anymore. But of course, those are still available.

  • HSEARCH-2588: some methods in the Sort DSL allowed to specify the sort field type explicitly, which was useful when sorting on fields added through custom field bridges. We improved support for MetadataProvidingFieldBridge, so that you now only need to implement this interface on your custom field bridges, and won’t need anything special when using the sort DSL anymore. As a result, the byField(String, SortField.Type) and andByField(String, SortField.Type) methods in the sort DSL have been deprecated.

  • HSEARCH-2587: an NPE could occur when using the Sort DSL to sort on embedded (@IndexedEmbedded) fields; this is now fixed.

  • HSEARCH-2576: unnecessary memory allocations within queries have been removed, resulting in better query throughput.

  • And several improvements for other solutions integrating Hibernate Search, like HSEARCH-2597, HSEARCH-2561, HSEARCH-2418 or HSEARCH-2585.

Elasticsearch-specific improvements

  • HSEARCH-2453: Elasticsearch authentication is now fully implemented: there are dedicated username and password properties so that you don’t need to embed the credentials in the URL anymore.

  • HSEARCH-2593: when using automatic node discovery on Elasticsearch, a scheme (HTTP or HTTPS) cannot be detected automatically for newly discovered nodes; by default it is HTTP. You can now customize this to use HTTPS instead: just add hibernate.search.default.elasticsearch.discovery.default_scheme https to your configuration.

  • HSEARCH-2590: queries using pagination could allocate too much memory in certain circumstances; this is now fixed.

What’s new on 5.6?

5.6.1.Final contains exclusively backported fixes from 5.7. You can see a full list of changes since 5.6.0.Final in the release notes.

What’s next?

We’re now focusing on the next two big improvements:

  • support for Elasticsearch 5 without losing support for Elasticsearch 2 (HSEARCH-2434, HSEARCH-2581). This will be in a 5.8 release.

  • streamlining our APIs, in particular to better fit remote indexing services (like Elasticsearch), but also to support Lucene 6. Due to the breaking changes this requires, we will include these changes in a new major release, Hibernate Search 6.0.

How to get these releases

All versions are available on Hibernate Search’s web site.

Ideally use a tool to fetch it from Maven central; these are the coordinates:

<dependency>
   <groupId>org.hibernate</groupId>
   <artifactId>hibernate-search-orm</artifactId>
   <version>5.7.0.Final</version>
</dependency>

To use the experimental Elasticsearch integration you’ll also need:

<dependency>
   <groupId>org.hibernate</groupId>
   <artifactId>hibernate-search-elasticsearch</artifactId>
   <version>5.7.0.Final</version>
</dependency>

Downloads from Sourceforge are available as well.

Feedback, issues, ideas?

To get in touch, use the following channels:

After a relatively quiet candidate release, we just published Hibernate Search version 5.6.0.Final with experimental Elasticsearch integration, along with 5.7.0.CR1.

Version 5.6.0.Final brings the latest bugfixes for our experimental Elasticsearch integration. This is the version to use with Hibernate ORM versions 5.0 and 5.1.

Version 5.7.0.CR1 brings the exact same changes as 5.6.0.Final, and now features the compatibility with Hibernate ORM version 5.2.7 (but not lower than 5.2.3, see more on this further down).

What’s new on 5.6 since the candidate release?

Below are the main changes since the candidate release.

For a full list of changes since 5.6.0.CR1, please see the release notes. For a full list of changes since 5.5, please see this list of tickets on our JIRA instance.

Common changes

  • HSEARCH-2547: nesting an @IndexedEmbedded with includePaths within an @IndexedEmbedded without an includePaths will now work properly.

  • HSEARCH-2535: @Facet with string encoding will now work properly on multi-valued properties (such as String[] or List<String>).

Elasticsearch-specific changes

  • HSEARCH-2501: the handling of @CalendarBridge.resolution with Elasticsearch was aligned on the one with Lucene: they are now consistent.

  • HSEARCH-2531: with Elasticsearch, index names can now be overridden using configuration properties, just like when using Lucene. Thanks to Cary Yu for reporting the issue!

  • HSEARCH-2519 and HSEARCH-2520: the Elasticsearch VALIDATE and MERGE index management strategies now handle analyzer definitions. This will only affect you if you were using @AnalyzerDef-defined analyzers on entities that are indexed in Elasticsearch. If you are, please be aware that the MERGE strategy may now close/reopen your index automatically during bootstrap! See the reference documentation for more information about the MERGE strategy.

What’s new on 5.7?

The main change in 5.7.0.CR1 since 5.7.0.Beta2 is the upgrade to Hibernate ORM 5.2.7. For a full list of changes, see the release notes.

Please note that Hibernate Search 5.7 now requires Hibernate ORM 5.2.3 onwards, and cannot be used with previous Hibernate ORM versions.

When will 5.7 be released?

5.7 is beginning its candidate release period, meaning that as far as we are concerned, it is already ready. We are giving a last chance for the community to report bugs, and unless we are delayed by major bugs, the actual 5.7.0.Final release should happen in mid-February.

How to get these releases

All versions are available on Hibernate Search’s web site.

Ideally use a tool to fetch it from Maven central; these are the coordinates:

<dependency>
   <groupId>org.hibernate</groupId>
   <artifactId>hibernate-search-orm</artifactId>
   <version>5.6.0.Final</version>
</dependency>

Or, for Hibernate Search 5.7:

<dependency>
   <groupId>org.hibernate</groupId>
   <artifactId>hibernate-search-orm</artifactId>
   <version>5.7.0.CR1</version>
</dependency>

To use the experimental Elasticsearch integration you’ll also need:

<dependency>
   <groupId>org.hibernate</groupId>
   <artifactId>hibernate-search-elasticsearch</artifactId>
   <version>5.6.0.Final</version>
</dependency>

Change the version to 5.7.0.CR1 in order to test the Hibernate ORM 5.2 integration.

Downloads from Sourceforge are available as well.

Feedback, issues, ideas?

To get in touch, use the following channels:

Hibernate Search 5.5.6.Final is out

Posted by    |       |    Tagged as Hibernate Search Releases

Hibernate Search 5.5.6.Final is here!

This is a maintenance release and contains exclusively bugfixes.

What’s new?

  • HSEARCH-2494: @TikaBridge will now work correctly on properties of type byte[].

  • HSEARCH-2535: @Facet with string encoding will now work properly on multi-valued properties (such as String[] or List<String>).

  • HSEARCH-2486: @ContainedIn in a superclass will now be taken into account even if the concrete class does not carry any Hibernate Search annotation.

  • HSEARCH-2479: building phrase queries with the Hibernate Search DSL used to trigger an IllegalArgumentException in some specific cases; this has been fixed.

  • …​ and more: the full change log can be found on our JIRA instance.

Thanks to Julien Bénichou, Andrew Robie and Timo Tretter for reporting issues, and even fixing one!

How to get this release

All versions are available on Hibernate Search’s web site.

Ideally use a tool to fetch it from Maven central; these are the coordinates:

<dependency>
   <groupId>org.hibernate</groupId>
   <artifactId>hibernate-search-orm</artifactId>
   <version>5.5.6.Final</version>
</dependency>

Downloads from Sourceforge are available as well.

Feedback, issues, ideas?

To get in touch, use the following channels:

It’s finally here! We just published the first candidate release of Hibernate Search with experimental Elasticsearch integration (5.6.0.CR1), along with 5.7.0.Beta2.

Version 5.6.0.CR1 brings the latest bugfixes and previously missing features for our experimental Elasticsearch integration. This is the version to use with Hibernate ORM versions 5.0 and 5.1.

Version 5.7.0.Beta2 brings the exact same changes as 5.6.0.CR1, and it still features the compatibility with Hibernate ORM version 5.2 that was introduced with 5.7.0.Alpha1.

What’s new?

  • HSEARCH-2348: Elasticsearch queries are now logged at the DEBUG level on the org.hibernate.search.fulltext_query logger, just as embedded Lucene queries.

  • HSEARCH-2449: more configuration options have been added regarding the connection to Elasticsearch, including the size of connection pools, the connection/read timeouts, and the automatic discovery of new nodes in the Elasticsearch cluster for client-side loadbalancing and (limited) failover.

  • HSEARCH-2219: analyzer definitions are now automatically translated and added to the Elasticsearch settings when Elasticsearch indexes are created. Please note there are still some limitations, most notably the definitions are not updated automatically (no support for the MERGE schema management strategy). See the documentation for more information about analyzers with Elasticsearch.

  • HSEARCH-2387: Elasticsearch mappings generated by Hibernate Search used to be strictly static. This can be an issue in some cases, which is why opt-in dynamic mappings are now available, either through a global option (see org.hibernate.search.elasticsearch.cfg.ElasticsearchEnvironment.DYNAMIC_MAPPING) or locally on metadata-providing field bridges (builder.field( name, FieldType.OBJECT ).mappedOn( Elasticsearch.class ).dynamic( DynamicType.TRUE )). Thanks to Alex Laptseu for reporting this!

  • …​ and much more, mainly bug fixes. The full change log can be found on our JIRA instance.

When will 5.6 be released?

As far as we are concerned, 5.6 is ready. Version 5.6.0.CR1 is the last opportunity for the community to test it and report bugs before the release which, if all goes well, will happen in early January.

What about Elasticsearch 5 support?

As mentioned in the blog post about the previous release, Elasticsearch 5.x is not supported yet.

Quoting:

The main reason is it brings several backward-incompatible changes that would require quite a bit of work if we still want to support the 2.x series. And we don’t want to postpone the Hibernate Search 5.6.0 release any more.

Our plan is to release a 5.6 supporting Elasticsearch 2.x, and add Elasticsearch 5 support in Hibernate Search 6.0 or, maybe, in an early 5.8 release. You may refer to HSEARCH-2434 to track the status of Elasticsearch 5.0 support.

When will 5.7 be released?

We are planning to publish a candidate release for 5.7 soon after 5.6.0.Final has been released.

How to get these releases

All versions are available on Hibernate Search’s web site.

Ideally use a tool to fetch it from Maven central; these are the coordinates:

<dependency>
   <groupId>org.hibernate</groupId>
   <artifactId>hibernate-search-orm</artifactId>
   <version>5.6.0.CR1</version>
</dependency>

Or, for Hibernate Search 5.7:

<dependency>
   <groupId>org.hibernate</groupId>
   <artifactId>hibernate-search-orm</artifactId>
   <version>5.7.0.Beta2</version>
</dependency>

To use the experimental Elasticsearch integration you’ll also need:

<dependency>
   <groupId>org.hibernate</groupId>
   <artifactId>hibernate-search-elasticsearch</artifactId>
   <version>5.6.0.CR1</version>
</dependency>

Change the version to 5.7.0.Beta2 in order to test the Elasticsearch integration within Hibernate Search 5.7.

Downloads from Sourceforge are available as well.

Feedback, issues, ideas?

To get in touch, use the following channels:

Today we are releasing two new versions of Hibernate Search: 5.6.0.Beta4 and 5.7.0.Beta1!

Version 5.6.0.Beta4 brings the latest bugfixes and previously missing features for our experimental Elasticsearch integration. This is the version to use with Hibernate ORM versions 5.0.x and 5.1.x.

Version 5.7.0.Beta1 brings the exact same changes as 5.6.0.Beta1, but on top of the compatibility with Hibernate ORM version 5.2.x that was introduced with 5.7.0.Alpha1.

What’s new?

  • HSEARCH-402: A new async reader strategy has been added for the Lucene indexing service, bringing performance boosts when you are okay with your queries being run on an out-of-date index (how much out-of-date is configurable).

  • HSEARCH-2260: A new VALIDATE index schema management strategy has been added for Elasticsearch, allowing you to automatically check on startup that your Hibernate Search mappings are in line with the Elasticsearch mappings.

  • Issues with @IndexedEmbedded in the Elasticsearch integration have be addressed: everything should now work properly, with the notable exception of @IndexedEmbedded.indexNullAs (not to be confused with @Field.indexNullAs).

  • HSEARCH-2235: You can now configure Hibernate Search to send requests to Elasticsearch servers in round-robin, enabling load-balancing. Failover is not supported yet, but we’ll be working on it.

  • HSEARCH-2360: Elasticsearch projections now use source filtering, greatly reducing the bandwidth needs when retrieving results.

  • We now test our Elasticsearch integration against version 2.4.2, which fixed an issue with date formats that impacted Hibernate Search. We strongly recommend to update your 2.4.x instances to the lastest available version in the 2.4.x series.

  • …​ and much more. The full change log can be found on our JIRA instance or on our GitHub repository.

When will this 5.6.0 be released?

We’ve been caught up in the polishing work with the Elasticsearch integration lately, but we’re seeing the end of the tunnel: the list of open tasks is getting shorter and shorter. The first release candidate for Hibernate Search 5.6.0 will land by the end of next week.

So, if you haven’t tested 5.6 already, now’s the time! Should you find any bug, please report them on our JIRA instance.

What about Elasticsearch 5 support?

Please be aware that we’re not currently supporting Elasticsearch 5.x. The main reason is it brings several backward-incompatible changes that would require quite a bit of work if we still want to support the 2.x series. And we don’t want to postpone the Hibernate Search 5.6.0 release any more.

Our plan is to release a 5.6.0 supporting Elasticsearch 2.x, and add Elasticsearch 5 support in Hibernate Search 6.0 or, maybe, in an early 5.8 release. You may refer to HSEARCH-2434 to track the status of Elasticsearch 5.0 support.

When will 5.7.0 be released?

Everything is going smoothly with this version, and very few bugs have been reported. As soon as 5.6.0 will be released, we’ll publish the candidate release for 5.7.0.

How to get these releases

All versions are available on Hibernate Search’s web site.

Ideally use a tool to fetch it from Maven central; these are the coordinates:

<dependency>
   <groupId>org.hibernate</groupId>
   <artifactId>hibernate-search-orm</artifactId>
   <version>5.6.0.Beta4</version>
</dependency>

Or, for Hibernate Search 5.7:

<dependency>
   <groupId>org.hibernate</groupId>
   <artifactId>hibernate-search-orm</artifactId>
   <version>5.7.0.Beta1</version>
</dependency>

To use the experimental Elasticsearch integration you’ll also need:

<dependency>
   <groupId>org.hibernate</groupId>
   <artifactId>hibernate-search-elasticsearch</artifactId>
   <version>5.6.0.Beta4</version>
</dependency>

(Change the version to 5.7.0.Beta1 in order to test the Elasticsearch integration within Hibernate Search 5.7)

Downloads from Sourceforge are available as well.

Introducing Hibernate Search Sort DSL

Posted by    |       |    Tagged as Hibernate Search

With Elasticsearch support coming as a technological preview in Hibernate Search 5.6, you would think we’re leaving out other features. Well, think again! Enters the Sort DSL, which will work with Elasticsearch of course, but also with the good ol' Lucene backend.

The point here is to provide an API to build sort descriptions easily, without knowing everything about Hibernate Search features added on top of Lucene, such as DistanceSortField. And while we’re at it, we’re making it a modern, fluid API.

Most common case: sorting by field

The QueryBuilder interface now has an additional sort() method:

QueryBuilder builder = fullTextSession.getSearchFactory()
  .buildQueryBuilder().forEntity(Book.class).get();
Query luceneQuery = builder.all().createQuery();
FullTextQuery query = fullTextSession.createFullTextQuery( luceneQuery, Book.class );
Sort sort = builder
  .sort()
    .byField("author").desc() // Descending order
    .andByField("title") // Default order (ascending)
  .createSort();
query.setSort(sort);
List results = query.list();

Of course, other kinds of sort are available. Let’s have a look!

Sorting by relevance

The relevance sort is also available with byScore(). Obviously, there’s one key difference with that one: the sort is descending by default, so you get the most relevant results (higher scores) first. If you need the least relevant results, fear not, we got you covered with byScore().asc().

Sorting by distance

If your entity has some spatial fields you may also build spatial sorts:

  .sort()
    .byDistance()
      .onField("location")
      .fromLatitude(24.0d)
      .andLongitude(32.0d)
  .createSort()

Stabilizing with byIndexOrder()

byIndexOrder offers an arbitrary, yet deterministic sort. This comes handy when you want to stabilize your sort:

  .sort()
    .byField("title")
    .andByIndexOrder()
  .createSort()

That way, if there are two books with the same title in your index, they will always keep the same relative order from one query to another.

Handling missing values

What if you’re sorting books by publishing date, and some of them haven’t even been published yet? No worry, you may decide whether the unpublished books will appear first or last:

  .sort()
    .byField("publishingDate_sort").desc() // Most recently published first
      .onMissingValue().sortFirst() // Not published yet => put this upper on the list
    .andByField("custom_id_sort") // Default for the case when multiple books have no publishing date
  .createSort()

Accessing native features

Let’s assume you’re using an external backend, such as Elasticsearch. You may want to take advantage of a brand-new feature that appeared in the last snapshot of this backend, that feature you just spotted this morning and that would really save you of a lot of trouble on your project. But, hey, the Hibernate Search team is not on the same time zone, and even if they’re providing fast support, you’re not getting the feature pushed into Hibernate Search in time to meet your deadline. Which is this evening, by the way.

Well, guess what: you can use that feature anyway. The sorting API also allows using native sorts. When using the Elasticsearch backend, it means passing the JSON description of this sort, which will be added to the Elasticsearch query as is:

  .sort()
    .byNative("authors.name", "{'order':'asc', 'mode': 'min'}")
    .andByField("title")
  .createSort()

Next…​

Of course, one could point out that this API is not really backend-independent. The API itself, its interfaces and methods, mostly are, but the returned type (Sort) is clearly bound to Apache Lucene.

Well, one day at a time: the API in its current form can be adapted to be completely backend-agnostic, so it’s paving the way to Hibernate Search 6.x, while still requiring no change to any other contract such as FullTextQuery.setSort(Sort). And that means it’s available directly in 5.6.0.Beta3!

So be sure to check it out, and to check the documentation for more information. Or, you know, since it’s a fluid API, you can simply use your IDE autocomplete feature and see what’s available!

In any case, feel free to contact us for any question, problem or simply to give us your feedback!

back to top