Red Hat

In Relation To Hibernate Search

In Relation To Hibernate Search

While the team has been busy implementing great new features such as the Elasticsearch integration for the next 5.6 release, some of you provided interesting feedback on our stable release.

The summary of the feedback I heard is that migrating to the new sorting requirements can be confusing, and there were some issues with our Faceting implementation.

Hibernate Search version 5.5.3.Final is available now, fixing the reported issues and improving the error messages around sorting.

The changelog is rather small, so this time I’ll post it verbatim:

  • HSEARCH-1917 - Cannot index null or empty values for faceted fields

  • HSEARCH-2082 - Documentation refers to @SortField when it should be @SortableField

  • HSEARCH-2085 - Typo in hibernate-search-engine logger

  • HSEARCH-2086 - Long and Date range faceting doesn’t honor hasZeroCountsIncluded

  • HSEARCH-2179 - Hanging during shutdown of SyncWorkProcessor

  • HSEARCH-2193 - LuceneBackendQueueTask does not release the Directory lock on update failures

  • HSEARCH-2200 - Typo in log message

  • HSEARCH-2240 - Parallel service lookup might fail to find the service

  • HSEARCH-2199 - Allows the use of CharFilter in the programmatic API of SearchMapping

  • HSEARCH-2084 - Upgrade to WildFly 10.0.0.Final

  • HSEARCH-2089 - Ensure the performance tests do not use the WildFly embedded version of Search

  • HSEARCH-1951 - Improve resulting error message when applying the wrong Sort Type

  • HSEARCH-2090 - Using the wrong header in the distribution/pom.xml

  • HSEARCH-2241 - Clarify deprecation of setFilter() method on FullTextQuery

Spot inefficient sorting operations easily in test suites

While Hibernate Search already would log a warning when forced to perform a query using a sub-optimal sorting strategy, that wasn’t making it very easy to spot mapping or usage mistakes.

Set this property:

    hibernate.search.index_uninverting_allowed = false

and you’ll have your tests fail with an exception rather than log warnings.

This property is not new in this release, but it’s worth reminding as it makes it much easier to validate your migrations from previous versions.

<dependency>
   <groupId>org.hibernate</groupId>
   <artifactId>hibernate-search-orm</artifactId>
   <version>5.5.3.Final</version>
</dependency>
<dependency>
   <groupId>org.hibernate</groupId>
   <artifactId>hibernate-core</artifactId>
   <version>5.0.6.Final</version>
</dependency>
<dependency>
   <groupId>org.apache.lucene</groupId>
   <artifactId>lucene-core</artifactId>
   <version>5.3.1</version>
</dependency>

What are we working on?

The Elasticsearch integration is almost feature complete, we expect to be able to release a Beta1 version in some weeks.

How to get this release

Everything you need is available on Hibernate Search’s web site. Download the full distribution from here, or get it from Maven Central using the above coordinates, and don’t hesitate to reach us in our forums or mailing lists.

Having fixed several issues and tasks since the previous milestone, it’s time to publish our third milestone towards Elasticsearch integration: Hibernate Search version 5.6.0.Alpha3 is now available!

Migration from Hibernate Search 5.5.x

Even if you’re not interested in the new Elasticsearch support, you might want to try out this version as it benefits from Apache Lucene 5.5.0.

If you ignore the new features and want to simply use Lucene in embedded mode the migration is easy, and as usual we are maintaining notes regarding relevant API changes in the Migration Guide to Hibernate Search 5.6.

Elasticsearch support progress

  • you can now use the Analyzers from Elasticsearch

  • Multiple operations will now be sent to Elasticsearch as a single batch to improve both performance and consistency

  • Spatial indexing and querying is now feature complete

  • We’ll wait for Elasticsearch to be "green" before attempting to use it at boot

  • Many improvements in the query translation

  • Error capture and reporting was improved

  • the Massindexer is working now, but is not yet using efficient bulk operations

  • the Elasticsearch extensions are now included in the WildFly modules

How to get this release

Everything you need is available on Hibernate Search’s web site.

Get it from Maven Central using the above coordinates.

Downloads from Sourceforge are available as well.

Feedback

Feedback always welcome!

Please let us know of any problem or suggestion by creating an issue on JIRA, or by sending an email to the developer’s developer’s mailing lists, or posting on the forums.

We also monitor Stack Overflow; when posting on SO please use the tag hibernate-search.

In this post, I’d like you to meet Martin, who, in spite of his young age, has been very active in the Hibernate Search project development, implementing some interesting extensions or helping with pull request reviewing.

Because I’d love to see more university students getting involved with open source software, I took this opportunity and interviewed Martin about this experience.

  1. Hi, Martin. You are one of the youngest contributors we’ve ever had. Can you please introduce yourself?

    Hi, Vlad. I am a 22-year-old Master’s Degree student at University of Bayreuth, Germany and have been interested in Hibernate Search and Fulltext Search (Lucene, Solr) for quite some time now. I am also a firm believer of Open Source and have actually always wanted to become a contributor of a tool (or software) many other developers use in their projects. Knowing that a piece of code you wrote is running in other systems is quite the rewarding feeling.

  2. I understand that you took part in the Google Summer of Code event. Can you tell us a little bit about this program?

    Yes, I took part in last year Google Summer of Code program and was mentored by Sanne Grinovero while working on adapting Hibernate Search to work with any JPA provider. It gave me the opportunity to dive more deeply into the codebase as it allowed me to concentrate on nothing but my project work-wise. In general Google Summer of Code is one of the best learning experiences any student that wants to get into Open Source can have.

  3. Contributing to an open-source project is a great learning experience. Has this activity helped you improve your skills?

    Definitely. While building new features or tracking down bugs, you encounter loads of different pieces of code you have to work through. With that comes learning new technologies and APIs. Also, the general process of submitting JIRA issues, discussing them and implementing the solutions is something you can learn while working on an open source project. Trying out the process yourself is invaluable and cannot be compared to just learning them on paper. This is also something I always tell to new coders: Try it out or you will not get it 100%.

  4. Do you think the entry barrier is high for starting contributing to an open source project? How should we encourage students to getting involved with open source?

    In the case of the Hibernate team, I can only say that it was quite easy to get into contact with the other developers. I just got onto IRC and asked questions about problems I had. They helped me with every question I had, so I stuck around. Then, I started reporting issues or making feature requests and was immediately incorporated into discussions. So no, the barrier is not high (at least for me in the case of the Hibernate team).

    I think open source needs to be encouraged more at a university level. I think many students don’t realize what they are missing. Yes, open standards are encouraged and teaching uses open APIs all over the place, but universities tend to keep much of the work that is suitable for open source behind closed doors (btw: I don’t think that closed source is always a bad thing, but it sometimes is in the way of innovation).

  5. What are your plans for the future?

    Firstly, I want to finish my Masters degree at University. I haven’t fully decided yet, whether I want to stay at University or not. Time will tell, I guess. Secondly, I want to keep contributing to Hibernate Search and finish merging the features of last years Google Summer of Code into the core code base.

Thank you, Martin, and keep up the good work.

During my talk at VoxxedVienna on using Hibernate Search with Elasticsearch earlier this week, there was an interesting question which I couldn’t answer right away:

"When running a full-text query with a projection of fields, is it possible to return the result as a list of POJOs rather than as a list of arrays of Object?"

The answer is: Yes, it is possible, result transformers are the right tool for this.

Let’s assume you want to convert the result of a projection query against the VideoGame entity shown in the talk into the following DTO (data transfer object):

public static class VideoGameDto {

    private String title;
    private String publisherName;
    private Date release;

    public VideoGameDto(String title, String publisherName, Date release) {
        this.title = title;
        this.publisherName = publisherName;
        this.release = release;
    }

    // getters...
}

This is how you could do it via a result transformer:

FullTextEntityManager ftem = ...;

QueryBuilder qb = ftem.getSearchFactory()
    .buildQueryBuilder()
    .forEntity( VideoGame.class )
    .get();

FullTextQuery query = ftem.createFullTextQuery(
    qb.keyword()
        .onField( "tags" )
        .matching( "round-based" )
        .createQuery(),
    VideoGame.class
    )
    .setProjection( "title", "publisher.name", "release" )
    .setResultTransformer( new BasicTransformerAdapter() {
        @Override
        public VideoGameDto transformTuple(Object[] tuple, String[] aliases) {
            return new VideoGameDto( (String) tuple[0], (String) tuple[1], (Date) tuple[2] );
        }
    } );

List<VideoGameDto> results = query.getResultList();

I’ve pushed this example to the demo repo on GitHub.

There are also some ready-made implementations of the ResultTransformer interface which you might find helpful. So be sure to check out its type hierarchy. For this example I found it easiest to extend BasicTransformerAdapter and implement the transformTuple() method by hand.

To the person asking the question: Thanks, and I hope this answer is helpful to you!

Today we’re proud to announce the first Alpha release sporting experimental integration with Elasticsearch!

We also updated to use Apache Lucene 5.5 - the latest stable release of our favourite search engine - as of course we’re not abandoning our traditional users!

What is the Hibernate Search / Elasticsearch integration?

Both Hibernate Search and Elasticsearch can do much more, but for the purpose of explaining this integration at an high level I’ll simplify their definitions as:

Hibernate Search

is a popular extension of the super popular Hibernate ORM framework, which makes it easy to index and query your entities using Apache Lucene.

Elasticsearch

is a very popular REST server which wraps the capabilities of Apache Lucene and makes it easier to scale this service horizontally.

Until today when using Hibernate Search you’d be using Apache Lucene directly, in what we will now call "embedded mode". In this mode a query is executed by the same process of your application, and while indexing happens in background still the overhead of such processing is happening within the same server and within the same JVM process as your Hibernate powered application.

With the Elasticsearch integration, rather than indexing your entities directly by managing the Lucene resources, we will be sending RPCs to an Elasticsearch cluster to achieve a very similar purpose: after all it is also Lucene based, so the feature match is extremely close!

This means that we’re able to transparently map all the current features to this new alternative backend, and by so doing give you more architectural choices at minimum required changes in your applications: the goal is that for most users the differences will be mostly in configuration details.

When using Elasticsearch we will need to send RPCs over the network to run queries and index updates, but on the other hand you benefit from Microservices - style decoupling and all the nice features that Elasticsearch can provide in terms of running and managing an horizontally scalable cluster.

In a nutshell…​

Elasticsearch will manage the index for you, Hibernate ORM will manage your objects and the transactions, Hibernate Search will transparently keep Elasticsearch synchronized to your database transactions and let you query your Domain Model returning managed objects, but moving the heavy lifting to an independent Elasticsearch service.

Elasticsearch integration status

This is literally being developed right now, so do not expect this to be feature complete nor reliable enough to run in a production system. Still, we already have a great set of features working so it’s a nice time to start playing with it and hopefully provide some feedback.

The documentation sports a new Elasticsearch chapter, which is a good place to start.

Configuring the Elasticsearch backend

Dependencies

You’ll need a new dependency, named org.hibernate:hibernate-search-backend-elasticsearch.

The Maven dependencies for Hibernate Search with Elasticsearch:

<dependency>
   <groupId>org.hibernate</groupId>
   <artifactId>hibernate-search-backend-elasticsearch</artifactId>
   <version>5.6.0.Alpha2</version>
</dependency>
<dependency>
   <groupId>org.hibernate</groupId>
   <artifactId>hibernate-search-orm</artifactId>
   <version>5.6.0.Alpha2</version>
</dependency>

Configuration options

Point it to your Elasticsearch node

hibernate.search.elasticsearch.host http://127.0.0.1:9200

Enable the backend to be used for all your indexed entities

hibernate.search.default.indexmanager elasticsearch

Allow it to destroy and create indexes

hibernate.search.elasticsearch.index_management_strategy CREATE_DELETE

For details about these configuration options see the Reference documentation.

Updating the indexes

As with Lucene in embedded mode, the indexes are updated automatically when you create or update entities which are mapped to Hibernate Search using the same annotations already familiar from our traditional index mapping (see Mapping entities to the index structure).

Running a query on an Elasticsearch mapped entity

In many cases the existing way (see Querying) of running queries should work: we do automatically translate the most common types of Apache Lucene queries and many of the queries generated by the Hibernate Search DSL.

On top of translating Lucene queries, you can directly create Elasticsearch queries by using either its String format or a JSON format:

Creating an Elasticsearch native query from a string:

FullTextSession fullTextSession = Search.getFullTextSession(session);
QueryDescriptor query = ElasticsearchQueries.fromQueryString("title:tales");
List<?> result = fullTextSession.createFullTextQuery(query, ComicBook.class).list();

Creating an Elasticsearch native query from JSON:

FullTextSession fullTextSession = Search.getFullTextSession(session);
QueryDescriptor query = ElasticsearchQueries.fromJson(
      "{ 'query': { 'match' : { 'lastName' : 'Brand' } } }");
List<?> result = session.createFullTextQuery(query, GolfPlayer.class).list();

Remaining work ahead

This is an early preview, and while we’re proud of some of the progress there are several areas which still need much coding. On the other hand, implementing some of these is not very hard: this might be the perfect time to join the project.

Please check with JIRA and the mailing lists for updates, but at the time of writing this at least the following features are known to not work yet:

  • Analyzer definitions are not being applied

  • Spatial queries need more work

  • Filters can’t be applied yet

  • Faceting is mostly implemented

  • Scheduled index optimisation is not applied

  • Query timeouts

  • Delete by queries

  • Resolution for Date type mapping is ignored

  • Scrolling on large results won’t work yet

  • MoreLikeThis queries

  • Mixing Lucene based indexes and Elasticsearch based indexes

Any aspect related to performance and efficiency will also be looked at only at the end of basic feature development.

API Changes

In the 5.x series we will keep backward compatibility.

That might come at a cost of not perfect Hibernate Search / Elasticsearch integration API wise. This is something we will address in the 6.x series. But our focus is on offering the right set of features and get feedback in 5.x before improving the APIs.

In a nutshell, 6.x will depend on how you use this feature in 5.6.

How to get this release

Everything you need is available on Hibernate Search’s web site.

Get it from Maven Central using the above coordinates.

Downloads from Sourceforge are available as well, but these don’t contain the Elasticsearch integration components yet. Similarly the WildFly modules also are not including the new Elasticsearch extensions yet.

Feedback

Feedback always welcome!

Please let us know of any problem or suggestion by creating an issue on JIRA, or by sending an email to the developer’s developer’s mailing lists, or posting on the forums.

We also monitor Stack Overflow; when posting on SO please use the tag hibernate-search.

Eclipse tools for Hibernate Search

Posted by Dmitry Bocharov    |       |    Tagged as Hibernate Search JBoss Tools

I’m glad to announce the first release of the Eclipse plugin for Hibernate Search. In this post I want to describe its features and ask you for any comments, positive or (even more important for me) negative.

Installation

The plugin was made as a feature of jbosstools-hibernate plugin, which can be downloaded and installed on its own or together with the full JBoss Tools distribution. After that you can install the Hibernate Search plugin via the Eclipse Marketplace.

All requirements, such as eclipse version and platform support, are listed in the link.

In order to work with Hibernate Search you have to set the Hibernate configuration properties hibernate.search.default.directory_provider and hibernate.search.default.indexBase.

Functionality

The plugin was thought to be some kind of a Luke tool inside Eclipse. It was thought to be more convenient than launching a separate application, and picks up the configuration directly from your Hibernate configuration.

Three options were added to the console configurations: Index rebuild, Explore documents and Try analysers.

Index Rebuild Action screenshot

Index rebuild

When introducing Hibernate Search in an existing application, you have to create an initial Lucene index for the data already present in your database.

The option "Rebuild index" will do so by re-creating the Lucene index in the directory specified by the hibernate.search.default.indexBase property.

Screenshot of Hibernate Search indexed entities
Screenshot for Hibernate Search configuration properties

Explore Documents

After creating the initial index you can now inspect the Lucene Documents it contains.

All entities annotated as @Indexed are displayed in the Lucene Documents tab. Tick the checkboxes as needed and load the documents. Iterate through the documents using arrows.

Screenshot of Lucene Documents inspection

Try the Analyzers

The "try analyzers" instrument allows you to view the result of work of different Lucene Analyzers. The combo-box contains all classes in the workspace which extend org.apache.lucene.analysis.Analyzer, including custom implementations created by the user. While you type the text you want to analyse, the result immediately appears in the AnalysisResultTab view.

Screenshot of Try Analyzers

Possible issues

One problem which you might have is that "Index rebuild" option seems to do nothing. As a temporary workaround try to set the Hibernate configuration property "hibernate.search.autoregister_listeners" to "true" explicitly.

If you have any other problems, such as unexpected behaviour, strange windows with exceptions or any errors in the Error log view feel free to contact me directly anywhere or just create an issue in the plugin github page.

Plans

  • Make options "Index rebuild" and "Explore documents" available not only for configurations, but also for concrete entities under session factory.

  • Make Lucene Documents view more comfortable to use and add there more features from Luke tool, for example, the ability to search over documents.

  • Increasing stability of the plugin and implementing your suggestions!

Version 5.5.2.Final is now available, our latest stable version sporting integration with Hibernate ORM 5 and Apache Lucene 5.3 - the state of the art.

Creating this version to be compatible with these two great OSS projects kept us busy for a good deal of this past year; I remember discussing this option with superstar OSS contributors Uwe Schindler (Apache Lucene developer) and Gustavo Nalle (Infinispan developer) at FOSDEM in January 2015! I am grateful to both for their guidance and suggestions, as driving progress forward is sometimes challenging when we strive to keep backwards compatibility as best as we can.

On top of that, our same small but amazing team as been working hard on Hibernate OGM 5, a bit of Hibernate Validator, incredible performance improvements on Hibernate ORM "classic" and is still tinkering on the internals of Hibernate Search to make an Elasticsearch backend an alternative to plain Lucene.

Feel like helping with the Elasticsearch integration?

Version 5.6 will feature an experimental integration with Elasticsearch. Early feedback would be very welcome! If you feel like helping, quite some code is integrated in the master branch already and there are clear TODO and FIXME comments to be found in the code.

Feel free to join the hacking! But best to let us know which area you feel like working on to avoid duplication of efforts and conflicting patches.

The 5.5.2.Final release and our great community

Technically I pushed the release button the 24th, but I couldn’t publish the blog until now because of travels.

On top of the improvements from previous 5.5.1.Final release which sported significant performance improvements, this version now incorporates several important corrections by Yoann Rodière and Guillaume Smet around metadata which was producing incorrect range queries for embedded numeric types.

Yoann also contributed another great performance improving patch, by limiting the recursion triggered by @ContainedIn more strictly, by considering the indexed paths and not just the recursion depth. You might not notice this if you have very simple flat models, but this can provide a very significant boost for all those of you using Hibernate Search to index rich models with extensive usage of @ContainedIn annotations.

Many thanks to Yoann and Guillaume, they are "power users" who do the right thing: their hands-on regular feedback is invaluable, not to mention they send such great patches for everybody else to benefit from. Not least, this is very encouraging for us: let us know which great things you’re building with these libraries!

<dependency>
   <groupId>org.hibernate</groupId>
   <artifactId>hibernate-search-orm</artifactId>
   <version>5.5.2.Final</version>
</dependency>
<dependency>
   <groupId>org.hibernate</groupId>
   <artifactId>hibernate-core</artifactId>
   <version>5.0.6.Final</version>
</dependency>
<dependency>
   <groupId>org.apache.lucene</groupId>
   <artifactId>lucene-core</artifactId>
   <version>5.3.1</version>
</dependency>

How to get this release

Everything you need is available on Hibernate Search’s web site. Download the full distribution from here, or get it from Maven Central using the above coordinates, and don’t hesitate to reach us in our forums or mailing lists.

Hibernate Search 5.5.1.Final is now available!

Feedback about our recent 5.5.0.Final release has been great, and while the good news is that nobody reported significant issues, some people also pointed out that the new sorting system was a bit limited.

So we’ve been working on enhancing the FieldBridge API to make sure that those more expert users who implement their own bridging would have a better control on how sorting works as well.

We’ve also started some work to push the performance higher, and overall I’m proud to state that this 5.5.1.Final release is including some small internal polish, but results in measurable improvements.

Sorting & FieldBridge improvements

If you create custom FieldBridge implementations, you can now declare field metadata to benefit from the improved sorting performance.

Since Hibernate Search 5.5, we recommend you explicitly mark which fields will be used for sorting via the @SortableField annotation.

This has an effect on indexing and triggers improved query performance as long as Hibernate Search understands it can use the new more effective sorting strategy.

As kindly reported by Ashot Golovenko, when implementing a custom FieldBridge there was no way to let the Hibernate Search Engine component know which custom created fields are valid to sort on, so even if your implementation was clever enough to index fields appropriately to take advantage of the new sorting capabilities, this wouldn’t be used at query time.

This has been fixed by introducing an extension to FieldBridge: org.hibernate.search.bridge.MetadataProvidingFieldBridge which allows you to configure the metadata correctly. Beware though: this approach is meant for expert users, and the Query engine is going to trust that the metadata you define is actually reflected by what your bridge implementation writes into the index. In case of mismatches, you’ll have runtime exceptions during query time when sorting on a field which wasn’t indexed as declared.

Spot inefficient sorting operations easily in test suites

While Hibernate Search already would log a warning when forced to perform a query using a sub-optimal sorting strategy, that wasn’t making it very easy to spot mapping or usage mistakes.

You can now set this property:

    hibernate.search.index_uninverting_allowed = false

and you’ll have your tests fail with a reasonable exception rather than log the warning.

Performance improvements

There are many internal improvements related to performance.

The most interesting one is that now we’ll be able to automatically skip scoring in various index housekeeping operations. This implies you’ll see lower CPU usage on some index write and update operations, and also improved query performance when certain automatic filtering needs to be applied on your queries, such as for narrowing down the entity types, apply sharding related filtering.

The sorting operations have been improved as well: we can now skip the index uninverting process when sorting on distances during spatial queries, or when sorting by scores.

Memory usage has been reduced as well! Special thanks to Andrej Golovnin to have diagnosed and reported HSEARCH-2029, that fix alone will reduce our permanent memory usage.

We also reduced allocation of several short lived but heavy objects being used during indexing and query execution, overall this should improve the efficiency of the JVM.

The performance work is an on-going challenge: I’m quite happy to see very respectable figures, but we’re planning even more improvements, and if you have profiling data or other useful data to share don’t be selfish and share it! Always happy to improve it further.

Components upgrades

Several components were upgraded; most notably we’re now using the latest Apache Lucene version 5.3.1.

  • Upgrade Narayana to 5.2.5.Final

  • Upgrade JGroups to 3.6.6.Final

  • Upgrade Hibernate ORM to 5.0.4.Final

  • Upgrade Apache Lucene to 5.3.1

  • Upgrade to Hibernate Commons Annotations 5.0.1.Final

<dependency>
   <groupId>org.hibernate</groupId>
   <artifactId>hibernate-search-orm</artifactId>
   <version>5.5.1.Final</version>
</dependency>
<dependency>
   <groupId>org.hibernate</groupId>
   <artifactId>hibernate-core</artifactId>
   <version>5.0.4.Final</version>
</dependency>
<dependency>
   <groupId>org.apache.lucene</groupId>
   <artifactId>lucene-core</artifactId>
   <version>5.3.1</version>
</dependency>

Next?

We’re working on version 5.6, as previously announced it’s going to sport an experimental integration with Elasticsearch.

How to get this release

Everything you need is available on Hibernate Search’s web site. Download the full distribution from here, or get it from Maven Central using the above coordinates, and don’t hesitate to reach us in our forums or mailing lists.

Hibernate Search 5.5 Final is out

Posted by Davide D'Alto    |       |    Tagged as Hibernate Search Releases

I’m happy to announce the latest final release of Hibernate Search: Hibernate Search 5.5 Final. We mainly consolidated the features included in the latest candidate release and worked on fixing some bugs.

As a reminder on versions:

Hibernate Search now requires at least Hibernate ORM 5.0.0.Final, and at the time of writing only Infinispan 8.0.1.Final supports real time replication of an Apache Lucene 5.3 index.

<dependency>
   <groupId>org.hibernate</groupId>
   <artifactId>hibernate-search-orm</artifactId>
   <version>5.5.0.Final</version>
</dependency>
<dependency>
   <groupId>org.hibernate</groupId>
   <artifactId>hibernate-core</artifactId>
   <version>5.0.1.Final</version>
</dependency>
<dependency>
   <groupId>org.infinispan</groupId>
   <artifactId>infinispan-directory-provider</artifactId>
   <version>8.0.1.Final</version>
</dependency>
<dependency>
   <groupId>org.apache.lucene</groupId>
   <artifactId>lucene-core</artifactId>
   <version>5.3.0</version>
</dependency>

WildFly 10, Lucene 5

Last week, we released Hibernate Search 5.4 Final with the minimal set of changes to make it work with Hibernate ORM 5. This version uses Lucene 5.3; if you haven’t upgraded already, we recommend to start with version 5.4.0.Final, first.

Hibernate Search 5.5 will be included in WildFly 10.

Out of the box indexing of java.time types

Hibernate Search is now able to automatically provide a sensible mapping for java.time.Year, java.time.Duration java.time.Instant, java.time.LocalDate, java.time.LocalTime, java.time.LocalDateTime, java.time.LocalTime, java.time.MonthDay, java.time.OffsetDateTime, java.time.OffsetTime, java.time.Period, java.time.YearMonth, java.time.ZoneDateTime, java.time.ZoneId, java.time.ZoneOffset.

That means that it includes an out of the box FieldBridge for each of these types, and allows transparent indexing and querying of properties of these types. You can of course customize the indexing by providing your own FieldBridge, as usual.

This feature is only available if you are running on a Java 8 runtime, although all other features of Hibernate Search are still backwards compatible with Java 7.

Sorting improvements

Since Apache Lucene 5.0 (and including 5.3 as we currently require) we can provide a significant performance improvement if you let us know in advance which fields you intend to use for sorting.

For this purpose a new annotation org.hibernate.search.annotations.SortableField is available. If you start using this annotation in your projects you’ll benefit from improved performance, but for those who don’t want to update their mapping yet we will fallback to the older strategy.

This subject is discussed in detail in this follow-up post.

Encoding null tokens in your index

When using @Field(indexNullAs=) to encode some marker value in the index, the type of the marker is now required to be of a compatible field type as all other values which are indexed in that same field.

This was problematic for `NumericField`s, the ones optimised for range queries on numbers, as we would previously allow you to encode a string keyword like 'null': this is no longer allowed, you will have to pick a number to be used to represent the null value.

As an example for an "age" property you might want to use:

@Field(indexNullAs = "-1")
Integer nullableAge;

How to get this release

Everything you need is available on Hibernate Search’s web site. Download the full distribution from SourceForge, or get it from Maven Central, and don’t hesitate to reach us in our forums or mailing lists.

Order, ooorder! Sorting results in Hibernate Search 5.5

Posted by Gunnar Morling    |       |    Tagged as Hibernate Search

"Order, ooorder!" - Sometimes not only the honourable members of the House of Commons need to be called to order, but also the results of Hibernate Search queries need to be ordered in a specific way.

To do so, just pass a Sort object to your full-text query before executing it, specifying the field(s) to sort on:

FullTextSession session = ...;
QueryParser queryParser = ...;

FullTextQuery query = session.createFullTextQuery( queryParser.parse( "summary:lucene" ), Book.class );
Sort sort = new Sort( new SortField( "title", SortField.Type.STRING, false ) );
query.setSort( sort );
List<Book> result = query.list();

As of Lucene 5 (which is what Hibernate Search 5.5 is based on), there is a big performance gain if the fields to sort on are known up front. In this case these fields can be stored as so-called "doc value fields", which is much faster and less memory-consuming than the traditional approach of index un-inverting.

For that purpose, Hibernate Search provides the new annotation @SortableField (and it’s multi-valued companion, @SortableFields) for tagging those fields that should be available for sorting. The following example shows how do it:

@Entity
@Indexed(index = "Book")
public class Book {

    @Id
    private Integer id;

    @Field
    @SortableField
    @DateBridge(resolution = Resolution.DAY)
    private Date publicationDate;

    @Fields({
        @Field,
        @Field(name = "sortTitle", analyze = Analyze.NO, store = Store.NO, index = Index.NO)
    })
    @SortableField(forField = "sortTitle")
    private String title;

    @Field
    private String summary;

    // constructor, getters, setters ...
}

@SortableField is used next to the known @Field annotation. In case a single field exists for a given property (e.g. publicationDate) just specifying the @SortableField annotation is enough to make that field sortable. If several fields exist (see the title property), specify the field name via @SortableField#forField().

Note that sort fields must not be analyzed. In case you want to index a given property analyzed for searching purposes, just add another, un-analyzed field for sorting as it is shown for the title property. If the field is only needed for sorting and nothing else, you may configure it as un-indexed and un-stored, thus avoid unnecessary index growth.

For using the configured sort fields when querying nothing has changed. Just specify a Sort with the required field(s):

FullTextQuery query = ...;
Sort sort = new Sort(
    new SortField( "publicationDate", SortField.Type.LONG, false ),
    new SortField( "sortTitle", SortField.Type.STRING, false )
);
query.setSort( sort );

Now what happens if you sort on fields which you have not explicitly declared as sortable, e.g. when migrating an existing application over to Hibernate Search 5.5? The good news is that the sort will be applied as expected from a functional perspective. Hibernate Search detects the missing sort fields and transparently falls back to index-univerting.

But be aware that this comes with a performance penalty (also it is quite memory-intensive as uninverting is RAM-only operation) and it even might happen that this functionality will be removed from Lucene in a future version altogether. Thus watch out for messages like the following in your log files and follow the advice to declare the missing sort fields:

WARN ManagedMultiReader:142 - HSEARCH000289: Requested sort field(s) summary_forSort are not configured for entity \
type org.hibernate.search.test.query.Book mapped to index Book, thus an uninverting reader must be created. You \
should declare the missing sort fields using @SortField.

When migrating an existing application, be sure to rebuild the affected index(es) as described in the reference guide.

With all the required sort fields configured, your search results with be in order, just as the British parliament members after being called to order by Mr. Speaker.

back to top