In Relation To

Back to home

Hibernate Search 6.1.0.Beta1 is out, with distributed indexing that scales dynamically & Jakarta EE artifacts!

Posted by    |      

Hibernate Search is a library that integrates Hibernate ORM with Apache Lucene or Elasticsearch by automatically indexing entities, enabling advanced search functionality: full-text, geospatial, aggregations and more. For more information, see Hibernate Search on hibernate.org.

We just published Hibernate Search 6.1.0.Beta1, a beta release of the next minor version of Hibernate Search.

Among the main changes: asynchronous, distributed automatic indexing now scales to more or fewer nodes dynamically, supports multi-tenancy, and pauses during mass indexing; new Jakarta EE artifacts are available alongside the traditional ones for Java EE, the Elasticsearch backend accepts JSON to customize the mapping, and more.

Hibernate Search 6.1.0.Beta1 also includes upgrades to Hibernate ORM 5.6.1.Final and Lucene 8.11, among others.

What’s new

Hibernate Search 6.1 is still in development: some features are still incomplete or may change in a backward-incompatible way.

Dependency upgrades

Hibernate ORM (HSEARCH-4379)

Hibernate Search 6.1 now relies on Hibernate ORM 5.6.

Lucene (HSEARCH-4367)

The Lucene backend now uses Lucene 8.11.

Others

Asynchronous, distributed automatic indexing: improvements

Dynamic scaling through dynamic sharding of the outbox

In the announcement post for Hibernate Search 6.1.0.Alpha1, we introduced coordination between application nodes to implement asynchronous, distributed automatic indexing.

At that time, coordination supported static sharding only: the number of nodes was fixed and you had to manually assign a shard to each node participating in automatic indexing.

As of HSEARCH-4140, static sharding is still available but optional: by default Hibernate Search will use dynamic sharding, which means it will dynamically scale up or down, adapting the number and assignment of shards automatically according to the application nodes currently connected to the database.

For more information, see this section of the documentation, but the short version is that starting with 6.1.0.Beta1, the only configuration you need to enable asynchronous, distributed automatic indexing is this (you will also need to add tables to your database schema):

hibernate.search.coordination.strategy = outbox-polling
Since 6.1.0.Alpha1, the strategy was renamed from database-polling to outbox-polling, and most related configuration properties were renamed. See Breaking changes.

Head to this section of the documentation for information on how to configure coordination more finely.

Compatible with multi-tenancy

As of HSEARCH-4316, the outbox-polling coordination strategy will now work correctly for applications relying on Hibernate ORM’s multi-tenancy, provided they list all expected tenant IDs in their configuration.

Pausing of automatic indexing while mass indexing

As of HSEARCH-4358, automatic indexing will be suspended while mass indexing is in progress, and resume after mass indexing finishes.

For information about why this is necessary and how to tune the behavior, see this section of the documentation.

Custom ES index mapping

Since HSEARCH-4253, you can provide Hibernate Search with JSON files containing part of the desired mapping of your indexes, and Hibernate Search will automatically merge this mapping with the one it generated, so that it gets pushed to Elasticsearch when it creates/updates the indexes.

# To configure the defaults for all indexes:
hibernate.search.backend.schema_management.mapping_file = custom/index-mapping.json
# To configure a specific index:
hibernate.search.backend.indexes.<index name>.schema_management.mapping_file = custom/index-mapping.json

This is especially useful to set top-level mapping attributes, for example disabling the _source field:

{
  "_source": {
    "enabled": false
  }
}

Jakarta EE

As of HSEARCH-4240, beside the traditional artifacts targeting Java EE (JPA, CDI, …​), Hibernate Search now provides alternative, experimental artifacts that target Jakarta EE 9.1 (Jakarta Persistence 3, Jakarta CDI 3).

These artifacts have their artifact ID suffixed with "-jakarta", similarly to the equivalent artifacts for Hibernate ORM (which you should use together).

In Hibernate Search 6.1.0.Beta1, this includes the following Jakarta artifacts:

  • org.hibernate.search:hibernate-search-mapper-orm-jakarta:6.1.0.Beta1

  • org.hibernate.search:hibernate-search-mapper-orm-coordination-outbox-polling-jakarta:6.1.0.Beta1

  • org.hibernate.search:hibernate-search-mapper-orm-batch-jsr352-core-jakarta:6.1.0.Beta1

  • org.hibernate.search:hibernate-search-mapper-orm-batch-jsr352-jberet-jakarta:6.1.0.Beta1

  • org.hibernate.search:hibernate-search-v5migrationhelper-orm-jakarta:6.1.0.Beta1

Note that in order to switch to Jakarta EE, you will need to switch to these artifacts, but also:

  • Use Jakarta EE-specific artifacts for your other Java EE dependencies, such as Hibernate ORM.

  • Convert your code to Jakarta EE APIs. This should be rather straightforward: simply replace import javax.persistence.<something> with import jakarta.persistence.<something> everywhere in your code (same with javax.enterprise, javax.transaction, etc.).

  • Find a new version of your framework or application server that supports Jakarta EE. This might be hard to come by, as Jakarta EE is still young, but some application servers already provide at least a preview.

Java modules

As of HSEARCH-3274, where possible, Hibernate Search now provides multi-release JARs with a full Java module definition that includes all the dependencies.

This excludes the Lucene backend in particular, because Lucene 8 has split packages.

Breaking changes

  • HSEARCH-4368: The database-polling coordination strategy was renamed to outbox-polling. You should update your Maven dependency from hibernate-search-mapper-orm-coordination-database-polling to hibernate-search-mapper-orm-coordination-outbox-polling, and the value of the configuration property hibernate.search.coordination.strategy from database-polling outbox-polling.

  • HSEARCH-4372: Several configuration properties changed:

    • hibernate.search.coordination.shards.total_count is now hibernate.search.coordination.event_processor.shards.total_count.

    • hibernate.search.coordination.shards.assigned is now hibernate.search.coordination.event_processor.shards.assigned.

    • hibernate.search.coordination.shards.static is no longer useful: sharding will automatically be considered static if you set either hibernate.search.coordination.event_processor.shards.total_count or hibernate.search.coordination.event_processor.shards.assigned.

    • hibernate.search.coordination.processors.indexing.enabled is now hibernate.search.coordination.event_processor.enabled.

    • hibernate.search.coordination.processors.indexing.polling_interval is now hibernate.search.coordination.event_processor.polling_interval.

    • hibernate.search.coordination.processors.indexing.batch_size is now hibernate.search.coordination.event_processor.batch_size.

  • HSEARCH-4209, HSEARCH-4295, HSEARCH-4347: The table holding the outbox events has changed both its name and content, and a new "agent" table is now necessary. You will need to drop your current outbox event table and re-create the necessary tables.

Other improvements and bug fixes

  • HSEARCH-4369: The outbox event processor will no longer wait the duration of the polling interval (defaults to 100 ms) between processing two batches of events.

  • HSEARCH-4194: If processing an outbox event fails, Hibernate Search will now wait for a bit before trying again. Set configuration property hibernate.search.coordination.event_processor.retry_delay to control the delay.

  • HSEARCH-4282: The transaction timeout for event processing can now be raised through configuration property hibernate.search.coordination.event_processor.transaction_timeout.

  • HSEARCH-4307: Hibernate Search’s detection of its own version (for logging on startup) should now work even on native images.

  • HSEARCH-4331: We added a few default identifier bridges.

    All types that used to be supported for document identifiers in Hibernate Search 5 should now be supported in Hibernate Search 6 as well (though possibly with a different format), with the exception of java.util.Class.

  • HSEARCH-4354: Hibernate Search will no longer retrieve and try to interpret Hibernate ORM metadata for properties that are not involved in indexing. This should reduce the impact of metadata parsing bugs in the future.

  • HSEARCH-4033: Entities with @IdClass can now be mapped provided they have a unique, immutable property annotated with @DocumentId.

  • HSEARCH-4273: MassIndexer#cacheMode is now honored. The parameter previously had no impact whatsoever.

  • HSEARCH-4363: Fixed a NullPointerException that occurred from time to time on shutdown.

  • HSEARCH-4365: When indexing, Hibernate Search will now ignore deleted entities that were erroneously left in the entity graph.

  • HSEARCH-4366: Exceptions thrown while processing entities for indexing will now include context showing on which part of the entity graph the exception occurred.

And more. For a full list of changes since the previous releases, please see the release notes.

How to get this release

All details are available and up to date on the dedicated page on hibernate.org.

Getting started, migrating

For new applications, refer to the getting started guide.

For existing applications, Hibernate Search 6.1 is a drop-in replacement for 6.0, assuming you also upgrade the dependencies. Information about deprecated configuration and API is included in the migration guide.

Feedback, issues, ideas?

To get in touch, use the following channels:

Back to top