Red Hat

In Relation To Discussions

In Relation To Discussions

Today we’ll be talking about Hibernate Validator and how you can provide your own constraints and/or validators in a fully self-contained manner. Meaning packaging it all into its own JAR file, in a way that others can use your library by simply adding it to the classpath.

This functionality is based on Hibernate Validator usage of Java’s ServiceLoader mechanism that allows to register additional constraint definitions. But more on the details later.

What can be a real life scenario for building your own library with constraints and sharing it? Well, let’s say that you are building some library with data classes that user might want to validate. As it may be tough to keep track of all such libraries and write/maintain all those constraints for them - Hibernate Validator provides authors of such libraries a possibility to write and share their own validation extensions. Which can be picked up by Hibernate Validator and used to validate your data classes.

Also this ServiceLoader mechanism allows to solve another problem. As you are trying to be a good developer and provide end users of your library only with the relevant classes and hide implementation details from them, you may not want to expose your validator implementation by mentioning it in the validatedBy() parameter of the @Constraint annotation. By using the approach described in this post you can achieve all these thing.

For our examples we will be creating Maven projects with two modules - one will contain validators and represent a "library" that can be shared, another module will be a consumer of this library and will contain some tests.

Enough of the talking, let’s validate some beans! That’s why we all gathered here, right? :)

Using custom annotations and validators

First let’s consider a case of adding your own constraint annotation and a corresponding validator.

Time, it needs time …​

While Hibernate Validator 5.4 supports a wide range of the Java 8 date/time API (and Bean Validation 2.0 will move this support to the specification level), there are some types not supported, one of them being Duration. This type is not describing a point in time so the regular date/time constraints (@Future / @Past) do not make sense for it. So if for instance we wanted to validate that a given duration has a specified minimum length, a new constraint is needed. Let’s call it @DurationMin.

Our new constraint annotation might look like this:

DurationMin.java
@Documented
@Constraint(validatedBy = { })
@Target({ METHOD, FIELD, ANNOTATION_TYPE, CONSTRUCTOR, PARAMETER, TYPE_USE })
@Retention(RUNTIME)
@Repeatable(List.class)
@ReportAsSingleViolation
public @interface DurationMin {

    String message() default "{com.acme.validation.constraints.DurationMin.message}";
    Class<?>[] groups() default { };
    Class<? extends Payload>[] payload() default { };

    long value() default 0;
    ChronoUnit units() default ChronoUnit.NANOS;

    /**
     * Defines several {@code @DurationMin} annotations on the same element.
     */
    @Target({ METHOD, FIELD, ANNOTATION_TYPE, CONSTRUCTOR, PARAMETER, TYPE_USE })
    @Retention(RUNTIME)
    @Documented
    @interface List {
        DurationMin[] value();
    }
}

Now that we have an annotation, we need to create a corresponding constraint validator. To do that you need to implement the ConstraintValidator<DurationMin, Duration> interface, which contains two methods:

  • initialize() - initializes the validator based on annotation parameters

  • isValid() - performs actual validation

An implementation might look like this:

DurationMinValidator.java
public class DurationMinValidator implements ConstraintValidator<DurationMin, Duration> {

    private Duration duration;

    @Override
    public void initialize(DurationMin constraintAnnotation) {
        this.duration = Duration.of( constraintAnnotation.value(), constraintAnnotation.units() );
    }

    @Override
    public boolean isValid(Duration value, ConstraintValidatorContext context) {
        // null values are valid
        if ( value == null ) {
            return true;
        }
        return duration.compareTo( value ) < 1;
    }
}

As we are creating a new constraint annotation, we should also provide a default message for it. This can be done by placing a ContributorValidationMessages.properties property file in the classpath. This property file should contain a key/message pair, where key is the one used in annotation declaration (in our case it’s com.acme.validation.constraints.DurationMin.message) and message is the one you would like to show when validation fails. Our property file looks like so:

ContributorValidationMessages.properties
com.acme.validation.constraints.DurationMin.message = must be greater than or equal to {value} {units}

The bundle ContributorValidationMessages is queried by Hibernate Validator if the standard ValidationMessages bundle doesn’t contain a given message key, allowing library authors to provide default messages for their constraints as part of their JAR.

If you leave everything else as is, your constraint annotation will live its own life without knowing about the presence of validator. Hibernate Validator will not know of the validator as well. So to make sure that Hibernate Validator discovers your DurationMinValidator, you need to create the file META-INF/services/javax.validation.ConstraintValidator and put the fully qualified name of the validator implementation in it:

META-INF/services/javax.validation.ConstraintValidator
com.acme.validation.validators.DurationMinValidator

After all of this, your new constraint annotation on Duration elements can be used like this:

Task.java
public class Task {

    private String taskName;
    @DurationMin(value = 2, units = ChronoUnit.HOURS)
    private Duration timeSpent;

    public Task(String taskName, Duration timeSpent) {
        this.taskName = taskName;
        this.timeSpent = timeSpent;
    }
}

The project structure should look similar to next one:

project structure, align=

The whole source code presented here can be found in the hibernate-demos repository on GitHub.

Use standard constraints for non standard classes

Now let’s consider the case where you would want a standard Bean Validation constraint to support some other type, besides the ones that are already supported.

ThreeTen Extra types validation

As we were talking about date/time related validation, let’s stay on the same topic for this example as well. In this section we will look at ThreeTen Extra types - a great library that provides additional date and time classes to complement those already present in Java.

Bean Validation provides support for validating temporal types via the @Past/@Future annotations. So we would want to use these annotations on ThreeTen Extra types as well. To keep this example simple we will provide validators only for YearWeek and YearQuarter.

Let’s start with implementing ConstraintValidator<Future, YearWeek> interface:

FutureYearWeekValidator.java
public class FutureYearWeekValidator implements ConstraintValidator<Future, YearWeek> {

    @Override
    public void initialize(Future constraintAnnotation) {
    }

    public boolean isValid(YearWeek value, ConstraintValidatorContext context) {
        if ( value == null ) {
            return true;
        }
        return YearWeek.now().isBefore( value );
    }
}

The next step is to provide a list of implemented validators in META-INF/services/javax.validation.ConstraintValidator file:

META-INF/services/javax.validation.ConstraintValidator
com.acme.validation.validators.FutureYearQuarterValidator
com.acme.validation.validators.FutureYearWeekValidator
com.acme.validation.validators.PastYearQuarterValidator
com.acme.validation.validators.PastYearWeekValidator

After this we can package it all in a JAR file and we are ready to use our validators and share them with the world!

In the end our project structure should look similar to this:

project structure, align=

Now you can place @Past/@Future annotations on YearQuarter and YearWeek types like this:

PastEvent.java
public static class PastEvent {

    @Past
    private YearWeek yearWeek;
    @Past
    private YearQuarter yearQuarter;

    public PastEvent(YearWeek yearWeek, YearQuarter yearQuarter) {
        this.yearWeek = yearWeek;
        this.yearQuarter = yearQuarter
    }
}
FutureEvent.java
public static class FutureEvent {

    @Future
    private YearWeek yearWeek;
    @Future
    private YearQuarter yearQuarter;

    public FutureEvent(YearWeek yearWeek, YearQuarter yearQuarter) {
        this.yearWeek = yearWeek;
        this.yearQuarter = yearQuarter
    }
}

You also can find this example at GitHub.

Conclusion

So, as you can see, custom constraint validators can be built and shared in a fully self-contained way. And it can be done in a few simple steps:

  • create a validator implementing the ConstraintValidator interface

  • reference this validator’s fully qualified name in a META-INF/services/javax.validation.ConstraintValidator file

  • (optional) define custom/default messages by adding a ContributorValidationMessages.properties file

  • package it all as a JAR

  • you are ready to share your constraints, people can add them by simply ading your JAR to the classpath

Meet Kevin Peters

Posted by    |       |    Tagged as Discussions Hibernate ORM Interview

In this post, I’d like you to meet Kevin Peters, a Software Developer from Germany and Hibernate aficionado.

Kevin Peters, align=

Hi, Kevin. Would you like to introduce yourself and tell us a little bit about your developer experience?

My name is Kevin Peters, and I live in Germany where I work as a Software Developer. My first contact with the Java language was around 2005 during my vocational training, and I fell in love with it immediately.

I worked for several companies leveraging Java and Spring to implement ERP extensions, customizing eCommerce systems and PIM solutions. Nearly one year ago, I joined the GBTEC Software + Consulting AG, one of the leading suppliers of business process management (BPM) software, and there we are now reimplementing a BPM system in a cloud-based manner using Dockerized Spring Boot microservices.

You have recently mentioned on Twitter a DataSource proxy solution for validating auto-generated statements. Can you tell us what about this tool and how it works?

We use Spring Data JPA with Hibernate as JPA provider to implement our persistence layer, and we really enjoy the convenience coming along with it. But we also know about the "common" obstacles like Cartesian Products or the N+1 query problem while working with an ORM framework.

In our daily technical discussions and during knowledge transfer sessions we try to raise awareness for these topics among our colleagues, and in my opinion, the best way to achieve this is implementing tests and real world code examples showing that practically.

I started to prepare a small mapping example for one of our technical meetings, called "techtime", to demonstrate the "unordered element collection recreation" issue, and I wanted to show the unexpected amount of queries fired in this simple use case.

Fortunately, I came across the ttddyy/datasource-proxy GitHub project which helped me a lot to make that problem tangible. The datasource-proxy project empowers you to wrap your existing datasource with a proxy and allows you to count all executed queries separated by query type (e.g. INSERT, UPDATE, etc.). With that opportunity you can not only write tests which assert that you are doing the right thing within your use cases, you can also check if you are doing it in an effective way and avoid the traps I did mention before.

At the time when our Coding Architect Ingo Griebsch suggested to use this approach to enhance our test environment by automating the hunt for performance penalties, you caught us talking about your article on Twitter.

Proxies are a great way to add cross-cutting concerns without cluttering business logic. For instance, FlexyPool brings Monitoring and Fallback capabilities to connection pools. Are you using Proxies for other concerns as well, like logging statements?

There are many ways to enrich application code with proxies, facades or aspects. Starting with small things like logging with a facade like SLF4J, using Spring Security for access control, Hystrix service-to-service communication or even "basic" stuff like transactions in Spring Data, all these features are working with proxies, and we won’t miss them anymore.

Why did you choose Hibernate for that particular project, and did it meet your expectations, especially when it comes to application performance?

Hibernate provides a lot of convenience to us, especially if we combine it with Spring Data JPA. But the fact I enjoy most is that you can still switch to Hibernate specific features like Hibernate Named Queries or special Hibernate annotations.

It’s important to know when you can relax using "magic" ORM features and when the opposite is needed - forgo bidirectional relations and write HQL instead or even using database native queries to receive complex data. In our opinion, Hibernate offers the best balance between convenience and performance if one knows how to use it.

Hence, we have a quite complex data model and customers which store a lot of data it’s vital for our software to fetch and write data in a performant way in every of our use cases. And in case of any doubts, at least your articles help us getting things done right.

We always value feedback from our users, so can you tell us what you’d like us to improve or are there features that we should add support for?

In general, we love the feature set of Hibernate. Only the support of UNION HQL queries/Criteria API would be an awesome feature that we missed recently.

Thank you, Kevin, for taking your time. It is a great honor to have you here. To reach Kevin, you can follow him on Twitter.

Meet Marco Pivetta

Posted by    |       |    Tagged as Discussions Hibernate ORM Interview

In this post, I’d like you to meet Marco Pivetta, who is one of the maintainer of Doctrine, a suite of PHP projects that were inspired by Hibernate ORM.

Marco Pivetta, align=

Hi, Marco. Would you like to introduce yourself and tell us a little bit about your developer experience?

I’m Marco "Ocramius" Pivetta, an Italian PHP consultant, currently living in Germany. Yes, the nickname is weird, but it comes from an era of Quake 3 Arena, Unreal Tournament & co.

I’ve been tinkering with computers since I was a child, and have been working with PHP for more than half my life now, developing a love-hate relationship with the language. Interestingly, I didn’t start with the usual Joomla/Wordpress/Drupal/etc, but built a quite complex website that interacted with a browser game called "OGame", and scraped game information through a Firefox addon that would then provide an additional information to the players.

The reason why this project ("stogame") is important for me is that it included extremely challenging problems to be solved for a rookie with no help at all, and is still one of the most complex projects that I worked on:

  • XSS/SQL injections - had those, wasn’t fun

  • queuing mechanisms to sync browser extensions and the website - invented my own system

  • optimizing queries and indexes on ~60Gb of MySQL MyISAM tables

  • disaster recoveries on such a system - had those too, wasn’t fun either

  • real-time push mechanisms for clients via BOSHXMPP

  • simplistic prediction engine to aid players in decision making

All of the above were built by 15-years-old-me by just spending countless sleepless nights on it, and also jeopardizing my school evaluations. Still, this was before libraries, design patterns, mentoring, Github: only me, some friends, and a good amount of design and prediction work.

I then moved on, gave up on the project, failed university (I’m a terrible student), got a few jobs and started using frameworks. Eventually, I got to work with all of the typical DB abstraction approaches:

  • Active Record (with ZendFramework)

  • Table Data Gateway - in a custom solution

  • Data Mapper - in a Java EE project

I liked the JPA approach in the Java EE project so much that I started looking for a PHP analogue solution for my daytime job, and ended up discovering Doctrine 2.

Since then, I started getting more and more involved with the project, starting from answering questions on the mailing list and StackOverflow. Benjamin Eberlei, who was the lead on the project at that time, pushed me towards contributing with actual code changes back in 2011.

Eventually, I became part of the maintainers of the project, and that also boosted my career, allowing me to become a consultant for Roave, which allows me to see dozens of different projects, teams and tools every month, as well as a public speaker.

You are one of the developers of Doctrine ORM framework. Can you please tell us what’s the goal of Doctrine?

I am actually not one of the developers, but one of the current maintainers. The initial designers of the current Doctrine 2 ORM, as far as I know, are Jonathan Wage, Guilherme Blanco, Benjamin Eberlei and Roman Borschel. I can probably still answer the question: Doctrine ORM tries to abstract the "database thinking" away from PHP software projects, while still being a leaky abstraction on purpose.

To clarify, most PHP developers are used to developing applications from the database up to the application layer, rather than from the domain logic down, and that’s a quite widespread problem that leads to hardly maintainable and unreadable code. This tool gets rid of most of those problems, by still allowing developers to access the database directly when needed.

Ruby on Rails employs the Active Record pattern. Why did Doctrine choose the ORM paradigm instead?

Interestingly, Doctrine 1.x was an Active Record library, and also a quite good one, but it became evident quite quickly that the JPA specification and Data Mapper plus Unit of Work were better solutions altogether.

Specifically, the Data Mapper approach allows consumers of the library to write abstractions that decouple the tool from the domain almost completely (there are always limitations to this). The Unit of Work pattern has an increased memory impact for PHP applications, but also massively reduces required query operations (via in-memory identity maps) while adding some transactional boundaries, and that is a big win for most PHP apps, which often don’t even use transactions at all.

There are more advantages, but I personally wouldn’t ever consider using Active Record again due to its limitations and inherent framework coupling. This doesn’t mean that Active Record doesn’t work, but I’ve been burnt many more times with AR than with DM.

Since Hibernate ORM has been influencing Doctrine, can you tell use the similarities and differences between these two frameworks?

Doctrine is hugely inspired by Hibernate and the JPA, although we couldn’t really copy things, both due to licensing issues and life-cycle differences in Java and PHP software.

Doctrine resembles Hibernate in the Unit of Work, mappings, basic event system, second level cache and the DQL language (HQL in Hibernate). We even designed an annotation system for PHP, since the language doesn’t support them, and it currently is the de-facto standard for custom annotations in PHP libraries, and we initially only needed this to simulate inline mappings like Hibernate allows them.

Where things differ a lot are flexibility and lifecycle, since Java is an AOT-compiled language with a powerful JIT and generally deployed in long-running applications.

PHP is an interpreted language, and its strength is also its pitfall: the typical share-nothing architecture allows for short-lived, memory-safe, retry-able application runs. That also means that we have no connection pooling, and the ORM internals are much more inflexible and less event-driven than Hibernate’s due to memory and execution time constraints. That also means that we rarely encounter memory issues due to large Unit of Work instances, and connections and entity instances aren’t shared across separate web application page loads, and slow ORM will unlikely slow down an entire application server.

Another huge difference is managed state: DETACHED makes little sense in the PHP world, since a detached entity may only come from serialized state. In Doctrine 3.x, we are planning to remove support for detaching entities, since storing serialized objects in PHP is generally leading to security issues and more trouble.

As you can see, the differences are indeed mostly in the lifecycle, but each language and framework has its strengths and pitfalls.

We always value feedback from our users, so can you tell us what you’d like us to improve or are there features that we should add support for?

I’m probably being weird here, but I don’t lack any particular features from either ORM at this time. What would be interesting is reducing support for entity and transaction lifecycle events, since most consumers of these ORMs tend to code application and domain logic in those, while they were mostly intended for technical tasks, such as creating audit logs and executing pre- and post- DB cleanup tasks.

A possible improvement is to explore saving/loading of single aggregate-root-acting entities attached to a Unit of Work, which is only responsible for tracking state in child aggregates. This is only to prevent sharing entity references across aggregates, and to prevent DB transactions from crossing aggregate root boundaries.

Thank you, Marco, for taking your time. It is a great honor to have you here. To reach Marco, you can follow him on Twitter.

The MySQL Dialect refactoring

Posted by    |       |    Tagged as Discussions Hibernate ORM

Starting with Hibernate ORM 5.2.8, MariaDB gets its own Hibernate dialects.

Why?

While working on the new MariaDB Dialects, I realized that the MySQL Dialects would benefit from simplifying the version hierarchy.

Previously, the MySQL Dialects used to looks like that:

MySQL Dialects before refactoring, align=

As you can see, because of the various MySQL storage engines (e.g. MyISAM and InnoDB), the class hierarchy has diverged in multiple branches. Once we integrated Hibernate Spatial, the MySQL Dialects have become even more convoluted.

For this reason, we created the HHH-11473 Jira issue, which is fixed in Hibernate 5.2.9.

How do we stand now?

After refactoring, the MySQL Dialects look as follows:

MySQL Dialects after refactoring, align=

The following Dialects have been deprecated, therefore, they were not added to the class diagram above:

MySQLMyISAMDialect

Use MySQLDialect instead, as well as the hibernate.dialect.storage_engine=myisam Environment Variable or System Property.

MySQLInnoDBDialect

Use MySQLDialect instead, as well as the hibernate.dialect.storage_engine=innodb Environment Variable or System Property.

MySQL5InnoDBDialect

Use MySQL5Dialect instead, as well as the hibernate.dialect.storage_engine=innodb Environment Variable or System Property.

MySQL57InnoDBDialect

Use MySQL57Dialect instead.

MySQL5InnoDBSpatialDialect

Use MySQL5SpatialDialect instead, as well as the hibernate.dialect.storage_engine=innodb Environment Variable or System Property.

MySQL56InnoDBSpatialDialect

Use MySQL56SpatialDialect which defaults to InnoDB by default.

The MySQLStorageEngine abstraction encapsulates the difference between various storage engines, By delegating this responsibility to a new abstraction, the MySQL Dialect hierarchy got a lot simpler.

Traditionally, MySQL used the non-transactional MyISAM storage engine, and this is the default storage engine for all Dialects that are older than MySQL55Dialect. From MySQL55Dialect onwards, the InnoDB storage engine is used by default.

You can always override the default storage engine by providing the hibernate.dialect.storage_engine Environment Variable or System Property. Unlike other Hibernate configuration properties, this one must not be provided via persistence.xml because the Dialect is bootstrapped prior to the configuration management mechanism.

Conclusion

The deprecated Dialects will be available for a while, but they will surely be removed in a future version of Hibernate, so you better use the new ones instead. This refactoring is useful for two reasons. First, supporting MySQL 8.0 requires a single Dialect, not two. Second, it’s easier for our users as well since the choice is much more straightforward now since there is only one Dialect associated to a given MySQL version.

Hibernate Community Newsletter 4/2017

Posted by    |       |    Tagged as Discussions Hibernate ORM

Welcome to the Hibernate community newsletter in which we share blog posts, forum, and StackOverflow questions that are especially relevant to our users.

Looking for your feedback

We are looking for your feedback in about Hibernate bootstrap in cloud environments. Check out this article for more details. If you have any idea or proposal, don’t hesitate to use the comments section below the aforementioned article.

Articles

Sometimes, it’s easy to miss the basic concepts, and relational databases are no different. Check out this article about how does a relational database work.

We released dedicated Dialects for MariaDB, so you don’t have to use the MySQL-specific Dialects when working with MariaDB.

Integration testing is of paramount importance when building an enterprise application. However, many projects rely on in-memory databases (e.g. H2, HSQLDB) for testing, while in production they use Oracle, SQL Server, PostgreSQL or MySQL. In this article, you’ll find how you can run integration tests faster using tmpfs and Docker.

If you’re using JCache through Spring, Hibernate, and Ehcache, this article explains how you can prevent spontaneous cache creation.

Emmanouil Gkatziouras wrote two articles about Hibernate and Hazelcast as a 2nd-level caching provider:

Russ Thomas wrote https://sqljudo.wordpress.com/2014/12/29/what-every-dba-and-swe-should-know-about-ef/[a comprehensive list of ORM Anti-Patterns. Although the article was written for Entity Framework, the tups apply to JPA or Hibernate.

For our Portuguese readers, Rhuan Henrique Rocha da Silva wrote an article about the meaning of mappedBy in JPA and Hibernate.

Thorben Janssen wrote an article about adding Full-Test Search capabilities to a Hibernate application.

Questions and answers

MariaDB Dialects

Posted by    |       |    Tagged as Discussions Hibernate ORM

Starting with Hibernate ORM 5.2.8, MariaDB gets its own Hibernate dialects.

About MariaDB

MariaDB is a MySQL fork that emerged in 2009 as a drop-in replacement for MySQL. While for a while, MariaDB and MySQL offered similar functionalities, with time, both MariaDB and MySQL have diverged.

For this reason, we created the HHH-11457 issue, which is fixed in the Hibernate ORM 5.2.8.

Dialect variants

For the moment, you can use one of the following two options:

MariaDBDialect

which is the base class for all MariaDB dialects and it works with any MariaDB version

MariaDB53Dialect

which is intended to be used with MariaDB 5.3 or newer versions

In time, we will add new Dialects based on newer capabilities introduced by MariaDB.

Connection properties

While to connect to a MySQL application, the connection properties look as follows:

  • 'db.dialect' : 'org.hibernate.dialect.MySQL57InnoDBDialect',

  • 'jdbc.driver': 'com.mysql.jdbc.Driver',

  • 'jdbc.user' : 'hibernate_orm_test',

  • 'jdbc.pass' : 'hibernate_orm_test',

  • 'jdbc.url' : 'jdbc:mysql://127.0.0.1/hibernate_orm_test'

For MariaDB, the connection properties look like this:

  • 'db.dialect' : 'org.hibernate.dialect.MariaDB53Dialect',

  • 'jdbc.driver': 'org.mariadb.jdbc.Driver',

  • 'jdbc.user' : 'hibernate_orm_test',

  • 'jdbc.pass' : 'hibernate_orm_test',

  • 'jdbc.url' : 'jdbc:mariadb://127.0.0.1/hibernate_orm_test'

While the URL includes the mariadb database identifier, the MariaDB53Dialect supports Time and Timestamp with microsecond precision, just like MySQL57InnoDBDialect.

Conclusion

If you are using MariaDB, it’s best to use the MariaDB-specific Dialects from now on since it’s much easier to match the MariaDB version with its appropriate Hibernate Dialect.

Recently, the team has been discussing improvements around Hibernate (ORM) usage within cloud based apps and microservices. In particular the fundamental assumption that things will break regularly on these platforms and that services should be resilient to failures.

The problem

In microservices or cloud architectures, services are started in different orders (usually beyond your control). It is possible for the app using Hibernate ORM to be started before the database. At the moment, Hibernate ORM does not like that and will explicitly fail (exception) if it can’t connect to the database.

Another related concern is to consider what is happening if the database is stopped for a while after the app running Hibernate ORM has started and resume working shortly after.

Solution 1: Hibernate waits and retries at boot time

Some users have asked us to delay and retry the connection process in case the database is not present at boot time. That would work and solve the bootstrap problem. It would not solve the database gone while running the app but here at least you have your transaction and the error propagation mechanism covering you. Plus at development time, the boot time problem gone would be quite nice already.

I understand that this is probably a quick win to implement this, but better be sure of the problem before adding that feature. It feels to me that Hibernate ORM bootstrap is not the ideal area to fix that problem. But at the end of the day if it helps enough, it would be worth it.

We are exploring that option and considering alternatives and that’s where we need your feedback.

Wait and retry vs platform notification

In this blog post, I mention the wait and retry approach. It can be replaced by a notification from the cloud platform when a service is up / down.

This avoids the regular polling process at the cost of having to rely on various integrations from various cloud platforms.

Solution 1.b: The connection pool waits and retries

It probably would be better if the connection pool Hibernate ORM uses, implements that logic but there are more than one connection pool Hibernate supports. That’s a minor variation on solution 1.

Solution 2: Hibernate boots in non-functioning mode

If Hibernate ORM cannot connect to the datasbase, it continues its bootstrap process. If an EntityManager is asking for a connection while the database is still unavailable, a well defined exception is raised. To not flood the system, a wait and retry system for connection checking would be in place to only try a few times even when lots of EntityManager are requested.

There are some subtle difficulties here on concurrency and on the fact that we use info from the bootstrap connection to configure Hibernate ORM. The most visible option guessed from the connection is the dialect to use. On the other hand, stopping the app boot process while waiting and retrying like solution 1 proposes is probably not without its challenges.

The exception raised by Hibernate ORM upon DB inaccessibility needs to be treated properly by the application (framework) being used. Like a global try catch that moves the application in degraded mode or propagating the execution error to the client (e.g. HTTP error 500). It might even be helpful if Hibernate ORM was exposing the not ready status via an explicit API.

This could be tied to a health check from the cloud platform. The application would report the not ready but trying status via a /health endpoint that the orchestrator would use.

On database connection breaking

There are many reasons for failing to connect to a database:

  1. Host unreachable

  2. DB server denying access temporarily (e.g. load)

  3. Incorrect port setting

  4. Incorrect credentials

  5. And many more

Should the system go into the wait and retry mode for cases 3, 4, 5? Or should it refuse to deploy?

Solution 3: the smart app (framework)

Another solution is for the app to have a smart overall bootstrap logic. It tries to eagerly start but if a Hibernate ORM connection error occurs, only the inbound request framework is started. It will regularly try and boot and in the mean time return HTTP 500 errors or similar.

This requires an app framework that could handle that. It embeds circuit breaker logic in the app and can better react to specific errors. I wonder how common such frameworks are though.

This is in spirit the same solution as solution 2 except it is handled at the higher level of the app (framework) vs Hibernate ORM.

Solution 4: the cloud / MSA platform restart the apps

An arguably better solution would be for the cloud platform to handle these cases and restart apps that fail to deploy in these situations. It likely requires some kind of service dependency management and a bit of smartness from the cloud infra. The infrastructure would upon specific error code thrown at boot time, trigger a wait and retry deployment logic. There is also a risk of a dependency circularity leading to a never starting system.

I guess not all cloud infra offer this and we would need an alternative solution. OpenShift let’s you express dependencies to make sure a given service is started before another. The user would have to declare that dependency of course.

Solution 5: proxy!

Another solution is to put proxies either before the app inbound requests and/or between the app and the database. Proxy is the silver bullet that lots for cloud platforms uses to solve world hunger in the digital universe.

How many proxies and routing logic does it take to serve a "Hello world!" in the cloud?
Who proxies the proxies?

:)

This approach has the advantage of not needing customized apps or libraries. The inconvenience is more intermediary points between your client and the app or data.

If the proxy is before the application, then it needs a health check or a feedback from the boot system to wait and retry the re-deployment of the application on a regular basis. I’m again not certain cloud infrastructures offer all of this infrastructure.

If the proxy is between Hibernate ORM and the database (like HAProxy for MySQL), you’re still facing some timeout exception on the JDBC side. Which means the application will fail to boot. But at least, the proxy could implement the wait and retry logic.

My questions to you

Do you have any input on this subject:

  • what’s your opinion?

  • what’s your experience when deploying cloud apps?

  • any alternative solution you have in mind?

  • any resource you found interesting covering this subject?

  • would you benefit from solution 1?

  • would you benefit from solution 2?

Any feedback to help us think this problem further is what we need :)

Building Multi-Release JARs with Maven

Posted by    |       |    Tagged as Discussions

Java 9 comes with a new feature very useful to library authors: multi-release JARs (JEP 238).

A multi-release JAR (MR JAR) may contain multiple variants of one and the same class, each targeting a specific Java version. At runtime, the right variant of the class will be loaded automatically, depending on the Java version being used.

This allows library authors to take advantage of new Java versions early on, while keeping compatibility with older versions at the same time. If for instance your library performs atomic compare-and-set operations on variables, you may currently be doing so using the sun.misc.Unsafe class. As Unsafe has never been meant for usage outside the JDK itself, Java 9 comes with a supported alternative for CAS logics in form of var handles. By providing your library as an MR JAR, you can benefit from var handles when running on Java 9 while sticking to Unsafe when running on older platforms.

In the following we’ll discuss how to create an MR JAR using Apache Maven.

Structure of a Multi-Release JAR

Multi-Release JARs contain several trees of class files. The main tree is at the root of the JAR, whereas version-specific trees are located under META-INF/versions, e.g. like this:

JAR root
- Foo.class
- Bar.class
+ META-INF
   - MANIFEST.MF
   + versions
      + 9
         - Bar.class

Here the Foo and the Bar class from the JAR root will be used on Java runtimes which are not aware of MR JARs (i.e. Java 8 and earlier), whereas Foo from the JAR root and Bar from META-INF/versions/9 will be used under Java 9 and later. The JAR manifest must contain an entry Multi-Release: true to indicate that the JAR is an MR JAR.

Example: Getting the Id of the Current Process

As an example let’s assume we have a library which defines a class providing the id of process (PID) it is running in. PIDs shall be represented by a descriptor comprising the actual PID and a String describing the provider of the PID:

src/main/java/com/example/ProcessIdDescriptor.java
package com.example;

public class ProcessIdDescriptor {

    private final long pid;
    private final String providerName;

    // constructor, getters ...
}

Up to Java 8, there is no easy way to obtain the id of the running process. One rather hacky approach is to parse the return value of RuntimeMXBean#getName() which is "pid@hostname" in the OpenJDK / Oracle JDK implementation. While that behavior is not guaranteed to be portable across implementations, let’s use it as the basis for our default ProcessIdProvider:

src/main/java/com/example/ProcessIdProvider.java
package com.example;

public class ProcessIdProvider {

    public ProcessIdDescriptor getPid() {
        String vmName = ManagementFactory.getRuntimeMXBean().getName();
        long pid = Long.parseLong( vmName.split( "@" )[0] );
        return new ProcessIdDescriptor( pid, "RuntimeMXBean" );
    }
}

Also let’s create a simple main class for displaying the PID and the provider it was retrieved from:

src/main/java/com/example/Main.java
package com.example;

public class Main {

    public static void main(String[] args) {
        ProcessIdDescriptor pid = new ProcessIdProvider().getPid();

        System.out.println( "PID: " + pid.getPid() );
        System.out.println( "Provider: " + pid.getProviderName() );
    }
}

Note how the source files created so far are located in the regular src/main/java source directory.

Now let’s create another variant of ProcessIdDescriptor based on Java 9’s new ProcessHandle API, which eventually provides a portable way for obtaining the current PID. This source file is located in another source directory, src/main/java9:

src/main/java9/com/example/ProcessIdProvider.java
package com.example;

public class ProcessIdProvider {

    public ProcessIdDescriptor getPid() {
        long pid = ProcessHandle.current().getPid();
        return new ProcessIdDescriptor( pid, "ProcessHandle" );
    }
}

Setting up the build

With all the source files in place, it’s time to configure Maven so an MR JAR gets built.

Three steps are required for that. The first thing is to compile the additional Java 9 sources under src/main/java9. I hoped I could simply set up another execution of the Maven compiler plug-in for that, but I could not find a way which only would compile src/main/java9 but not the ones from src/main/java for second time.

As a work-around, the Maven Antrun plug-in can be used for configuring a second javac run just for the Java 9 specific sources:

pom.xml
...
<properties>
    <java9.sourceDirectory>${project.basedir}/src/main/java9</java9.sourceDirectory>
    <java9.build.outputDirectory>${project.build.directory}/classes-java9</java9.build.outputDirectory>
</properties>
...
<build>
    ...
    <plugins>
        ...
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-antrun-plugin</artifactId>
            <executions>
                <execution>
                    <id>compile-java9</id>
                    <phase>compile</phase>
                    <configuration>
                        <tasks>
                            <mkdir dir="${java9.build.outputDirectory}" />
                            <javac srcdir="${java9.sourceDirectory}" destdir="${java9.build.outputDirectory}"
                                classpath="${project.build.outputDirectory}" includeantruntime="false" />
                        </tasks>
                    </configuration>
                    <goals>
                        <goal>run</goal>
                    </goals>
                </execution>
            </executions>
        </plugin>
        ...
    </plugins>
    ...
</build>
...

This uses the target/classes directory (containing the class files emitted by the default compilation) as the classpath, allowing to refer to classes common for all Java versions supported by our MR JAR, e.g. ProcessIdDescriptor. The compiled classes go into target/classes-java9.

The next step is to copy the compiled Java 9 classes into target/classes so they will later be put to the right place within the resulting JAR. The Maven resources plug-in can be used for that:

pom.xml
...
<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-resources-plugin</artifactId>
    <executions>
        <execution>
            <id>copy-resources</id>
            <phase>prepare-package</phase>
            <goals>
                <goal>copy-resources</goal>
            </goals>
            <configuration>
                <outputDirectory>${project.build.outputDirectory}/META-INF/versions/9</outputDirectory>
                <resources>
                    <resource>
                        <directory>${java9.build.outputDirectory}</directory>
                    </resource>
                </resources>
            </configuration>
        </execution>
    </executions>
</plugin>
...

This will copy the Java 9 class files from target/classes-java9 to target/classes/META-INF/versions/9.

Finally, the Maven JAR plug-in needs to be configured so the Multi-Release entry is added to the manifest file:

pom.xml
...
<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-jar-plugin</artifactId>
    <configuration>
        <archive>
            <manifestEntries>
                <Multi-Release>true</Multi-Release>
                <Main-Class>com.example.Main</Main-Class>
            </manifestEntries>
        </archive>
        <finalName>mr-jar-demo.jar</finalName>
    </configuration>
</plugin>
...

And that’s it, we got everything together to build a multi-release JAR. Trigger the build via mvn clean package (using Java 9) to create the JAR in the target directory.

In order to take a look whether the JAR contents is alright, list its contents via jar -tf target/mr-jar-demo.jar. You should see the following:

...
com/example/Main.class
com/example/ProcessIdDescriptor.class
com/example/ProcessIdProvider.class
META-INF/versions/9/com/example/ProcessIdProvider.class
...

Eventually, let’s execute the JAR via java -jar target/mr-jar-demo.jar and examine its output. When using Java 8 or earlier, you’ll see the following:

PID: <some pid>
Provider: RuntimeMXBean

Whereas on Java 9, it’ll be this:

PID: <some pid>
Provider: ProcessHandle

I.e. the ProcessIdProvider class from the JAR root will be used on Java 8 and earlier, and the one from META-INF/versions/9 on Java 9.

Conclusion

While javac, jar, java and other JDK tools already support multi-release JARs, build tools like Maven still need to catch up. Luckily, it can be done using some plug-ins for the time being, but it’s my hope that Maven et al. will provide proper support for creating MR JARs out of the box some time soon.

Others have been thinking about the creation of MR JARs, too. E.g. check out this post by my colleague David M. Lloyd. David uses a separate Maven project for the Java 9 specific classes which are then copied back into the main project using the Maven dependency plug-in. Personally, I prefer to have all the sources within one single project, as I find that a tad simpler, though it’s not without quirks either. Specifically, if you have both, src/main/java and src/main/java9, configured as source directories within your IDE, you’ll get an error about the duplicated class ProcessIdProvider. This can be ignored (you might also remove src/main/java9 as a source directory from the IDE if you don’t need to touch it), but it may be annoying to some.

One could think about having the Java 9 classes in another package, e.g. java9.com.example and then using the Maven shade plug-in to relocate them to com.example when building the project, though this seems quite a lot of effort for a small gain. Ultimately, it’d be desirable if IDEs also added support for MR JARs and multiple compilations with different source and target directories within a single project.

Any feedback on this or other approaches for creating MR JARs is welcome in the comments section below. The complete source code of this blog post can be found on GitHub.

Hibernate Community Newsletter 3/2017

Posted by    |       |    Tagged as Discussions Hibernate ORM

Welcome to the Hibernate community newsletter in which we share blog posts, forum, and StackOverflow questions that are especially relevant to our users.

Articles

If you’re using MySQL, then the GenerationType.AUTO identifier strategy is not the best option. Check out this article for more details and a very simple workaround.

Injecting a JPA/Hibernate Entity Managers wit CDI and Weld is extremely easy. Check out this article for more details.

Concurrency Control is a very difficult topic, and relational databases are no different. If you wonder how different database systems prevent Phantom reads or you are curious about how Two-phase Locking and MVCC work, you should definitely read this article.

Arno Huetter wrote a list of tips to improve application performance when you’re using JPA and Hibernate.

If you’re working on a database system which does not allow you to create temporary tables, then rest assured. Hibernate 5.2.8 adds support for non-temporary table bulk-id strategies.

If you want to separate the entity validation logic from the entity data structures, Hibernate Validator is a very attractive solution.

Time to upgrade

This article is about the HHH-11262 JIRA issue which now allows the bulk-id strategies to work even when you cannot create temporary tables.

Class diagram

Considering we have the following entities:

Class diagram, align=

The Person entity is the base class of this entity inheritance model, and is mapped as follows:

@Entity(name = "Person")
@Inheritance(
    strategy = InheritanceType.JOINED
)
public class Person
    implements Serializable {

    @Id
    private Integer id;

    @Id
    private String companyName;

    private String name;

    private boolean employed;

    //Getters and setters omitted for brevity

    @Override
    public boolean equals(Object o) {
        if ( this == o ) {
            return true;
        }
        if ( !( o instanceof Person ) ) {
            return false;
        }
        Person person = (Person) o;
        return Objects.equals(
            getId(),
            person.getId()
        ) &&
        Objects.equals(
            getCompanyName(),
            person.getCompanyName()
        );
    }

    @Override
    public int hashCode() {
        return Objects.hash(
            getId(), getCompanyName()
        );
    }
}

Both the Doctor and Engineer entity classes extend the Person base class:

@Entity(name = "Doctor")
public class Doctor
    extends Person {
}

@Entity(name = "Engineer")
public class Engineer
    extends Person {

    private boolean fellow;

    //Getters and setters omitted for brevity
}

Inheritance tree bulk processing

Now, when you try to execute a bulk entity query:

int updateCount = session.createQuery(
    "delete from Person where employed = :employed" )
.setParameter( "employed", false )
.executeUpdate();

Hibernate executes the following statements:

create temporary table
    HT_Person
(
    id int4 not null,
    companyName varchar(255) not null
)

insert
into
    HT_Person
    select
        p.id as id,
        p.companyName as companyName
    from
        Person p
    where
        p.employed = ?

delete
from
    Engineer
where
    (
        id, companyName
    ) IN (
        select
            id,
            companyName
        from
            HT_Person
    )

delete
from
    Doctor
where
    (
        id, companyName
    ) IN (
        select
            id,
            companyName
        from
            HT_Person
    )

delete
from
    Person
where
    (
        id, companyName
    ) IN (
        select
            id,
            companyName
        from
            HT_Person
    )

HT_Person is a temporary table that Hibernate creates to hold all the entity identifiers that are to be updated or deleted by the bulk id operation. The temporary table can be either global or local, depending on the underlying database capabilities.

What if you cannot create a temporary table?

As the HHH-11262 issue describes, there are use cases when the application developer cannot use temporary tables because the database user lacks this privilege.

In this case, we defined several options which you can choose depending on your database capabilities:

  • InlineIdsInClauseBulkIdStrategy

  • InlineIdsSubSelectValueListBulkIdStrategy

  • InlineIdsOrClauseBulkIdStrategy

  • CteValuesListBulkIdStrategy

InlineIdsInClauseBulkIdStrategy

To use this strategy, you need to configure the following configuration property:

<property name="hibernate.hql.bulk_id_strategy"
          value="org.hibernate.hql.spi.id.inline.InlineIdsInClauseBulkIdStrategy"
/>

Now, when running the previous test case, Hibernate generates the following SQL statements:

select
    p.id as id,
    p.companyName as companyName
from
    Person p
where
    p.employed = ?

delete
from
    Engineer
where
        ( id, companyName )
    in (
        ( 1,'Red Hat USA' ),
        ( 3,'Red Hat USA' ),
        ( 1,'Red Hat Europe' ),
        ( 3,'Red Hat Europe' )
    )

delete
from
    Doctor
where
        ( id, companyName )
    in (
        ( 1,'Red Hat USA' ),
        ( 3,'Red Hat USA' ),
        ( 1,'Red Hat Europe' ),
        ( 3,'Red Hat Europe' )
    )

delete
from
    Person
where
        ( id, companyName )
    in (
        ( 1,'Red Hat USA' ),
        ( 3,'Red Hat USA' ),
        ( 1,'Red Hat Europe' ),
        ( 3,'Red Hat Europe' )
    )

So, the entity identifiers are selected first and used for each particular update or delete statement.

The IN clause row value expression has long been supported by Oracle, PostgreSQL, and nowadays by MySQL 5.7. However, SQL Server 2014 does not support this syntax, so you’ll have to use a different strategy.

InlineIdsSubSelectValueListBulkIdStrategy

To use this strategy, you need to configure the following configuration property:

<property name="hibernate.hql.bulk_id_strategy"
          value="org.hibernate.hql.spi.id.inline.InlineIdsSubSelectValueListBulkIdStrategy"
/>

Now, when running the previous test case, Hibernate generates the following SQL statements:

select
    p.id as id,
    p.companyName as companyName
from
    Person p
where
    p.employed = ?

delete
from
    Engineer
where
    ( id, companyName ) in (
        select
            id,
            companyName
        from (
        values
            ( 1,'Red Hat USA' ),
            ( 3,'Red Hat USA' ),
            ( 1,'Red Hat Europe' ),
            ( 3,'Red Hat Europe' )
        ) as HT
            (id, companyName)
    )

delete
from
    Doctor
where
    ( id, companyName ) in (
         select
            id,
            companyName
        from (
        values
            ( 1,'Red Hat USA' ),
            ( 3,'Red Hat USA' ),
            ( 1,'Red Hat Europe' ),
            ( 3,'Red Hat Europe' )
        ) as HT
            (id, companyName)
    )

delete
from
    Person
where
    ( id, companyName ) in (
        select
            id,
            companyName
        from (
        values
            ( 1,'Red Hat USA' ),
            ( 3,'Red Hat USA' ),
            ( 1,'Red Hat Europe' ),
            ( 3,'Red Hat Europe' )
        ) as HT
            (id, companyName)
    )

The underlying database must support the VALUES list clause, like PostgreSQL or SQL Server 2008. However, this strategy requires the IN-clause row value expression for composite identifiers so you can use this strategy only with PostgreSQL.

InlineIdsOrClauseBulkIdStrategy

To use this strategy, you need to configure the following configuration property:

<property name="hibernate.hql.bulk_id_strategy"
          value="org.hibernate.hql.spi.id.inline.InlineIdsOrClauseBulkIdStrategy"
/>

Now, when running the previous test case, Hibernate generates the following SQL statements:

select
    p.id as id,
    p.companyName as companyName
from
    Person p
where
    p.employed = ?

delete
from
    Engineer
where
    ( id = 1 and companyName = 'Red Hat USA' )
or  ( id = 3 and companyName = 'Red Hat USA' )
or  ( id = 1 and companyName = 'Red Hat Europe' )
or  ( id = 3 and companyName = 'Red Hat Europe' )

delete
from
    Doctor
where
    ( id = 1 and companyName = 'Red Hat USA' )
or  ( id = 3 and companyName = 'Red Hat USA' )
or  ( id = 1 and companyName = 'Red Hat Europe' )
or  ( id = 3 and companyName = 'Red Hat Europe' )

delete
from
    Person
where
    ( id = 1 and companyName = 'Red Hat USA' )
or  ( id = 3 and companyName = 'Red Hat USA' )
or  ( id = 1 and companyName = 'Red Hat Europe' )
or  ( id = 3 and companyName = 'Red Hat Europe' )

This strategy has the advantage of being supported by all the major relational database systems (e.g. Oracle, SQL Server, MySQL, and PostgreSQL).

CteValuesListBulkIdStrategy

To use this strategy, you need to configure the following configuration property:

<property name="hibernate.hql.bulk_id_strategy"
          value="org.hibernate.hql.spi.id.inline.CteValuesListBulkIdStrategy"
/>

Now, when running the previous test case, Hibernate generates the following SQL statements:

select
    p.id as id,
    p.companyName as companyName
from
    Person p
where
    p.employed = ?

with HT_Person (id,companyName ) as (
    select id, companyName
    from (
    values
        (?, ?),
        (?, ?),
        (?, ?),
        (?, ?)
    ) as HT (id, companyName) )
delete
from
    Engineer
where
    ( id, companyName ) in (
        select
            id, companyName
        from
            HT_Person
    )

with HT_Person (id,companyName ) as (
    select id, companyName
    from (
    values
        (?, ?),
        (?, ?),
        (?, ?),
        (?, ?)
    ) as HT (id, companyName) )
delete
from
    Doctor
where
    ( id, companyName ) in (
        select
            id, companyName
        from
            HT_Person
    )


with HT_Person (id,companyName ) as (
    select id, companyName
    from (
    values
        (?, ?),
        (?, ?),
        (?, ?),
        (?, ?)
    ) as HT (id, companyName) )
delete
from
    Person
where
    ( id, companyName ) in (
        select
            id, companyName
        from
            HT_Person
    )

The underlying database must support the CTE (Common Table Expressions) that can be referenced from non-query statements as well, like PostgreSQL since 9.1 or SQL Server since 2005. The underlying database must also support the VALUES list clause, like PostgreSQL or SQL Server 2008.

However, this strategy requires the IN-clause row value expression for composite identifiers, so you can only use this strategy only with PostgreSQL.

Conclusion

If you can use temporary tables, that’s probably the best choice. However, if you are not allowed to create temporary tables, you must pick one of these four strategies that works with your underlying database. Before making your mind, you should benchmark which one works best for your current workload. For instance, CTE are optimization fences in PostgreSQL, so make sure you measure before taking a decision.

If you’re using Oracle or MySQL 5.7, you can choose either InlineIdsOrClauseBulkIdStrategy or InlineIdsInClauseBulkIdStrategy. For older version of MySQL, then you can only use InlineIdsOrClauseBulkIdStrategy.

If you’re using SQL Server, InlineIdsOrClauseBulkIdStrategy is the only option for you.

If you’re using PostgreSQL, then you have the luxury of choosing any of these four strategies.

back to top