Red Hat

In Relation To Vlad Mihalcea

In Relation To Vlad Mihalcea

Introduction

When you’re using JDBC or if you are generating SQL statements by hand, you always know what statements are sent to the database server. Although there are situations when a native query is the most obvious solution to a given business use case, most statements are simple enough to be generated automatically That’s exactly what JPA and Hibernate do, and the application developer can focus on entity state transitions instead.

Nevertheless, the application developer must always assert that Hibernate generates the expected statements, as well as the number of statements being generated (to avoid N+1 query issues).

Proxying the underlying JDBC Driver or DataSource

In production, it’s very common to Proxy the underlying Driver Connection providing mechanism so that the application benefits from connection pooling, or for monitoring connection pool usage. For this purpose, the underlying JDBC Driver or DataSource can be proxied using tools such as P6spy or datasource-proxy. In fact, this is also a very convenient way of logging JDBC statements along with their bind parameters.

While for many application, it’s not an issue to add yet another dependency when you are developing an open source framework you strive to minimize the number of dependencies your project needs to depend on. Luckily, for Hibernate, we don’t even need to use an external dependency for intercepting JDBC statements, and this post is going to show you how easily you can tackle this requirement.

StatementInspector

For many use cases, the StatementInspector is the only thing you need to capture all SQL statements that are executed by Hibernate. The StatementInspector must be provided during SessionFactory bootstrapping as follows:

public class SQLStatementInterceptor {

    private final LinkedList<String> sqlQueries = new LinkedList<>();

    public SQLStatementInterceptor(SessionFactoryBuilder sessionFactoryBuilder) {
        sessionFactoryBuilder.applyStatementInspector(
        (StatementInspector) sql -> {
            sqlQueries.add( sql );
            return sql;
        } );
    }

    public LinkedList<String> getSqlQueries() {
        return sqlQueries;
    }
}

With this utility we can easily verify the Oracle follow-on-locking mechanism which is caused by the FOR UPDATE clause restrictions imposed by the database engine:

sqlStatementInterceptor.getSqlQueries().clear();

List<Product> products = session.createQuery(
    "select p from Product p", Product.class )
.setLockOptions( new LockOptions( LockMode.PESSIMISTIC_WRITE ) )
.setFirstResult( 40 )
.setMaxResults( 10 )
.getResultList();

assertEquals( 10, products.size() );
assertEquals( 11, sqlStatementInterceptor.getSqlQueries().size() );

So far, so good. But as simple as the StatementInspector may be, it does not mix well with JDBC batching. StatementInspector intercepts the prepare phase, whereas for batching we need to intercept the addBatch and executeBatch method calls.

Even without native support for such a feature, we can easily design a custom ConnectionProvider that can intercept all PreparedStatement method calls.

First, we start with the ConnectionProviderDelegate which is capable of substituting any other ConnectionProvider that would otherwise be picked by Hibernate (e.g. DatasourceConnectionProviderImpl, DriverManagerConnectionProviderImpl, HikariCPConnectionProvider) for the current configuration properties.

public class ConnectionProviderDelegate implements
        ConnectionProvider,
        Configurable,
        ServiceRegistryAwareService {

    private ServiceRegistryImplementor serviceRegistry;

    private ConnectionProvider connectionProvider;

    @Override
    public void injectServices(ServiceRegistryImplementor serviceRegistry) {
        this.serviceRegistry = serviceRegistry;
    }

    @Override
    public void configure(Map configurationValues) {
        @SuppressWarnings("unchecked")
        Map<String, Object> settings = new HashMap<>( configurationValues );
        settings.remove( AvailableSettings.CONNECTION_PROVIDER );
        connectionProvider = ConnectionProviderInitiator.INSTANCE.initiateService(
                settings,
                serviceRegistry
        );
        if ( connectionProvider instanceof Configurable ) {
            Configurable configurableConnectionProvider = (Configurable) connectionProvider;
            configurableConnectionProvider.configure( settings );
        }
    }

    @Override
    public Connection getConnection() throws SQLException {
        return connectionProvider.getConnection();
    }

    @Override
    public void closeConnection(Connection conn) throws SQLException {
        connectionProvider.closeConnection( conn );
    }

    @Override
    public boolean supportsAggressiveRelease() {
        return connectionProvider.supportsAggressiveRelease();
    }

    @Override
    public boolean isUnwrappableAs(Class unwrapType) {
        return connectionProvider.isUnwrappableAs( unwrapType );
    }

    @Override
    public <T> T unwrap(Class<T> unwrapType) {
        return connectionProvider.unwrap( unwrapType );
    }
}

With the ConnectionProviderDelegate in place, we can now implement the PreparedStatementSpyConnectionProvider which, using Mockito, it returns a Connection spy instead of an actual JDBC Driver Connection object:

public class PreparedStatementSpyConnectionProvider
        extends ConnectionProviderDelegate {

    private final Map<PreparedStatement, String> preparedStatementMap = new LinkedHashMap<>();

    @Override
    public Connection getConnection() throws SQLException {
        Connection connection = super.getConnection();
        return spy( connection );
    }

    private Connection spy(Connection connection) {
        if ( new MockUtil().isMock( connection ) ) {
            return connection;
        }
        Connection connectionSpy = Mockito.spy( connection );
        try {
            doAnswer( invocation -> {
                PreparedStatement statement = (PreparedStatement) invocation.callRealMethod();
                PreparedStatement statementSpy = Mockito.spy( statement );
                String sql = (String) invocation.getArguments()[0];
                preparedStatementMap.put( statementSpy, sql );
                return statementSpy;
            } ).when( connectionSpy ).prepareStatement( anyString() );
        }
        catch ( SQLException e ) {
            throw new IllegalArgumentException( e );
        }
        return connectionSpy;
    }

    /**
     * Clears the recorded PreparedStatements and reset the associated Mocks.
     */
    public void clear() {
        preparedStatementMap.keySet().forEach( Mockito::reset );
        preparedStatementMap.clear();
    }

    /**
     * Get one and only one PreparedStatement associated to the given SQL statement.
     *
     * @param sql SQL statement.
     *
     * @return matching PreparedStatement.
     *
     * @throws IllegalArgumentException If there is no matching PreparedStatement or multiple instances, an exception is being thrown.
     */
    public PreparedStatement getPreparedStatement(String sql) {
        List<PreparedStatement> preparedStatements = getPreparedStatements( sql );
        if ( preparedStatements.isEmpty() ) {
            throw new IllegalArgumentException(
                    "There is no PreparedStatement for this SQL statement " + sql );
        }
        else if ( preparedStatements.size() > 1 ) {
            throw new IllegalArgumentException( "There are " + preparedStatements
                    .size() + " PreparedStatements for this SQL statement " + sql );
        }
        return preparedStatements.get( 0 );
    }

    /**
     * Get the PreparedStatements that are associated to the following SQL statement.
     *
     * @param sql SQL statement.
     *
     * @return list of recorded PreparedStatements matching the SQL statement.
     */
    public List<PreparedStatement> getPreparedStatements(String sql) {
        return preparedStatementMap.entrySet()
                .stream()
                .filter( entry -> entry.getValue().equals( sql ) )
                .map( Map.Entry::getKey )
                .collect( Collectors.toList() );
    }

    /**
     * Get the PreparedStatements that were executed since the last clear operation.
     *
     * @return list of recorded PreparedStatements.
     */
    public List<PreparedStatement> getPreparedStatements() {
        return new ArrayList<>( preparedStatementMap.keySet() );
    }
}

To use this custom provider, we just need to provide an instance via the hibernate.connection.provider_class configuration property:

private PreparedStatementSpyConnectionProvider connectionProvider =
    new PreparedStatementSpyConnectionProvider();

@Override
protected void addSettings(Map settings) {
    settings.put(
            AvailableSettings.CONNECTION_PROVIDER,
            connectionProvider
    );
}

Now, we can assert that the underlying PreparedStatement is batching statements according to our expectations:

Session session = sessionFactory().openSession();
session.setJdbcBatchSize( 3 );

session.beginTransaction();
try {
    for ( long i = 0; i < 5; i++ ) {
        Event event = new Event();
        event.id = id++;
        event.name = "Event " + i;
        session.persist( event );
    }
}
finally {
    connectionProvider.clear();
    session.getTransaction().commit();
    session.close();
}

PreparedStatement preparedStatement = connectionProvider.getPreparedStatement(
    "insert into Event (name, id) values (?, ?)" );

verify(preparedStatement, times( 5 )).addBatch();
verify(preparedStatement, times( 2 )).executeBatch();

The PreparedStatement is not a mock but a real object spy, which can intercept method call while also propagating the call to the underlying actual JDBC Driver PreparedStatement object.

Although getting the PreparedStatement by its associated SQL String is useful for the aforementioned test case, we can also get all executed PreparedStatements like this:

List<PreparedStatement> preparedStatements = connectionProvider.getPreparedStatements();
assertEquals(1, preparedStatements.size());
preparedStatement = preparedStatements.get( 0 );

verify(preparedStatement, times( 5 )).addBatch();
verify(preparedStatement, times( 2 )).executeBatch();

Hibernate Community Newsletter 12/2016

Posted by    |       |    Tagged as Discussions Hibernate ORM

Welcome to the Hibernate community newsletter in which we share blog posts, forum, and StackOverflow questions that are especially relevant to our users.

Articles

Hibernate Community Newsletter 11/2016

Posted by    |       |    Tagged as Discussions Hibernate ORM

Welcome to the Hibernate community newsletter in which we share blog posts, forum, and StackOverflow questions that are especially relevant to our users.

Articles

In this post, I’d like you to meet Mark Paluch, who, among other projects, is one of our Hibernate OGM project contributors.

  1. Hi, Mark. Would you like to introduce yourself and tell us what you are currently working on?

    I am Mark Paluch, and I am working for Pivotal Software as Spring Data Engineer. I am a member of the JSR 365 EG (CDI 2.0), project lead of the lettuce Redis driver, and I run a couple of other open source projects. I enjoy tinkering on Internet of Things projects in my spare time. Before I joined Pivotal, I worked since the early 2000’s as a freelancer in a variety of projects using Java SE/EE and web technologies. My focus lies now on Spring Data with Redis, Cassandra, and MongoDB in particular.

  2. You have contributed a lot to the Hibernate OGM Redis module. Can you please tell us a little bit about Redis?

    I was not the first one bringing up the idea of Redis support in Hibernate OGM. In fact, Seiya Kawashima did a pretty decent job with his pull-request but at some point, Hibernate OGM development and the PR diverged. I came across the pull request and picked it up from there.

    Redis is an in-memory data structure store, used as database, cache and message broker. It originated from a key-value store but evolved by supporting various data structures like lists, sets, hashes and much more. Redis is blazing-fast although it runs mostly single-threaded. Its performance originates in a concise implementation and that all operations are performed in-memory. This does not mean that Redis has no persistence. Redis is configured by default to store data on disk and disk I/O is asynchronous. Redis facilitates through its versatile nature an enormous number of use-cases such as Caching, queues, remote locking, just storing data and much more. An important fact to me is always that I’d never use Redis for data I cannot recover as wiping data from Redis is just too easy but using it as semi-persistent a store is the perfect use.

  3. You are also the author of the Lettuce open source project. How does it compare to Hibernate OGM?

    Hibernate OGM and lettuce are projects with different aims. Lettuce is a driver/Java-binding for Redis. It gives Java developers access to the Redis API using synchronous, asynchronous and reactive API bindings. You can invoke the Redis API with lettuce directly and get the most out of Redis if you need it. Any JDBC driver is situated on a similar abstraction level as lettuce except for some specific features. lettuce does not require connection-pooling and dealing with broken connections as it allows users to benefit from auto-reconnection and thread-safe connections. Hibernate OGM Redis uses this infrastructure and provides its data mapping features on top of lettuce.

  4. What benefit do you think Hibernate OGM offers to application developers compared to using the NoSQL API directly?

    Each NoSQL data store has its own, very specific API. Native APIs require developers not only get familiar with the data store traits but also with its API. Redis API comes with over 150 commands that translate to 650+ commands with sub-command permutations.

    Every Redis command is very specific and behaves on its own. The Redis command documentation provides detailed insight to commands, but users are required to spend a fair amount of their time to get along with the native API.

    Hibernate OGM applies elements from the JPA spec to NoSQL data stores and comes up with an API that Java EE/JPA developers are familiar. Hibernate OGM lowers barriers to entry. Hibernate OGM comes with a purpose of mapping data into a NoSQL data store. Mapping simple JPA entities to the underlying data store works fine but sometimes, like associations or transactions, do not map well to MongoDB and Redis. Users of Hibernate OGM need to be aware of the underlying persistence technology to get familiar with its concepts and strengths as well with their limitations.

    I also see a great advantage of the uniform configuration mechanism of Hibernate OGM. Every individual datastore driver comes up with its individual configuration method. Hibernate OGM unifies the styles into a common approach. One item on my wish list for Hibernate OGM is JDNI/WildFly configuration support to achieve similar flexibility as it is possible with JDBC data sources.

  5. Do you plan on supporting Hibernate OGM in Spring Data as well?

    Hibernate OGM and Spring Data follow both the idea of supporting NoSQL data stores. Hibernate OGM employs several features from NoSQL data stores to enhance its data mapping centering around JPA. JPA is an inherently relational API, which talks about concepts that are not necessarily transferable to the NoSQL world. Spring Data comes with modules for various popular data stores with a different approach to providing a consistent programming model for the supported stores but not try to force everything into a single abstracting API. Spring Data Modules provide multiple levels of abstraction on top of the NoSQL data store APIs. Core concepts of NoSQL data stores are exposed through an API that commonly looks and feels like Spring endpoints. Hibernate OGM can already be used together with Spring Data JPA. A good use-case is the Spring Data Repositories abstraction which provides a uniform interface to access data from various data stores that do not require users to write a query language and leverages store-specific features.

Thank you, Mark, for taking your time. It is a great honor to have you here. To reach Mark, you can follow him on Twitter.

Meet Sergey Chernolyas

Posted by    |       |    Tagged as Discussions Hibernate OGM Interview

In this post, I’d like you to meet Sergey Chernolyas who is one of our Hibernate OGM project contributors.

  1. Hi, Sergey. You are one of the people who contributed to the Hibernate OGM project. Can you please introduce yourself?

    Hi, Vlad! My name is Sergey Chernolyas. I am from Russia, and I am 38 years old. I have been working with Java technologies since 2000. During my career, I got four certificates on Java technologies from Oracle and got involved in many development and integration projects.

  2. Can you tell us what project are you currently working on and if it uses Hibernate OGM?

    Now, I am working on a new module for Hibernate OGM, which aims to integrate the OrientDB NoSQL database. With this module, OGM will support a total of 7 NoSQL databases. Although at my current job, my work is not related to NoSQL solutions or Hibernate OGM, I am interested in this topic, and that’s why I pushed myself to learn Hibernate OGM and exploring NoSQL databases.

  3. Can you tell us a little about OrientDB?

    OrientDB is a graph-oriented and document-oriented database, and it is built using Java technologies. Briefly, the main advantages of using OrientDB are:

    1. It can operate in several modes: as an in-memory database, or through a network connection, or it can be store data in a local file.

    2. It offers join-less entity associations.

    3. It supports stored procedures that may be written in Java, JavaScript and any other language implementing the JSR-223 specification (e.g. Groovy, JRuby, etc.).

    4. It has good performance and is Big Data-oriented.

      For more details about OrientDB, you can visit the official documentation. Recently, the OrientDB team released the 2.2 GA version, so it’s worth giving it a try.

  4. What is the main benefit of using Hibernate OGM for accessing OrientDB over using their native API?

    The main benefit of using Hibernate OGM over the native API is the standard way for application development. Also, Hibernate OGM hides many low-level operations for creating and managing database connections, or for executing queries.

    While implementing the first version of the OrientDB Hibernate OGM module, I was faced with some OrientDB issues that prevented me integrate all features that ought to be supported by any Hibernate OGM module. Luckily, the OrientDB team was helpful and supportive, and I hope that by the time I finish this integration, the OrientDB team had already fixed my previously reported issues.

Thank you, Sergey for taking your time, and keep up the good work.

Hibernate Community Newsletter 10/2016

Posted by    |       |    Tagged as Discussions Hibernate ORM

Welcome to the Hibernate community newsletter in which we share blog posts, forum, and StackOverflow questions that are especially relevant to our users.

Events

C2B2 and Red Hat are hosting a Hibernate Q&A event, so if you are in London on May 24th, you now have the opportunity of meeting some of the developers that are behind Hibernate ORM, OGM, Search, and Validator.

Articles

Videos

The Devoxx France 2016 videos are available on YouTube, so, if you know French, you should definitely watch Emmanuel Bernard' talks on Hibernate projects latest features, as well as the one about the Elasticsearch support that is being integrated into Hibernate Search.

If you want to learn how to boost your data access layer performance, you should check my High-Performance Hibernate talk as well.

Hibernate Community Newsletter 9/2016

Posted by    |       |    Tagged as Discussions Hibernate ORM

Welcome to the Hibernate community newsletter in which we share blog posts, forum, and StackOverflow questions that are especially relevant to our users.

Articles

Presentations

Both Emmanuel and I have been presenting at Devoxx France, and we are going to share the videos once they are available. Meanwhile, you can checkout the slides for the High-Performance Hibernate presentation.

Hibernate Community Newsletter 8/2016

Posted by    |       |    Tagged as Discussions Hibernate ORM

Welcome to the Hibernate community newsletter in which we share blog posts, forum, and StackOverflow questions that are especially relevant to our users.

Articles

Presentations

Hibernate Community Newsletter 7/2016

Posted by    |       |    Tagged as Discussions Hibernate ORM

Welcome to the Hibernate community newsletter in which we share blog posts, forum, and StackOverflow questions that are especially relevant to our users.

Articles

back to top