Red Hat

In Relation To Hardy Ferentschik

In Relation To Hardy Ferentschik

Hibernate Search monitoring and statistics

Posted by    |       |    Tagged as Hibernate Search

Emmanuel mentioned in his previous Search post the new Statistics interface which is new in Hibernate search 3.3 (latest version 3.3.0.Beta1). I thought it is time to write a little bit more about it. The API is actually self-explanatory:

package org.hibernate.search.stat;
...
public interface Statistics {
    /**
     * Reset all statistics.
     */
    void clear();

    /**
     * Get global number of executed search queries
     *
     * @return search query execution count
     */
    long getSearchQueryExecutionCount();

    /**
     * Get the total search time in nanoseconds.
     */
    long getSearchQueryTotalTime();

    /**
     * Get the time in nanoseconds of the slowest search.
     */
    long getSearchQueryExecutionMaxTime();

    /**
     * Get the average search time in nanoseconds.
     */
    long getSearchQueryExecutionAvgTime();

    /**
     * Get the query string for the slowest query.
     */
    String getSearchQueryExecutionMaxTimeQueryString();

    /**
     * Get the total object loading in nanoseconds.
     */
    long getObjectLoadingTotalTime();

    /**
     * Get the time in nanoseconds for the slowest object load.
     */
    long getObjectLoadingExecutionMaxTime();

    /**
     * Get the average object loading time in nanoseconds.
     */
    long getObjectLoadingExecutionAvgTime();

    /**
     * Gets the total number of objects loaded
     */
    long getObjectsLoadedCount();

    /**
     * Are statistics logged
     */
    public boolean isStatisticsEnabled();

    /**
     * Enable statistics logs (this is a dynamic parameter)
     */
    public void setStatisticsEnabled(boolean b);

    /**
     * Returns the Hibernate Search version.
     *
     * @return the Hibernate Search version
     */
    String getSearchVersion();

    /**
     * Returns a list of all indexed classes.
     *
     * @return list of all indexed classes
     */
    Set<String> getIndexedClassNames();

    /**
     * Returns the number of documents for the given entity.
     *
     * @param entity the fqc of the entity
     *
     * @return number of documents for the specified entity name
     *
     * @throws IllegalArgumentException in case the entity name is not valid
     */
    int getNumberOfIndexedEntities(String entity);

    /**
     * Returns a map of all indexed entities and their document count in the index.
     *
     * @return a map of all indexed entities and their document count. The map key is the fqc of the entity and
     *         the map value is the document count.
     */
    Map<String, Integer> indexedEntitiesCount();
}

Access to the statistics is via SearchFactory.getStatistics(). The information about which classes are indexed and how many entities are in the index will always be available. However, the query and object loading timings will only be collected if the property hibernate.search.generate_statistics is set in your configuration. I am thinking about introducing an additional interface in order to make this separation more obvious. WDYT?

The new statistic and monitoring functionality does not end here. You can also enable access to the statistics via JMX. Setting the property hibernate.search.jmx_enabled will automatically register the StatisticsInfoMBean with the MBeanServer. On top of this MBean there are two more MBeans - IndexControlMBean and IndexingProgressMonitorMBean - which will or will not be available depending on your configuration and the state of the application.

The IndexControlMBean allows you to build, optimize and purge the index for a given entity. Indexing occurs via the mass indexing API. A requirement for this bean to be registered in JMX is, that the Hibernate SessionFactory is bound to JNDI via the hibernate.session_factory_name property. Refer to the Hibernate Core manual for more information on how to configure JNDI. The IndexControlMBean API are for now just experimental.

Last but not least, the IndexingProgressMonitorMBean. This MBean is an implementation of the MassIndexerProgressMonitor interface. If hibernate.search.jmx_enabled is set and the mass indexer API is used the indexing progress can be followed via this MBean. The bean will only be bound to JMX while indexing is in progress. Once indexing is completed the MBean is not longer available. Again, this API is for now experimental.

Do you think this new monitoring and statistic API it is valuable? Are you missing any functionality? Let us know and use the Search forum or Jira to suggest new features or to report a bug.

Enjoy!

There are no functional changes between CR1 and this Final release. Some minor issues in the @Min, @Max and @EmailValidator got fixed (HV-335, HV-339) as well as some documentation typos. We also spend some time improving the parsing and validation speed by reviewing and improving some reflection based code (HV-340, HV-341, HV-342). Review the full release notes for details.

Download as usual either from the JBoss Maven repo or from SourceForge.

There are already several issues on the 4.2 roadmap, but we are welcoming ideas and suggestions for the next release on the Validator Forum. One of the major new features will most likely be method-level validation (as specified in Appendix C of the Bean Validation specification) as well.

Stay tuned!

Amendment: Gunnar did a great job summarizing the new features of Hibernate Validator 4.1 on his blog. And of course we covered already most of the new features here as well. See programmatic constraint configuration, ResourceBundleLocator and @ScriptAssert.

Hibernate Validator 4.1.0.CR1

Posted by    |       |    Tagged as Hibernate Validator

Hibernate Validator 4.1.0.CR1 is now available. This is the last milestone build before the Final release. Not much has happened between Beta2 and CR1. The programmatic API as described in the previous release blog entry stayed as is and there were only three minor bug fixes. See release notes for details.

Download as usual either from the JBoss Maven repo or from SourceForge.

Ideas and suggestions are welcome on the Validator forum. Let us know what you want to see in Validator post 4.1.0. In case you find a bug create a jira issue in HV.

Enjoy!

Hibernate Validator 4.1.0.Beta2 is out. A total of 29 issues got resolved since the last release and you can review then in the Jira changelog. Amongst others we added the @ScriptAssert constraints as promised in the previous release blog. Initially we wanted to release 4.1.0.CR directly, but we added a new programmatic constraint configuration API (HV-274) for which we want to first gather some feedback.

The new API is centered around ConstraintMapping which is the entry point to a fluent API allowing the configuration of constraints. The API is comparable to the one described in Emmanuel's blog Hibernate Search Programmatic Mapping API. But let's look at an example:

ConstraintMapping mapping = new ConstraintMapping();
mapping.type( Car.class )
    .property( "manufacturer", FIELD )
        .constraint( NotNullDef.class )
    .property( "licensePlate", FIELD )
        .constraint( NotNullDef.class )
        .constraint( SizeDef.class )
            .min( 2 )
            .max( 14 )
    .property( "seatCount", FIELD )
        .constraint( MinDef.class )
            .value ( 2 )
.type( RentalCar.class )
    .property( "rentalStation", METHOD)
        .constraint( NotNullDef.class );

As you can see you can configure constraints on multiple classes and properties using method chaining. The constraint definition classes NotNullDef, SizeDef and MinDef are helper classes which allow to configure constraint parameters in a type-safe fashion. Definition classes exists for all built-in constraints in the org.hibernate.validator.cfg.defs package. For a custom constraint you can either create your own definition class extending ConstraintDef or you can use GenericConstraintDef (we are also thinking about providing another annotation processor which at compile time could generate the required helper classes):

ConstraintMapping mapping = new ConstraintMapping();
mapping.type( Car.class )
    .property( "licensePlate", FIELD )
        .constraint( GenericConstraintDef.class )
            .constraintType( CheckCase.class )
            .param( "value", CaseMode.UPPER );

Last but not least, you can also define cascading constraints as well as the default group sequence of an entity.

ConstraintMapping mapping = new ConstraintMapping();
mapping.type( Car.class )
    .valid( "driver", FIELD )
.type( RentalCar.class)
    .defaultGroupSequence( RentalCar.class, CarChecks.class );

Since the programmatic configuration is not part of the official Bean Validation specification you will have to get hold of HibernateValidatorConfiguration during the bootstrapping process in order to set your ConstraintMapping:

HibernateValidatorConfiguration config = Validation.byProvider( HibernateValidator.class ).configure();
config.addMapping( mapping );
ValidatorFactory factory = config.buildValidatorFactory();
Validator validator = factory.getValidator();

WDYT? Is this API useful? Easy to use? Anything we should consider/change?

If you want to play with this release, you can download it from the JBoss Maven repo or from SourceForge.

Enjoy!

A Validation stickler!

Posted by    |       |    Tagged as

One unsolved problem with Bean Validation is how to apply a constraint on the elements of a collection (or iterable). Think about a User entity with a list of emails (primary, secondary, ...). This could be modeled like this:

public class User {
    @NotEmpty
    private String name;

    @NotEmpty
    @Email
    private List<String> emailList;
    ...
}

The problem is that validating an instance of this User would currently throw an UnexpectedTypeException, since there is no built-in ConstraintValidator<Email, Collection<String>> which could validate emailList (see also HV-264). One way to solve the problem within the current framework is to implement a ConstraintValidator<Email, Iterable<String>>. Great, but it only solves the problem for @Email, not for the generic problem: How can I apply a constraint on all elements of a collection?.

Not thinking about language limitations, the following solution would be nice:

public class User {
    @NotEmpty
    private String name;

    @NotEmpty
    private List<@Email String> emailList;
     ...
}

Of course this is not possible in the current version of Java. However, there is JSR 308, which allows the above example and which will hopefully be part Java 7. For the Bean Validation specification it makes sense to wait for this feature (see BVAL-202), but should we have a interim solution for Hibernate Validator. This is what HV-296 is about. HV-296 discusses several solutions to the problem. Two solutions I would like to introduce here:

The payload solution

For this solution a custom payload class would be introduced. If this payload is specified on a constraint, the constraints will be applied on the elements of the collection:

public class User {
    @NotEmpty
    private String name;

    @NotEmpty
    @Email(payload=OnElements.class)
    private List< String> emailList; 
    ...
}

This seems to be the least intrusive solution and hopefully within the rules of the specification: Payloads are typically used by validation clients to associate some metadata information with a given constraint declaration. Payloads are typically non-portable.

The @ValidateEach annotation solution

For this solution we would introduce a new annotation @ValidateEach (or maybe @ApplyOn).

@ValidateElements
public @interface ValidateEach {
	Email[] value();
}

@ValidateElements is a marker annotation used by the validator engine to determine annotations used for applying built-in constraints on collections. Our usecase looks like this now:

public class User {
    @NotEmpty
    private String name;

    @NotEmpty
    @ValidateEach(@Email)
    private List< String> emailList;
    ...
}

Problematic with this solution is that for each constraint we have to place the corresponding ValidateEach annotation in its own package. This could be avoided if we give each annotation a distinctive name, for example ValidateEachEmail, ValidateEachSize, etc. Alternatively we could have one single ValidateEach annotation and attributes for each built-in constraint:

@ValidateElements
public @interface ValidateEach {
       Email[] email() default{};
       Size[] size() default {};
       NotNull[] notNull() default {};
       ...
}

This would reduce the required number of annotations, but it would introduce a special case for custom constraints.

You get the full history on how we arrived at these solutions by reading the irc log attached to HV-296. If you have an opinion on the matter, make sure to leave a comment, post on the forum or send an email to hibernate-dev.

Hibernate Validator 4.1.0.Beta1

Posted by    |       |    Tagged as

Time has not been standing still for Hibernate Validator and I am happy to announce the 4.1.0.Beta1 release of Hibernate Validator. The focus towards the final 4.1.0 release is to add new custom constraints, provide enhancement which go beyond the Bean Validation specification and give the developers an annotation processor to verify the correct placement of constraints. You can see what is already implemented in this beta release by referring to the Jira release notes. The roadmap towards 4.1.0 can be seen here. On the custom constraint side we added @CreditCardNumber, @NotBlank and @URL and are planning to add @ScriptAssert for the next release. We also introduced a new interface ResourceBundleLocator which allows users to make a better use of ResourceBundleMessageInterpolator. Instead of just loading resource bundles from the classpath using ResourceBundle.getBundle, you can now provide a custom implementation of ResourceBundleLocator providing the ResourceBundleMessageInterpolator with the message resource bundles to use:

HibernateValidatorConfiguration configuration = Validation.byProvider(HibernateValidator.class).configure();
ResourceBundleLocator defaultResourceBundleLocator = configure.getDefaultResourceBundleLocator(); 
ResourceBundleLocator myResourceBundleLocator = new MyCustomResourceBundleLocator(defaultResourceBundleLocator);
configuration.messageInterpolator(new ResourceBundleMessageInterpolator(myResourceBundleLocator));

Thanks to the great work of Gunnar Morling developers can now also use the new Hibernate Validator Annotation Processor. By adding the jar file hibernate-validator-annotation-processor-4.1.0.Beta1.jar to the classpath of their IDE project, they can get feedback about invalid placements of constraints. For example @Valid on a primitive type or a @Pattern constraint on a non string attribute. Prerequisite for this to work is that you are using a JDK 6. The configuration of the annotation processor is the same as for the Hibernate Metamodel Generator and can be seen here. The fully qualified name of the processor is org.hibernate.validator.ap.ConstraintValidationProcessor.

It would be great to get some feedback on the usability of the annotation processor. Also let us know which other custom constraints you would like to see in Hibernate Validator. Use the forum to discuss your ideas or create a jira issue in HV.

Enjoy, Hardy

P.S. Download is of course as usual either from the JBoss Maven repo or from SourceForge.

Hibernate Search 3.2.0 Beta1

Posted by    |       |    Tagged as Hibernate Search

It has been quite some time since the latest Hibernate Search release, but we are happy to announce the first beta release of version 3.2.0 with tons of bug fixes and exciting new features. In fact there are so many new features that we are planning to write a whole series of blog entries covering the following topics:

  • The new API for programmatic configuration of Hibernate Search via org.hibernate.search.cfg.SearchMapping.
  • Ability to rebuild the index in parallel threads using the MassIndexer API. This can be as simple as fullTextSession.createIndexer().startAndWait(), but of course there are plenty of options to fine-tune the behavior.
  • Clustering via JGroups as an alternative to the existing JMS solution. The values for the hibernate.search.worker.backend option are jgroupsSlave and jgroupsMaster in this case.
  • Dynamic boosting via the new @DynamicBoost annotation.

Most of these new features are already documented in the Hibernate Search documentation available in the distribution packages. However, there might be still some gaps in the documentation. If you find any let us know via the Forum or Jira. Smaller new features are:

  • New built-in field bridges for the java.util.Calendar and java.lang.Character
  • Ability to configure Lucene's LockFactory using hibernate.search.<index>.locking_strategy with the values simple, native, single or none.
  • Ability to share IndexWriter instances across transactions. This can be activated via the hibernate.search.<indexname>.exclusive_index_use flag.

Of course we also fixed several bugs of which the following are worth mentioning explicitly:

  • HSEARCH-391 Multi level embedded objects don't get an index update
  • HSEARCH-353 Removing an entity and adding another with same PK (in same TX) will not add second entity to index

For a full changelog see the Jira release notes. Last but not least, Hibernate Search depends now on Hibernate Core 3.5 beta2 and Lucene 2.4.1 and is aligned with JPA 2.0 CR1.

Special thanks to our contributors Sanne Grinovero and Amin Mohammed-Coleman who put a lot of effort into this release.

enjoy!

Hibernate Static Metamodel Generator Annotation Processor

Posted by    |       |    Tagged as

Steve mentioned in his Hibernate 3.5.0 Beta2 blog entry briefly the Hibernate Static Metamodel Generator Annotation Processor. Time to have a closer look what this annotation processor actually does.

Typesafe Criteria API

Let's start with a look at one of the new features of JPA 2 - the type-safe Criteria API. The key to this new API are metamodel classes generated for their corresponding managed entities. These metamodel classes allow the construction of object based queries. Lets look at an example:

@Entity public class Order {
    @Id 
    Integer id;
    @ManyToOne 
    Customer customer;
    @OneToMany 
    Set<Item> items;
    BigDecimal totalCost;
    ...
}

In the above example we have a managed entity Order with an id, a reference to a customer, a list of order items and the total costs. The corresponding metamodel class looks like this:

@StaticMetamodel(Order.class)
public class Order_ {
    public static volatile SingularAttribute<Order, Integer> id;
    public static volatile SingularAttribute<Order, Customer> customer;
    public static volatile SetAttribute<Order, Item> items;
    public static volatile SingularAttribute<Order, BigDecimal> totalCost;
}

Having such a metamodel class allows for queries of the form:

CriteriaBuilder cb = ordersEntityManager.getCriteriaBuilder();
CriteriaQuery<Order> cq = cb.createQuery(Order.class);
SetJoin<Order, Item> itemNode = cq.from(Order.class)
	.join(Order_.orderItems);
cq.where( cb.equal(itemNode.get(Item_.id), 5 ) ) 
	.distinct(true);

I don't want to go any deeper into the actual Criteria API here. Refer to JSR-317 for more information. Instead I would like to answer the question on how to generate this metamodel classes? Of course a developer could manually maintain them, but that would hardly be productive. A better solution is to utilize the annotation processing facilities officially available since Java 6. In Java 6 javac behaves analogously to the apt command in Java 5. This is specified by JSR-269.

Hibernate Static Metamodel Generator

Hibernate Static Metamodel Generator is an implementation of such an annotation processor with the task of creating the static metamodel classes for entities (as seen in the example above). In the following I will show how to integrate the annotation processor into your build environment. The good news is that in most cases the annotation processor will automatically run provided the annotation processor jar is on the classpath. This happens due to Java's Service Provider contract and the fact the the Hibernate Static Metamodel Generator jar files contains the file javax.annotation.processing.Processor in the META-INF/services listing org.hibernate.jpamodelgen.JPAMetaModelEntityProcessor as annotation processor.

Maven

There are several ways of running the annotation processor as part of a Maven build. It will automatically run if you are using a JDK 6 compiler and the annotation processor jar is on the classpath. In case you have more than one annotation processors on your classpath you can explicitly pass the processor option to the compiler plugin:

<plugin>
    <artifactId>maven-compiler-plugin</artifactId>
    <configuration>
        <source>1.6</source>
        <target>1.6</target>
        <compilerArguments>
        	<processor>org.hibernate.jpamodelgen.JPAMetaModelEntityProcessor</processor>
        </compilerArguments>
    </configuration>
</plugin>

The maven-compiler-plugin approach has the disadvantage that the maven compiler plugin does currently not allow to specify multiple compiler arguments (MCOMPILER-62) and that messages from the Messenger API are suppressed (MCOMPILER-66). A better approach is to disable annotation processing for the compiler plugin:

<plugin>
    <artifactId>maven-compiler-plugin</artifactId>
    <configuration>
        <source>1.6</source>
        <target>1.6</target>
        <compilerArgument>-proc:none</compilerArgument>
    </configuration>
</plugin>

and use the maven-annotation-plugin for annotation processing (you will need the following additional maven repositories - maven-annotation-plugin and jfrog):

<plugin>
    <groupId>org.bsc.maven</groupId>
    <artifactId>maven-processor-plugin</artifactId>
    <executions>
        <execution>
            <id>process</id>
            <goals>
                <goal>process</goal>
            </goals>
            <phase>generate-sources</phase>
            <configuration>
                <!-- source output directory -->
                <outputDirectory>target/metamodel</outputDirectory>
            </configuration>
        </execution>
    </executions>
</plugin>
<plugin>
    <groupId>org.codehaus.mojo</groupId>
    <artifactId>build-helper-maven-plugin</artifactId>
    <version>1.3</version>
    <executions>
        <execution>
            <id>add-source</id>
            <phase>generate-sources</phase>
            <goals>
                <goal>add-source</goal>
            </goals>
            <configuration>
                <sources>
                    <source>target/metamodel</source>
                </sources>
            </configuration>
        </execution>
    </executions>
</plugin>

Last, but not least one could also use the antrun-plugin to run the annotation processor. This approach can be seen here.

Ant

As mentioned before, the annotation processor will run automatically in a Java 6 environment. In case you want configure the processor explicitly or disable it you can use the compilerarg option of the Javac Task. The supported command javac line options are listed here.

Eclipse

Of course you also want to have annotation processing available in your favorite IDE. In Eclipse (at least since the latest Galileo release) exists an additional configuration section under Java Compiler where you can configure all kinds of aspects of annotation processing. Just check the Enable annotation processing option, configure the directory for the generated sources and finally add the Hibernate Static Metamodel Generator and JPA 2 jar files to the factory path.

Idea

Idea does not have an explicit configuration section for annotation processing yet. Instead do you have to rely on the fact that the processing will automatically occur in the case the Hibernate Static Metamodel Generator is on the classpath and you are using a Java 6 compiler. You can, however, specify additional compiler command line options under the Compiler - Java Compiler section.

Feeback

Hopefully this gets you started with the Hibernate Static Metamodel Generator. It would be great to get some feedback. The source code is currently part of the Hibernate svn repository. The trunk is http://anonsvn.jboss.org/repos/hibernate/jpamodelgen/trunk/. The latest SNAPSHOT of the jar file can be retrieved from the JBoss snapshot repository. Bugs should be reported in METAGEN.

Enjoy!

Hibernate Validator 4.0.2

Posted by    |       |    Tagged as

Hibernate Validator 4.0.2 is now available for download. This release was necessary for the JCP integration efforts for JSR-303 and addresses a potential NullPointerException (HV-263). Refer to the 4.0.2 release notes for a full list of changes.

Enjoy!

Hibernate Validator 4.0.1

Posted by    |       |    Tagged as

Hibernate Validator 4.0.1 is now available and can be downloaded from here. This release addresses issues discovered by the JCP integration efforts for JSR-303. Of course we used this opportunity to also fix some bugs identified by the community. Thanks for all the feedback.

Refer to the 4.0.1 release notes for a full list of changes.

Enjoy!

back to top