IntelliJ now has support for the new JPA 2.0 typesafe query facility I've been blogging about. It's very important that this stuff works smoothly with tooling, so it's great to see that the tooling vendors are on this early.

UPDATE: Even better, here and here are some screenshots of the CDI support in IntelliJ. Looks great!

Linda has written up the new typesafe query API. I previously blogged the reasoning behind this stuff here and here.

An open issue that Linda doesn't mention is query execution. I'm trying to convince the rest of the group that we should carry the typesafety all the way through to the query result set. Here's what I wrote to the group a few weeks ago:

Folks, I figured out a refactoring that gives us a way to do typesafe result sets, avoiding the use of Result. In this new approach, CriteriaQuery and Query would both have a type parameter. You could write code like this, if you have a single selection:

  CriteriaQuery<Order> q = qb.create(Order.class);
  Root<Order> order = q.from(Order.class);
  q.select(order);

  Query<Order> eq = em.createQuery(q);
  List<Order> res= eq.getTypedResultList();

like this, if you have multiple selections and an object to wrap them in:

  CriteriaQuery<OrderProduct> q = qb.create(OrderProduct.class);
  Root<Order> order = q.from(Order.class);
  Join<Item, Product> product = order.join(Order_.items)
                                     .join(Item_.product);
  q.select( qb.construct(OrderProduct.class, order, product) );

  Query<OrderProduct> eq = em.createQuery(q);
  List<OrderProduct> res= eq.getTypedResultList();

Or, if you don't have a nice wrapper class like OrderProduct, you can fall back to use Result:

  CriteriaQuery<Result> q = qb.create();
  Root<Order> order = q.from(Order.class);
  Join<Item, Product> product = order.join(Order_.items)
                                     .join(Item_.product);
  q.select( qb.result(order, product) );

  Query<Result> eq = em.createQuery(q);
  List<Result> res= eq.getTypedResultList();

This change let's people directly get typesafe lists of entities or wrappers, which is something that many people have asked for!

The big point about this API is that I can't write a query which selects Foo and then try to put it in a List<Bar>. It's truly typesafe, end-to-end.

The sticking point with this is that javax.persistence.Query does not currently have the needed type parameter, and there are millions of queries written to the JPA 1.0 APIs which would suddenly spit compiler warnings if we added a type parameter. So we might have to introduce a new interface like TypesafeQuery or something.

There's been plenty of discussion in the JPA group about my typesafe criteria proposal. My new favorite feature of the Java language is javax.annotation.Processor. Java 6 annotation processors are derived from the APT tool that existed in JDK 5, but are built into javac. Really, the name annotation processor is misleading, since this feature is only incidentally related to annotations. The Processor is really a fairly general purpose compiler plugin. If, like me, you've never been a fan of code generation, now is the time to reconsider. A Java 6 Processor can:

• analyze the compiler's metamodel of the Java source code that is being compiled
• search the source path for other metadata, such as XML
• generate new types, which will also be compiled, or other files

Best of all, this functionality requires no special tool or commandline options to javac. All you need to do is put the jar containing your Processor in the classpath, and the compiler does the rest!

In the typesafe query API, I want to use this to work around Java's lack of a typesafe metamodel for fields and methods of a class. The basic idea is that the compiler plugin will generate a metamodel type for each persistent class in the application.

Suppose we have the following persistent class:

@Entity
public class Order {
    @Id long id;
	
    boolean filled;
    Date date;

    @OneToMany Set<Item> items;

    @ManyToOne Shop shop;
	
    //getters and setters...
}

Then a class named Order_ would be generated, with a static member of each persistent attribute of Order, that the application could use to refer to the attributes in queries.

After several iterations, we've settled on the following format for the generated type:

import javax.jpa.metamodel.Attribute;
import javax.jpa.metamodel.Set;
import javax.jpa.metamodel.Metamodel;

@Metamodel
public abstract class Order_ {
    public static Attribute<Order, Long> id;
    public static Attribute<Order, Boolean> filled;
    public static Attribute<Order, Date> date;
    public static Set<Order, Item> items;
    public static Attribute<Order, Shop> shop;
}

The JPA provider would be responsible for initializing the values of these members when the persistence unit is initialized.

Now, criteria queries would look like the following:

Root<Item> item = q.addRoot(Item.class);
Path<String> shopName = item.get(Item_.order)
                            .get(Order_.shop)
                            .get(Shop_.name);
q.select(item)
 .where( qb.equal(shopName, "amazon.com") );

Which is equivalent to:

select item 
from Item item
where item.shop.name = 'amazon.com'

Or like:

Root<Order> order = q.addRoot(Order.class);
Join<Item, Product> product = order.join(Order_.items)
		                   .join(Item_.product);
		
Path<BigDecimal> price = product.get(Product_.price);
Path<Boolean> filled = order.get(Order_.filled);
Path<Date> date = order.get(Order_.date);
		
q.select(order, product)
 .where( qb.and( qb.gt(price, 100.00), qb.not(filled) ) )
 .order( qb.ascending(price), qb.descending(date) );

Which is equivalent to:

select order, product 
 from Order order 
    join order.items item
    join item.product product
 where
    product.price > 100 and not order.filled
 order by
    product.price asc, order.date desc

The queries are almost completely typesafe. Because of the generic type parameters of Attribute:

• I can't pass an attribute of Order to a Join or Path that represents an Item, and
• I can't try to perform a comparison like gt() on a Path that represents a boolean attribute, or not() on a Path that represents an attribute of type Date.

There's some skeptics in the expert group, but my feeling is that once people get used to the idea that type generation is no longer something that gets in your way during development, we're going to see a lot more frameworks using this kind of approach. I certainly think this API is a big improvement over the previous proposal:

Root item = q.addRoot(Item.class);
Path shopName = item.get("order")
                    .get("shop")
                    .get("name");
q.select(item)
 .where( qb.equal(shopName, "amazon.com") );

Or:

Root order = q.addRoot(Order.class);
Join product = order.join("items")
		    .join("product");
		
Path price = product.get("price");
Path filled = order.get("filled");
Path date = order.get("date");
		
q.select(order, product)
 .where( qb.and( qb.gt(price, 100.00), qb.not(filled) ) )
 .order( qb.ascending(price), qb.descending(date) );

Both these queries are riddled with non-typesafe method invocations which can't be validated without executing the query.

The public draft of the JPA 2.0 specification is already out and includes a much-awaited feature: an API that lets you create queries by calling methods of Java objects, instead of by embedding JPA-QL into strings that are parsed by the JPA implementation. You can learn more about the API proposed by the public draft at Linda's blog.

There's several reasons to prefer the API-based approach:

• It's easier to build queries dynamically, to handle cases where the query structure varies depending upon runtime conditions.
• Since the query is parsed by the Java compiler, no special tooling is needed in order to get syntactic validation, autocompletion and refactoring support.

(Note that JPA-QL syntax validation and autocompletion is available is some IDEs - in JBoss Tools, for example.)

There's two major problems with criteria query APIs in the Java language:

• The queries are more verbose and less readable.
• Attributes must be specified using string-based names.

The first problem isn't really solvable without major new language features (usually described as DSL support). The second problem could easily be solved by adding a typesafe literal syntax for methods and fields to Java. This is now a sorely needed feature of the language, it's especially useful in combination with annotations.

There have been some previous efforts to work around the lack of method and field literals. One recent example is LIQUidFORM. Unfortunately that particular approach forces you to represent every persistent attribute as a public getter method, which is not a restriction that is acceptable in the JPA specification.

I've proposed a different approach to the JPA EG. This approach comes in three layers:

• A metamodel API for JPA
• A query API where types and attributes are specified in terms of metamodel API objects
• Support for third-party tooling which would generate a typesafe metamodel from the entity classes

Let's go layer-by layer.

The Metamodel

The metamodel API is a bit like the Java reflection API, except that it is provided by the JPA persistence provider, is aware of the JPA metadata, and uses generics in a clever way. (Also it uses unchecked exceptions.)

For example, to obtain an object that represents an entity, we call the MetaModel object:

import javax.jpa.metamodel.Entity;
...
Entity<Order> order = metaModel.entity(Order.class);
Entity<Item> item = metaModel.entity(Item.class);
Entity<Product> item = metaModel.entity(Product.class);

To obtain attributes of the entity, we need to use string-based names, as usual:

import javax.jpa.metamodel.Attribute;
import javax.jpa.metamodel.Set;
...
Set<Order, Item> orderItems = order.set("items", Item.class);
Attribute<Item, Integer> itemQuantity = item.att("quantity", Integer.class);
Attribute<Item, Product> itemProduct = item.att("product", Product.class);
Attribute<Product, BigDecimal> productPrice = product.att("price", BigDecimal.class)

Notice how the metamodel types which represent attributes are parameterized not only by the type of the attribute they represent, but also by the type that they belong to.

Also notice that this code is non-typesafe and can fail at runtime if no persistent attribute with the given type and name exists in the entity class. This is the only non-typesafe code we'll see - our goal is keep the rest of the API completely typesafe. How does that help us? Well, the trick here is to notice that the metamodel objects represent completely static information about a persistent classes, state that doesn't change at runtime. So we can:

• obtain and cache these objects at system intialization time, forcing any errors to occur upfront, or even
• let a tool that has access to our persistent classes generate the code that obtains and caches metamodel objects.

That's much better than having these errors occur at query execution time, as they do in the previous criteria query proposal.

The metamodel API is generally useful, even independent of the query API. Currently it's very difficult to write generic code that interacts with JPA because JPA metadata may be partitioned between annotations and various XML documents.

But, of course, the most popular use of the metamodel is to build queries.

Queries

To construct a query, we pass metamodel objects to the QueryBuilder API:

Query query = queryBuilder.create();

Root<Order> orderRoot = query.addRoot(order);
Join<Order, Item> orderItemJoin = orderRoot.join(orderItems);
Join<Item, Product> itemProductJoin = orderItemJoin.join(itemProduct);

Expression<Integer> quantity = orderItemJoin.get(itemQuantity);
Expression<BigDecimal> price = itemProductJoin.get(productPrice);

Expression<Number> itemTotal = queryBuilder.prod(quantity, price);
Expression<Boolean> largeItem = queryBuilder.gt(itemTotal, 100);

query.restrict(largeItem)
     .select(order)
     .distinct(true);

For comparison, here is the same query expressed using the API proposed in the public draft:

Query query = queryBuilder.createQueryDefinition();

DomainObject orderRoot = query.addRoot(Order.class);
DomainObject orderItemJoin = orderRoot.join("items");
DomainObject itemProductJoin = orderItemJoin.join("product");

Expression quantity = orderItemJoin.get("quantity");
Expression price = itemProductJoin.get("price");

Expression itemTotal = quantity.times(price);
Predicate largeItem = queryBuilder.greaterThan(100);

query.where(largeItem);
     .selectDistinct(order);

Of course, this query could be written more compactly in either API, but I'm trying to draw attention to the generic types of the objects that make up the query. The type parameters prevent me from writing something like this:

orderItemJoin.get(productPrice); //compiler error

The use of generics means the compiler can detect when we try to create a path expression by combining a queried entity of one type and an attribute of some other type. The metamodel object productPrice has type Attribute<Product, BigDecimal> and therefore cannot be passed to the get() method of orderItemJoin. get() only accepts Attribute<Item, ?>, since orderItemJoin is of type Join<Order, Item>.

Expressions are also parameterized by the expression type, so the compiler detect mistakes like:

queryBuilder.gt(stringExpression, numericExpression); //error

Indeed, the API has sufficient typesafeness that it's more or less impossible to build an unexecutable query.

Generating a typesafe metamodel

It's completely possible to build queries with only the metamodel API and the query API. But to really make the most of these APIs, the final piece of the puzzle is a little code generation tool. This tooling doesn't need to be defined by the JPA specification, and different tools don't need to generate exactly the same code. Nevertheless, the generated code will always be portable between all JPA implementations. All the tool does is reflect upon the persistent entities and create a class or classes that statically cache references to the metamodel Entity and Attribute objects.

Why do we need this code generator? Because writing Attribute<Item, Integer> itemQuantity = item.att("quantity", Integer.class); by hand is tedious and slightly error prone, and because your refactoring tool probably isn't smart enough to change the string based name when you refactor the name of the attribute of the persistent class. Code generation tools don't make these kind of errors, and they don't mind re-doing their work from scratch each time you ask them to.

In a nutshell: the tool uses the non-typesafe metamodel API to build a typesafe metamodel.

The most exciting possibility is that this code generation tool could be an APT plugin for javac. You wouldn't have to run the code generator explicitly, since APT is now fully integrated into the Java compiler. (Or, it could be an IDE plugin.)

But didn't code generation tools go out of fashion recently? Wasn't one of the great features of ORM solutions like Hibernate and JPA that they didn't rely upon code generation? Well, I'm a great believer in using whatever tool is the right solution to the problem at hand. Code generation has certainly been applied to problems where it wasn't the best solution. On the other hand, I don't see anyone bashing ANTLR or JavaCC for their use of code generation to solve the problem they address. In this case, we're working around a specific problem in the Java type system: the lack of a typesafe metamodel (reflection is one of the worst-designed language features). And code generation is simply the only solution that works. Indeed, for this problem it works well.

Don't worry, the generated code won't be hard to understand ... it might look something like this, for example:

public class persistent {
	static Metamodel metaModel;

	public static Entity<model.Order> order = metaModel.entity(model.Order.class);
	public static class Order {
		public static Attribute<model.Order, Long> id = order.id(Long.class);
		public static Set<model.Order, model.Item> items = order.set("items", model.Item.class);
		public static Attribute<model.Order, Boolean> filled = order.att("filled", Boolean.class);
		public static Attribute<model.Order, Date> date = order.att("date", Date.class);
	}

	public static Entity<model.Item> item = metaModel.entity(model.Item.class);
	public static class Item {
		public static Attribute<model.Item, Long> id = item.id(Long.class);
		public static Attribute<model.Item, model.Product> product = item.att("product", model.Product.class);
		public static Attribute<model.Item, model.Order> order = item.att("order", model.Order.class);
		public static Attribute<model.Item, Integer> quantity = item.att("quantity", Integer.class);
	}

	public static Entity<model.Product> product = metaModel.entity(model.Product.class);
	public static class Product {
		public static Attribute<model.Product, Long> id = product.id(Long.class);
		public static Set<model.Product, model.Item> items = product.set("items", model.Item.class);
		public static Attribute<model.Product, String> description = product.att("description", String.class);
		public static Attribute<model.Product, BigDecimal> price = product.att("price", BigDecimal.class);
	}

}

This class just let's us refer to attributes of the entities easily. For example, we could type persistent.Order.id to refer to the id attribute of Order. Or persistent.Product.description to refer to the description of the Product.