Help

The public draft of the JPA 2.0 specification is already out and includes a much-awaited feature: an API that lets you create queries by calling methods of Java objects, instead of by embedding JPA-QL into strings that are parsed by the JPA implementation. You can learn more about the API proposed by the public draft at Linda's blog.

There's several reasons to prefer the API-based approach:

  • It's easier to build queries dynamically, to handle cases where the query structure varies depending upon runtime conditions.
  • Since the query is parsed by the Java compiler, no special tooling is needed in order to get syntactic validation, autocompletion and refactoring support.

(Note that JPA-QL syntax validation and autocompletion is available is some IDEs - in JBoss Tools, for example.)

There's two major problems with criteria query APIs in the Java language:

  • The queries are more verbose and less readable.
  • Attributes must be specified using string-based names.

The first problem isn't really solvable without major new language features (usually described as DSL support). The second problem could easily be solved by adding a typesafe literal syntax for methods and fields to Java. This is now a sorely needed feature of the language, it's especially useful in combination with annotations.

There have been some previous efforts to work around the lack of method and field literals. One recent example is LIQUidFORM. Unfortunately that particular approach forces you to represent every persistent attribute as a public getter method, which is not a restriction that is acceptable in the JPA specification.

I've proposed a different approach to the JPA EG. This approach comes in three layers:

  • A metamodel API for JPA
  • A query API where types and attributes are specified in terms of metamodel API objects
  • Support for third-party tooling which would generate a typesafe metamodel from the entity classes

Let's go layer-by layer.

The Metamodel

The metamodel API is a bit like the Java reflection API, except that it is provided by the JPA persistence provider, is aware of the JPA metadata, and uses generics in a clever way. (Also it uses unchecked exceptions.)

For example, to obtain an object that represents an entity, we call the MetaModel object:

import javax.jpa.metamodel.Entity;
...
Entity<Order> order = metaModel.entity(Order.class);
Entity<Item> item = metaModel.entity(Item.class);
Entity<Product> item = metaModel.entity(Product.class);

To obtain attributes of the entity, we need to use string-based names, as usual:

import javax.jpa.metamodel.Attribute;
import javax.jpa.metamodel.Set;
...
Set<Order, Item> orderItems = order.set("items", Item.class);
Attribute<Item, Integer> itemQuantity = item.att("quantity", Integer.class);
Attribute<Item, Product> itemProduct = item.att("product", Product.class);
Attribute<Product, BigDecimal> productPrice = product.att("price", BigDecimal.class)

Notice how the metamodel types which represent attributes are parameterized not only by the type of the attribute they represent, but also by the type that they belong to.

Also notice that this code is non-typesafe and can fail at runtime if no persistent attribute with the given type and name exists in the entity class. This is the only non-typesafe code we'll see - our goal is keep the rest of the API completely typesafe. How does that help us? Well, the trick here is to notice that the metamodel objects represent completely static information about a persistent classes, state that doesn't change at runtime. So we can:

  • obtain and cache these objects at system intialization time, forcing any errors to occur upfront, or even
  • let a tool that has access to our persistent classes generate the code that obtains and caches metamodel objects.

That's much better than having these errors occur at query execution time, as they do in the previous criteria query proposal.

The metamodel API is generally useful, even independent of the query API. Currently it's very difficult to write generic code that interacts with JPA because JPA metadata may be partitioned between annotations and various XML documents.

But, of course, the most popular use of the metamodel is to build queries.

Queries

To construct a query, we pass metamodel objects to the QueryBuilder API:

Query query = queryBuilder.create();

Root<Order> orderRoot = query.addRoot(order);
Join<Order, Item> orderItemJoin = orderRoot.join(orderItems);
Join<Item, Product> itemProductJoin = orderItemJoin.join(itemProduct);

Expression<Integer> quantity = orderItemJoin.get(itemQuantity);
Expression<BigDecimal> price = itemProductJoin.get(productPrice);

Expression<Number> itemTotal = queryBuilder.prod(quantity, price);
Expression<Boolean> largeItem = queryBuilder.gt(itemTotal, 100);

query.restrict(largeItem)
     .select(order)
     .distinct(true);

For comparison, here is the same query expressed using the API proposed in the public draft:

Query query = queryBuilder.createQueryDefinition();

DomainObject orderRoot = query.addRoot(Order.class);
DomainObject orderItemJoin = orderRoot.join("items");
DomainObject itemProductJoin = orderItemJoin.join("product");

Expression quantity = orderItemJoin.get("quantity");
Expression price = itemProductJoin.get("price");

Expression itemTotal = quantity.times(price);
Predicate largeItem = queryBuilder.greaterThan(100);

query.where(largeItem);
     .selectDistinct(order);

Of course, this query could be written more compactly in either API, but I'm trying to draw attention to the generic types of the objects that make up the query. The type parameters prevent me from writing something like this:

orderItemJoin.get(productPrice); //compiler error

The use of generics means the compiler can detect when we try to create a path expression by combining a queried entity of one type and an attribute of some other type. The metamodel object productPrice has type Attribute<Product, BigDecimal> and therefore cannot be passed to the get() method of orderItemJoin. get() only accepts Attribute<Item, ?>, since orderItemJoin is of type Join<Order, Item>.

Expressions are also parameterized by the expression type, so the compiler detect mistakes like:

queryBuilder.gt(stringExpression, numericExpression); //error

Indeed, the API has sufficient typesafeness that it's more or less impossible to build an unexecutable query.

Generating a typesafe metamodel

It's completely possible to build queries with only the metamodel API and the query API. But to really make the most of these APIs, the final piece of the puzzle is a little code generation tool. This tooling doesn't need to be defined by the JPA specification, and different tools don't need to generate exactly the same code. Nevertheless, the generated code will always be portable between all JPA implementations. All the tool does is reflect upon the persistent entities and create a class or classes that statically cache references to the metamodel Entity and Attribute objects.

Why do we need this code generator? Because writing Attribute<Item, Integer> itemQuantity = item.att("quantity", Integer.class); by hand is tedious and slightly error prone, and because your refactoring tool probably isn't smart enough to change the string based name when you refactor the name of the attribute of the persistent class. Code generation tools don't make these kind of errors, and they don't mind re-doing their work from scratch each time you ask them to.

In a nutshell: the tool uses the non-typesafe metamodel API to build a typesafe metamodel.

The most exciting possibility is that this code generation tool could be an APT plugin for javac. You wouldn't have to run the code generator explicitly, since APT is now fully integrated into the Java compiler. (Or, it could be an IDE plugin.)

But didn't code generation tools go out of fashion recently? Wasn't one of the great features of ORM solutions like Hibernate and JPA that they didn't rely upon code generation? Well, I'm a great believer in using whatever tool is the right solution to the problem at hand. Code generation has certainly been applied to problems where it wasn't the best solution. On the other hand, I don't see anyone bashing ANTLR or JavaCC for their use of code generation to solve the problem they address. In this case, we're working around a specific problem in the Java type system: the lack of a typesafe metamodel (reflection is one of the worst-designed language features). And code generation is simply the only solution that works. Indeed, for this problem it works well.

Don't worry, the generated code won't be hard to understand ... it might look something like this, for example:

public class persistent {
	static Metamodel metaModel;

	public static Entity<model.Order> order = metaModel.entity(model.Order.class);
	public static class Order {
		public static Attribute<model.Order, Long> id = order.id(Long.class);
		public static Set<model.Order, model.Item> items = order.set("items", model.Item.class);
		public static Attribute<model.Order, Boolean> filled = order.att("filled", Boolean.class);
		public static Attribute<model.Order, Date> date = order.att("date", Date.class);
	}

	public static Entity<model.Item> item = metaModel.entity(model.Item.class);
	public static class Item {
		public static Attribute<model.Item, Long> id = item.id(Long.class);
		public static Attribute<model.Item, model.Product> product = item.att("product", model.Product.class);
		public static Attribute<model.Item, model.Order> order = item.att("order", model.Order.class);
		public static Attribute<model.Item, Integer> quantity = item.att("quantity", Integer.class);
	}

	public static Entity<model.Product> product = metaModel.entity(model.Product.class);
	public static class Product {
		public static Attribute<model.Product, Long> id = product.id(Long.class);
		public static Set<model.Product, model.Item> items = product.set("items", model.Item.class);
		public static Attribute<model.Product, String> description = product.att("description", String.class);
		public static Attribute<model.Product, BigDecimal> price = product.att("price", BigDecimal.class);
	}

}

This class just let's us refer to attributes of the entities easily. For example, we could type persistent.Order.id to refer to the id attribute of Order. Or persistent.Product.description to refer to the description of the Product.

26 comments:
 
16. Dec 2008, 06:22 CET | Link

I like the idea and I agree some kind of code generation supported by a metmodel is the only viable solution within the current Java Model - trick like liquidform or db4o's query tricks is simply hacks.

And it even works well with dynamically built queries since the compiler can still type check based on generics. The JPA metamodel should though also allow to call metaModel.entity() on non-entity classes such as @MappedSuperclass to allow common querying on base attributes.

 

--max

ReplyQuote
 
16. Dec 2008, 06:28 CET | Link
Jevgeni Kabanov | ekabanov(AT)gmail.com

We did a similar thing for SQL with Squill (http://squill.dev.java.net). We were considering also a wrapper for Hibernate/JPA, but if someone else is working on that already, that's great! You might also be interested in our paper on typesafe DSLs in Java: http://www.ekabanov.net/kabanov-raudjarv-pppj08.pdf.

 
16. Dec 2008, 20:48 CET | Link

Is this something that would be JPA-only or is it possible to be applied to ordinary hibernate projects as well?

 
16. Dec 2008, 20:50 CET | Link

I don't understand what you mean ... what is an ordinary hibernate project? Why could it not use JPA APIs?

 
17. Dec 2008, 21:06 CET | Link
AndyT (lordpixel)

We've been wondering whether to switch from using the Hibernate APIs to JPA, and what's been holding us back is the lack of User Type support in JPA.

The feedback I've gotten is people dislike having to sprinkle Hibernate annotations among the JPA ones. The objection is really that this information isn't optional - the application won't work without the user types, and so if we're going to have to import and use Hibernate annotations anyway, why not just stick with Hibernate top to bottom. I suppose we could layer that on in XML, but people like the idea of maintaining 2 files even less!

Just an example of something that makes the Hibernate APIs sticky - not sure if that's a good thing in and of itself.

I suppose it is orthogonal - presumably we could still use this query API even if we use Hibernate in other parts of the code base?

 
18. Dec 2008, 02:31 CET | Link

There's no reason why we couldn't do what we did with Hibernate annotations and make the criteria API a separate package from the EntityManager API.

However, FYI, it's very likely that Hibernate4 will be packaged quite differently, with the JPA APIs (plus extensions) as the native view of Hibernate, and support for stuff like the old Hibernate XML as the add-on package.

Personally, I'm deeply, deeply unconvinced by the oh, we occasionally have to use non-standard features of Hibernate, so therefore we should never use the standard JPA APIs for anything argument.

 
18. Dec 2008, 05:27 CET | Link
AndyT (lordpixel)

I suppose it depends on your definition of occasionally...

Almost every mapping we have has a user type in it. So while it is fair to say we don't use a lot of non-standard Hibernate features, the one or two we do use we use pervasively.

Then again, there's another question lurking below the surface here: is there some philosophical objection to user types that is keeping them out of JPA or is it simply priorities?

From our point of view their lack is a glaring, inexplicable omission, but then we use them everywhere, so we're bound to think that, aren't we? Any different perspective?

 
18. Dec 2008, 06:31 CET | Link

Well, from my point of view it would be extremely difficult to come up with a UserType API that accommodated the differences between the implementations of various JPA solutions. If you look at the hibernate UserType, it contains a whole lot of stuff that is just very specific to how Hibernate is implemented.

And I don't see the job of porting what ... 10? .... user type implementations from one JPA provider to another as especially difficult...

 
18. Dec 2008, 07:56 CET | Link
AndyT (lordpixel)

I believe you - from what I understand it took a long time for Hibernate to 'apply' the user types as consistently and transparently as it does. e.g. for the longest time I don't think they worked in all cases with Queries.

But then are you leaving this as yes, they're useful, yes, they're widely used but unfortunately they're just too hard to do right?

Maybe what's needed is a richer set of built in types? e.g. here are some of our reasonably generic user types:

  • TrimmedStringType -> gets rid of all the useless whitespace when someone has modeled a column as char(10) instead of varchar(10)
  • BooleanYNType -> maps bools to a char(1) column with Y/N
  • AbstractEnumType -> maps db values to typesafe enums (the old Java 1.4 pattern, but probably could be adapted for Java 5 enums)
  • PropertyEditorType -> uses Java Beans PropertyEditors to convert a class to a String representation for storage and vice-versa
  • NumericType -> See this issue

That's a mixed bag. The first two aim to deal with legacy schema issues, the next two are trying to automagically map from classes to a varchar representation and the last is a workaround.

I wonder if anyone has data on what the top 10 most common user types are? Are we all coding the same thing over and over?

 
18. Dec 2008, 08:03 CET | Link
AndyT (lordpixel)

On second thoughts, I reject my own idea - if I look at the balance of the rest of our user types, they're all really specific to our code base, so there would always need to be a user type facility in whatever implementation we might use.

It'd be great if it were portable, but as you say, they are not the most challenging things to reimplement.

 
18. Dec 2008, 23:12 CET | Link
AndyT (lordpixel)

Having wandered off topic and talked about something else, I find I want to comment on the proposal.

Overall it is something I think I would use, certainly the type safety and automated refactoring is great. Having the meta model generated helps massively with the typos and Don't Repeat Yourself aspect.

The fact it is in a separate package feels weird. There would be a learning curve, but I don't think it is insurmountable.

I guess if we have field literal's it'd look something like mypackage.Order#id rather than persistent.Order.id, right? But you're correct I simply can't think of a way around this at present that doesn't involve making everything public. It is frustrating because all of the data is in the class file, there's just no syntax to refer to it.

One idea this sparked... would the meta model be richer than just the Type and Name of each Attribute?

Hibernate knows all sorts of interesting things about my code, because it parses in the XML and annotations, but traditionally getting at this has been kind of tricky:

e.g. I have some code like this that figures out the database table name for an entity for doing SQL queries using Hibernate:

org.hibernate.cfg.Configuration cfg = ...

String tableName = cfg.getClassMapping(someEntityClass.getName()).getTable().getName()

Another example: the other day I was working on figuring out if the primary key of a persistent class was a composite key or not. Hibernate knows but it is a bit tricky to navigate the configuration model and find out.

Would you see a meta model API as a natural place to expose all of this juicy stuff that the ORM tool knows, or is that a bridge too far?

 
19. Dec 2008, 01:58 CET | Link

Actually we can make attribute references look like Person_.name, Order_.date, etc. Stylistically, this is really nice, and really close to what you would type if Java had method/attribute literals. I will do another post on that.

JPA annotations come in two layers:

  1. information about the model semantics, e.g. @ManyToMany, @Id, @Emddable
  2. information about how the model maps to the database, e.g. @Column, @Table, @GeneratedValue

The current proposed metamodel API gives you access to everything in the first layer, but to nothing in the second layer.

 
19. Dec 2008, 09:39 CET | Link

I like the idea of type safe attributes provided throw your meta model proposition. My fear is adding another artefact (while inevitable for the task at hand) will drive us far from the POJO model. Yes a generation tool like wsimport will come at handy, but I already hear screaming detractors voices about tool reliance and that the javac should be the only tool to hand coded class's. Certainly the JAVA language have to address those limitations.. Until that happen I support this proposal.

Warm Regards, Daoud AbdelMonem Faleh.

19. Dec 2008, 16:06 CET | Link

Hi guys, I like to subscribe your site, but the Subscrie to this site link (http://relation.to/service/Feed/atom) doesn't work.

- Patrick

19. Dec 2008, 21:12 CET | Link
Nik

Define doesn't work? 404? Parsing error? Works for me, have you tried with another reader?

 
20. Dec 2008, 10:24 CET | Link
My fear is adding another artefact (while inevitable for the task at hand) will drive us far from the POJO model.

It's still a POJO model, since this proposal does not affect the implementation of your model classes.

Yes a generation tool like wsimport will come at handy, but I already hear screaming detractors voices about tool reliance and that the javac should be the only tool to hand coded class's.

So I've just spent several days researching the capabilities of javac Processors, and it's very clear that you can generate the necessary types as part of the compilation process. So no tool will be required. (This is a very exciting new language feature of Java 6, by the way.

 
20. Dec 2008, 20:50 CET | Link
Gavin King wrote on Dec 20, 2008 04:24:
My fear is adding another artefact (while inevitable for the task at hand) will drive us far from the POJO model. It's still a POJO model, since this proposal does not affect the implementation of your model classes. Yes a generation tool like wsimport will come at handy, but I already hear screaming detractors voices about tool reliance and that the javac should be the only tool to hand coded class's. So I've just spent several days researching the capabilities of javac Processors, and it's very clear that you can generate the necessary types as part of the compilation process. So no tool will be required. (This is a very exciting new language feature of Java 6, by the way.

Good to know :)

 
31. Dec 2008, 13:28 CET | Link

This is just like EnitySpaces for .NET Dynamic queries, with one exception they include metadata in their entity classes as inner class, so you access it like

entity.MetaModel.Price==10

On a separate note it would be nice if we could use a builder pattern to resemble something like

select(items).where(item.price>10).and(item.price<10)

Greg

 
30. Mar 2009, 20:48 CET | Link
Vladimir Kovalyuk | gmail(AT)kovalyuk

Gavin,

I believe metamodel API should be aware of @Column annotation. I believe the classes should be serializable in order to unbound it from persistence context.

1000 for fields and method literals in Java. But I won't obtain my hope until I understand why new version of Java language has to be compatible with old JVMs (at the time when the runtimes are not compatible with each other).

I like the idea of LiquidForms. I employed some techniques it uses. Unfortunately it isn't possible to build metamodel API the same way because we have to refuse primitive types in properties. The problem is fundamental - CGLIB can't instrument all the invocations which return primitive types with the equivalent that return Object in order to insert them into IdentityMap.

Thank you for your effort in changing JPA criteria API.

P.S. I would like to read about your attention to Wicket. I'm done with JSF after more than 2 years of thinking how to do this and that when it is just impossible or expensive. I know you like JSF and made some contribution into JSF 2.0 spec. That's why it would be interesting to read about how would you compare JSF 2 with current Wicket capabilities.

 
30. Mar 2009, 20:51 CET | Link
Vladimir Kovalyuk | gmail(AT)kovalyuk

Sorry, forget to mention why. I believe metamodel API should be aware of @Column annotation because it would come in handy when we want to perform some sort of validation on the client side. It is possible with annotations right now. But that way does not accomodate orm.xml metadata.

 
30. Oct 2009, 06:42 CET | Link

I have a few concerns about the static metadata model:

- The generated metadata classes with one underscore in name are jpa specific; for example what happens if someone wants to create strongly-typed xpath api, would that implementation's static metadata model have two underscores in the class name? Also, the generated metadata class could not be able to be used in other places with strings are currently used (e.g. apache's BeanComparator).

- Probably won’t survive refactoring well (unless refactoring the entity's field would refactor the static metadata class’s field usages).

I would propose a more different approach of invoking methods on a proxy instance to form path expressions (instead accessing fields on a static metadata model). This would be strongly typed, would work with both jpa and hibernate (and potentially with other criteria use cases).

The code would look like:

ProxyQueryFactory queryFactory = new JpaProxyQueryFactoryImpl(entityManager);
ProxyQuery<Customer> query = queryFactory.createQuery(Customer.class);
Customer a = query.getRootProxy();

query.andWhere(a.getFirstName()).equalTo("John");

List<Customer> results = query.select().find();

jpql produced: select a from Customer a where a.firstName = :p1

This would be akin to mock object techniques where a method calls are invoked a proxy for a class and the resulting calls are recorded.

I’ve made a somewhat complete prototype at: http://www.cementframework.org/repos/pub/querybyproxy/trunk/

It looks like another framework has also independently came up with a similar concept: http://code.google.com/p/liquidform/

 
30. Oct 2009, 06:50 CET | Link
Unfortunately that particular approach forces you to represent every persistent attribute as a public getter method, which is not a restriction that is acceptable in the JPA specification.

A transient public method could be linked to a persistent field with a private access with via an annotation.

Also it begs the question of whether a field is really private if one is using it with any criteria expression (is read access prohibited or is is not!).

 
30. Oct 2009, 11:52 CET | Link
another framework ... came up with a similar concept

Yes, actually more than one. I added your project to my list of similar projects.

 
21. Jan 2010, 04:33 CET | Link
According to literals:
Yes, I'm waiting for Java literals to utilize then in Java ORMs.
I think that this Java Feature request for enhancement http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=5043025 is this what you mean: literals, am I right?
I've already voted for this requst - vote for it too! ;]

Smiles and Regards,
Adaslaw
Poland
24. May 2011, 04:30 CET | Link

On second thoughts, I reject my own idea - if I look at the balance of the rest of our user types, they're all really specific to our code base, so there would always need to be a user type facility in whatever implementation we might use.

It'd be great if it were portable, but as you say, they are not the most challenging things to reimplement.

 
02. Nov 2011, 02:49 CET | Link
Xavier Jodoin | xjodoin(AT)gmail.com

You can use TorpedoQuery to create type safe hibernate query.

Entity entity = from(Entity.class);
where(entity.getCode()).eq("mycode");
org.torpedoquery.jpa.Query<Entity> select = select(entity);
List<Entity> result = select.list(entityManager);
Post Comment