Hibernate, Toplink Essentials, Openjpa, Eclipselink (jpa implementation compare)

2015-12-03 by kaqi



This article is a response to the lack of information on the net about the performance differences among the 4 most well known Java Persistence API (JPA) implementations: Toplink Essentials, EclipseLink, Hibernate and OpenJPA.
Besides results and conclusions, the full test code is also available in case you want to repeat the test yourself.

I wrote a relatively simple program which executes some queries and inserts in a MySQLdatabase through JPA. Four fixed-time tests were done with exactly thesame code, just changing the JPA implementation library and thepersistence.xml. I monitored the resources used by the JVM and countedthe inserts and queries executed during the tests. Finally, I show heremy conclusions and also the results of these tests, so that you candraw your own. I consider the differences found among the differentimplementations truly relevant.
For the tests performed for thisarticle, nothing except JPA was used. No web pages, no web orapplication server. Just java threads, JPA and MySQL. I give moredetails in the next sections.

Note: In case you are using JPA with Axis and/or the Google Web Toolkit (GWT), this other article focused on working with JPA, Axis and GWT could be of interest for you.

Description of hardware and software

Thetests have been done in an Acer Extensa 5620G laptop, with a pentiumCore 2 Duo T5250 Processor with 2 Gb Ram DDR2, being monitored by astandard PC.
For the tests I have used the following software:
Ubuntu 8.10 Intrepid IbexMySQL database, version 5.0 (installed from the official Ubuntu repositories).Java Virtual Machine 1.6Driver jdbc for MySQL 5.1.Eclipse GanymedeThe employees database example for MySQL, courtesy of Patrick Crews and Giuseppe Maxia (url below in the references section)JConsole for resources monitoringGIMP 2 to capture screensThedatabase and the JVM were running in the Acer machine. But bothJConsole and GIMP were executed in a PC (also equiped with Ubuntu 8.10)connected via tcp/ip to the test machine. I did it so that I did notoverload the machine running the tests.Versions of the JPA implementations tested:Hibernate EntityManager and Annotations 3.4.0Toplink Essentials version 2 build 41Openjpa 1.2.0Eclipselink 1.0.2Description of the codeThe code developed for the tests is available to download here. All you have to do is import the zip file in Eclipse.You will need at least one of the JPA implementation libraries. You candownload them from the urls in the references section below.The code is made up of two type of threads, one for inserting and one for querying.
Insertingthread gets an arbitrary employee and makes a copy of him/her, lettingMySQL generate a new emp_no. This was the only modification I did tothe employees database: the emp_no is auto-generated.
Querying threads execute all these queries in sequence:A query returning the number of female employees.A query returning the number of male employees.A query returning all employees hired since an arbitray date.A query returning all employees born after an arbitrary date.A query returning all women who have earned more than an arbitrary salary.Ihave also created an independent class JPAManager, which is in chargeof creating the static EntityManagerFactory and the EntityManager foreach of the threads. You have the details of that class here.

This is the starting sequence:

Whenthe program starts, it waits 2 minutes for the monitoringinfraestructure to be ready (connecting JConsole to the JVM, basically).Itthen starts 2 of the so-called inserting threads. I start the insertingthreads before the querying threads trying that the queries do notalways return the same (which will eventually happen, anyway).Afterstarting the inserting threads, the program starts running 18 of thequerying threads, inserting a pause of 10 seconds before starting next.This is so that they do not execute the same query at the same time.Theprogram runs the threads for 30 minutes. After that time, it sends astop signal to the threads, which will safely make them stop after thenext inserting or querying round. The main program waits 15 minutes forthe threads to stop and the jvm memory to stabilize.Before stopping, the threads provide information about the number of inserts/queries they have executed.Theonly change from test to test was the JPA implementation library andthe persistence.xml. It is important to notice that the persistence.xmlwas left by default for each of the implementations, omitting onpurpose any kind optimization that the implementation could accept.

ResultsThese were the results of the tests per JPA implementation library. Notice that the time was fixed: 30 minutes running.
 Number of queries+inserts executedNumber of queries executedNumber of inserts executedMax mem occupied during the test(Mb)Mem occupied after the test(Mb)
Toplink Essentials

The maximum memory occupied is the maximum amount that the JVM reserved during the test.
The memory occupied after the test is the amount of memory that remained reserved after finishing the test.
I have emphasized the highest and lowest values for each of the columns.
You can see this graphically in the following images showing the data monitored during the different tests.  /2014th7cj/d/file/p/20151020/3lby3zgzv4o.pnghttp://2.bp.blogspot.com/_VnVvmksSsVY/SXMNKB2YF4I/AAAAAAAAAKM/B13HTy_yq-Q/s1600-h/hibernate-complete.pnghttp://1.bp.blogspot.com/_VnVvmksSsVY/SXMNKrN1TuI/AAAAAAAAAKc/YGW3IPjy84U/s1600-h/toplink-complete.pnghttp://4.bp.blogspot.com/_VnVvmksSsVY/SXMNJg0AXSI/AAAAAAAAAKE/KJ2Ud70nyX8/s1600-h/eclipselink-complete.pngConclusions
My intention is that anyone can draw their own conclusions looking at the results or using the code to do a test of their own.
Nevertheless, I consider that there are a number of conclusions that one can draw watching the monitored data:
Thereis not an implementation that clearly has the best performance. Somehad a very good CPU or memory performance and some did it very wellwhen inserting or querying. But none of them was outstanding as a whole.Thenumber of records inserted by Hibernate was extremely higher than itwas for any other implementation (4 times more compared to Eclipselinkand 24 times more compared to OpenJPA). However, Hibernate was also theJPA implementation that executed the lowest number of queries, althoughthe differences in this value (3080 for Hibernate vs 3740 for ToplinkEssentials) are not so extreme as for the number of inserts.Hibernatewas also the implementation that consumed more memory. But having intoaccount that it inserted many more records than the others, it soundsreasonable.OpenJPA had the lowest value of inserts+queries.The number of inserts executed by OpenJPA was extremely low, compared to the others.The usage of CPU in the case of Toplink Essentials and Eclipselink was extremely low.Note for the JPA implementations responsible/developers:I am aware that some optimization can be obtained by changing thepersistence.xml and/or changing the code somehow. If you give me someadvice on how to improve the performance of any of the implementations,I will be glad to update this post with that information.

Ubuntu: http://www.ubuntu.com/
Employees database: http://dev.mysql.com/doc/employee/en/employee.html, https://launchpad.net/test-db/
Openjpa: http://openjpa.apache.org/
Toplink Essentials: http://www.oracle.com/technology/products/ias/toplink/jpa/download.html
Hibernate JPA: http://www.hibernate.org/397.html
Eclipselink: http://www.eclipse.org/eclipselink/
MySQL: http://www.mysql.com/
Eclipse: http://www.eclipse.org/ Journal is from http://terrazadearavaca.blogspot.com/2008/12/jpa-implementations-comparison.html