Sunday

Apache Ignite with JPA : A missing element

In this article, author of the book "High Performance in-memory computing with Apache Ignite", discussing the use of JPA (Hibernate OGM) with Apache Ignite. Part of this article taken from the Persistence chapter of the book.

UP1: The article also published in DZONE.
Often the first step to developing an Enterprise information system is creating the domain model, that is, listing the entities in the domain and defining the relationships between them. A domain model is a conceptual image of the problem your system is trying to solve. Domain model elements can be linked by relationships. Usually, relational objects are represented in a tabular format, while application object model are represented in an interconnected graph of the object format. While storing and retrieving an object model from the relational database, a few mismatch occurs such as Granularity, SubTypes etc. To solve the mismatches between relational and object model, JPA provides a collection of APIs and methods to manipulates with the persistence store. The JPA specification only defines relational database access, but its API and many of its annotations are not relational specific. There are a few factors we have to take into account before applying JPA into any NoSQL database:

1. Pure relational concepts may not apply well in NoSQL
  • a. Table, Column, Joins.
2. JPA queries may not be suitable for NoSQL. NoSQL data modeling is typically driven by application-specific access patterns.

Note that, if your dataset is by nature non-domain model centric, then JPA is not for you.

Anyway, Apache Ignite provides in-memory KeyValue store and it’s quite well fit for using JPA. Other NoSQL vendor like Infinspan, Oracle NoSQL, Ehcache also supported by JPA persistence as well. There are a few NoSQL/JPA solutions available in today's market.
• Kundera
  • o One of the very first JPA implementations for NoSQL databases.
  • o Supports Cassandra, MongoDB, Hbase, Redis, Oracle NoSQL DB etc.
• DataNucleus
  • o Persistence layer behind Google App engine
  • o Supports MongoDB, Cassandra, Neo4J
• Hibernate OGM
  • o Using Hibernate ORM engine to persists entities in NoSQL database.
  • o Supports MongoDB, Cassandra, Neo4j, Infinspan, Ehcache
  • o Experipental support for Apache Ignite.
Hibernate OGM talks to NoSQL database via store-specific dialects. Hibernate OGM or Hibernate Object Grid Mapper also supports several ways for searching entities and returning them as Hibernate managed objects:

1. JP-QL queries (we convert them into a native backend query)
2. datastore specific native queries
3. full-text queries, using Hibernate Search as indexing engine

So, for Apache Ignite we are going to give a try to use JPA by Hibernate OGM framework. Note that, Apache Ignite support by Hibernate OGM still in development stage and not recommended to use in production. The project is available at Github repositories and any contributions are welcome. Anyway, you can also contribute to code review of this project with others by this URL. High-level view of the Hibernate OGM are shown below:

In the next few section we will cover the following topics:

• Clone and build the module Ignite for Hibernate OGM framework.
• Create a new maven project for using Hibernate OGM with Ignite.
• Persisting a few entities in Ignite through JPA.

Before we start, make sure the prerequisites of the project in your workstation:
1. Java JDK 1.8
2. Ignite version 1.7.0
3. Apache Maven version >3.0.3

Step 1:
Let’s set up our sandbox first. Clone or download the Hibernate OGM framework source code from the GitHub repositories.

git clone git@github.com:Z-z-z-z/hibernate-ogm.git hibernate-ogm

Step 2:

Modify the pom.xml, comment the following modules as follows:
<module>infinispan</module> 
<module>infinispan-remote</module> 
<module>mongodb</module>
<module>neo4j</module>
<module>couchdb</module>
<module>cassandra</module>
<module>redis</module>
We donot need these above modules in our project. Make sure that, you have the ignite module on pom.xml file.

Step 3:
Build the project with the following command:
mvn clean install -Dmaven.test.skip=true -DskipDocs -DskipDistro
If everything goes well, you should have all the necessary libraries in your local maven repositories.

Step 4:
Clone or download the ignite-jpa repository from the GitHub. If you create your own maven project, add this dependencies of your pom.xml.
<dependency>
    <groupId>org.hibernate.ogm</groupId>
    <artifactId>hibernate-ogm-ignite</artifactId>
    <version>5.1.0-SNAPSHOT</version>
</dependency>
<dependency>
    <groupId>org.hibernate.javax.persistence</groupId>
    <artifactId>hibernate-jpa-2.1-api</artifactId>
    <version>1.0.0.Final</version>
</dependency>
The dependencies are:
1. The hibernate OGM Ignite module for working with Apache Ignite cache. This will pull in all other required modules such as Hibernate OGM core.
2. Hibernate JPA API to working with JPA.

The domain model:
Our example domain model is consisted of two entities: Breed and Dog.

The association between Breed and Dog is a ManyToOne. One Dog can have only one breed and so on.
Step 5:
Now let’s map the domain model by creating the entity Java classes and annotating them with the required meta information. Let’s strat with the Breed class.

@Entity(name = "BREED")
public class Breed {
    @Id
    @GeneratedValue(generator = "uuid")
    @GenericGenerator(name="uuid", strategy="uuid2")
    private String id;

    private String name;

    public String getId() { return id; }
    public void setId(String id) { this.id = id; }

    public String getName() { return name; }
    public void setName(String name) { this.name = name; }

    @Override
    public String toString() {
        return "Breed{" +
                "id='" + id + '\'' +
                ", name='" + name + '\'' +
                '}';
    }
}

The entity is marked as a JPA annotation of @Entity, while other properties such as ID are annoted by the @ID.
By the @ID annotation, Hibernate will take care to generate the primary key or the key value for the entity object. @GeneratedValue UUID will generate a UUID value as a entity identifier.
Create another class named Dog and add the following contents on it.

@Entity
public class Dog {
    @Id
    @GeneratedValue(strategy = GenerationType.TABLE, generator = "dog")
    public Long getId() { return id; }
    public void setId(Long id) { this.id = id; }
    private Long id;

    public String getName() { return name; }
    public void setName(String name) { this.name = name; }
    private String name;

    @ManyToOne
    public Breed getBreed() { return breed; }
    public void setBreed(Breed breed) { this.breed = breed; }
    private Breed breed;

    @Override
    public String toString() {
        return "Dog{" +
                "id=" + id +
                ", name='" + name + '\'' +
                ", breed=" + breed +
                '}';
    }
}

We also annotated the Dog entity with @Entity and @ID annotation. Also we add one @ManyToOne annotation to make the association with Breed entity.

Step 6:
Let’s create the cache configuration class and the persistence.xml. Create a Ignite cache configuration class with name ConfigurationMaker as follows:
public class ConfigurationMaker implements IgniteConfigurationBuilder {
    @Override
    public IgniteConfiguration build() {
        IgniteConfiguration config = new IgniteConfiguration();
        config.setPeerClassLoadingEnabled(true);
        config.setClientMode(false);
        TcpDiscoverySpi discoSpi = new TcpDiscoverySpi();
        TcpDiscoveryMulticastIpFinder ipFinder = new TcpDiscoveryMulticastIpFinder();
        ArrayList addrs = new ArrayList<>();
        addrs.add("127.0.0.1:47500..47509");
        ipFinder.setAddresses(addrs);
        discoSpi.setIpFinder(ipFinder);
        config.setDiscoverySpi(discoSpi);

        CacheConfiguration accountCacheCfg = new CacheConfiguration()
                .setName("BREED")
                .setAtomicityMode(TRANSACTIONAL)
                .setIndexedTypes(
                        String.class, Breed.class
                );

        config.setCacheConfiguration(accountCacheCfg);
        return config;
    }
}

The above class represented the Ignite Cache configuration, instead of using spring configuration. We have explained the cache configurarion in chapter one. Let’s create the persistence.xml file in /ignite-jpa/src/main/resources/META-INF/persistence.xml directory.

<?xml version="1.0"?>
<persistence xmlns="http://java.sun.com/xml/ns/persistence"
             xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
             xsi:schemaLocation="http://java.sun.com/xml/ns/persistence http://java.sun.com/xml/ns/persistence/persistence_2_0.xsd"
             version="2.0">

    <persistence-unit name="ogm-jpa-tutorial" transaction-type="RESOURCE_LOCAL">
        <provider>org.hibernate.ogm.jpa.HibernateOgmPersistence</provider>
        <properties>

            <property name="com.arjuna.ats.jta.jtaTMImplementation" value="com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionManagerImple"/>
            <property name="com.arjuna.ats.jta.jtaUTImplementation" value="com.arjuna.ats.internal.jta.transaction.arjunacore.UserTransactionImple"/>

            <property name="hibernate.ogm.datastore.provider" value="IGNITE_EXPERIMENTAL"/>
            <property name="hibernate.ogm.ignite.configuration_class_name" value="com.blu.imdg.exampleOgm.ConfigurationMaker"/>
        </properties>
    </persistence-unit>
</persistence>

If you have familiar with JPA before, this persistence definition unit should look very common to you. The main difference to using the classic Hibernate ORM on top of a relational database is the specific provider class we need to specify for Hibernate OGM: org.hibernate.ogm.jpa.HibernateOgmPersistence. Also Note that, we are using RESOURCE_LOCAL instead of JTA. If you want to use JTA, you should have provided a particular JTA implementation such as JBOSS. In addition, we have also specified the following configurations:

• DataStore provide: IGNITE_EXPERIMENTAL
• Configuration_class_name : Ignite configuration (ConfigurationMaker)

Step 7:
Let’s now persist a set of entities and retrieve them. Create a class with name TestOgm with following the following content:
public class TestOgm {
    public static void main(String[] args) throws SystemException, NotSupportedException, HeuristicRollbackException, HeuristicMixedException, RollbackException {
        EntityManagerFactory emf = Persistence.createEntityManagerFactory("ogm-jpa-tutorial");

        EntityManager em =  emf.createEntityManager();
        em.getTransaction().begin();

        Breed collie = new Breed();
        collie.setName("breed-collie");
        em.persist(collie);

        Dog dina = new Dog();
        dina.setName("dina");
        dina.setBreed(collie);
        //persis dina
        em.persist(dina);
        em.getTransaction().commit();
        //get ID dina
        Long dinaId = dina.getId();
        // query
        Dog ourDina =  em.find(Dog.class, dinaId);
        System.out.println("Dina:" + ourDina);

        em.close();

    }

    private static TransactionManager extractJBossTransactionManager(EntityManagerFactory factory) {
        SessionFactoryImplementor sessionFactory = (SessionFactoryImplementor) ( (HibernateEntityManagerFactory) factory ).getSessionFactory();
        return sessionFactory.getServiceRegistry().getService( JtaPlatform.class ).retrieveTransactionManager();
    }
}
First we have created a EntityManagerFactory with parameter “ogm-jpa-tutorial”. Next, we derived our EntityManager from the factory, this EntityManager will be our entry point for persistence entities. We opened a transaction from the EntityManager and create our Breed with name breed-collie. Persist the breed-collie with the entityManager persist() method. Also created an another instance of Dog: dina and associated with it breed-collie. Next we persist the dog dina in cache with the same method persist() and retrieve the instance by the find() method.
Step 8:
Let’s build and run the application. Before run the class TestOgm, we have to run an instance of the Ignite node. Run the following command to start an instance of Ignite node.

mvn exec:java -Dexec.mainClass=com.blu.imdg.StartCacheNode
Now run the following command to execute the TestOgm class as follows:
mvn exec:java -Dexec.mainClass=com.blu.imdg. exampleOgm.TestOgm
You should notice a lot of log into console, with the following entries:
Two entries have been flushed into the Ignite cache and retrieve the dog Dina from the cache. Let’s explorer the cache through Ignite Visor.
Two different cache has been created for the entities: Breed and Dog. If we scan the cache entries of the Dog cache, we should find the following entity on it.

Entity Dina has been persisted into the cache with the key of the Breed collie. Unfortunately, Hibernate HQL or Search is not working in this experimental version of this Hibernate OGM Ignite module. All the hibernate features are under development and will be supported soon.

You can learn even more from book

Friday

Black Friday discount on High performance in-memory computing with Apache Ignite

Black Friday discount on High-performance in-memory computing with Apache Ignite. 40% discount https://leanpub.com/ignite/c/MKU2EsYCj6nN


Wednesday

Tip: SQL client for Apache Ignite cache

In this article, author of the book "High Performance in-memory computing with Apache Ignite", will share a tip to using SQL client for query Apache Ignite cache. Part of this article taken from the Installation and the first Ignite application chapter of the book.

Apache Ignite provides SQL queries execution on the caches, SQL syntax is an ANSI-99 compliant. Therefore, you can execute SQL queries against any caches from any SQL client which supports JDBC thin client. This section is for those, who feels comfortable with SQL rather than execute a bunch of code to retrieve data from the cache. Apache Ignite out of the box shipped with JDBC driver that allows you to connect to Ignite caches and retrieve distributed data from the cache using standard SQL queries. Rest of the section of this chapter will describe how to connect SQL IDE (Integrated Development Environment) to Ignite cache and executes some SQL queries to play with the data. SQL IDE or SQL editor can simplify the development process and allow you to get productive much quicker.

Most database vendors have their own front-end specially developed IDE for their database. Oracle has SQL developer and Sybase has Interactive SQL so on. Unfortunately, Apache Ignite doesn’t provide any SQL editor to work with Ignite caches, however, GridGain (commercial version of the Apache Ignite) provides a commercial GridGain web console application to connect to Ignite cluster and run SQL analytics on it. As far as I work with a multi-platform database in my daily works, the last couple of years I am using Dbeaver6 to work with different databases. A couple of words about Dbeaver, it’s open-source multi-platform database tool for Developers, Analytics or Database administrators. It supports a huge range of Databases and also let you connect to any Database with JDBC thin client (if the database supports JDBC). Anyway, you can also try SQuirrel SQL client or Jetbrains DataGrip to connect to Ignite cluster, they all supports JDBC.

Note that, Cache updates are not supported by SQL queries, for now, you can only use SELECT queries.

How SQL/Text queries work in Ignite: it’s interesting to know how a query is processing under the hood of the Ignite. There are three main approaches to process SQL/Text query in Ignite:

- In-memory Map-Reduce: If you are executing any SQL query against Partitioned cache, Ignite under the hood split the query into in-memory map queries and a single reduce query. The number of map queries depends on the size of the partitions and number of the partitions of the cluster. Then all the map queries are executed on all data nodes of participating caches, providing results to the reducing node, which will, in turn, run the reduce query over these intermediate results. If you are not familiar with Map-Reduce pattern, you can imagine it as a Java Fork-join process.

- H2 SQL engine: if you are executing SQL queries against Replicated or Local cache, Ignite admit that all the data is available locally and runs a simple local SQL query in the H2 database engine. Note that, in replicated cache, every node contains replica data for other nodes. H2 database is free database written in Java and can work as an embedded mode. Depending on the configuration, every Ignite node can have an embedded h2 SQL engine.

- Lucene engine: in Apache Ignite, each node contains a local Lucene engine that stores the index in memory that reference in local cache data. When any distributed full-text queries are executed, each node performs the search in local index via IndexSearcher and send the result back to the client node, where the result aggregated.

Note that, Ignite cache doesn't contain the Lucene index, instead, Ignite provides an in-memory GridLuceneDirectory directory which is the memory-resident implementation to store the Lucene index in memory. GridLuceneDirectory is very much similar to the Lucene RAMDirectory.

To running SQL queries on caches, we already added a complete Java application (HelloIgniteSpring) in chapter installation. You can run the application by the following command.

java -jar .\target\HelloIgniteSpring-runnable.jar

At this moment, we are not going to details all the concepts of Ignite cache queries here. We will have a detailed look at Ignite SQL queries on chapter four. For now, after running the HelloIgniteSpring application, it always put a few Person objects into cache named testCache. Object Person has attributes like name and age as follows:


Property Name Property Age
1 Shamim 37
2 Mishel 2
3 Scott 55
4 Tiger 5

After completing the configuration of the Dbeaver SQL client, we will run a few SQL queries against the above objects. Now it’s the time to download the Dbeaver and complete the JDBC configuration on it.

Step 1:
Download the Dbeaver Enterprise edition (it’s free but not an open source product) for your
operating system from the following URL:

http://dbeaver.jkiss.org/download/enterprise/

Step 2:
Install the Dbeaver, please refer to the install section of the Dbeaver site, if you will encounter any problems during the installation.

Step 3:
Compile the maven chapter-installation project, if you didn’t do it before.

Step 4:
Run the HelloIgniteSpring application with the following command:

java -jar ./target/HelloIgniteSpring-runnable.jar

You should have the following output in your console:


If you are curious about the code, please refer to the chapter-installation.

Step 5:
Now, let’s configure the JDBC driver for the Dbeaver. Go to Database -> Driver Manager -> New In the Settings section, fill in the requested information as follow:


Add all the libraries shown in the above screenshot. Copy and rename the file ∼/ignite-book- code-samples/chapters/chapter-installation/src/main/resources/default-config.xml into default-config-dbeaver.xml somewhere in your file system. Change the clientMode properties value to true in the default-config-dbeaver.xml file. Add the file path to the URL template as shown in the above screenshot and click ok.

Step 6:
Create a New connection based on the Ignite Driver manager. Go to the Database->New Connection. Select Ignite drive manager from the drop down list and click next. You should have the following screen before you.


Click the Test connection button for a quick test. If everything is done properly, you should have the next screen shot with the success notification.

Click ok and go through all the next step to complete the connection.

Step 7:
Create a new SQL editor and type the following SQL query on Dbeaver.

SELECT name FROM Person;

Step 8:
Run the script by pressing the button command+x and you should have the following result.

The above query returns all the cache objects from the cache testCache. You can also execute the following query:

SELECT name FROM Person p WHERE p.age BETWEEN 30 AND 60;

It should return the result with the following person

Shamim 
Scott

Ignite SQL engine is fully ANSI-99 compliant and let you run any SQL query like analytical or Ad-hoc queries. You can also try to configure Oracle SQL developer or IntellijIdea as a SQL client to work with Apache Ignite.

If you like this article, you would also like the book