Sunday

Pitfalls of the MyBatis Caches with Apache Ignite

UPD1: This blog has been published in Java Dzone https://dzone.com/articles/pitfalls-of-the-mybatis-caches-with-apache-ignite
UPD2: This blog also published in Habrahabr for Russian reader https://habrahabr.ru/company/at_consulting/blog/280452
UPD3: See also the sample chapter of the book "High performance in-memory computing with Apache Ignite" here.

A week ago, MyBatis and Apache ignite announced of support apache ignite as a MyBatis cache (L2 cache).
technically MyBatis support two levels of Caches:
  1. Local cache, which is always enable by default
  2. L2 cache, optional
As Apache Ignite project is fast growing with it's various functionality, in this blog post we are going to examine the MyBatis support in some details.
The second level cache stores the entity data, but NOT the entities or objects themselves. The data is stored in a 'serialised' format which looks like a hash map where the key is the entity Id, and the value is a list of primitive values.
Here is an example how the cache entries looks like in Apache ignite:
Where
Cache Key: CacheKey [idHash=1499858, hash=2019660929, checksum=800710994, count=6, multiplier=37, hashcode=2019660929, updateList=[com.blu.ignite.mapper.UserMapper.getUserObject, 0, 2147483647, SELECT * FROM all_objects t where t.OBJECT_TYPE='TABLE' and t.object_name=?, USERS, SqlSessionFactoryBean]]
Value class: java.util.ArrayList
Cache Value: [UserObject [idHash=243119413, hash=1658511469, owner=C##DONOTDELETE, object_type=TABLE, object_id=94087, created=Mon Feb 15 13:59:41 MSK 2016, object_name=USERS]]
As for Example, i selected the 'all_objects' objects and the following query from the Oracle Database
SELECT count(*) FROM all_objects;

SELECT * FROM all_objects t where t.OBJECT_TYPE='TABLE' and t.object_name='EMP';

SELECT * FROM all_objects t where t.OBJECT_TYPE='TABLE';
In my case, this given query execution time is ~660 ms in average.
SELECT count(*) FROM all_objects;
And the next following query execution time is more than 700ms:
SELECT t.object_type, count(*) FROM all_objects t group by t.OBJECT_TYPE;
lets add apache ignite as a second level cache and examine the result. If you want to know how to install and configure apache ignite with spring and myBatis, please refer to my previous blog post. Moreover, all the source you can find in github repositories.
As a quick start, lets add the myBatis maven dependency in project.
<dependency>
    <groupId>org.mybatis.caches</groupId>
    <artifactId>mybatis-ignite</artifactId>
    <version>1.0.0-beta1</version>
</dependency>
Then, just specify it in the mapper XML as follows
<mapper namespace="com.blu.ignite.mapper.UserMapper">

    <cache type="org.mybatis.caches.ignite.IgniteCacheAdapter" />

    <select id="getUserObject" parameterType="String" resultType="com.blu.ignite.dto.UserObject" useCache="true">
        SELECT * FROM all_objects t where t.OBJECT_TYPE='TABLE' and t.object_name=#{objectName}
    </select>
    <select id="getAllObjectsTypeByGroup" parameterType="String" resultType="com.blu.ignite.dto.UobjectGroupBy" useCache="true">
        SELECT t.object_type, count(*) as cnt FROM all_objects t group by t.OBJECT_TYPE
    </select>

    <select id="allObjectCount" parameterType="String" resultType="String" useCache="true">
        SELECT count(*) FROM all_objects
    </select>
</mapper>
Also i have the following java mapper
public interface UserMapper {
    User getUser( String id);
    List<string> getUniqueJob();
    UserObject getUserObject(String objectName);
    String allObjectCount();
    List<uobjectgroupby> getAllObjectsTypeByGroup();
}
and the web service as follows:
@WebService(name = "BusinessRulesServices",
        serviceName="BusinessRulesServices",
        targetNamespace = "http://com.blu.rules/services")
public class WebServices {
    private UserServices userServices;

    @WebMethod(operationName = "getUserName")
    public String getUserName(String userId){
        User user = userServices.getUser(userId);
        return user.getuName();
    }
    @WebMethod(operationName = "getUserObject")
    public UserObject getUserObject(String objectName){
        return userServices.getUserObject(objectName);
    }
    @WebMethod(operationName = "getUniqueJobs")
    public List<string> getUniqueJobs(){
        return userServices.getUniqueJobs();
    }
    @WebMethod(exclude = true)
    public void setDao(UserServices userServices){
        this.userServices = userServices;
    }
    @WebMethod(operationName = "allObjectCount")
    public String allObjectCount(){
        return userServices.allObjectCount();
    }
    @WebMethod(operationName = "getAllObjectsTypeCntByGroup")
    public List<uobjectgroupby> getAllObjectsTypeCntByGroup(){
        return userServices.getAllObjectCntbyGroup();
    }

}
If i will invoke the web method 'getAllObjectsTypeCntByGroup' in soupUI, first time it will get very high response time, approximately 1700 ms, because the result is not in the cache. From the second times, response time will be ~4 to ~5 ms.

Invoke web method first time will look like this:
Response time of the second or later invoke of web method
In apache ignite cache entry will look like as follows:

Cache Key: CacheKey [idHash=46158416, hash=1558187086, checksum=2921583030, count=5, multiplier=37, hashcode=1558187086, updateList=[com.blu.ignite.mapper.UserMapper.getAllObjectsTypeByGroup, 0, 2147483647, SELECT t.object_type, count(*) as cnt FROM all_objects t group by t.OBJECT_TYPE, SqlSessionFactoryBean]]
Value class: java.util.ArrayList
Cache Value: [UobjectGroupBy [idHash=2103707742, hash=1378996400, cnt=1, object_type=EDITION], UobjectGroupBy [idHash=333378159, hash=872886462, cnt=444, object_type=INDEX PARTITION], UobjectGroupBy [idHash=756814918, hash=1462794064, cnt=32, object_type=TABLE SUBPARTITION], UobjectGroupBy [idHash=931078572, hash=953621437, cnt=2, object_type=CONSUMER GROUP], UobjectGroupBy [idHash=1778706917, hash=1681913927, cnt=256, object_type=SEQUENCE], UobjectGroupBy [idHash=246231872, hash=1764800190, cnt=519, object_type=TABLE PARTITION], UobjectGroupBy [idHash=1138665719, hash=1030673983, cnt=4, object_type=SCHEDULE], UobjectGroupBy [idHash=232948577, hash=1038362844, cnt=1, object_type=RULE], UobjectGroupBy [idHash=1080301817, hash=646054631, cnt=310, object_type=JAVA DATA], UobjectGroupBy [idHash=657724550, hash=1248576975, cnt=201, object_type=PROCEDURE], UobjectGroupBy [idHash=295410055, hash=33504659, cnt=54, object_type=OPERATOR], UobjectGroupBy [idHash=150727006, hash=499210168, cnt=2, object_type=DESTINATION], UobjectGroupBy [idHash=1865360077, hash=727903197, cnt=9, object_type=WINDOW], UobjectGroupBy [idHash=582342926, hash=1060308675, cnt=4, object_type=SCHEDULER GROUP], UobjectGroupBy [idHash=1968399647, hash=1205380883, cnt=1306, object_type=PACKAGE], UobjectGroupBy [idHash=1495061270, hash=1345537223, cnt=1245, object_type=PACKAGE BODY], UobjectGroupBy [idHash=1328790450, hash=1823695135, cnt=228, object_type=LIBRARY], UobjectGroupBy [idHash=1128429299, hash=1267824468, cnt=10, object_type=PROGRAM], UobjectGroupBy [idHash=760711193, hash=1240703242, cnt=17, object_type=RULE SET], UobjectGroupBy [idHash=317487814, hash=61657487, cnt=10, object_type=CONTEXT], UobjectGroupBy [idHash=1079028994, hash=1960895356, cnt=229, object_type=TYPE BODY], UobjectGroupBy [idHash=276147733, hash=873140579, cnt=44, object_type=XML SCHEMA], UobjectGroupBy [idHash=24378178, hash=1621363993, cnt=1014, object_type=JAVA RESOURCE], UobjectGroupBy [idHash=1891142624, hash=90282027, cnt=10, object_type=DIRECTORY], UobjectGroupBy [idHash=902107208, hash=1995006200, cnt=593, object_type=TRIGGER], UobjectGroupBy [idHash=142411235, hash=444983119, cnt=14, object_type=JOB CLASS], UobjectGroupBy [idHash=373966405, hash=1518992835, cnt=3494, object_type=INDEX], UobjectGroupBy [idHash=580466919, hash=1394644601, cnt=2422, object_type=TABLE], UobjectGroupBy [idHash=1061370796, hash=1861472837, cnt=37082, object_type=SYNONYM], UobjectGroupBy [idHash=1609659322, hash=1543110475, cnt=6487, object_type=VIEW], UobjectGroupBy [idHash=458063471, hash=1317758482, cnt=346, object_type=FUNCTION], UobjectGroupBy [idHash=1886921697, hash=424653540, cnt=7, object_type=INDEXTYPE], UobjectGroupBy [idHash=1455482905, hash=1776171634, cnt=30816, object_type=JAVA CLASS], UobjectGroupBy [idHash=49819096, hash=2110362533, cnt=2, object_type=JAVA SOURCE], UobjectGroupBy [idHash=1916179950, hash=1760023032, cnt=10, object_type=CLUSTER], UobjectGroupBy [idHash=1138808674, hash=215713426, cnt=2536, object_type=TYPE], UobjectGroupBy [idHash=305229607, hash=340664529, cnt=23, object_type=JOB], UobjectGroupBy [idHash=1365509716, hash=623631686, cnt=12, object_type=EVALUATION CONTEXT]]

Performance gain:
With simple calculation we can define the performance gain that we have got: Response Time without cache/Response Time with cache = 1589ms/6ms ~265X faster, or (Response Time without cache - Response Time with Cache)/ Response Time with cache * 100 = (1589-6)/6*100 ~ 26383% percent faster.

Conclusion: Expensive database operation can be reduce by using L2 cache, properly using L2 cache in MyBatis can increase the application performance from 10 to 20 times. Apache Ignite in memory data grid is a very suitable candidate for this purpose, certainly, you can also use Hazelcast, EhCache or any other Caching tools.

If you like this article, you would also like the book

Thursday

Quick start with In memory Data Grid, Apache Ignite

UP1: For complete quick start guide, see also the sample chapter of the book "High performance in-memory computing with Apache Ignite" here. Even you can find the sample examples from the GitHub repository.

IMDG or In memory data grid is not an in-memory relational database, an NoSQL database or a relational database. It is a different breed of software datastore. The data model is distributed across many servers in a single location or across multiple locations. This distribution is known as a data fabric. This distributed model is known as a ‘shared nothing’ architecture. IMDG has following characteristics:

  1. All servers can be active in each site.
  2. All data is stored in the RAM of the servers.
  3. Servers can be added or removed non-disruptively, to increase the amount of RAM available.
  4. The data model is non-relational and is object-based. 
  5. Distributed applications written on the platform independent language.
  6. The data fabric is resilient, allowing non-disruptive automated detection and recovery of a single server or multiple servers.

Most of all time we use IMDG for web session management of Application server and as a distributed cache or L2 cache. Hazelcast community addition was our all time favourite IMDG tools, but from the last few releases of hazelcast community edition, it's performance not happy us at all. As a quick alternative of HazelCast, we decided to make a try with Apache ignite. This post is dedicated to apache ignite and be used for quick startup guide. For installation I will use 2 virtual machines of Redhat operating system with following configurations:

CPU: 2
RAM: 4
HDD: 25 GB
OS: Redhat Santiago

From a lot of features of Apache ignite6 we will only examines following features:
  1. Prepare operating system
  2. Using Spring for using DataGrid
  3. MyBatis Cache configuration
  4. Spring Caching

Instaling apache ignite:
Pre require-ties:
1) Java 1.7 and above
2) open ports: 47500..47509, 8080 (for Rest interface), 47400, 47100:47101, 48100:48101, 31100:31101

After installing JDK in operating system, we have to open ports that mentioned above. By following commands we can manipulates iptables.
vi /etc/sysconfig/iptables
-A INPUT -m state --state NEW -m tcp -p tcp --dport 47500:47509 -j ACCEPT
-A INPUT -m state --state NEW -m tcp -p tcp --dport 47400 -j ACCEPT
-A INPUT -m state --state NEW -m tcp -p tcp --dport 47100 -j ACCEPT
-A INPUT -m state --state NEW -m tcp -p tcp --dport 47101 -j ACCEPT
-A INPUT -m state --state NEW -m tcp -p tcp --dport 48100 -j ACCEPT
-A INPUT -m state --state NEW -m tcp -p tcp --dport 48101 -j ACCEPT
-A INPUT -m state --state NEW -m tcp -p tcp --dport 31100 -j ACCEPT
-A INPUT -m state --state NEW -m tcp -p tcp --dport 31101 -j ACCEPT

/etc/init.d/iptables restart
Installation of Apache ignite in few machines:
1) Lets download the ignite 1.5.0final version from the following links.
2) Unzip the archive any where in os such as /opt/apache-ignite
3) Add environmental path IGNITE_HOME to the home directory of the apache ignite.
4) copy the folder $IGNITE_HOME/libs/optional/ignite-rest-http to /home/user/apache-ignite-fabric-1.5.0/libs, it will enable the apache ignite through rest interface.
5) Run the command ignite.sh examples/config/example-cache.xml to start the apache ignite.
If every thing goes fine you should see the following log in your console
[12:32:01] Ignite node started OK (id=ceb614ca)
[12:32:01] Topology snapshot [ver=4, servers=2, clients=0, CPUs=3, heap=2.0GB]
and ignite will also available through http by URL http://host:port/ignite?cmd=version
Using Spring for using DataGrid:
First of all we have to build maven project to write down bunch of code to examine features of apache Ignite.
1) Add the following dependencies to the pom.xml
<dependency>
            <groupId>org.apache.ignite</groupId>
            <artifactId>ignite-core</artifactId>
            <version>${ignite.version}</version>
        </dependency>
        <dependency>
            <groupId>org.apache.ignite</groupId>
            <artifactId>ignite-spring</artifactId>
            <version>${ignite.version}</version>
        </dependency>
        <dependency>
            <groupId>org.apache.ignite</groupId>
            <artifactId>ignite-indexing</artifactId>
            <version>${ignite.version}</version>
        </dependency>
        <!-- myBatis -->
        <dependency>
            <groupId>org.mybatis.caches</groupId>
            <artifactId>mybatis-ignite</artifactId>
            <version>1.0.0-beta1</version>
        </dependency>
        <dependency>
            <groupId>org.mybatis</groupId>
            <artifactId>mybatis-spring</artifactId>
            <version>1.2.4</version>
        </dependency>
        <dependency>
            <groupId>org.mybatis</groupId>
            <artifactId>mybatis</artifactId>
            <version>3.3.1</version>
        </dependency>
        <!-- Oracle 12-->
        <dependency>
            <groupId>com.oracle</groupId>
            <artifactId>ojdbc6</artifactId>
            <version>11.2.0.3</version>
        </dependency>
Please, note that Oracle JDBC client jar should be in local maven repositories. In my case i use Oracle 11.2.02 client.
2) Add spring-context.xml file in resources directory with following contexts
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:context="http://www.springframework.org/schema/context"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:mvc="http://www.springframework.org/schema/mvc"
       xmlns:cache="http://www.springframework.org/schema/cache"
       xsi:schemaLocation="
        http://www.springframework.org/schema/beans
        http://www.springframework.org/schema/beans/spring-beans.xsd
        http://www.springframework.org/schema/cache
        http://www.springframework.org/schema/cache/spring-cache-3.1.xsd
        http://www.springframework.org/schema/context
        http://www.springframework.org/schema/context/spring-context.xsd ">
    <!-- Enable annotation-driven caching. -->
    <cache:annotation-driven/>

    <context:property-placeholder location="classpath:jdbc.properties"/>
    <!-- beans -->

    <bean id="ignite.cfg" class="org.apache.ignite.configuration.IgniteConfiguration">
        <property name="gridName" value="TestGrid"/>
        <!-- Enable client mode. -->
        <property name="clientMode" value="true"/>

        <property name="cacheConfiguration">
            <list>
                <!-- Partitioned cache example configuration (Atomic mode). -->
                <bean class="org.apache.ignite.configuration.CacheConfiguration">
                    <!--<property name="atomicityMode" value="ATOMIC"/>-->
                    <!-- Set cache mode. -->
                    <property name="cacheMode" value="PARTITIONED"/>
                    <property name="backups" value="1"/>
                    <property name="statisticsEnabled" value="true" />
                </bean>
            </list>
        </property>
        <!-- Explicitly configure TCP discovery SPI to provide list of initial nodes. -->
        <property name="discoverySpi">
            <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
                <property name="ipFinder">
                    <!-- Uncomment static IP finder to enable static-based discovery of initial nodes. -->
                    <!--<bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">-->
                    <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.multicast.TcpDiscoveryMulticastIpFinder">
                        <property name="addresses">
                            <list>
                                <!-- In distributed environment, replace with actual host IP address. -->
                                <value>Add your node ip address</value>
                                <value>add your node ip address</value>
                            </list>
                        </property>
                    </bean>
                </property>
            </bean>
        </property>
    </bean>
    <bean id="sqlSessionFactory" class="org.mybatis.spring.SqlSessionFactoryBean">
        <property name="dataSource" ref="dataSource" />
        <property name="mapperLocations" value="classpath*:com/blu/ignite/dao/*Mapper.xml"/>
    </bean>
    <bean id="dataSource" class="oracle.jdbc.pool.OracleDataSource" destroy-method="close">
        <property name="URL" value="${jdbc.url}" />
        <property name="user" value="${jdbc.username}"/>
        <property name="password" value="${jdbc.password}"/>
        <property name="connectionCachingEnabled" value="true"/>
    </bean>
</beans>
Lets examine a few configuration properties:
property name="clientMode" value="true" - this property will force the current application to run as client.
property name="cacheMode" value="PARTITIONED" - cache mode will be partitioned, cache mode can be replicated also.
property name="backups" value="1" - always there will be one redundant element of cache in another node.
property name="statisticsEnabled" value="true" - this property will activate the cache statistics.

3) now lets write some
public class SpringIgniteRun {
    public static void main(String[] args) throws Exception{
        System.out.println("Run Spring example!!");
        ApplicationContext ctx = new ClassPathXmlApplicationContext("spring-core.xml");

        IgniteConfiguration igniteConfiguration = (IgniteConfiguration) ctx.getBean("ignite.cfg");
        Ignite ignite = Ignition.start(igniteConfiguration);
        // get or create cache
        IgniteCache cache = ignite.getOrCreateCache("myCacheName");
        for(int i = 1; i < 1000; i++){
            cache.put(i, Integer.toString(i));
        }
        for(int i =1; i<1000;i++){
            System.out.println("Cache get:"+ cache.get(i));
        }
        Thread.sleep(20000); // sleep for 20 seconds
        // statistics
        System.out.println("Cache Hits:"+ cache.metrics(ignite.cluster()).getCacheHits());
        ignite.close();
    }
}
above code is self explained, we just create a cache named "myCacheName" and add 1000 String value of Integer. After inserting the value to cache, we also read the elements from cache and check the statistics. through ignitevisorcmd you can also monitor the data grid, follows you can find screen shot of the statistics of the grid
MyBatis Cache configuration: Now lets add MyBatis ORM l2 cache and examine how it's works.
<bean id="servicesBean" class="com.blu.ignite.WebServices">
        <property name="dao" ref="userServicesBean"/>
    </bean>
    <bean id="userServicesBean" class="com.blu.ignite.dao.UserServices">
        <property name="userMapper" ref="userMapper"/>
    </bean>

    <bean id="sqlSessionFactory" class="org.mybatis.spring.SqlSessionFactoryBean">
        <property name="dataSource" ref="dataSource" />
        <property name="mapperLocations" value="classpath*:com/blu/ignite/dao/*Mapper.xml"/>
    </bean>
    <bean id="dataSource" class="oracle.jdbc.pool.OracleDataSource" destroy-method="close">
        <property name="URL" value="${jdbc.url}" />
        <property name="user" value="${jdbc.username}"/>
        <property name="password" value="${jdbc.password}"/>
        <property name="connectionCachingEnabled" value="true"/>
    </bean>


    <bean id="userMapper" autowire="byName" class="org.mybatis.spring.mapper.MapperFactoryBean">
        <property name="mapperInterface" value="com.blu.ignite.mapper.UserMapper" />
        <property name="sqlSessionFactory" ref="sqlSessionFactory" />
    </bean>
    <bean class="org.mybatis.spring.mapper.MapperScannerConfigurer">
        <property name="basePackage" value="com.blu.ignite.mapper" />
    </bean>
We add SQLsessionFactory, MyBatis mapper and Service Bean. Now lets add the *.Mapper.xml
<?xml version="1.0" encoding="UTF-8" ?>

<!DOCTYPE mapper
        PUBLIC "-//mybatis.org//DTD Mapper 3.0//EN" "http://mybatis.org/dtd/mybatis-3-mapper.dtd">

<mapper namespace="com.blu.ignite.mapper.UserMapper">

    <cache type="org.mybatis.caches.ignite.IgniteCacheAdapter" />

    <select id="getUser" parameterType="String" resultType="com.blu.ignite.dto.User" useCache="true">
      SELECT * FROM users WHERE id = #{id}
    </select>

    <select id="getUniqueJob" parameterType="String" resultType="String" useCache="false">
        select unique job from emp order by job desc
    </select>

</mapper>
Full sql (DDL/DML) scripts of emp and dept tables will be find in the directory com/blu/ignite/scripts I have created a simple web service to get the users and the unique jobs for employees. Here is the code for the web service as follows:
@WebService(name = "BusinessRulesServices",
        serviceName="BusinessRulesServices",
        targetNamespace = "http://com.blu.rules/services")
public class WebServices {
    private UserServices userServices;

    @WebMethod(operationName = "getUserName")
    public String getUserName(String userId){
        User user = userServices.getUser(userId);
        return user.getuName();
    }
    @WebMethod(operationName = "getUniqueJobs")
    public List getUniqueJobs(){
        return userServices.getUniqueJobs();
    }
    @WebMethod(exclude = true)
    public void setDao(UserServices userServices){
        this.userServices = userServices;
    }

}
Invoke of the web method getUserName will query the database and cache the query result in Ignite cache. Spring Caching: With spring caching you could achieve caching of the return value of any spring bean method. Apache Ignite will create the cache by the name of the cache which you will provide by the annotation @Cacheable("returnHello") For example, if I have such method as follows:
@Cacheable("returnHello")
    public String sayhello(String str){
        System.out.println("Client says:"+ str);

        return "hello"+str;
    }
The first time when the method will be invoked, a replicated cache with argument name will create in ignite, next time every invocation of the above method will return the value from the cache.
For now it's enough. Soon i will return back with some new features of apache ignite. Full source code of the project will found in the github. If you like this article, you would also like the book