Tuesday

Book: High performance in-memory computing with Apache Ignite has been published


Notice: If you encounter any difficulties purchasing the book (Leanpub is having issues with processing payments from the Russian Federation). Please contact us by hppignitesales@gmail.com

The book "High performance in-memory computing with apache Ignite" has been released and available at http://leanpub.com/ignite
Buy now by 27$







The goal of the book is to provide a guide for those who really need to implement the In-memory platform in their projects. At the same time, the idea behind the book is not writing a manual.

This book wraps all the topics like in-memory data grid, highly available service grid, streaming and in-memory computing use cases from high-performance computing to get the performance gain. The book will be particularly useful for those, who have the following use cases:

  • You have database bottleneck in your application and want to solve the problem.
  • You have a high volume of ACID transactions in your system.
  • You want to develop and deploy microservices in distributed fashion.
  • You have existing Hadoop ecosystem (OLAP) and want to improve the performance of the Map/Reduce jobs without making any changes in your existing Map/Reduce jobs.
  • You want to share Spark RDD directly in-memory (without storing the state to disk), which can dramatically increase the performance of the Spark jobs.
  • You are planning to migrate to microservices and the web session clustering is the problem for you.
  • You are planning to process continuous never-ending streams and complex events of data in scalable and fault-tolerant fashion.
  • You want to use distributed computations in parallel fashion to gain high performance, low latency, and linear scalability.
  • You heard about Off-heap memory but don't know how to use it in your application.

For every topic, a complete application is delivered, which will help the audience to quick start with the topic. The book is a project-based guide, where each chapter focuses on the complete implementation of a real-world scenario, the commonly occurring challenges in each scenario has also discussed, along with tips and tricks and best practices on how to overcome them. Every chapter is independent and a complete project.

Who is this book for

Target audience of this book will be IT architect, team leaders, a programmer with minimum programming knowledge, who want to get the maximum performance from their applications.

No excessive knowledge is required, though it would be good to be familiar with JAVA and Spring framework. The book is also useful for any reader, who already familiar with Oracle Coherence, Hazelcast, Infinispan or memcached.

Happy Reading.

Sunday

Modern application design with In-memory data fabrics


In this article, author of the book "High Performance in-memory computing with Apache Ignite", discussing the design of modern application architecture with Apache Ignite. Part of this article taken from the book.

Let’s take a quick look at an architecture of a traditional system. The traditional application architecture uses data stores which have synchronous read-write operations. This is useful for data consistency and data durability, but it is very easy to get bottleneck if there are a lot of transactions waiting in the queue. Consider the following traditional architecture as shown below.

High-volume transaction processing.

In-memory data grid adds an additional layer within an environment, which uses the Random-Access Memory (RAM) of the server to store most of all data required by the applications. In-memory data grid sits between the application servers and the data store. In-memory data grid uses a cache of frequently accessed data by the client in the active memory and then can access the persistence store whenever needed and even asynchronously send and receive updates from the persistence store. An application architecture with in-memory data grid is shown below.

By using in-memory data grid, data moves closer to the application endpoints. This approach reduces the response times and can lower transactions times from a few seconds to fractions of a second. This way, application can support extremely large numbers of concurrent transactions involving terabytes of operational data, providing a faster, more reliable transactional experience for customers. It is also a more modern scalable data management system than traditional RDBMS, able to elastically scale as demand increases.

Resilient web acceleration.

With in-memory data grid like Apache Ignite, you can provide fault tolerance of your web application and accelerate your web application performance. Without changing any code, you can share session states between web applications through caches.
This above approach provides the highest level of high availability of a system and a customer experience. Since Ignite is an in-memory solution, the performance of the web session clustering and replication mechanism of user web sessions are very high.

Event processing & real-time analysis.

Data tells the story of what's happing with your business on the background right now. With the IoT as a continuous data source, the opportunities to take advantage of the hot data is greater than ever. Traditional data management system cannot process big data fast enough to notify the business of important events as they occur: such as online credit card fraud detection or risk calculation. Apache Ignite allows processing continuous never-ending streams of data in scalable and fault-tolerant fashion in-memory, rather than analyzing data after it's reached the database.
Not only does this enable you to correlate relationships and detect meaningful patterns from significantly more data that you can process it faster and much more efficiently. Apache Ignite in-memory data grid can manage a tremendous amount of incoming data and push notifications to the business application when changes occur with the server. The Apache Ignite continuous queries capability allows systems to quickly access a large amount of incoming never ending data and take action.

Microservices in distributed fashion.

Microservice architecture has a number of benefits and enforces a level of modularity that in practice is extremely difficult to achieve with a monolithic code base. In-memory data grid like Apache Ignite can provide independent cache nodes to corresponding microservices in the same distributed cluster and gives you a few advantages over traditional approaches.
It allows you to use maximum uses of the data fabrics/grid resources. Services running on the in-memory cluster is much faster than the disk-based application server. Apache Ignite microservice based service grid provides automatically deploy any number of distributed service instance in the cluster.

BigData accelerator.

Hadoop has been widely used for its ability to economically store and analyze large data sets and has long passed the point of being nascent technology. However, it’s batch scheduling overhead and disk-based data storage have made it unsuitable for use in analyzing live, real-time data in the production environment. One of the main factors that limit performance scaling of Hadoop and Map/Reduce is the fact that Hadoop relies on a file system that generates a lot of input/output (I/O) files. An alternative is to store the needed distributed data within the memory. Placing Map/Reduce in- memory with the data it needs eliminates file I/O latency.
Apache Ignite has offered a set of useful components allowing in-memory Hadoop job executing and file system operations. Apache Ignite Hadoop accelerator can automatically deploy all necessary executable programs and libraries for the execution of MapReduce across the JVMs, greatly reducing startup time down to milliseconds. This speeds up analysis by avoiding delays in accessing secondary storage. Also, because the execution engine is integrated with the in-memory data grid, key/value pairs hosted within the data grid can be efficiently read into the execution engine to minimize access time.

Cache as a Service.

Data-driven applications that take too long to load are boring and frustrating to use. Four out of five online users will click away if a page stalls while loading. In-memory data grid can provide a common caching layer across the organization, which can allow multiple applications to access managed in-memory cache.

You can isolate the caching layer from the applications by separating the caching layer from the applications. Any applications (Java, .Net, C++) across the organization can store and read data from the cache. By using in-memory data grid as a service, it’s not necessary to build and deploy local caching infrastructure for each application. Applications can use Apache Ignite as cache-aside or write behind to their data base or load data from the database into the cache. It’s eliminate the complexity of management of the hundred or more separate caching infrastructures.

These are some of the ways in-memory grids like Apache Ignite have served as an essential, architectural component for transforming the way businesses use their data to do business. But that’s not all folks! We will cover a lot of in-memory data grid use cases and application architecture in more depth through the book.