Remote Dictionary Server(Redis) and Redis Enterprise Cluster

1. Overview

An in-memory data structure project implementing a distributed, in-memory key-value database with optional durability. Redis supports different kinds of abstract data structures, such as strings, lists, maps, sets, sorted sets, HyperLogLogs, bitmaps, streams, and spatial indexes. 

2. Description

2.1 Differences with other database systems

2.1.1 Can be considered either a store and a cache at the same

• Using a design where data is always modified and read from the main memory(RAM)
• Also Stored on disk in a format that is unsuitable for random access of data but only to reconstruct the data back in memory once the system restart

2.1.2 Specific operations that are performed on given abstract data types

• Data must be stored in a way which suitable later for fast retrieval without help from the database system in the form of secondary indexes aggregations or other common features of traditional RDBMS

2.1.3 Makes heavy use of the Fork(system call) to duplicate the process holding the data 

• So that the parent continues to serve clients
• While the child process creates a copy of the data on disk

 

2.2 Redis Architecture

2.2.1 Components of Redis

• Redis Client
• Redis Server

These clients and server can be on the same computer or two different computers.

Redis server is used to store data in memory. It controls all type of management and forms the main part of the architecture. You can create a Redis client or Redis console client when you install Redis application or you can use.

2.3 Features of Redis

Feature	Description
Speed	Redis stores the whole dataset in primary memory(DRAM). Redis supports pipelining of commands and facilitates you to use multiple values in a single command to speed up communication with the client libraries
Persistence	While all the data lives in memory, changes are asynchronously saved on disk using flexible policies based on elapsed time and/or number of updates since last save Redis supports an append-only file persistence mode
Data Structures	Redis supports various types of data structures such as strings, hashes, sets, lists, sorted sets with range queries, bitmaps, hyperloglogs and geospatial indexes with radius queries
Atomic Operations	Redis operations working on the different Data Types are atomic, so it is safe to set or increase a key, add and remove elements from the set, increase a count, and etc.
Supported Languages	Redis supports a lot of languages such as ActionScript, C, C++, C#, Clojure, Common Lisp, D, Dart, Erlang, Go, Haskell, Haxe, Io, Java, JavaScript (Node.js), Julia, Lua, Objective-C, Perl, PHP, Pure Data, Python, R, Racket, Ruby, Rust, Scala, Smalltalk and Tcl
Master/Slave Replication	Redis follows a very simple and fast Master/Slave replication It takes only one line in the configuration file to set it up 21 seconds for a Slave to complete the initial sync of 10MM key set on an Amazon EC2 instance
Sharding	Redis support sharding which is very easy to distribute the dataset across multiple Redis instances, like other key-value stores
Portable	Redis is written in ANSI C and works in most POSIX systems like Linux, BSD, Mac OS X, Solaris, and WIN32.

 

2.4 Redis Enterprise Cluster Architecture

• A simple database which is a single master shard
• A highly available(HA) database which is a pair of master and slave shards
• A clustered database, which contains multiple master shards, each managing a subset of the dataset(or in Redis terms, a different range of "hash-slots")
• An HA clustered database which multiple pairs of master/slave shards

2.4.1 Each database can be built in one of the following forms

• A Redis on RAM database
• A Redis on Flash database, in which Flash(SSD or persistent memory) is used as a RAM extender
• A Memcached database(in reality, A Redis on RAM database, accessible through the Memcached protocol)

2.4.2 Each database can be accessed using one of the following methods

• Database endpoint: Simply connect your application to your database endpoint, and Redis Enterprise will handle all the scaling and failover operations in a transparent manner
• Sentinel API: Use the sentinel protocol to connect to the right node in the cluster in order to access your database
• OSS Cluster API: Use the cluster API to directly connect to each shard of your cluster without any additional hops

2.4.3 Shared-Nother, Linearly Scalable, Multi-Tenant, Symmetric Architecture

Redis Enterprise cluster is built on a complete separation between the data path components (i.e proxies and shards) and the control/management path components (i.e. cluster management processes), which provides the following benefits

Feature	Description
Performance	Data path entities need not deal with control and management duties The architecture guarantees that any processing cycles are dedicated to serving users' requests, improving the overall performance Each Redis shard in a Redis Enterprise cluster works as if it were a standalone Redis instance The shard doesn't need to monitor other Redis instances, has no need to deal with failure or partition events and is unaware of which hash-slot are being managed.
Availability	The application continues to access data from its Redis database, even as sharding, resharding, and re-balancing takes place No manual changes are needed to ensure data access
Security	Redis Enterprise prevents configuration commands from being executed via the regular Redis APIs Any configuration operation is allowed through a secure UI, CLI or API interface that follows the role-based authorization controls The proxy-based architecture ensures that only certified connections can be created with each shard, and only certified requests can be received by Redis shards
Manageability	Database provisioning, configuration changes, software updates, and more are done with a single command(via UI or API) in a distributed manner and without interrupting user traffic

2.4.4 Redis Enterprise Cluster Components

Redis Enterprise cluster is built on a symmetric architecture, with all nodes containing the following components

Component	Description
Redis Shard	An OSS Redis instance with either a master or slave role that is part of the database
Zero-Latency Proxy	The proxy runs on each node of the cluster based on a cut-through, multi-threaded, lock-free stateless architecture. Hides cluster complexity from the application/user Maintains the database endpoint Request forwarding Supports the Memcached protocol Manages data encryption through SSL Provides strong, client-based SSL authentication Enable Redis acceleration through pipelining and connection management
Cluster Manager	Contains a set of distributed processes that together manage the entire cluster lifecycle The cluster manager entity is completely separated from the data path components(such as proxies and Redis shard) and has the following responsibilities Database provisioning and de-provisioning resource management watchdog processes auto-scaling re-sharding re-balancing
Secure REST API	All the management and control operations on the Redis Enterprise cluster are performed through a dedicated and secure API that is resistant to attacks and provides better control of cluster admin operations. The ability of provision and de-provision Redis resources at a very high rate, and with almost no dependency on the underlying infrastructure Very suitable for the new generation of microservices-based environments

3. References

https://en.wikipedia.org/wiki/Redis

https://www.javatpoint.com/redis-tutorial

https://redislabs.com/redis-enterprise/technology/redis-enterprise-cluster-architecture/

https://www.webopedia.com/TERM/P/provisioning.html

https://www.slideshare.net/BadoMaturin/discover-some-big-data-architectural-concepts-with-redis

http://qnimate.com/overview-of-redis-architecture/