Redis in Cloud Foundry

NoSQL Databases

Redis is a distinguished member of a whole new category of databases called NoSQL, which stands for "Not Only" SQL. As we will see in a moment the NoSQL family is by no means a uniform one. There is in fact lots of diversity but in general they have the following characteristics:

• They have no fixed schema. You can even modify it as you go along
• They are more scalable. How big is the largest Oracle database you have come across? 100TB and 4 nodes? Some NoSQL databases can scale to PB and beyond
• They are able to handle unstructured data
• And they are usually cheaper to manage ... btw, I am not questioning your Oracle DBA's salary

All this sounds great, but everything in life has a trade-off. What are the downsides?

• They can have duplicated data
• Most of them offer "eventual" consistency, ie during a "hopefully" short period of time a record is likely to have different values in different nodes ... I can see in your eyes what you are thinking. You are hoping your bank is not using one of these for your credit card details, right?
• They offer worse data integrity
• It is harder to achieve ACID (Atomicity, Consistency, Isolation, Durability) compliance.

Maybe you were wondering at the beginning why NoSQL stands for "Not-Only" SQL instead of "No SQL". It is not hard to see why after reading the four points above.

NoSQL databases are typically classified in 5 types:

• Key-Value (Ex: Redis). They are used to manage transient data like user sessions and profiles
• Document (Ex: MongoDB). Very widely used: blogs, CMS and heavy read analytics
• Columnar (Ex: Cassandra). They are great for heavy write use cases, such as log aggregation, write intensive analytics
• Graph(Ex:Neo4j). They track relationships between items and are heavily used with recommendation engines
• Map-Reduce (Ex: Hadoop)

It is very common for modern applications to use more than one NoSQL database, each for a different purpose and often in conjunction with a traditional SQL database as well. Hence, you will hear the term "Polyglot persistence".

Redis

In future posts we will cover MongoDB and other databases but for now let's look into Redis and how to drive from Python in Cloud Foundry.

Redis is an in-memory database. As you can guess this is good and bad. Redis is very fast but if you are unlucky you can loose data. It features replication and on-disk persistence that could prevent you from loosing some data as opposed to all the data. This limitation means that Redis is often used for caching transient data such as user session data.

Redis is primarily a key-value store. However the "value" part of the equation can be a more elaborated structure such as a list, a set (with unique but unordered elements), a hash (similar to a dictionary), etc

If you want to learn more about the different data structures available and how to use them you can follow the official Redis tutorial. This tutorial runs on a real publicly available Redis database.

In the Pied Piper program we have used Python for all our exercises. It was important for us to learn how to interact with Redis using Python code. The code we used can be accessed in this Github repo:
https://github.com/cermegno/Redis-test

The contents of the repository are:

• Redis-test.py - Cookbook script with sample Redis operations with keys, lists, hashes and sets. You can uncomment whatever lines you are interested on to examine its behaviour and copy and paste the different Python commands to bring Redis functionality into your own apps
• The repo also contains a number of step scripts that lead you to create a website that displays the agenda for an event
• Sample code to that allows your app to determine whether it is running in Cloud Foundry or in your laptop and connect to the relevant Redis database

Now that you have seen what the environment of our sample app will look like, let's look at the code to use it.

import os 

if 'VCAP_SERVICES' in os.environ:
    VCAP_SERVICES = json.loads(os.environ['VCAP_SERVICES'])
    CRED = VCAP_SERVICES["rediscloud"][0]["credentials"]
    r = redis.Redis(host=CRED["hostname"], port=CRED["port"], password=CRED["password"])
else:
    r = redis.Redis(host='127.0.0.1', port='6379')

First it checks whether "VCAP_SERVICES" variable is present in the environment. This surely means it is running in Cloud Foundry. If it is present then it will create a dictionary with its contents, read the hostname and credentials and establish a connection. Otherwise establishes a connection to a local Redis database running in your laptop ... if you have one