Cloud Foundry High Availability features

By Al
Cloud Foundry has several High Availability features which are essential on an Enterprise application platform. We can classify them into two categories depending on at what level they are implemented.


  • Elastic Runtime features. These are features that are implemented at the application instance level
    • Availability Zones. Administrators can define multiple Availability Zones which typically correspond to chunks of the infrastructure that are independent from each other, such as different physical servers sitting on a separate racks with different power sources, etc. When you define an application with multiple instances, Cloud Foundry automatically distributes the app across the available Availability Zones. This ensures that if an AZ goes down either accidentally or during maintenance the app can keep running
    • App instance fails. Diego sends "heartbeat" messages to the "Health Manager" component of the platform. If the "actual state" is different from "desired state" it instructs the "Cloud Controller" to restart the instance


  • BOSH Features. BOSH is a major differentiator of Cloud Foundry. It has the ability to interface downwards with the IaaS atop which the platform sits. This ability to "abstract" the underlying IaaS is what makes Cloud Foundry a multi-cloud platform. There are multiple choices of (VMware, Openstack, ...). BOSH also has the notion of what the "Desired State" is. To clarify this is the desired state of the VM's that conform the platform
    • Elastic Runtime process goes down. BOSH is aware of the many process that Elastic Runtime needs to do its job. Some examples are the processes I mentioned above, such as "Health Manager", "Cloud Controller" or "Diego". If any of these process dies it will attempt to restart it and it will notify the Ops team
    • VM goes down. VMs have a BOSH agent running on them. If the agent or the VM goes down BOSH restarts the VM, through a conveniently named process called "the resurrector" ... you gotta love CF terminology !!
In the video below I demonstrate how Elastic Runtime restarts a failed application instance. The demonstration is a bit convoluted as it is not easy to crash an app instance. The way I simulate it is by killing the main process in the instance. For example a Python app requires a "Procfile" file which defines what needs to be run as soon as the instance is app. For example a "Procfile" could read:


web: python app.py

This means that as soon as the container is up, the instance will attempt to run "app.py". From that point on the health of "app.py" is monitored as there is no point in keeping the instance app if that process is not running. In the video I leverage the ability to SSH into an instance to run the "kill" command and show how the instance gets restarted immediately.


Comments

Post a Comment

Popular posts from this blog

Sending PowerStore alerts via SNMP

By Al
Image
In this previous article we discussed how to leverage PowerStore's powerful API to collect audit logs and send them to a Syslog server . We used Logstash to do that because of its extensive list of plugins Sometimes I come across organizations that still have a requirement to send storage array Alerts to SNMP. As in the Syslog case, ultimately most modern monitoring tools today support collecting information via REST API because that's the way the application world is going, i.e. the cloud native way ... it would be fun to see Prometheus monitoring Kubernetes via SNMP :) However there is still some requirement out there for SNMP traps, so we will explore how to do it. For simplicity we will use the same toolset that we used in the previous article. I encourage you to revisit quickly the article as we did an extensive introduction to Logstash, with practical examples as well as a description of the logs available in PowerStore. I won't cover those aspects in this article I
Read more

Sending PowerStore logs to Syslog

By Al
Image
In this article we will explore how to get the logs from a DellEMC PowerStore array to a Syslog server. For this purpose we will use the PowerStore’s REST API, which is a great piece of engineering and a joy to work with as a developer. If you want to learn more about the PowerStore REST API I strongly recommend you quickly skim through the 2 articles I have written about the REST API. Part 1 - http://anzpiper.blogspot.com/2020/08/intro-to-dellemc-powerstore-rest-api.html Part 2 - http://anzpiper.blogspot.com/2020/08/powerstore-rest-api-best-practices.html In particular the second article demonstrates the capabilities of the REST API query language. This is a great feature I will use heavily in the last section so I strongly recommend you read that one at least As a side note, Syslog was developed in 1980. So this year it has turn 40 years old! That is a long time by any measure ... many of my colleagues were not even born in 1980. But in the technology scale it looks even scarier. 198
Read more

Electronic Nose - eNose

By Al
Image
In the previous article I provided details on VeeFarm which was one of the best projects we saw in ANZ Pied Piper 2019 edition and one of the best projects in the history of the program. The program inspired me and others to work together to showcase a great idea and the Pied Piper program itself during our presales conference. An idea I suggested to enhance VeeFarm was an electronic nose that could be used to detect the optimal ripening of the produce to advise the owner that it is the right time to pick it up the produce. This is an idea I had in the backburner for a while but never found the time to work on. Vee gave me the perfect excuse. The electronic nose, or eNose for short, is a custom PCB that hosts a range of gas sensors. For the proof of concept, I used the cheap MQ sensors. These are designed mostly for hobbyists and makers and they barely cost $1 per sensor. As one could expect, tweaking and calibrating these sensors it is even harder than the most expensive counter
Read more