Cloud data co-location prototype

By Al
The cloud data co-location use case is a very interesting one. Potentially the biggest reasons why organizations choose not to move a workload to the cloud include:
  • compliance / regulatory requirements
  • lock-in risks and difficulty/costs of migrating to another public/private cloud
  • unpredictable and/or high costs
These reasons also play an important part in the ongoing workload repatriation activity we are seeing in the market. The idea that "cloud is not a destination, it's an operating model" is finally sinking in. But, when we talk about destinations, it doesn't all have to be public or private. There is a third location that plays an interesting part

Last year I participated in a project that studied the feasibility of implementing "cloud data co-location". This basically consists on hosting a traditional storage array in a datacenter that is physically close to a public cloud and connect to it through a high-throughput low-latency link. Some providers like Equinix in APJ host one (or sometimes many) public clouds in their datacenters and offer a "direct-connect" link to these clouds. This allows you to leverage all the compute offerings from the public cloud while retaining control of your data outside (but near to) the cloud

Since the data is already sitting outside the cloud there is less risk of lock in, and certainly no astronomical egress charges. It is also much easier to satisfy regulatory requirements. Finally, costs can be lower than storage offerings from the cloud itself specially when high performance is required

In this article you can see the conclusions of the project, but in summary it provides better performance than the public cloud and it can make sense for many use cases. One important observation from our tests was that NAS protocols (NFS and CIFS) provide better performance than iSCSI

Recently, with a few colleagues from APJ, we created a group of Infrastructure-as-Code enthusiasts and we gave ourselves the name of IaC Avengers. As a foundational project we all run a two-day hackathon whose objective was to build a prototype for the cloud data colo use case. We referred to it as "Project Nebula"



For this we used Ansible with a very simple web frontend. The idea was to create a VM on any cloud, including vSphere (whether VMC or on-prem) and then provision and NFS share and automate the mounting on the VM. We also worked on day-2 operations such as adding storage to an existing VM. From a storage perspective we provided options for DellEMC Unity and for VxFlex (ScaleIO).

In this video you can see a demo of the prototype in action for vSphere with Unity and VxFlex.



Comments

Post a Comment

Popular posts from this blog

Sending PowerStore alerts via SNMP

By Al
Image
In this previous article we discussed how to leverage PowerStore's powerful API to collect audit logs and send them to a Syslog server . We used Logstash to do that because of its extensive list of plugins Sometimes I come across organizations that still have a requirement to send storage array Alerts to SNMP. As in the Syslog case, ultimately most modern monitoring tools today support collecting information via REST API because that's the way the application world is going, i.e. the cloud native way ... it would be fun to see Prometheus monitoring Kubernetes via SNMP :) However there is still some requirement out there for SNMP traps, so we will explore how to do it. For simplicity we will use the same toolset that we used in the previous article. I encourage you to revisit quickly the article as we did an extensive introduction to Logstash, with practical examples as well as a description of the logs available in PowerStore. I won't cover those aspects in this article I
Read more

Electronic Nose - eNose

By Al
Image
In the previous article I provided details on VeeFarm which was one of the best projects we saw in ANZ Pied Piper 2019 edition and one of the best projects in the history of the program. The program inspired me and others to work together to showcase a great idea and the Pied Piper program itself during our presales conference. An idea I suggested to enhance VeeFarm was an electronic nose that could be used to detect the optimal ripening of the produce to advise the owner that it is the right time to pick it up the produce. This is an idea I had in the backburner for a while but never found the time to work on. Vee gave me the perfect excuse. The electronic nose, or eNose for short, is a custom PCB that hosts a range of gas sensors. For the proof of concept, I used the cheap MQ sensors. These are designed mostly for hobbyists and makers and they barely cost $1 per sensor. As one could expect, tweaking and calibrating these sensors it is even harder than the most expensive counter
Read more

Use Vagrant to deploy to AWS

By Al
Image
In the Pied Piper program we run through a Vagrant tutorial as we found it a great tool to help set up other labs. We are not going to cover Vagrant in detail in this blog as there are plenty of good examples out there. If you are interested in the exercises we did you can visit the repository: https://github.com/cermegno/vagrant-lesson However, something I felt is not sufficiently document and would make a good candidate for a post is how to use Vagrant to deploy to AWS and particularly from Windows. I found a couple of posts that provided good detail but none of them prevented me from running into Windows specific issues. If you are running it from Linux, this article will still help you as I will point out the differences anyway. Install prerequisites Firstly, as the Vagrant documentation states, "providers" other than VirtualBox require a plug in. This is how you install the AWS provider: vagrant plugin install vagrant-aws Deploying to AWS EC2 generally mea
Read more