Stateful Apps and Support of Persistent Storage

Stateful Apps and Support of Persistent Storage
June 8, 2020 Karan Ingale

With enterprises and ISVs adopting containers and Kubernetes (k8s) to increase the agility and scalability of their applications, they would want more applications to be deployed on k8s. Applications can be a mix of stateful and stateless. Until recently, only stateless applications were supported by k8s. However, with the advent of persistent storage on k8s, stateful applications will be supported. In the pet Vs cattle analogy of service, you would want to treat the storage as cattle.

CNCF supports a lot of options (the last time I checked, there were 40+) for cloud-native storage options. The cloud-native storage needs to scale horizontally, there should be no single point of failure, it needs to be resilient and self-heal, and it needs to be decoupled from the underlying platform. 

Let’s look at the landscape of containerized applications, they vary from AI model training, Machine Learning, Big data analytics, IoT, data warehousing, databases, and many more. If they were to be modernized, a special storage arrangement will have to be implemented. That is where the Kubernetes persistent storage object model comes in along with CSI (Container Storage Interface). It provides an ideal IaaC based model to provide persistent storage options to the applications mentioned above.

Dynamic storage provisioning options provided by CSI providers (e.g. Portworx and others) are ideal to define and provision the persistent storage for the application workloads. CloudHedge Cruize can be configured to interface with CSI providers’ API with cloud-based Persistent Storage options (Azure Disk, Google Persistent Disk, Amazon EBS, etc.) and automatically detects all the available storage options.

Such CSI providers usually have API and CLI interfaces which can be leveraged and CloudHedge Cruize can provide the necessary GUI interfaces to interact and integrate with them, thus providing simple user experience. CloudHedge automates application containerization using Discover and Transform, we also support configuration of persistent storage (via Cruize) before deploying (drag and drop) the applications on k8s.

During the assessment of application using Discover, CloudHedge identifies applications that are making persistent writes to the disk and detects the need for persistent storage. It also captures the application’s characteristics and captures the metadata. CloudHedge Transform creates the deployment spec for such applications in order to make the external storage available to the container as external volumes. Then Cruize enables the options to provision static or dynamic persistent storage. 

Implementation:

Discover:

  • CloudHedge helps migrate monolith on-premise applications into container-based applications.
  • CloudHedge is able to identify applications that are making persistent writes to the disk and detect the need for persistent storage.
  • CloudHedge Discover feature captures these application characteristics in the discover metadata.

Transform:

CloudHedge Transform feature creates the deployment spec (Dockerfile) for such applications in order to make the external storage available to the container as external volumes.

Cruize:

Following use case is where dynamic provisioning can be envisioned

  • CloudHedge Cruize consumes the external storage-specific metadata generated by Discover and the CSI available options and thus dynamically providing the needed persistent storage options required for the application container.

Following use case is where static provisioning can be envisioned

  • Solution Architects can also use Cruize UI and choose the external storage options, provide the specs (type – SSD, etc; location; etc.)