Introducing Dell Container Storage Modules (CSM), Part2 – The ‘What’
In the first post of the series, which you can read all about here, I discussed some of the challenges that are associated with managing the storage / Data Protection […]
Dell Storage, PowerStore, PowerFlex PowerMax & PowerScale, Virtualization & Containers Technologies
In the first post of the series, which you can read all about here, I discussed some of the challenges that are associated with managing the storage / Data Protection […]
In the first post of the series, which you can read all about here, I discussed some of the challenges that are associated with managing the storage / Data Protection aspects of Kubernetes, now, let’s discuss our solutions:
Enter CSM or, Introduction to Container Storage Modules
Remember the 2019 session and the in-depth thinking we had gone thorught about our customer’s real world needs? The Kubernetes ecosystem is growing rapidly and when it comes to storage integration, CSI plugins offer a way to expose block and file storage systems to containerized workloads on Container Orchestration Systems (COs) like Kubernetes. Container Storage Modules (CSM) improves the Observability, usability, and data mobility for stateful applications using Dell Technologies storage portfolio and extends Kubernetes storage features beyond what is available in the Container Storage Interface (CSI) specification. CSM along with the underlying CSI plugins are pioneering application-aware/application consistent backup and recovery solutions from the most comprehensive enterprise-grade storage and data protection for Kubernetes. CSM extends enterprise storage capabilities to Kubernetes. It reduces management complexity so developers can independently consume enterprise storage with ease and automate daily operations such as provisioning, snapshotting, replication, observability, authorization, and resiliency. CSM is open-source and freely available from https://github.com/dell/csm.
Dell EMC Container Storage Modules (CSM) brings powerful enterprise storage features and functionality to Kubernetes for easier adoption of cloud-native workloads, improved productivity, and scalable operations. This release delivers software modules for storage management that provide developers with access to build automation for enhanced IT needs and other critical enterprise storage features like data replication across data centers, role-based access control (RBAC) authorization, observability, and resiliency for disaster recovery and avoidance. Improved resource utilization Enables automated access to any of our portfolio storage systems into k8s environment
Enables the K8 environment manager to directly allocate storage and services
Extend Enterprise Storage to Kubernetes – by accelerating adoption of cloud-native workloads with proven enterprise storage with proven enterprise storage
Empower Developers – Improve productivity by reducing development lifecycles
Automate storage operations – Integrate enterprise storage with existing Kubernetes toolsets for scalable operations
The modules are separated into these six specific capabilities. Observability – delivers a single pane to view the whole CSM environment for the K8s/container administrator, via Grafana and Prometheus dashboards that K8s admins are familiar with in monitoring persistent storage performance Replication – enables array replication capabilities to K8s users with support for stretched and replica K8s clusters. Authorization – provides storage and provides Kubernetes administrators the ability to apply RBAC and usage rules for our CSI Drivers. Resiliency – Enables K8s node failover by monitoring persistent volume health Designed to make Kubernetes Applications, including those that utilize persistent storage, more resilient to node failures. The module is focused on detecting; Node failures (power failure), K8s control plane network failures and Array I/O network failures, and move the protected pods to hardware that is functioning correctly. Volume Placement – Intelligent volume placement for Kubernetes workloads, optimized based on available capacity Snapshots- CSI based snapshots for operational recovery and data repurposing Snapshots feature is part of the CSI plugins of the different Dell EMC arrays and takes advantage of the state-of-the-art snapshot technology to protect and repurpose data. In addition to point-in-time recovery, these snapshots are writable and can be mounted for test and dev and analytics use cases without impacting the production volumes. These modules are planned for RTS, but there is a rolling release prioritized based upon customer demand by storage platform – applicable to PowerScale, PowerStore, PowerMax, PowerFlex, and Unity XT. Available on RTS:
The publicly accessible repository for CSM is available at https://github.com/dell/csm. Please see the documentation for a complete set of material on CSM: https://dell.github.io/csm-docs/
Below, you can see an overview demo of CSM
watched it? Awesome, now let’s go deeper into the modules:
Observability
CSM for Observability is part of the CSM (Container Storage Modules) open-source suite of Kubernetes storage enablers for Dell EMC products. It is an OpenTelemetry agent that collects array-level metrics for Dell EMC storage so they can be scraped into a Prometheus database. With CSM for Observability, you will gain visibility not only on the capacity of the volumes/file shares you manage with Dell CSM CSI (Container Storage Interface) drivers but also their performance in terms of bandwidth, IOPS, and response time. Thanks to pre-packaged Grafana dashboards, you will be able to go through these metrics history and see the topology between a Kubernetes PV (Persistent Volume) and its translation as a LUN or file share in the backend array. This module also allows Kubernetes admins to collect array level metrics to check the overall capacity and performance directly from the Prometheus/Grafana tools rather than interfacing directly with the storage system itself. Metrics data is collected and pushed to the OpenTelemetry Collector, so it can be processed, and exported in a format consumable by Prometheus. CSM for Observability currently supports PowerFlex and PowerStore.
Key High-Level Features:
And PowerFlex:
Replication
CSM for Replication is the module that allows provisioning of replicated volumes using Dell storage.
CSM for Replication currently supports PowerMax and PowerStore.
Key High-Level Features:
Replication of PersistenVolumes (PV) across Kubernetes clusters Multi/single cluster topologies
Replication action execution (planned/unplanned failover, sync, pause, resume)
Async/Sync/Metro configurations support (PowerStore only supports Async)
repctl – CLI tool that helps with replication related procedures across multiple k8s clusters
The publicly accessible repository for CSM is available at https://github.com/dell/csm-replication.
Please see the documentation for a complete set of material on CSM Replication: https://dell.github.io/csm-docs/docs/replication/
Video1 – Introduction, and Architecture (using PowerMax as the example)
Below, you can see an end-to-end demos on how to configure CSM replication for PowerStore and perform failover & failback operations of WordPress and MySQL DB, using PowerStore Async replication.
Installing
Performing Failover & Failback (Reprotect)
Using PowerMax?
Authorization
No man (or a customer) is an island, Kubernetes comes in many flavors, here at Dell Technologies, we offer a wide variety of solutions for the customer, starting from just storage arrays for every need (from PowerStore to PowerFlex to PowerMax to PowerScale and ECS) to turnkey solutions like vXRAIL with/without VCF, deep integration with our storage arrays to anything from upstream Kubernetes to RedHat Openshift, with deep integration to the OpenShift Operator, or vSphere with Tanzu, just so we can meet you where you today AND tomorrow.
With Dell Technologies’ broad portfolio designed for modern and flexible IT growth, customers can employ end-to-end storage, data protection, compute and open networking solutions that support rapid container adoption. Developers can create and integrate modern data applications by relying on accessible open-source integrated frameworks and tools across bare metal, virtual, and containerized platforms. Dell enables support for organizational autonomy and real-time benefits for container and Kubernetes platforms with and adherence to IT best practices based on an organization’s own design needs.
In the next post, we will be covering the ‘How’ to install the new CSI 2,0 Common installer and the CSM modules.