Dell Storage, PowerStore, PowerFlex PowerMax & PowerScale, Virtualization & Containers Technologies
The Kubernetes Eco-system, is growing rapidly and so far, where it comes to storage integration, the only way to integrate was via the CSI plugin which I blogged about here […]
The Kubernetes Eco-system, is growing rapidly and so far, where it comes to storage integration, the only way to integrate was via the CSI plugin which I blogged about here
But, as we get more and more customers who are adopting Kubernetes, it’s only natural that they will want more and more functionality to be presented, far above and beyond what CSI is capable of..
Enter Project Karavi
The ultimate source of wisdom, aka, Wikipedia, tells us that Karavi is an island in Greek and we thought it will be a cool name to leverage to explain what we are doing beyond just CSI and data protection, it’s important to note that for us, Karavi is a journey that will covers many aspects and this is the beginning of this journey..
In the fast-moving, competitive, and technology-driven business environment, companies are adopting containers/Kubernetes to accelerate cloud-native-application development and value release to the customers. Kubernetes use cases outgrew the basic webserver type of apps and today include mission-critical workloads such as databases. Customers expect enterprise-class features such as encryption, replication (Disaster Recovery), etc. to be offered for Kubernetes, even if the Container Storage Interface does remain ignorant of such capabilities. The Karavi project objective is to expose enterprise storage features to Kubernetes users.
Project Karavi aims at improving the observability, usability and data mobility for stateful applications with Dell Technologies Storage portfolio. Project Karavi together with the CSI plugins and the pioneering app-aware, app-consistent backup and recovery solutions form the most comprehensive enterprise grade Storage and Data Protection solutions for Kubernetes from Dell Technologies.
Project Karavi is currently in tech-preview and includes Karavi Observability
Karavi Observability is part of the Project Karavi that covers Kubernetes storage features and functionality for Dell EMC storage portfolio. Karavi Observability provides standardized approaches for storage observability.
This project brings capacity and performance metrics collection as well as basic storage topology by pulling the data from the storage array and the CSI drivers and sending it to Grafana and Prometheus, community-driven open source tools which became the de-facto standard for collecting and visualizing various Kubernetes ecosystem metrics.
Karavi Observability currently has the following services:
|
Karavi Observability Service |
Description |
Repository |
| Karavi Metrics for PowerFlex | Karavi Metrics for PowerFlex captures telemetry data about Kubernetes storage usage and performance obtained through the CSI (Container Storage Interface) Driver for Dell EMC PowerFlex. The metrics service pushes it to the OpenTelemetry Collector, so it can be processed, and exported in a format consumable by Prometheus. Prometheus can then be configured to scrape the OpenTelemetry Collector exporter endpoint to provide metrics so they can be visualized in Grafana. Please visit the repository for more information. | Karavi Metrics for PowerFlex |
| Karavi Topology | Karavi Topology provides Kubernetes administrators with the topology data related to containerized storage that are provisioned by a CSI (Container Storage Interface) Driver for Dell EMC storage products. Please visit the repository for more information. | Karavi Topology |
Each of the services listed above can be deployed independently by the documentation provided in the associated repositories. Alternatively, the services can all be deployed together as part of a single deployment of the Karavi Observability solution.
This project is deployed using Helm. Usage information and available release versions can be found here: Karavi Observability.
This application requires the following third-party components to be deployed in the same Kubernetes cluster as the karavi-metrics-powerflex service:
Prometheus
As part of the Karavi Observability deployment, the OpenTelemetry Collector gets deployed. The OpenTelemetry Collector is what the Karavi metrics services use to push metrics to that can be consumed by Prometheus. This means that Prometheus must be configured to scrape the metrics data from the OpenTelemetry Collector.
The Grafana dashboards require Grafana to be deployed in the same Kubernetes cluster as the Karavi Observability services.
Once Grafana and Prometheus are properly configured, customers can import the pre-built observability dashboards, of course, each dashboard can be edited, and customers can build their own customized dashboards
| I/O Performance by node dashboard | Provides visibility into the I/O performance metrics (IOPS, bandwidth, latency) by Kubernetes node
|
| I/O Performance by volume dashboard | Provides visibility into the I/O performance metrics (IOPS, bandwidth, latency) by volume
|
| Storage consumption dashboard | Provides visibility into the total, used, and available capacity for a storage class and associated underlying storage construct.
|
| Topology dashboard | Provides visibility into Dell EMC CSI (Container Storage Interface) driver provisioned volume characteristics in Kubernetes correlated with volumes on the storage system.
|
Together with the new PowerFlex HTLM5 UI, Karavi Observability provides end to end performance monitoring solution for Kubernetes cloud native applications running on PowerFlex, from the PowerFlex storage building blocks – to the application persistent volumes.
For more information:
Official GitHub Page https://github.com/dell/karavi/
Dell Container Community : https://www.dell.com/community/Containers/bd-p/Containers
You can watch a demo, showing how it all works, below
