A few months ago, we introduced ‘Project Karavi’. The first module we introduced, was the ‘Obsevarability’ one With the new module we are releasing today, this project has matured to […]
With the new module we are releasing today, this project has matured to have a new, official name: Container Storage Module!
Dell EMC Container Storage Modules (CSM) aims at improving the observability, usability and data mobility for stateful applications with Dell Technologies Storage portfolio. CSM together with the CSI plugins and the pioneering app-aware, app-consistent backup and recovery solutions form the most comprehensive enterprise grade Storage and Data Protection solutions for Kubernetes from Dell Technologies.
CSM Authorization is part of the CSM open source suite of Kubernetes storage enablers for Dell EMC products. CSM Authorization provides storage and provides Kubernetes administrators the ability to apply RBAC for Dell EMC CSI Drivers. It does this by deploying a proxy between the CSI driver and the storage system to enforce role-based access and usage rules.
This module is currently supported with DELL EMC PowerFlex systems. Additional storage arrays will be added in the future.
Storage administrators of compatible storage platforms will be able to apply quota and RBAC rules that instantly and automatically restrict cluster tenant’s usage of storage resources. Users of storage through CSM Authorization do not need to have storage admin root credentials to access the storage system.
Kubernetes administrators will have an interface to create, delete, and manage roles/groups that storage rules may be applied. Administrators and/or users may then generate authentication tokens that may be used by tenants to use storage with proper access policies being automatically enforced.
The following diagram shows a high-level overview of CSM Authorization with a tenant-app that is using a Dell CSI driver to perform storage operations through the CSM Authorization proxy-server to access the Powerflex storage system. All requests from the CSI driver will contain the token for the given tenant that was granted by the Storage Administrator.
With the help of CSM authorization customers can:
- Segregate the array usage between multiple tenants
- Control storage consumption with the help of quota
- Ensure that a tenant cannot access storage of other tenants
- Create, update or revoke storage access at any point in time
- Audit storage access by the tenants
- Hide the array management credentials from the CSI driver and replace it with a JSON web token
The key concept behind CSM Authorization is to minimize the storage administration exposure from the Kubernetes storage consumers.
We are segregating the responsibilities between the Kubernetes administrator and the Storage administrator by adding a component that acts as a proxy between the Kubernetes clusters and the management interfaces of backend arrays.
The storage admin will deploy CSM authorization and operate the component on several aspects like:
- maintaining the list of tenants and JSON Web Token to authenticate the users used by the CSI driver for queries.
- maintaining the list of arrays and credentials to access them,
- and, last but not least it will maintain the relationships and quotas between those two.
The steps of a normal workflow are:
- The storage admin creates the different elements (quota, storage, systems, tokens, etc.)
- a cluster requests to create / delete a volume or a snap
- the proxy intercepts the query and validates it (is it authorized, do we have the quota for it) ; if invalid we return a payload with the details of the reject error to the user
- in case of success the proxy forwards the request to Powerflex
- in case of error we forward the error to the querier; in case success we return the success and we update the quota accordingly
The CSM Authorization is published in several different packages, either as Kubernetes application that you can load on an existing cluster or as an all-in-one package to be installed on separate host.
The standalone package contains everything necessary to run CSM authorization:
- k3s Kubernetes distribution to run the application
- the application itself as a set of Pods, services, etc.
- karavictl to manage the different aspects of the application
- and even the patch for the CSI drivers to connect to the application
Once downloaded you can install it with:
rpm -ivh karavi-authorization-0.1-0.x86_64.rpm
Now that the installation is done you can check the application is running with:
Configure CSM auth
The first step is adding an array. The array is the entrypoint for other objects like access or tenants to be hooked to.
To add a storage system we run the following command:
karavictl storage create –endpoint https://********/ –insecure –system-id ********** –type powerflex –user admin
In case you put the wrong password karavi will prompt you to reenter a correct password.
We can check the configured systems with :
karavictl storage list
here we can see the details of the registered PowerFlex system.
The next step is to configure the storage access roles:
The first parameter is the role name, for example and R&D role for my development team, next, we specify the array type and our specific array ID, next we specify the storage pool and the allocated quota for this role
karavictl role create –role=RnD=powerflex=0ca110b75dd4300f=SP1=44000000
we can check the roles with:
karavictl role list
Next, we need to create a tenant object, this can be our Kubernetes or Openshift cluster
karavictl tenant create -n openshift -4.6
and now, we can Link our tenant to the role we’ve just created:
karavictl rolebinding create –tenant openshift-4.6–role RnD
Next we generate a token from our CSM machine and Creating the new secret with the token in the namespace of the CSI driver.
karavictl generate token –addr *********:443 -t openshift -4.6 | jq -r “.Token” | kubectl apply -n vxflexos -f –
we Check that the installer is ready:
and then go for it:
curl http://CSM.local/install | sh
After the installation, we have an additional container in the controller pods, this is the authorization proxy which intercepts the query and validates it against the CSM machine.
In the screenshot below, the Kubernetes user tries to create a PVC larger than its allocated quota, as a result, the volume hasn’t been created.
By navigating to the CSM authorization logs, we can see that this tenant reached hid limit, and as a result he cannot consume more storage from that PowerFlex storage system,
In this case, he needs to remove existing persistent volumes or contact the storage admin and for more storage capacity.
Below you can see a demo of how it all works together
In addition to this, a new pre-release version (v0.3.0) of CSM for Observability has also been released with support for the 1.4 release of the CSI driver for PowerFlex and CSM for Authorization: https://github.com/dell/karavi-observability.
(you can read about the first CSM module, here Obsevarability’ one)
CSM for authorization gives full control to the storage admin to create secured & multi-tenant access control from different clusters.
Project CSM together with the CSI plugins and the pioneering app-aware, app-consistent backup and recovery solutions form the most comprehensive enterprise grade Storage and Data Protection solutions for Kubernetes from Dell Technologies.
In the future we will add support for additional Dell arrays like PowerMax, PowerStore, PowerScale and Unity.
As well as intelligent volume placement between different arrays
If you want to know more feel free to check the GitHub project https://github.com/dell/karavi/ or reach out directly from the Dell container community https://www.dell.com/community/Containers/bd-p/Containers