Introducing DELL EMC Container Storage Module Authorization

A few months ago, we introduced ‘Project Karavi’. The first module we introduced, was the ‘Obsevarability’ one

With the new module we are releasing today, this project has matured to have a new, official name: Container Storage Module!

Dell EMC Container Storage Modules (CSM) aims at improving the observability, usability and data mobility for stateful applications with Dell Technologies Storage portfolio. CSM together with the CSI plugins and the pioneering app-aware, app-consistent backup and recovery solutions form the most comprehensive enterprise grade Storage and Data Protection solutions for Kubernetes from Dell Technologies.

CSM Authorization is part of the CSM open source suite of Kubernetes storage enablers for Dell EMC products. CSM Authorization provides storage and provides Kubernetes administrators the ability to apply RBAC for Dell EMC CSI Drivers. It does this by deploying a proxy between the CSI driver and the storage system to enforce role-based access and usage rules.

This module is currently supported with DELL EMC PowerFlex systems. Additional storage arrays will be added in the future.

Storage administrators of compatible storage platforms will be able to apply quota and RBAC rules that instantly and automatically restrict cluster tenant’s usage of storage resources. Users of storage through CSM Authorization do not need to have storage admin root credentials to access the storage system.

Kubernetes administrators will have an interface to create, delete, and manage roles/groups that storage rules may be applied. Administrators and/or users may then generate authentication tokens that may be used by tenants to use storage with proper access policies being automatically enforced.

The following diagram shows a high-level overview of CSM Authorization with a tenant-app that is using a Dell CSI driver to perform storage operations through the CSM Authorization proxy-server to access the Powerflex storage system. All requests from the CSI driver will contain the token for the given tenant that was granted by the Storage Administrator.

With the help of CSM authorization customers can:

Segregate the array usage between multiple tenants
Control storage consumption with the help of quota
Ensure that a tenant cannot access storage of other tenants
Create, update or revoke storage access at any point in time
Audit storage access by the tenants
Hide the array management credentials from the CSI driver and replace it with a JSON web token

Architecture

The key concept behind CSM Authorization is to minimize the storage administration exposure from the Kubernetes storage consumers.

We are segregating the responsibilities between the Kubernetes administrator and the Storage administrator by adding a component that acts as a proxy between the Kubernetes clusters and the management interfaces of backend arrays.

The storage admin will deploy CSM authorization and operate the component on several aspects like:

maintaining the list of tenants and JSON Web Token to authenticate the users used by the CSI driver for queries.
maintaining the list of arrays and credentials to access them,
and, last but not least it will maintain the relationships and quotas between those two.

The steps of a normal workflow are:

The storage admin creates the different elements (quota, storage, systems, tokens, etc.)
a cluster requests to create / delete a volume or a snap
the proxy intercepts the query and validates it (is it authorized, do we have the quota for it) ; if invalid we return a payload with the details of the reject error to the user
in case of success the proxy forwards the request to Powerflex
in case of error we forward the error to the querier; in case success we return the success and we update the quota accordingly

Installation

The CSM Authorization is published in several different packages, either as Kubernetes application that you can load on an existing cluster or as an all-in-one package to be installed on separate host.

The standalone package contains everything necessary to run CSM authorization:

k3s Kubernetes distribution to run the application
the application itself as a set of Pods, services, etc.
karavictl to manage the different aspects of the application
and even the patch for the CSI drivers to connect to the application

Once downloaded you can install it with:

rpm -ivh karavi-authorization-0.1-0.x86_64.rpm

Usage

Now that the installation is done you can check the application is running with:

karavictl cluster-info

Configure CSM auth

The first step is adding an array. The array is the entrypoint for other objects like access or tenants to be hooked to.

To add a storage system we run the following command:

karavictl storage create –endpoint https://********/ –insecure –system-id ********** –type powerflex –user admin

In case you put the wrong password karavi will prompt you to reenter a correct password.

We can check the configured systems with :

karavictl storage list

here we can see the details of the registered PowerFlex system.

The next step is to configure the storage access roles:

The first parameter is the role name, for example and R&D role for my development team, next, we specify the array type and our specific array ID, next we specify the storage pool and the allocated quota for this role

karavictl role create –role=RnD=powerflex=0ca110b75dd4300f=SP1=44000000

we can check the roles with:

karavictl role list

Next, we need to create a tenant object, this can be our Kubernetes or Openshift cluster

karavictl tenant create -n openshift -4.6

and now, we can Link our tenant to the role we’ve just created:

karavictl rolebinding create –tenant openshift-4.6–role RnD

Next we generate a token from our CSM machine and Creating the new secret with the token in the namespace of the CSI driver.

karavictl generate token –addr *********:443 -t openshift -4.6 | jq -r “.Token” | kubectl apply -n vxflexos -f –

we Check that the installer is ready:

curl http://CSM.local/install

and then go for it:

curl http://CSM.local/install | sh

After the installation, we have an additional container in the controller pods, this is the authorization proxy which intercepts the query and validates it against the CSM machine.

In the screenshot below, the Kubernetes user tries to create a PVC larger than its allocated quota, as a result, the volume hasn’t been created.

By navigating to the CSM authorization logs, we can see that this tenant reached hid limit, and as a result he cannot consume more storage from that PowerFlex storage system,

In this case, he needs to remove existing persistent volumes or contact the storage admin and for more storage capacity.

Below you can see a demo of how it all works together

In addition to this, a new pre-release version (v0.3.0) of CSM for Observability has also been released with support for the 1.4 release of the CSI driver for PowerFlex and CSM for Authorization: https://github.com/dell/karavi-observability.

(you can read about the first CSM module, here Obsevarability’ one)

CSM for authorization gives full control to the storage admin to create secured & multi-tenant access control from different clusters.

Project CSM together with the CSI plugins and the pioneering app-aware, app-consistent backup and recovery solutions form the most comprehensive enterprise grade Storage and Data Protection solutions for Kubernetes from Dell Technologies.

In the future we will add support for additional Dell arrays like PowerMax, PowerStore, PowerScale and Unity.

As well as intelligent volume placement between different arrays