Dell ObjectScale : Configuring Replication

A post by Anuraj PD

Dell ObjectScale is the next evolution in enterprise object storage, designed for Kubernetes.
Capable of scaling to any capacity and connecting sites from edge to core with a few simple clicks, ObjectScale serves as your globally accessible data lake for enterprise workloads such as cloud native, IoT, AI, big data analytics and archive applications. Dell EMC ObjectScale allows any organization to deliver scalable cloud services with the reliability and control of a private cloud infrastructure. ObjectScale enables you to easily manage globally distributed storage infrastructure under a single namespace with anywhere access to content. Dell offers ObjectScale Community Edition which gives up to 30TB of storage and no time limit for non-production use. You can read more about the Dell ObjectScale here.

ObjectScale is built with design principles, such as:

Global namespace with eventual consistency
Scale-out capabilities
Secure multi-tenancy
Superior performance for both small and large objects

In this demo we will configure ObjectScale replication and already deployed two ObjectScale System each running on separate Red Hat OpenShift clusters. Please refer our previous blog for the installation of ObjectScale on Red Hat OpenShift Cluster.

ObjectScale Replication allows you to manage and monitor replication policies and replicate bucket data. Replication between Object Stores is in compliance with S3 protocol of AWS. Replication is configured bucket to bucket by defining the replication rules on the source bucket.
Each source bucket can be configured to replicate some or all its data to one or more destination buckets. ObjectScale support unidirectional or bidirectional replication and in this demo, we will configure bidirectional replication of entire bucket. These features allow the customer to configure wide varieties of replication topologies to support different use cases.

The primary ObjectScale System is deployed in the namespace ocs01 on the OpenShift cluster named ocp01. Objectstore ocs01-objectstore01 is also created.

The ObjectScale management console is accessible using the URL https://objectscale01.vdi.xtremio and s3 endpoint is accessible using the URL https://ocs01s3.vdi.xtremio.

The secondary ObjectScale System is deployed in the namespace ocs02 on the OpenShift cluster named ocp02. Objectstore ocs02-objectstore01 is also created.

The ObjectScale management console is accessible using the URL https://objectscale02.vdi.xtremio and s3 endpoint is accessible using the URL https://ocs02s3.vdi.xtremio.

ObjectScale Federation service is for creating trust between ObjectScale instances. When you create a federation of ObjectScale systems it allows IAM entities to replicate from one ObjectScale system to other ObjectScale systems in the federation. Within a federation, an ObjectScale instance can either be the primary instance or a secondary instance. There can be two or more secondary instances within a federation, but there can only be a single primary instance. We will configure objectscale01 as the primary and objectscale02 as the secondary.

Configure the objectscale01 as the primary ObjectScale by going ObjectScale Systems -> Create Federation.

Before creating the federation, the type of the ObjectScale system will be shown as “Not Federated” and will change to “Primary” once the federation is created.

Join the secondary ObjectScale system objectscale02 to the primary ObjectScale system ObjectScale Systems -> Join Federation, save the federation signing xml file.

Upload the federation signing request file to the primary ObjectScale system by going to ObjectScale Systems -> New Remote Instance.

The primary ObjectScale generates a signed certificate that is named PostFederationSigning_<objectscale_system>.xml and automatically download the file.

From the secondary ObjectScale system, add a new remote instance by uploading the signed certificate that was downloaded from the primary ObjectScale system. The ObjectScale type will change from “Not federated” to “Secondary”.

From each ObjectScale system we must initiate trust of the remote instances. ObjectScale Systems -> Initiate Trust.

Once the Remote Instance on both sides have Trust Initiated Trust Status, the federation service on each side tries to communicate to the remote instance over the objectscale-gateway-internal endpoint. If this system-to-system communication is successful, it moves each remote instance to the Trusted Trust Status.

Initiate Trust from secondary ObjectScale system.

Once the trust is established between two different ObjectScale systems, the IAM entities like the account, user, role etc will be replicated between ObjectScale systems. Now we can create the IAM account in the primary ObjectScale system and it will be replicated to secondary ObjectScale system. Create IAM account – iamaccount01.

Add the account to ObjectStore in the primary ObjectScale system. Object Stores -> <Object Store Name> -> Accounts -> Add.

Account created in the primary ObjectScale system is successfully replicated and available in the secondary ObjectScale system.

Add the account to ObjectStore in the secondary ObjectScale system. Object Stores -> <Object Store Name> -> Accounts -> Add.

Create Role in the primary ObjectScale system and will be automatically replicated to secondary ObjectScale system.

Accounts -> <Account Name> -> Roles -> New Role.

Role is replicated to the secondary ObjectScale system.

Create user in the primary ObjectScale system and will be automatically replicated to secondary ObjectScale system.

Accounts -> <Account Name> -> Users -> New User

User is replicated to the secondary ObjectScale system.

Create S3 bucket in the primary ObjectScale system.

Object Stores -> <Object Store Name> -> Buckets -> New Bucket.

Enable the versioning on the bucket.

Create S3 bucket in the secondary ObjectScale system. Object Stores -> <Object Store Name> -> Buckets -> New Bucket.

Configure the replication rule on bucket in the primary ObjectScale system to replicate the objects to the bucket in the secondary ObjectScale system.

Object Stores -> <Object Store Name> -> Buckets -> <Bucket Name> ->Replication -> New Replication Rule.

In this demo we will configure bidirectional replication so we will configure the replication rule on the bucket in the secondary ObjectScale system to replicate the objects to the bucket in the primary ObjectScale system.

Configure both the ObjectScale system in the S3 browser to upload data to the buckets. Upload some sample data to the bucket and we can verify the objects are replicated to the destination buckets.