OSC S3 Bucket Extension
Introduction
This document describes the basic usage of the OSC S3 Bucket Extension.
This extension can be enabled and disabled directly in the OSC Dashboard, without editing the Shoot manifest. See Managing Cluster Services and the Simple Cluster Services page.
Enabling the Extension for a Shoot Cluster
For enabling this extension for a Shoot cluster,
the extension service named osc-s3-bucket-service needs to be added
to the extensions in the Shoot Custom Resource manifest:
You can issue this command to check the extensions on your Shoot cluster:
Disabling globally enabled extensions
To disable extensions which are enabled by default, add the following snippet to the shoot manifest:
Create an S3 Bucket
The following example S3Bucket manifest can be used
to request a new S3 bucket in a Shoot cluster :
In this example the default namespace is used
and therefore the namespace field is omitted in the further steps.
After the S3Bucket CR was created,
its status can be checked in the status of the CR.
If the bucket was created successfully and can be consumed,
the status of the S3Bucket CR will be set to Ready (.status.Ready):
The access details for the bucket can be fetched from the secret referenced
in the status of the S3Bucket CR (.status.credentialsSecret.name):
The secret contains the following information:
s3_access_key_id-
The access key for the s3 bucket ("user identity")
s3_bucket-
The bucket name (to avoid the upper limit of 64 characters for bucket names, the name is a sha224 hash)
s3_ca_bundle-
The CA bundle used by the MinIO API (used to sign the server certificates of the MinIO API and needed for validating the provided certificates)
s3_endpoint-
URL of the MinIO API
s3_secret_access_key-
The secret access key for the s3 bucket ("user password")
The secret can be directly consumed by workloads running in the same Kubernetes namespace. by mounting the secret into the workload container or using the content as environment variables (see the Kubernetes documentation for more details).
If the workload is running in another namespace than the secret, the workload needs to fetch the secret via the Kubernetes API or the credentials must be provided manually.
Access S3 Bucket
To access the S3 bucket access credentials are needed. To get these we have to check our created S3 bucket:
The access credentials can be fetched from the referenced Kubernetes secret as follows:
Example output:
The bucket name is a sha224 hash representation of the ACCESS_KEY_ID
to avoid the upper character limit for bucket names.
To list the buckets the MinIO CLI can be used (see the installation instructions). The installation procedure should create a configuration folder for the MinIO CLI.
For the S3 bucket, a configuration file can be created as follows:
The OSC S3 service uses a (server) certificate signed
by an internal OSC certificate authority.
Therefore the client which connects to the S3 bucket
needs to know this certificate authority.
The needed CA bundle is included in the Kubernetes secret
which contains the access credentials (the s3_ca_bundle key).
For the MinIO CLI the OSC CA bundle can be added as follows:
We can list now the bucket with the command
Example:
User Management
Create S3 User
S3 Users can be created by applying the following S3User
Custom Resource manifest:
After the S3User Custom Resource was created,
its status can be checked in the status of the CR.
Out of the box, no user credentials will be available for this user.
To create access credentials,
the S3User must first be authorized for a specific S3Bucket
with one of following permissions types:
readwritereadonlywriteonly
Authorize an S3User for an S3 Bucket
To authorize an S3User for a specific S3Bucket,
the S3Bucket CR must be updated to include the user in the users list:
The existence of the S3User CR is checked when
creating or updating the S3Bucket CR.
If the S3User CR does not exist, the request will fail.
A credential Secret for the authorized S3User will be created
in the same namespace as the S3User CR, not in the namespace of S3Bucket.
Therefore, it is possible to create an S3User in a different namespace
than the S3Bucket.
Information about the current S3Bucket authorization status
and the credential Secret name will be propagated
to the corresponding S3User BucketReference field:
The credential secret name is generated from the S3User CR name
by adding a random suffix.
In our example, the credential secret name is example-6n8dm.
The user credential Secret follows the same data fields naming style as the default user credential secret.
S3User deletion
Before deleting an S3User CR,
make sure it is not authorized in any S3Buckets.
If it is still authorized in some S3Bucket,
the deletion request will be rejected.
S3Bucket Storage Utilization
The .Status.Capacity.Used field indicates
the amount of storage consumed by respective S3Bucket CR.
This information provides visibility into the storage utilization
of the real S3 bucket storage associated with the S3Bucket CR.
Note that the size utilization in the .status.capacity.used field
is represented in Gibibytes (Gi) scale.
If the resulting size of all objects is less than 1 Gi,
it will be rounded up to 1 Gi.
For newly created S3Bucket CRs,
the initial capacity of 1Gi will be added immediately.
If the combined size of all objects exceeds 1 Gi but includes a fractional part,
it will be rounded up to the nearest integer.
Object Expiration
Currently, object expiration rules cannot be specified
via the S3Bucket CR (this is currently a feature on our roadmap).
Currently, object expiration rules must be set manually. For doing so, the MinIO CLI can be used, see the upstream documentation for Object Expiration.
For our example bucket one can define the corresponding lifecycle rule as follows:
For the example above:
For listing the current lifecycle rules:
For the example above:
Bucket Quota
Currently, bucket quotas cannot be specified
via the S3Bucket CR (this is currently a feature on our roadmap).
Currently, there is no way for users to define quotas for buckets.
Delete S3 Bucket
To delete s3 buckets simply issue:
Currently there is no support for updating S3 Bucket objects.
Be aware that deletion of s3Bucket CR will cause deletion of s3 bucket storage in MinIO and all data will lost.
Shoot cluster - S3 Bucket storage coupling
Note that based on the architecture design
of the osc-gardener-extension-shoot-s3-bucket,
each S3 bucket storage is uniquely coupled with the Shoot cluster
in which the respective S3Bucket custom resource (CR) was created.
Therefore, deleting the Shoot cluster (the Shoot CR)
will cause the deletion of all S3 buckets defined in it.
Therefore, all data in these buckets will be lost!
In case the Shoot cluster, in which an S3 bucket is defined, is not functional, but is not deleted, the S3 bucket is not affected or deleted. The same is true for hibernated Shoot cluster.
To keep the data stored in an S3 bucket of a Shoot cluster,
which is going to be deleted,
the data must be migrated manually to another S3 bucket.
To do this, another S3 bucket can be created
by creating an S3Bucket CR in another Shoot.
To migrate data between two buckets,
it is possible to use the MinIO mc CLI, s3cmd,
or any other tool supporting this functionality.
For instance, using the mc this could be done by
mc cp or mc mirror commands,
depending on your purpose.
The access credentials for both buckets are needed for the data migration. In case the access credentials for the first bucket are lost, the operations team needs to be contacted.