5.2. Exporting Data via S3¶

5.2.1. S3 Storage Infrastructure Overview¶

The object storage infrastructure consists of the following entities: object servers (OS), name servers (NS), S3 gateways (GW), and the block-level backend.

These entities run as services on the Acronis Cyber Infrastructure nodes. Each service should be deployed on multiple Acronis Cyber Infrastructure nodes for high availability.

• An object server stores actual object data received from S3 gateway. The data is packed into special containers to achieve high performance. The containers are redundant, you can specify the redundancy mode while configuring object storage. An object server also stores its own data in block storage with built-in high availability.
• A name server stores object metadata received from S3 gateway. Metadata includes object name, size, ACL (access control list), location, owner, and such. Name server (NS) also stores its own data in block storage with built-in high availability.
• An S3 gateway is a data proxy between object storage services and end users. It receives and handles Amazon S3 protocol requests and S3 user authentication and ACL checks. The S3 gateway uses the NGINX web server for external connections and has no data of its own (i.e. is stateless).
• The block-level backend is block storage with high availability of services and data. Since all object storage services run on hosts, no virtual environments (and hence licenses) are required for object storage.

5.2.2. Planning the S3 Cluster¶

Before creating an S3 cluster, do the following:

1. Define which nodes of the storage cluster will run the S3 storage access point services. It is recommended to have all nodes available in Acronis Cyber Infrastructure run these services.

2. Configure the network so that the following is achieved:

   • All components of the S3 cluster communicate with each other via the S3 private network. All nodes of an S3 cluster must be connected to the S3 private network. Acronis Cyber Infrastructure internal network can be used for this purpose.
   • The nodes running S3 gateways must have access to the public network.
   • The public network for the S3 gateways must be balanced by an external DNS load balancer.

   For more details on network configuration, refer to the Installation Guide.

3. All components of the S3 cluster should run on multiple nodes for high-availability. Name server and object server components in the S3 cluster are automatically balanced and migrated between S3 nodes. S3 gateways are not automatically migrated; their high availability is based on DNS records. You need to maintain the DNS records manually when adding or removing S3 gateways.

5.2.3. Sample S3 Storage¶

This section shows a sample object storage deployed on top of a storage cluster of five nodes that run various services. The final setup is shown on the figure below.

5.2.4. Creating the S3 Cluster¶

To set up object storage services on a cluster node, do the following:

1. On the INFRASTRUCTURE > Networks screen, make sure that the OSTOR private and S3 public traffic types are added to your networks.

2. In the left menu, click STORAGE SERVICES > S3.

3. Select one or more nodes and click Create S3 cluster in the right menu. To create a highly available S3 cluster, select at least three nodes. It is also recommended to enable HA for the management node prior to creating the S3 cluster. See Enabling High Availability for more details.

4. Make sure the correct network interface is selected in the drop-down list.

   If necessary, click the cogwheel icon and configure node’s network interfaces on the Network Configuration screen.

   

   Click Proceed.

5. In Tier, select the storage tier that will be used for the object storage.

6. In Failure domain, choose a placement policy for replicas.

7. In Data redundancy, select the redundancy mode that the object storage will use.

   

   You can change the redundancy mode later on the S3 > OVERVIEW > Settings panel. Click Proceed.

8. Specify the external (publicly resolvable) DNS name for the S3 endpoint that will be used by the end users to access the object storage. For example, s3.example.com. Click Proceed.

   Important

   Configure your DNS server according to the example suggested in the admin panel.

9. From the drop-down list, select an S3 endpoint protocol: HTTP, HTTPS or both.

   

   It is recommended to use only HTTPS for production deployments.

   If you have selected HTTPS, do one of the following:

   • Check Generate self-signed certificate to get a self-signed certificate for HTTPS evaluation purposes.

     Take note of the following:

     • S3 geo-replication requires a certificate from a trusted authority. It does not work with self-signed certificates.
     • To access the data in the S3 cluster via a browser, add the self-signed certificate to browser’s exceptions.

   • Acquire a key and a trusted wildcard SSL certificate for endpoint’s bottom-level domain. For example, the endpoint s3.storage.example.com would need a wildcard certificate for *.s3.storage.example.com with the subject alternative name s3.storage.example.com.

     If you acquired an SSL certificate from an intermediate certificate authority (CA), you should have an end-user certificate along with a CA bundle that contains the root and intermediate certificates. To be able to use these certificates, you need to merge them into a chain first. A certificate chain includes the end-user certificate, the certificates of intermediate CAs, and the certificate of a trusted root CA. In this case, an SSL certificate can only be trusted if every certificate in the chain is properly issued and valid.

     For example, if you have an end-user certificate, two intermediate CA certificates, and a root CA certificate, create a new certificate file and add all certificates to it in the following order:

     # End-user certificate issued by the intermediate CA 1
     -----BEGIN CERTIFICATE-----
     MIICiDCCAg2gAwIBAgIQNfwmXNmET8k9Jj1X<...>
     -----END CERTIFICATE-----
     # Intermediate CA 1 certificate issued by the intermediate CA 2
     -----BEGIN CERTIFICATE-----
     MIIEIDCCAwigAwIBAgIQNE7VVyDV7exJ9ON9<...>
     -----END CERTIFICATE-----
     # Intermediate CA 2 certificate issued by the root CA
     -----BEGIN CERTIFICATE-----
     MIIC8jCCAdqgAwIBAgICZngwDQYJKoZIhvcN<...>
     -----END CERTIFICATE-----
     # Root CA certificate
     -----BEGIN CERTIFICATE-----
     MIIDODCCAiCgAwIBAgIGIAYFFnACMA0GCSqG<...>
     -----END CERTIFICATE-----
     

     Upload the prepared certificate, and, depending on its type, do one of the following:

     • specify the passphrase (PKCS#12 files);
     • upload the SSL key.

     You can change the redundancy mode later on the S3 > OVERVIEW > Protocol settings panel. Click Proceed.

10. If required, click Configure Acronis Notary and specify Notary DNS name and Notary user key.

11. Click Done to create an S3 cluster.

After the S3 cluster is created, open the S3 Overview screen to view cluster status, hostname, used disk capacity, the number of users, I/O activity, and the state of S3 services.

To check if the S3 cluster is successfully deployed and can be accessed by users, visit https://<S3_DNS_name> or http://<S3_DNS_name> in your browser. You should receive the following XML response:

<Error>
<Code>AccessDenied</Code>
<Message/>
</Error>

To start using the S3 storage, you will also need to create at least one S3 user.

5.2.5. Managing S3 Users¶

The concept of S3 user is one of the base concepts of object storage along with those of object and bucket (container for storing objects). The Amazon S3 protocol uses a permission model based on access control lists (ACLs) where each bucket and each object is assigned an ACL that lists all users with access to the given resource and the type of this access (read, write, read ACL, write ACL). The list of users includes the entity owner assigned to every object and bucket at creation. The entity owner has extra rights compared to other users. For example, the bucket owner is the only one who can delete that bucket.

User model and access policies implemented in Acronis Cyber Infrastructure comply with the Amazon S3 user model and access policies.

User management scenarios in Acronis Cyber Infrastructure are largely based on the Amazon Web Services user management and include the following operations: create, query, and delete users as well as generate and revoke user access key pairs.

5.2.6. Managing S3 Buckets¶

All objects in Amazon S3-like storage are stored in containers called “buckets”. Buckets are addressed by names that are unique in the given object storage, so an S3 user of that object storage cannot create a bucket that has the same name as a different bucket in the same object storage. Buckets are used to:

• group and isolate objects from those in other buckets,
• provide ACL management mechanisms for objects in them,
• set per-bucket access policies, for example, versioning in the bucket.

In the current version of Acronis Cyber Infrastructure, you can enable and disable Acronis Notary for object storage buckets and monitor the space used by them on the STORAGE SERVICES > S3 > Buckets screen. You cannot create and manage object storage buckets from Acronis Cyber Infrastructure admin panel. However, you can do it via the Acronis Cyber Infrastructure user panel or by using a third-party application. For example, the applications listed below allow you to perform the following actions:

• CyberDuck: create and manage buckets and their contents.
• MountainDuck: mount object storage as a disk drive and manage buckets and their contents.
• Backup Exec: store backups in the object storage.

5.2.7. Best Practices for Using S3 in Acronis Cyber Infrastructure¶

This section offers recommendations on how to best use the S3 feature of Acronis Cyber Infrastructure.

5.2.8. Replicating S3 Data Between Datacenters¶

Acronis Cyber Infrastructure can store replicas of S3 cluster data and keep them up-to-date in multiple geographically distributed datacenters with S3 clusters based on Acronis Cyber Infrastructure. Geo-replication reduces the response time for local S3 users accessing the data in a remote S3 cluster or remote S3 users accessing the data in a local S3 cluster as they do not need to have an Internet connection.

Geo-replication schedules the update of the replicas as soon as any data is modified. Geo-replication performance depends on the speed of Internet connection, the redundancy mode, and cluster performance.

If you have multiple datacenters with enough free space, it is recommended to set up geo-replication between S3 clusters residing in these datacenters.

To set up geo-replication between S3 clusters, exchange tokens between datacenters as follows:

1. In the admin panel of a remote datacenter, open the STORAGE SERVICES > S3 > GEO-REPLICATION screen.

   

2. In the section of the home S3 cluster, click TOKEN and, on the Get token panel, copy the token.

3. In the admin panel of the local datacenter, open the STORAGE SERVICES > S3 > GEO-REPLICATION screen and click ADD DATACENTER.

   

4. Enter the copied token and click Done.

5. Configure the remote S3 cluster the same way.

5.2.9. Monitoring S3 Access Points¶

The S3 monitoring screen enables you to inspect the availability of each S3 component as well as the performance of NS and OS services (which are highly available).

If you see that some of the NS or OS services are offline, it means that the S3 access point does not function properly, and you should contact support consult the CLI guide for low-level troubleshooting. S3 gateways are not highly available, but DNS load balancing should be enough to avoid downtime if the gateway fails.

The performance charts represent the number of operations that the OS/NS services are performing.

5.2.10. Releasing Nodes from S3 Clusters¶

Before releasing a node, make sure that the cluster has enough nodes running name and object servers as well as gateways left.

To release a node from an S3 cluster, do the following:

1. On the STORAGE SERVICES > S3 > Nodes screen, check the box of the node to release.
2. Click Release.

5.2.11. Supported Amazon S3 Features¶

This section lists Amazon S3 operations, headers, and authentication schemes supported by the Acronis Cyber Infrastructure implementation of the Amazon S3 protocol.