Requirements
------------

Any operation or management request must be authenticated with a signed request via Signature Version 2 or 4 of the Amazon S3 protocol of the corresponding S3 system user. You can create system users on any storage node in the cluster with the ``ostor-s3-admin create-user -S`` command and parameter ``-e`` specifying the user email address:

::

   # ostor-s3-admin create-user -S -e user@example.com
   UserEmail:user@example.com
   UserId:a14040e0b2ef8b28
   KeyPair[0]:S3AccessKeyId:a14040e0b2ef8b28FZZ8
   KeyPair[0]:S3SecretAccessKey:dbwTnQTW602aAAdq8DQVFzB6yrTCFTNiGB8C8RFA
   Flags:system

With this user you will authenticate further REST API requests managing the S3 cluster. You can create multiple system accounts for different management operations.

Configuration
~~~~~~~~~~~~~

In addition, you need to create Acronis Storage directories to modify the default functionality.

Change to the document root directory of your WHMCS server (e.g., ``/srv/http``) and create the following directories in it:

- ``whmcs/includes/AcronisStorage``,
- ``whmcs/admin/AcronisStorage``.

Change to the directory ``whmcs/includes/AcronisStorage``.

The first file you need to create includes the S3 configuration. Create a configuration file ``S3_getConfig.php`` with the following contentss, replacing the variable ``s3_key`` with your ``S3AcessKeyId``, ``s3_secret`` with your ``S3SecretAccessKey``, ``s3_gateway`` with your configured S3 gateway, and ``whmcs_username`` with your WHMCS admin username:

::

   <?php
   
   // Return array with default configuration.
   if (!function_exists('S3_getConfig')) {
       function S3_getConfig() {
   
           // s3 login.
           $vars['s3_key'] = "939e2ac6916b57082P9O";
           $vars['s3_secret'] = "tVYF3kZD9zcTtl6q6QDTHaZKM2nuq4xVcl8ikJpd";
   
           // s3 gateway.
           $vars['s3_gateway'] = "http://s3.example.com";
   
           // whmcs login.
           $vars['whmcs_username'] = "admin";
   
           // Return config array.
           return $vars;
       }
   }
   
   ?>

Includes
~~~~~~~~

Shared functions required by API operations are provided in a number of standalone PHP include files. The first file returns the client information (e.g., email address) which further S3 API user management requests need for various operations. Create a file ``S3_getClient.php`` with the following contentss:

::

   <?php
   
   // API request to get whmcs client information.
   if (!function_exists('S3_getClient')) {
       function S3_getClient($userid, $whmcs_username) {
   
           // Get client details for user email.
           $command = 'GetClientsDetails';
           $data = array(
               'clientid' => $userid,
           );
           $results = localAPI($command, $data, $whmcs_username);
   
           // Return client information.
   	return $results;
       }
   }
   
   ?>

The next file adds notes to the client in WHMCS with the S3 access key pairs whenever a new user or access key pair is created. Create a file ``S3_addClientNote.php`` with the following contentss:

::

   <?php
   
   // API request to add note to client in whmcs.
   if (!function_exists('S3_addClientNote')) {
       function S3_addClientNote(
           $userid,
           $whmcs_username,
           $s3_client_userid,
           $s3_client_key,
           $s3_client_secret
       ) {
   
           // Add note only for non-empty users.
           if (!empty($s3_client_userid)) {
   	
               // Add note with the s3 access key and s3 secret.
               $command = 'AddClientNote';
               $data = array(
                   'userid' => $userid,
                   'notes' =>
                       "UserId: " . $s3_client_userid . "\n" .
                       "AWSAccessKeyId: " . $s3_client_key . "\n" .
                       "AWSSecretAccessKey: " . $s3_client_secret,
               );
               localAPI($command, $data, $whmcs_username);
           }
       }
   }
   
   ?>

The next file removes notes from the client in WHMCS with the S3 access key pairs whenever a user or access key pair is removed. Create a file ``S3_delClientNote.php`` with the following contents:

::

   <?php
   
   // whmcs database access.
   use WHMCS\Database\Capsule;
   
   // API request to remove note from client in whmcs.
   if (!function_exists('S3_delClientNote')) {
       function S3_delClientNote(
           $userid,
           $whmcs_username,
           $s3_client_userid,
           $s3_client_key
       ) {
   
           // Delete notes in database.
           $db = Capsule::connection()->getPdo();
           $db->exec('
               DELETE FROM
                 tblnotes
               WHERE
                 userid = ' . $userid . '
               AND
                 note LIKE "%' . $s3_client_userid . '%"
               AND
                 note LIKE "%' . $s3_client_key . '%"'
           );
       }
   }
   
   ?>

The last file is the cURL library for sending ``GET``, ``PUT``, ``POST``, and ``DELETE`` requests. Create a file ``S3_requestCurl.php`` with the following contents:

::

   <?php
   
   // API request to s3 gateway.
   if (!function_exists('S3_requestCurl')) {
       function S3_requestCurl($s3_key, $s3_secret, $s3_gateway, $s3_query, $method) {
   
           // Prepare signature.
           $s3_host  = parse_url($s3_gateway, PHP_URL_HOST);
           $s3_date  = date(DATE_RFC2822);
   
           // Generate signature.
           $s3_signature = hash_hmac('sha1', $method . "\n\n\n" . $s3_date . "\n" .
               current(explode('&', $s3_query)), $s3_secret, true);
           $s3_signature = base64_encode($s3_signature);
   
           // Curl init.
           $s3_curl = curl_init($s3_gateway . $s3_query);
   
           // Curl options.
           switch ($method) {
               case "PUT":
                   curl_setopt($s3_curl, CURLOPT_PUT, 1);
                   break;
               case "POST":
                   curl_setopt($s3_curl, CURLOPT_POST, 1);
                   break;
               case "DELETE":
                   curl_setopt($s3_curl, CURLOPT_CUSTOMREQUEST, "DELETE");
                   break;
           }
           curl_setopt($s3_curl, CURLOPT_RETURNTRANSFER, true);
           curl_setopt($s3_curl, CURLOPT_URL, $s3_gateway . $s3_query);
           curl_setopt($s3_curl, CURLOPT_HTTPHEADER, array(
               'Host: ' . $s3_host,
               'Date: ' . $s3_date,
               'Authorization: AWS ' . $s3_key . ':' . $s3_signature,
               'Content-Type:',
               'Expect:',
           ));
   
           // Call.
           $response = curl_exec($s3_curl);
           $response = json_decode($response, true);
   
           // Curl deinit.
           curl_close($s3_curl);
   
           // Return response.
           return $response;
       }
   }
   
   ?>

Hooks
~~~~~

Hooks allow you to execute custom code when certain events occur in WHMCS. You will need to add S3-related action links to the admin page in WHMCS.

Change to the directory ``whmcs/includes/hooks`` and create a file ``S3_adminAreaClientSummaryActionLinks.php`` with the following contents:

::

   <?php
   
   // Modify other actions admin page.
   function S3_adminAreaClientSummaryActionLinks($vars) {
   
       // Create additional links.
       $result[] = '<b>S3 - User Management</b>';
       $result[] = '<a href="AcronisStorage/S3_createUser.php?userid=' .
           $vars['userid'] . '"><img src="http://logo.Acronis.com/ogimage.png"
               width="16" height="16" border="0" align="absmiddle" /> Create User</a>';
       $result[] = '<a href="AcronisStorage/S3_deleteUser.php?userid=' .
           $vars['userid'] . '"><img src="http://logo.Acronis.com/ogimage.png"
               width="16" height="16" border="0" align="absmiddle" /> Delete User</a>';
       $result[] = '<a href="AcronisStorage/S3_enableUser.php?userid=' .
           $vars['userid'] . '"><img src="http://logo.Acronis.com/ogimage.png"
               width="16" height="16" border="0" align="absmiddle" /> Enable User</a>';
       $result[] = '<a href="AcronisStorage/S3_disableUser.php?userid=' .
           $vars['userid'] . '"><img src="http://logo.Acronis.com/ogimage.png"
               width="16" height="16" border="0" align="absmiddle" /> Disable User</a>';
       $result[] = '<a href="AcronisStorage/S3_generateAccessKey.php?userid=' .
           $vars['userid'] . '"><img src="http://logo.Acronis.com/ogimage.png"
               width="16" height="16" border="0" align="absmiddle" /> Generate Access Key</a>';
       $result[] = '<a href="AcronisStorage/S3_revokeAccessKey.php?userid=' .
           $vars['userid'] . '"><img src="http://logo.Acronis.com/ogimage.png"
               width="16" height="16" border="0" align="absmiddle" /> Revoke Access Key</a>';
       $result[] = '<a href="AcronisStorage/S3_queryUser.php?userid=' .
           $vars['userid'] . '"><img src="http://logo.Acronis.com/ogimage.png"
               width="16" height="16" border="0" align="absmiddle" /> Query User (on/off)</a>';
       $result[] = '<a href="AcronisStorage/S3_listUsers.php">
           <img src="http://logo.Acronis.com/ogimage.png"
               width="16" height="16" border="0" align="absmiddle" /> List Users (on/off)</a>';
       $result[] = '&nbsp;';
       $result[] = '<b>S3 - User Limits Management</b>';
       $result[] = '
           <form>
               <input name="userid" type="hidden" value="' . $vars['userid'] . '">
               <input name="ops-value" size="4">
               <select name="ops-name">
                   <option>default</option>
                   <option>get</option>
                   <option>put</option>
                   <option>list</option>
                   <option>delete</option>
               </select> ops/s
               <br />
               <input name="bandwidth-value" size="4">
               <select name="bandwidth-name">
                   <option>out</option>
               </select> bandwidth/s
               <br />
               <button type="submit"
                   formaction="AcronisStorage/S3_setLimitsForUser.php">Set</button>
               <button type="submit"
                   formaction="AcronisStorage/S3_getLimitsForUser.php">Get</button>
               <button type="submit"
                   formaction="AcronisStorage/S3_deleteLimitsForUser.php">Delete</button>
           </form>
       ';
       $result[] = '&nbsp;';
       $result[] = '<b>S3 - Bucket Limits Management</b>';
       $result[] = '
           <form>
               <input name="userid" type="hidden" value="' . $vars['userid'] . '">
               <input name="ops-value" size="4">
               <select name="ops-name">
                   <option>default</option>
                   <option>get</option>
                   <option>put</option>
                   <option>list</option>
                   <option>delete</option>
               </select> ops/s
               <br />
               <input name="bandwidth-value" size="4">
               <select name="bandwidth-name">
                   <option>out</option>
               </select> bandwidth/s
               <br />
               <input name="bucket" size="4"> bucket name
               <br />
               <button type="submit"
                   formaction="AcronisStorage/S3_setLimitsForBucket.php">Set</button>
               <button type="submit"
                   formaction="AcronisStorage/S3_getLimitsForBucket.php">Get</button>
               <button type="submit"
                   formaction="AcronisStorage/S3_deleteLimitsForBucket.php">Delete</button>
           </form>
       ';
       $result[] = '&nbsp;';
       $result[] = '<b>S3 - Usage Statistics</b>';
       $result[] = '
           <a href="AcronisStorage/S3_listStatsObjects.php">
               <img src="http://logo.Acronis.com/ogimage.png"
                   width="16" height="16" border="0" align="absmiddle" />
                       List Statistics Objects (on/off)
           </a>
           <p>
               <form>
                   <input name="object" size="15"> object name
                   <br />
                   <button type="submit"
                       formaction="AcronisStorage/S3_getStatsForObject.php">Get</button>
                   <button type="submit"
                       formaction="AcronisStorage/S3_deleteStatsForObject.php">Delete</button>
               </form>
           </p>
       ';
       $result[] = '&nbsp;';
   
       // Return links.
       return $result;
   }
   
   // Modify admin area.
   add_hook('AdminAreaClientSummaryActionLinks', 1, "S3_adminAreaClientSummaryActionLinks");
   ?>

The last file extends the admin summary page and displays S3 user information as well as user and bucket limits if the corresponding links are clicked. Create a file ``S3_adminAreaClientSummaryPage.php`` with the following contents:

::

   <?php
   
   // Modify admin client summary to show S3 information.
   function S3_adminAreaClientSummaryPage($vars) {
   
       // Sane default.
       $result = '
       <div class="row client-summary-panels">
       ';
   
       // Show users.
       if ($_SESSION['s3_list_users'] == 1) {
   
           // Table header.
           $result = $result . '
           <div class="col-lg-6 col-sm-12">
               <div class="clientssummarybox">
                   <div class="title">
                       S3 Users List
                   </div>
                   <table class="clientssummarystats" cellspacing="0" cellpadding="2">
                       <tr>
                           <td><b>UserId</b></td>
                           <td><b>UserEmail</b></td>
                       </tr>
           ';
   
           // One row per access key pair.
           foreach ($_SESSION['s3_list'] as $s3_row) {
               $result = $result . '
                       <tr class="altrow">
                           <td>' . $s3_row['UserId'] . '</td>
                           <td>' . $s3_row['UserEmail'] . '</td>
                       </tr>
               ';
           }
   
           // Table footer.
           $result = $result . '
                   </table>
               </div>
           </div>
           ';
       }
   
       // Show user.
       if ($_SESSION['s3_query_user'] == 1) {
   
           // Table header.
           $result = $result . '
           <div class="col-lg-6 col-sm-12">
               <div class="clientssummarybox">
                   <div class="title">
                       S3 Information for User: ' . $_SESSION['s3_userid'] . '
                   </div>
                   <table class="clientssummarystats" cellspacing="0" cellpadding="2">
                       <tr>
                           <td><b>AWSAccessKeyId</b></td>
                           <td><b>AWSSecretAccessKey</b></td>
                       </tr>
           ';
   
           // One row per access key pair.
           foreach ($_SESSION['s3_aws_access_keys'] as $s3_row) {
               $result = $result . '
                       <tr class="altrow">
                           <td>' . $s3_row['AWSAccessKeyId'] . '</td>
                           <td>' . $s3_row['AWSSecretAccessKey'] . '</td>
                       </tr>
               ';
           }
   
           // Table footer.
           $result = $result . '
                   </table>
               </div>
           </div>
           ';
       }
   
       // Table footer and next header.
       $result = $result . '
       </div>
       <div class="row client-summary-panels">
       ';
   
       // Show statistics list.
       if ($_SESSION['s3_stat_objects'] == 1) {
   
           // Table header.
           $result = $result . '
           <div class="col-lg-6 col-sm-12">
               <div class="clientssummarybox">
                   <div class="title">
                       S3 Statistics List
                   </div>
                   <table class="clientssummarystats" cellspacing="0" cellpadding="2">
                       <tr>
                           <td><b>Object Name</b></td>
                       </tr>
           ';
   
           // One row per access key pair.
           foreach ($_SESSION['s3_stat']['items'] as $s3_object) {
               $result = $result . '
                       <tr class="altrow">
                           <td>' . $s3_object . '</td>
                       </tr>
               ';
           }
   
           // Table footer.
           $result = $result . '
                   </table>
               </div>
           </div>
           ';
       }
   
       // Show limits for user.
       if (!empty($_SESSION['s3_limits_user'])) {
   
           // Table header.
           $result = $result . '
           <div class="col-lg-3 col-sm-6">
               <div class="clientssummarybox">
                   <div class="title">
                       S3 Limits for User
                   </div>
                   <table class="clientssummarystats" cellspacing="0" cellpadding="2">
                       <tr>
                           <td><b>Type</b></td>
                           <td><b>Name</b></td>
                           <td><b>Value</b></td>
                       </tr>
           ';
   
           // One row per access key pair.
           foreach ($_SESSION['s3_limits_user'] as $s3_limits => $s3_value) {
               list($s3_type, $s3_limit) = explode(":", $s3_limits);
               $result = $result . '
                       <tr class="altrow">
                           <td>' . $s3_type . '</td>
                           <td>' . $s3_limit . '</td>
                           <td>' . $s3_value . '</td>
                       </tr>
               ';
           }
   
           // Table footer.
           $result = $result . '
                   </table>
               </div>
           </div>
           ';
       }
   
       // Show limits for bucket.
       if (!empty($_SESSION['s3_limits_bucket'])) {
   
           // Table header.
           $result = $result . '
           <div class="col-lg-3 col-sm-6">
               <div class="clientssummarybox">
                   <div class="title">
                       S3 Limits for Bucket: ' . $_SESSION['s3_bucket'] . '
                   </div>
                   <table class="clientssummarystats" cellspacing="0" cellpadding="2">
                       <tr>
                           <td><b>Type</b></td>
                           <td><b>Name</b></td>
                           <td><b>Value</b></td>
                       </tr>
           ';
   
           // One row per access key pair.
           foreach ($_SESSION['s3_limits_bucket'] as $s3_limits => $s3_value) {
               list($s3_type, $s3_limit) = explode(":", $s3_limits);
               $result = $result . '
                       <tr class="altrow">
                           <td>' . $s3_type . '</td>
                           <td>' . $s3_limit . '</td>
                           <td>' . $s3_value . '</td>
                       </tr>
               ';
           }
   
           // Table footer.
           $result = $result . '
                   </table>
               </div>
           </div>
           ';
       }
   
       // Table footer and next header.
       $result = $result . '
       </div>
       <div class="row client-summary-panels">
       ';
   
       // Show statistics for object.
       if (!empty($_SESSION['s3_object_statistic'])) {
   
           // Table header.
           $result = $result . '
           <div class="col-lg-12 col-sm-24">
               <div class="clientssummarybox">
                   <div class="title">
                       S3 Statistics for Object: ' . $_SESSION['s3_object'] . '
                   </div>
                   <table class="clientssummarystats" cellspacing="0" cellpadding="2">
                       <tr>
                           <td><b>fmt_version</b></td>
                           <td><b>service_id</b></td>
                           <td><b>start_ts</b></td>
                           <td><b>period</b></td>
                           <td><b>bucket</b></td>
                           <td><b>epoch</b></td>
                           <td><b>user_id</b></td>
                           <td><b>tag</b></td>
                           <td><b>put</b></td>
                           <td><b>get</b></td>
                           <td><b>list</b></td>
                           <td><b>other</b></td>
                           <td><b>uploaded</b></td>
                           <td><b>downloaded</b></td>
                       </tr>
           ';
   
           // One row per access key pair.
           foreach ($_SESSION['s3_object_statistic']['items'] as $s3_object) {
               $result = $result . '
                       <tr class="altrow">
                           <td>' . $_SESSION['s3_object_statistic']['fmt_version']. '</td>
                           <td>' . $_SESSION['s3_object_statistic']['service_id']. '</td>
                           <td>' . $_SESSION['s3_object_statistic']['start_ts']. '</td>
                           <td>' . $_SESSION['s3_object_statistic']['period']. '</td>
                           <td>' . $s3_object['key']['bucket'] . '</td>
                           <td>' . $s3_object['key']['epoch'] . '</td>
                           <td>' . $s3_object['key']['user'] . '</td>
                           <td>' . $s3_object['key']['tag'] . '</td>
                           <td>' . $s3_object['counters']['ops']['put'] . '</td>
                           <td>' . $s3_object['counters']['ops']['get'] . '</td>
                           <td>' . $s3_object['counters']['ops']['list'] . '</td>
                           <td>' . $s3_object['counters']['ops']['other'] . '</td>
                           <td>' . $s3_object['counters']['net_io']['uploaded'] . '</td>
                           <td>' . $s3_object['counters']['net_io']['downloaded'] . '</td>
                       </tr>
               ';
           }
   
           // Table footer.
           $result = $result . '
                   </table>
               </div>
           </div>
           ';
       }
   
       // Table footer.
       $result = $result . '
       </div>
       ';
   
       // Return table.
       return $result;
   }
   
   // Modify admin area.
   add_hook('AdminAreaClientSummaryPage', 1, "S3_adminAreaClientSummaryPage");
   ?>

Statistics
~~~~~~~~~~

You need to have statistics collection enabled on your S3 gateway. The S3 gateway will save the statistics as regular storage objects. On each S3 storage node, create a file ``/var/lib/ostor/local/gw.conf`` with the following contents:

::

   # Enable usage statistics collection.
   S3_GW_COLLECT_STAT=1

Restart the S3 storage service to apply the configuration changes. Run the following command on all S3 storage nodes:

::

   # systemctl restart ostor-agentd.service

Now you can login to WHMCS. Additional links and S3 management options will be shown in the **Client Profile** section.

.. image:: ../../../images/stor_saas_whmcs_integration2.png
   :align: center
   :class: align-center