How to deploy files on Amazon's Simple Storage Service (S3) using S3 File System (s3fs)

Brief Description of our Project: the client wanted to push certain static HTML local files from the IMCE editor frontend directly into the Amazon S3 Bucket in order to Deploy Files to Production. Because these files are just static HTML, the Deploy Process needed only to simply push the Files on Amazon Bucket and keep track of the history of the changes (for roll back and logs purposes), something which was integrated into Github: to do git reset HEAD, git commit, git push. Therefore files were Uploaded simultaneously to Amazon and Github.

First you need to create an account with Amazon's Simple Storage Service (S3), then set up your Buckets one to keep files for Staging and the other Bucket to keep files for the Production. The files were first promoted to Staging and then Deployed to Production using Git pull.

Add, Install and Configure the S3 File System (s3fs) using composer:

composer require/s3fs

In the s3fs configuration you need to put the Amazon S3 credentials and we also had a custom Module to handle the settings related to which part of the file system needs to be integrated with S3.

Because this project was implemented for one of my client, I cannot share with you all the source code, but the main ideas behind the tool are:

• Achieve a file after it changes (roll back / logs purposes)
• Roll Back Changes
• Commit file to Git
• Push a bunch of files in Git
• Deploy the entire set of Changed to Amazon Staging Bucket

Here I share with you the code to push one File to Amazon:

  /**
   * Deploy file to Amazon S3.
   */
  public function deployS3File(File $file) {
    $uri = $file->getFileUri();
    $absolute_path = \Drupal::service('file_system')->realpath($uri);
    $file_key = file_uri_target($uri);

    try {
      $config = \Drupal::config('s3fs.settings')->get();

      /** @var \Drupal\s3fs\S3fsService $s3fs */
      $s3fs = \Drupal::service('s3fs');

      /** @var \Aws\S3\S3Client $s3 */
      $s3 = $s3fs->getAmazonS3Client($config);

      if (!empty($config['root_folder'])) {
        $file_key = "{$config['root_folder']}/$file_key";
      }
      $bucket = $config['bucket'];

      $s3->putObject(
        [
          'Bucket' => $bucket,
          'Key' => $file_key,
          'ACL' => 'public-read',
          'SourceFile' => $absolute_path,
        ]
      );
    }
    catch (S3fsException $e) {
      \Drupal::logger('imce_archive_plugin')
        ->error('S3fsException => ' . $e->getMessage());
    }
  }

It is straight forward to understand the code: First we need to build the Absolute File Path, then we need to pass also the Credential when we call the putObject() Method.

Class Aws\S3\S3Client is part of the AWS SDK for PHP, this Class has got lot of public Methods to interact with the Amazon Simple Storage Service (Amazon S3). I can list below some of the most popular Methods:

CompleteMultipartUpload ( array $params = [] )

Completes a multipart upload by assembling previously uploaded parts.

CopyObject ( array $params = [] )

Creates a copy of an object that is already stored in Amazon S3.

CreateBucket ( array $params = [] )

Creates a new bucket.

DeleteBucket ( array $params = [] )

Delete a bucket.

If you are interested about this Topic you can read the Entire APIs Documentation on Class Aws\S3\S3Client | AWS SDK for PHP 3.x