How to Install s3cmd in Linux and Manage Amazon s3 Buckets

S3 , – Simple Storage Service- is Amazon’s storage service that provides IT teams with a secure, scalable, and reliable way to store and retrieve files and folders on the cloud. S3 helps you make the most out of your data by making sure that it is available when needed and scalable as demand grows.

Conventionally, S3 is accessed from a web browser after signing in to your AWS account. This can also be achieved on command-line using a nifty tool called s3cmd. The s3cmd utility is an opensource command-line tool that allows you to access and manage your S3 service. It allows you to create/delete buckets, add/remote files and folders and list the contents of your S3 buckets.

This guide will seek to help you install s3cmd on Linux and help you make the most of it to manage your S3 account.

How to install s3cmd on Linux

Installing s3cmd on major Linux distributions such as Ubuntu, CentOS & RHEL is quite a walk in the park. This is because the s3cmd package is available on their repositories.

Let’s check out how you can install s3cmd on various distributions:

On Ubuntu / Debian flavors

For Debian / Ubuntu distributions use the APT package manage as follows:

$ sudo apt install s3cmd

On CentOS 8 / RHEL 8

On CentOS / RHEL distros install s3cmd using the command below:

$ sudo dnf install s3cmd

On SUSE Linux

For SUSE Enterprise server , start by adding the repository and using the zypper command-line tool as shown.

$ sudo zypper addrepo http://s3tools.org/repo/SLE_11/s3tools.repo

$ zypper install s3cmd

Installation from source

If your Linux distribution is not listed above, don’t fret. You can install from source which will work in all distributions.

First, download the zip file of the latest s3cmd version, which by this time is s3cmd 2.1.0.

$ wget https://sourceforge.net/projects/s3tools/files/s3cmd/2.1.0/s3cmd-2.1.0.zip

In your home directory, you should see the s3cmd-2.1.0.zip zip file.

Next, unzip the file.

$ unzip s3cmd-2.1.0

Navigate to the uncompressed folder

$ cd s3cmd-2.1.0

And execute the command shown to install from source.

$ sudo python3 setup.py install

Setting up the s3cmd environment

Before we begin interacting with Amazon S3, we need to set up the s3cmd tool by providing the necessary credentials such as Access Key & Secret access to our account. So, run the command below to configure the s3cmd environment.

$ s3cmd --configure

Provide the access key, secret access , region and encryption password as shown in the screenshot below.

Be sure to use the HTTPS protocol for enhanced security.

List s3 buckets

We have successfully configured the s3cmd tool, and we can now begin interacting with Amazon S3. Before you start interacting with S3, ensure that the user on your AWS has been configured with Amazon S3 IAM role.

To list the existing S3 buckets on your account, execute the command:

$ s3cmd ls

So far, we have none since we are just getting started out.

Let’s create one 🙂

Create an S3 bucket

To create a new S3 bucket, run the command shown. The command below creates a bucket in S3 called linuxways.

$ s3cmd mb s3://linuxways

You can head over to your AWS account and confirm that the bucket has been created as shown.

Upload a file to S3 bucket

To upload a regular file to your bucket, invoke the put option followed by the file and the path to S3 bucket. The command below uploads the file hello.sh to S3.

$ s3cmd put hello.sh s3://linuxways/

Upload a directory to S3 bucket

To upload a directory, use the -r flag for adding the directory recursively alongside its contents. Take care not to add the leading / as this will only add the directory’s contents and not the directory itself.

Here, we are uploading the Pictures directory that contains a JPG image file.

$ s3cmd put -r Pictures s3://linuxways/

List contents of S3 bucket

To view or display what your S3 bucket contains, run the command.

$ s3cmd ls s3://linuxways/

Delete a file or directory from an S3 bucket

If you wish to remove or delete a file you no longer need, use the del option as shown.

$ s3cmd del s3://linuxways/hello.sh

To delete a directory, use the same syntax as follows. However, this only works if the directory is empty.

$ s3cmd del s3://linuxways/Pictures

If a directory is not empty, use the -r option to recursively delete all its contents as shown.

$ s3cmd del -r s3://linuxways/Pictures/

Remove a bucket from S3

If you no longer require your bucket, you can delete it as follows.

$ s3cmd rb s3://linuxways

If the bucket still contains some files and folders, you will run into the error below informing you that the bucket is not empty.

The solution is to go back and delete all the files and directories in your bucket and try again.

Wrapping up

That was a summary of the s3cmd command-line tool and how you can use it to manage your Amazon S3 storage feature.