Cantech Cloud

How to Configure CMP S3 Bucket Storage on a Linux VM Using s3cmd and s3fs

Written by

Pavitra Italiya

in

Purpose

This KB explains how to configure CMP S3 bucket storage on a Linux VM using:

  • s3cmd for command-line S3 operations
  • s3fs for mounting the S3 bucket as a filesystem

Using this setup, you can upload, download, mount, and manage files in a CMP S3 bucket from a Linux server.

Prerequisites

Before starting, make sure you have the following:

  • A Linux VM, for example Ubuntu
  • CMP S3 Access Key
  • CMP S3 Secret Key
  • Target bucket name: linux-backup-test
  • Basic knowledge of Linux command-line operations
  • Internet connectivity from the Linux VM to the CMP S3 endpoint

Understanding CMP S3 Access Control

CMP S3 supports two access modes:

Bucket visibility

  • Public bucket
  • Private bucket

Access using keys

  • Access Key
  • Secret Key

The Access Key and Secret Key generated from CMP S3 provide access to CMP S3 resources.

Note: Keys created in CMP S3 work only within CMP S3.

How to Create Access Key and Secret Key in CMP

To access CMP S3 buckets securely, you need to generate an Access Key and Secret Key.

Steps:

  • Log in to the CMP Dashboard.
  • Navigate to Object Storage.
  • Click on the Access tab.
  • You will see fields like:

Region

User ID

Accounts

Status

  • Click on the required access section.
  • Click on Create First Key or Create Key if one already exists.
  • Copy the generated:

Access Key

Secret Key

Important: Save the Secret Key securely, as it is shown only once.

Installing s3cmd on Linux VM

Run the following commands:

Bash
sudo apt update
sudo apt install s3cmd -y

Verify the installation:

Bash
s3cmd –version

Configuring s3cmd for CMP S3

Run the configuration wizard:

Bash
s3cmd –configure

Enter the following details when prompted:

Prompt : Value
Access Key : <Your-Access-Key>
Secret Key : <Your-Secret-Key>
Default Region : in-west2
S3 Endpoint : s3.in-west2.purestore.io
DNS-style bucket+hostname:port template : %(bucket)s.s3.in-west2.purestore.io
Encryption password : Optional or leave blank
Path to GPG program : /usr/bin/gpg
Use HTTPS protocol : True
HTTP Proxy server name : Leave blank unless required
HTTP Proxy server port : 0

Bash
Note: Do not add https:// in the S3 Endpoint field during s3cmd –configure.

Validate s3cmd Configuration

Run the following command to check bucket access:

Bash
s3cmd ls

Expected output should show the available bucket, for example:

Bash
s3://linux-backup-test

Common Operations Using s3cmd

  • List Buckets
Bash
s3cmd ls
  • List Objects in Bucket
Bash
s3cmd ls s3://linux-backup-test
  • Create a Test File

Before uploading, create a test file on the server:

Bash
echo “This is a test file for CMP S3 upload” > today1234.txt

Verify the file:

Bash
ls -lh today1234.txt
cat today1234.txt
  • Upload a File

Upload the test file to the CMP S3 bucket:

Bash
s3cmd put today1234.txt s3://linux-backup-test

Verify the uploaded file:

Bash
s3cmd ls s3://linux-backup-test
  • Download a File

Download the file from the bucket:

Bash
s3cmd get s3://linux-backup-test/today1234.txt

To download the file to a specific directory:

Bash
mkdir -p /mnt/cmp-s3
s3cmd get s3://linux-backup-test/today1234.txt /mnt/cmp-s3/today1234.txt

Verify the downloaded file:

Bash
ls -lh /mnt/cmp-s3/today1234.txt
cat /mnt/cmp-s3/today1234.txt

Mount CMP S3 as a Filesystem Using s3fs

s3fs allows you to mount the CMP S3 bucket as a local directory on the Linux server.

  • Install s3fs
Bash
sudo apt update
sudo apt install s3fs -y
  • Create Credentials File

Create the password file using the Access Key and Secret Key:

Bash
echo ACCESS_KEY:SECRET_KEY > ~/.passwd-s3fs

Example:

Bash
echo ABCD123456789:xyzsecretkey123456789 > ~/.passwd-s3fs

Set secure permission:

Bash
chmod 600 ~/.passwd-s3fs
Bash
Important: Replace ACCESS_KEY and SECRET_KEY with the actual CMP S3 credentials.
  • Create Mount Directory

Create the directory where the bucket will be mounted:

Bash
mkdir -p /mnt/cmp-s3

Verify the directory:

Bash
ls -ld /mnt/cmp-s3
  • Mount the CMP S3 Bucket

Mount the bucket linux-backup-test on /mnt/cmp-s3:

Bash
s3fs linux-backup-test /mnt/cmp-s3 -o url=https://s3.in-west2.purestore.io -o use_path_request_style
  • Verify the Mount

Check whether the bucket is mounted successfully:

Bash
mount | grep s3fs

Expected output:

Bash
s3fs on /mnt/cmp-s3 type fuse.s3fs

You can also check using:

Bash
df -h | grep cmp-s3

List files from the mounted bucket:

Bash
ls -la /mnt/cmp-s3

File Operations Using Mounted S3 Directory

Once the bucket is mounted, you can manage files like a normal Linux directory.

  • Create a Local Test File

Create a local test file:

Bash
echo “This is a test upload using s3fs mount” > /root/localfile.txt

Verify it:

Bash
ls -lh /root/localfile.txt
cat /root/localfile.txt
  • Upload File to Bucket Using cp Command

Copy the local file to the mounted S3 bucket:

Bash
cp /root/localfile.txt /mnt/cmp-s3/

Verify:

Bash
ls -lh /mnt/cmp-s3/

You can also verify using s3cmd:

Bash
s3cmd ls s3://linux-backup-test
  • Create a New File Directly in Mounted Bucket

Create a file directly inside the mounted S3 bucket:

Bash
echo “Hello CMP S3” > /mnt/cmp-s3/hello.txt

Verify:

Bash
ls -lh /mnt/cmp-s3/hello.txt
cat /mnt/cmp-s3/hello.txt
  • Download/Copy File from Mounted Bucket to Local Server

Create a local download directory:

Bash
mkdir -p ~/Downloads

Copy the file from mounted bucket to local directory:

Bash
cp /mnt/cmp-s3/hello.txt ~/Downloads/

Verify:

Bash
ls -lh ~/Downloads/hello.txt
cat ~/Downloads/hello.txt
  • Delete a File from Bucket

Delete the file from the mounted S3 bucket:

Bash
rm /mnt/cmp-s3/hello.txt

Verify:

Bash
rm /mnt/cmp-s3/hello.txt

Or verify using:

Bash
s3cmd ls s3://linux-backup-test

Unmount the Bucket

When the work is completed, unmount the bucket:

Bash
fusermount -u /mnt/cmp-s3

If fusermount is not available, use:

Bash
umount /mnt/cmp-s3

Verify that it is unmounted:

Bash
mount | grep s3fs

If no output is shown, the bucket has been unmounted successfully.

Important Notes

  • Make sure the mount directory exists before running the s3fs mount command.
  • Keep the .passwd-s3fs file secure with 600 permission.
  • Do not share the Secret Key with anyone.
  • Use HTTPS endpoint for secure connectivity.
  • For persistent mount after reboot, an entry can be added in /etc/fstab with proper credential handling.
  • If s3cmd ls gives a connection refused error, check that use_https = True is configured in ~/.s3cfg.

Troubleshooting

Issue: Connection Refused

Error:

Bash
ERROR: Could not connect to server: [Errno 111] Connection refused

Check the ~/.s3cfg file:

Bash
cat ~/.s3cfg

Make sure this value is set:

Bash
use_https = True

Also confirm the endpoint:

Bash
host_base = s3.in-west2.purestore.io
host_bucket = %(bucket)s.s3.in-west2.purestore.io

Issue: File Not Found During Copy

Error:

Bash
cp: cannot stat ‘/path/to/localfile.txt’: No such file or directory

Reason:

/path/to/localfile.txt is only a sample path. You need to create an actual file first.

Create a test file:

Bash
echo “This is a test file” > /root/localfile.txt

Then copy it:

Bash
cp /root/localfile.txt /mnt/cmp-s3/

Issue: Mount Directory Does Not Exist

Create the mount directory:

Bash
mkdir -p /mnt/cmp-s3

Then run the mount command again.

Conclusion

CMP S3 bucket storage can be accessed from Linux using both s3cmd and s3fs.

  • s3cmd is useful for CLI-based upload/download operations.
  • s3fs is useful when you want to mount the S3 bucket as a local filesystem.

In this setup, the bucket linux-backup-test was configured and mounted successfully on:

Bash
/mnt/cmp-s3

More posts