Python aws-sdk Guide

Python (Boto3) SDK

How to run Amazon S3 examples using Python (Boto3) SDK on Titan Cloud Storage service

Amazon S3 is a cloud storage service from Amazon Web Services (AWS) that lets users store and get back any size of data from anywhere on the internet. It is scalable and highly available. Boto3 is the AWS SDK for Python. It gives you a simple and easy-to-use way to interact with different AWS services, like Amazon S3 (Simple Storage Service).
Boto3 has many functions and methods that developers can use to create, configure, and manage buckets and objects in Amazon S3. With Boto3, you can easily upload and download files to and from Amazon S3 buckets, manage bucket policies and permissions, and set up bucket notifications and lifecycle rules.

This quick start guide will show you how to install the Visual studio and run a few basic Python program examples.

Prerequisites

Prerequisites required are as follows:

  1. Python 3.7 or higher
  2. Boto3
  3. Access and Secret Keys

Once you have met these prerequisites, you can start using the AWS SDK for Python (Boto3) to interact with Amazon S3 and other AWS services from your Python code.

Goals

After finishing this article, you'll be able to:

  1. Initialize Titan cloud instance.
  2. Create a new bucket with a specified name.
  3. Checking if the bucket exists.
  4. Delete an existing bucket.
  5. Retrieve a list of all buckets in the account.
  6. Upload a new object to a bucket or replace an existing object with a new version.
  7. Retrieve an object from a bucket and download it to a local machine.
  8. Delete an object from a bucket.
  9. Delete many objects from a bucket.
  10. Retrieve a list of objects within a bucket.
  11. Make a copy of an existing object within the same bucket or in a different bucket.
  12. Check if an object exists in a bucket or not.
  13. Uploads a file to a bucket.
  14. Downloads an object from a bucket to a file.

Steps to run Amazon S3 examples using Python (Boto3) SDK on Titan Cloud Storage

Here are the steps to get started with using the Python SDK for Amazon S3 Titan cloud storage:

Step 1. Install Python

If you haven't already, you will need to install Python on your machine. You can download the latest version of Python from the official website at below link.
https://www.python.org/downloads/

Step 2. Install Boto3

Open a terminal or command prompt window and use the following command to install the Boto3 library:
pip install boto3

Step 3. Write your Python code

You can now write your Python code that uses the Boto3 library to interact with the AWS services you need. Make sure to import the necessary modules at the beginning of your script and use the appropriate methods and functions to interact with Titan Cloud Storage service.

Step 4.Run your Python code

Once you have written your Python code, save the file with a .py extension and navigate to the directory where the file is located in your terminal or command prompt window. Use the following command to run your code:
python my_script.py
This will execute your Python code and produce any output or results that you have defined in your script.
That's it! With these steps, you can install and run the AWS SDK for Python on your local machine, and start using the powerful AWS services in your Python projects.

Quick Start Examples

Boto3 makes it easy to interact with Amazon S3 using Python. It provides a wide range of features and capabilities that can assist developers in building powerful and scalable applications on AWS. Some of the key features of Boto3 with Amazon S3 include:

  1. Initializing Titan cloud instance.
  2. Checking if the bucket exists
  3. Creating a bucket.
  4. Listing buckets
  5. Listing bucket objects
  6. Removing an empty bucket
  7. Removing an object
  8. Removing objects
  9. Uploading and downloading files to and from the bucket
  10. Copying and moving files between S3 buckets or within the same bucket

Bucket operations

A place to store files is called an Amazon S3 bucket. Objects are the name for S3 files. This part explains how to use the AWS SDK for Python to do common tasks with S3 buckets.

1. Initializes a new Titan Cloud Connection Code:

The below code snippets Initializes a new Titan Cloud object storage client.
General Method:

boto3.client(*args, **kwargs)
*args-> one required argument
**kwargs-> several optional keyword arguments that can be used to customize the client's behavior.

Note: Before running this code, make sure to replace the endpoint, accessKey, and secretKey variables with the appropriate values for your Titan Cloud Storage account.

create_connection1.py

# Import the Boto3 library
import boto3
# Initialize the titan Cloud storage instance 
s3_titanobj = boto3.client('s3',
                  endpoint_url='https://demo.s3.titancloudstorage.com',
                  aws_access_key_id='titanadmin',
                  aws_secret_access_key='TitanDemo123')

print("You are connected successfully!!")

Run command:
>python create_connection1.py

2. Check if a bucket exists on Titan Cloud

The below code snippets checks if a bucket exists on the server. It returns True if the bucket exists.
General Method:
bucket_exists(bucket_name)

checkifbucketexists1.py

# Import the Boto3 library
import boto3

# Initialize the S3 client for Titan Cloud Storage
# 1st Block of code
s3 = boto3.client('s3',
                  endpoint_url='https://{YOUR_INSTANCE}.s3.titancloudstorage.com',
                  aws_access_key_id=‘<your_access_key>‘,
                  aws_secret_access_key=‘<your_secret_key>‘)

# 2nd Block of code
# Specify the name of the bucket to check
bucket_name = ‘new-awsboto3-bucket’
# Check if the bucket exists
response = None
try:
    response = s3.head_bucket(Bucket=bucket_name)
except Exception as e:
    print(f"Bucket {bucket_name} does not exist: {e}")
if response:
    print(f"Bucket {bucket_name} exists!")

Run command:
>python checkifbucketexists1.py

3. Create a bucket on Titan Cloud

The below code snippets create a bucket on the Titan Cloud server.
General Method:
S3Control.Client.create_bucket(**kwargs)

createbucket.py

# Import the Boto3 library
import boto3
# 1st Block of code
# Initialize the s3_titanobj client for Titan Cloud Storage
s3_titanobj = boto3.client('s3',
                  endpoint_url='https://{YOUR_INSTANCE}.s3.titancloudstorage.com',
                  aws_access_key_id=‘<your_access_key>‘,
                  aws_secret_access_key=‘<your_secret_key>‘)
# 2nd Block of code
# Specify the name of the bucket to check or create
bucket_name = 'new-awsboto3-bucket'

# Check if the bucket exists on titan cloud storage
response = None
try:
    response = s3_titanobj.head_bucket(Bucket=bucket_name)
except Exception as e:
    print(f"Bucket {bucket_name} does not exist: {e}")
if response:
    print(f"Bucket {bucket_name} exists!")
else:
    # 3rd  Block of code
    # Create the new bucket
    try:
        response = s3_titanobj.create_bucket(Bucket=bucket_name)
        if response:
            print(f"Bucket {bucket_name} created successfully!")
    except Exception as e:
        print(f"Error creating bucket {bucket_name}: {e}")

Run command:
>python createbucket1.py

4. Remove an empty bucket with Titan Cloud Storage

The below code snippets delete the specified S3 bucket. Before the bucket can be deleted, all of the objects in it, including all versions and delete markers, must be deleted on a Titan Cloud.
General Method:
S3.Client.delete_bucket(**kwargs)


Removebucket1.py

# Import the Boto3 library
import boto3
# 1st Block of code
# Initialize the S3 client for Titan Cloud Storage
s3_titanobj  = boto3.client('s3',
                  endpoint_url='https://{YOUR_INSTANCE}.s3.titancloudstorage.com',
                  aws_access_key_id=‘<your_access_key>‘,
                  aws_secret_access_key=‘<your_secret_key>‘)
# 2nd Block of code
# Specify the name of the bucket to check and remove
bucket_name = 'new-awsboto3-bucket'

# Check if the bucket exists
response = None
try:
    response = s3_titanobj.head_bucket(Bucket=bucket_name)
except Exception as e:
    print(f"Bucket {bucket_name} does not exist: {e}")
if response:
    print(f"Bucket {bucket_name} exists!")
    # 3rd Block of code
    # Remove the bucket
    try:
        response = s3_titanobj.delete_bucket(Bucket=bucket_name)
        if response:
            print(f"Bucket {bucket_name} removed successfully!")
    except Exception as e:
        print(f"Error removing bucket {bucket_name}: {e}")

Run command:
>python removebucket1.py

5. List buckets on Titan Cloud Storage

5. List buckets on Titan Cloud Storage 

Listbuckets1.py 

# Import the Boto3 library
import boto3
# 1st Block of code
# Initialize the S3 client for Titan Cloud Storage
s3_titanobj = boto3.client('s3',
                  endpoint_url='https://{YOUR_INSTANCE}.s3.titancloudstorage.com',
                  aws_access_key_id=‘<your_access_key>‘,
                  aws_secret_access_key=‘<your_secret_key>‘)
# 2nd Block of code
# List all buckets
response = s3_titanobj.list_buckets()

# Print each bucket name
for bucket in response['Buckets']:
    print(f"Bucket name: {bucket['Name']}")

Run command:
>python listbuckets1.py

5. Get specified bucket information on Titan Cloud

The below code snippets retrives the specified S3 bucket information on a Titan Cloud.
General Method:
S3Control.Client.get_bucket(**kwargs)

getbucket1.py

# Import the Boto3 library
import boto3
# 1st Block of code
# Initialize the S3 client for Titan Cloud Storage
s3_titanobj  = boto3.client('s3',
                  endpoint_url='https://{YOUR_INSTANCE}.s3.titancloudstorage.com',
                  aws_access_key_id=‘<your_access_key>‘,
                  aws_secret_access_key=‘<your_secret_key>‘)
# 2nd Block of code
# Specify the name of the bucket to check and remove
bucket_name = 'new-awsboto3-bucket'

# Check if the bucket exists
response = None
try:
    response = s3_titanobj.head_bucket(Bucket=bucket_name)
except Exception as e:
    print(f"Bucket {bucket_name} does not exist: {e}")
if response:
    print(f"Bucket {bucket_name} exists!")
    # 3rd Block of code
    # Remove the bucket
    try:
        response = s3_titanobj.delete_bucket(Bucket=bucket_name)
        if response:
            print(f"Bucket {bucket_name} removed successfully!")
    except Exception as e:
        print(f"Error removing bucket {bucket_name}: {e}")

Run command:
>python getbucket1.py

Object operations

Object operations refer to the various actions that can be performed on individual objects stored within a bucket in object storage systems.

1. Upload data to a bucket on Titan Cloud Storage

The code snippets below uploading data using AWS SDK for Python from a bucket on a Titan Cloud Storage service.
General Method:
s3.upload_file(file_name, bucket_name, object_name)

Uploadobject1.py 

# Import the Boto3 library
import boto3
# 1st Block of code
# Initialize the S3 client for Titan Cloud Storage
s3_titanobj = boto3.client('s3',
                  endpoint_url='https://{YOUR_INSTANCE}.s3.titancloudstorage.com',
                  aws_access_key_id=‘<your_access_key>‘,
                  aws_secret_access_key=‘<your_secret_key>‘)

# 2nd Block of code
# Specify the name of the bucket to upload to
bucket_name = 'aws-new-bucket'

# Specify the local file path of the file to upload
file_path = 'my-new-file.txt'

# Specify the key (i.e., filename) to use for the uploaded file
key = 'my-new-file.txt'

# Upload the file to the bucket
try:
    s3_titanobj.upload_file(file_path, bucket_name, key)
    print(f"File '{key}' uploaded to bucket '{bucket_name}' successfully!")
except Exception as e:
    print(f"Error uploading file '{key}' to bucket '{bucket_name}': {e}")

Run command:
>python uploadobject1.py

2. Download data from a bucket on Titan Cloud

The code snippets below used to download files from a bucket on a Titan Cloud.
General Method:
s3.download_file('BUCKET_NAME', 'OBJECT_NAME', 'FILE_NAME')

downloadobject1.py 

import boto3
# 1st Block of code
# Initialize the S3 client for Titan Cloud Storage
s3_titanobj = boto3.client('s3',
                  endpoint_url='https://{YOUR_INSTANCE}.s3.titancloudstorage.com',
                  aws_access_key_id=‘<your_access_key>‘,
                  aws_secret_access_key=‘<your_secret_key>‘)

# 2nd Block of code
# Specify the name of the bucket containing the object to download
bucket_name = 'my-bucket'

# Specify the key (i.e., filename) of the object to download
object_key = 'my-new-file.txt'

# Specify the local file path to save the downloaded file
local_file_path = 'my-new-file.txt'

# Download the object from the bucket
try:
    s3_titanobj.download_file(bucket_name, object_key, local_file_path)
    print(f"Object '{object_key}' downloaded from bucket '{bucket_name}' successfully!")
except Exception as e:
    print(f"Error downloading object '{object_key}' from bucket '{bucket_name}': {e}")

Run command:
>python downloadobject1.py

3. List all the objects in the bucket on Titan Cloud

The code snippets below list the items' details, optionally including bucket versions created on a Titan Cloud. To list every object in our S3 bucket that is currently available, we'll use the list_objects_v2() method:

General Method:
S3.list_objects_v2(bucket_name)

Listobjects1.py 

# Import the Boto3 library
import boto3
# 1st Block of code
# Initialize the S3 client for Titan Cloud Storage
s3_titanobj = boto3.client('s3',
                  endpoint_url='https://{YOUR_INSTANCE}.s3.titancloudstorage.com',
                  aws_access_key_id=‘<your_access_key>‘,
                  aws_secret_access_key=‘<your_secret_key>‘)
# 2nd Block of code
# Specify the name of the bucket to list objects from
bucket_name = 'aws-new-bucket'

# List all objects in the bucket
try:
    response = s3_titanobj.list_objects_v2(Bucket=bucket_name)
    if 'Contents' in response:
        for obj in response['Contents']:
            print(f"Object key: {obj['Key']}, size: {obj['Size']} bytes")
    else:
        print(f"No objects found in bucket {bucket_name}")
except Exception as e:
    print(f"Error listing objects in bucket {bucket_name}: {e}")	

Run command:
>python listobjects1.py

4. Remove an object from bucket on Titan Cloud

The below code snippets removes an object from the bucket information on a Titan Cloud. We'll use the client instance's delete_object() method to delete an object, using the bucket name and object key as parameters:
General Method:
S3.delete_object(bucket_name, object_key)

removeobject1.py 

# Import the Boto3 library
import boto3
# 1st Block of code
# Initialize the S3 client for Titan Cloud Storage
s3_titanobj = boto3.client('s3',
                  endpoint_url='https://{YOUR_INSTANCE}.s3.titancloudstorage.com',
                  aws_access_key_id=‘<your_access_key>‘,
                  aws_secret_access_key=‘<your_secret_key>‘)

# 2nd Block of code
# Specify the name of the bucket containing the object to remove
bucket_name = 'aws-new-bucket'

# Check if the bucket exists
try:
    response = s3_titanobj.head_bucket(Bucket=bucket_name)
    print(f"Bucket '{bucket_name}' exists!")
except Exception as e:
    print(f"Bucket '{bucket_name}' does not exist: {e}")
    exit()

# Specify the key (i.e., filename) of the object to remove
object_key = 'my-new-file.txt'

# Remove the object from the bucket
try:
    response = s3_titanobj.delete_object(Bucket=bucket_name, Key=object_key)
    print(f"Object '{object_key}' removed from bucket '{bucket_name}' successfully!")
except Exception as e:
    print(f"Error removing object '{object_key}' from bucket '{bucket_name}': {e}")
 
Run command:
>python removeobject1.py