DevOps(Day-43): S3 Programmatic access with AWS-CLI

TABLE OF CONTENTS

• S3

• Task-01: Creating a S3 bucket

• Task 2: Creating a EC2 snapshot and integrating to S3

S3

Amazon Simple Storage Service (AWS S3) is a scalable, high-speed, low-cost, and fully managed object storage service provided by Amazon Web Services (AWS). S3 provides customers with secure and durable storage of their data in the form of objects, which are organized into buckets and can be accessed through APIs and web-based interfaces.

S3 buckets can be used for a variety of purposes, including file storage, backup and archiving big data analytics, and content distribution.

The buckets are easily accessible from anywhere in the world, making it simple to store, access, and manage data on the cloud. S3 also provides a number of features that make it easy to manage and organize data, such as versioning, lifecycle policies, and access controls.

Task-01: Creating a S3 bucket

• Launch an EC2 instance using the AWS Management Console and connect to it using Secure Shell (SSH).

  1. Launch an EC2 instance by providing all the default configurations as we have seen in my previous blogs.

     

  2. Connect to the server using SSH. Open Command Prompt in your local system and navigate to the private key ".pem" file location.

  3. Copy the ssh command from the console and paste on CMD.

     

  4. Now you are connected to the EC2 instance.

     

     

• Create an S3 bucket and upload a file to it using the AWS Management Console.

  1. Navigate to the S3 page in the AWS console and provide the bucket name and choose the region for it.

     

     

     

  2. The S3 bucket is created now.

     

  3. Now upload a file to the S3 bucket.

     

  4. I have added my resume and a fileS3.txt to the bucket from my local system.

     

  5. The file is now uploaded to the S3 bucket.

     

• Access the file from the EC2 instance using the AWS Command Line Interface (AWS CLI).

  1. Navigate to your instance and install AWS CLI.

     

  2. Configure this with the IAM user having specific permission to have a connection between AWS CLI to the management console.

  3. Provide the necessary Access key IDs to build the connection.

     

  4. Now you can view the S3 bucket list and copy the file from S3 to EC2.

     

  5. You can view the file content now in AWS CLI.

Task 2: Creating a EC2 snapshot and integrating to S3

• Create a snapshot of the EC2 instance and use it to launch a new EC2 instance.

  1. Navigate to the snapshot section in the AWS console.

     

     

  2. Snapshot is now created.

     

  3. Now create an image from the snapshot.

     

     

     

  4. Navigate to AMIs section in EC2 dashboard and you can see the Machine image. Then create an instance from the image.

     

     

  5. You can see the new instance now.

     

• Download a file from the S3 bucket using the AWS CLI.

  1. Now, connect the instance from AWS CLI to the AWS management console.

     

• Verify that the contents of the file are the same on both EC2 instances.

  1. Download the file from S3 on the new instance and verify the content is matched with the old instance.

     

     

Some important AWS S3 commands are:-

1. aws s3 ls - This command lists all of the S3 buckets in your AWS account.

2. aws s3 mb s3://bucket-name - This command creates a new S3 bucket with the specified name.

3. aws s3 rb s3://bucket-name - This command deletes the specified S3 bucket.

4. aws s3 cp file.txt s3://bucket-name - This command uploads a file to an S3 bucket.

5. aws s3 cp s3://bucket-name/file.txt . - This command downloads a file from an S3 bucket to your local file system.

6. aws s3 sync local-folder s3://bucket-name - This command syncs the contents of a local folder with an S3 bucket.

7. aws s3 ls s3://bucket-name - This command lists the objects in an S3 bucket.

8. aws s3 rm s3://bucket-name/file.txt - This command deletes an object from an S3 bucket.

9. aws s3 presign s3://bucket-name/file.txt - This command generates a pre-signed URL for an S3 object, which can be used to grant temporary access to the object.

10. aws s3api list-buckets - This command retrieves a list of all S3 buckets in your AWS account, using the S3 API.

Thanks for reading my article. Have a nice day.

WRITTEN BY Biswaraj Sahoo --AWS Community Builder | DevOps Engineer | Docker | Linux | Jenkins | AWS | Git | Terraform | Docker | kubernetes

Empowering communities via open source and education. Connect with me over linktree: linktr.ee/biswaraj333