Anshul KicharaHere’s a step-by-step guide on how to implement file uploads using pre-signed URLs to a specific storage class, specifically with AWS S3. I’ll cover how to generate a pre-signed URL in Python and how to use it in Postman.
Architecture:
Create an IAM User:
- Sign in to the AWS Management Console.
- Navigate to IAM (Identity and Access Management):
- Open the IAM Console.
- Create a New User:
- Click on Users in the sidebar.
- Click the Add user button.
- Enter a user name (e.g., s3-uploader).
- Select Programmatic access for the access type to generate an access key ID and secret access key.
- Click Next: Permissions.
Create an S3 Bucket:
- Navigate to S3:
- Open the S3 Console.
- Create a New Bucket:
- Click on Create bucket.
- Enter a unique bucket name (e.g., data-from-resign).
- Choose a region.
- Configure options as needed (default settings are usually sufficient for this example).
- Click Create bucket.
[ Good Read: Comparison between Mydumper, mysqldump, xtrabackup]
You can check more info about: Pre-Signed URLs to a Specific Storage Class
.
Edit Cross-origin resource sharing (CORS):
[
{
"AllowedHeaders": [
"*"
],
"AllowedMethods": [
"PUT"
],
"AllowedOrigins": [
"https://example.com"
],
"ExposeHeaders": []
}
]
Attach a Custom Policy to the User:
- Create a Custom Policy:
- In the IAM Console, go to Policies.
- Click Create policy.
- Select the JSON tab and enter a policy that grants permission to upload files to your specific bucket. For example:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": "s3:PutObject",
"Resource": "arn:aws:s3:::data-from-presign/*"
}
]
}
- Click Next: Tags (optional) and then Next: Review.
- Provide a name (e.g., S3UploadPolicy) and description.
- Click Create policy.
## Attach the Policy to the User:
- Go to Users and select the user you created (s3-uploader).
- Click the Permissions tab.
- Click Add Permissions.
- Select Attach policies directly.
- Search for and select the policy you created (S3UploadPolicy).
- Click Next: Review and then Add permissions.
## Generate Programmatic Access Credentials:
- Get Access Keys
- Go to Users and select the user (s3-uploader).
- Click the Security credentials tab.
- Under Access keys, click Create access key.
- Download the CSV file containing the Access key ID and Secret access key or copy them. These are needed for programmatic access.
## Generate a Pre-Signed URL:
Using the AWS SDK (Boto3 for Python), generate a pre-signed URL. Here’s a Python script to do this:
import boto3
import botocore
# Assuming your S3 bucket name and image file name
ACCESS_KEY = 'access_key'
SECRET_ACCESS_KEY = 'secret_key'
BUCKET_NAME = 'data-from-presign'
OBJECT_KEY = 'image.png'
STORAGE_CLASS = 'ONEZONE_IA'
# Initialize a session using the AWS SDK for Python (Boto3)
session = boto3.Session(
aws_access_key_id=ACCESS_KEY,
aws_secret_access_key=SECRET_ACCESS_KEY,
region_name='ap-south-1' # Specify the region where your bucket is located
)
