Here we will see how to add files to S3 Bucket using Shell Script.
Pre Requisites:
- Create S3 Bucket
- Create an IAM user, get Access Key and Secret Key
Add files to S3 Bucket using Shell Script:
The shell script is the most prominent solution to push the files into S3 bucket if we consider this task as an independent. As part of this tutorial, I am going to push all the files under /opt/s3files directory to s3 bucket.
SendToS3.sh
bucket=my-files
files_location=/opt/s3files/
now_time=$(date +"%H%M%S")
contentType="application/x-compressed-tar"
dateValue=`date -R`
# your key goes here..
s3Key=AKIAWQEOCFWY224LOMFA
# your secrets goes here..
s3Secret=wu78rgdkjrhhdbcoUed+Xm1chCytUTiVeyOUDFJE
function pushToS3()
{
files_path=$1
for file in $files_path*
do
fname=$(basename $file)
logInfo "Start sending $fname to S3"
resource="/${bucket}/${now_date}/${fname}_${now_time}"
stringToSign="PUT\n\n${contentType}\n${dateValue}\n${resource}"
signature=`echo -en ${stringToSign} | openssl sha1 -hmac ${s3Secret} -binary | base64`
curl -X PUT -T "${file}" \
-H "Host: ${bucket}.s3.amazonaws.com" \
-H "Date: ${dateValue}" \
-H "Content-Type: ${contentType}" \
-H "Authorization: AWS ${s3Key}:${signature}" \
https://${bucket}.s3.amazonaws.com/${now_date}/${fname}_${now_time}
logInfo "$fname has been sent to S3 successfully."
done
}
pushToS3 $files_location
Run It:
Provide execute permissions to the file:
Output
$chmod 777 SendToS3.sh
$./SendToS3.sh
Resources:
Happy Learning 🙂