Here we will see how to add files to S3 Bucket using Shell Script.

Pre Requisites:

  • Create S3 Bucket
  • Create an IAM user, get Access Key and Secret Key

Add files to S3 Bucket using Shell Script:

The shell script is the most prominent solution to push the files into S3 bucket if we consider this task as an independent. As part of this tutorial, I am going to push all the files under /opt/s3files directory to s3 bucket.

SendToS3.sh
bucket=my-files
files_location=/opt/s3files/
now_time=$(date +"%H%M%S")
contentType="application/x-compressed-tar"
dateValue=`date -R`
# your key goes here..
s3Key=AKIAWQEOCFWY224LOMFA
# your secrets goes here..
s3Secret=wu78rgdkjrhhdbcoUed+Xm1chCytUTiVeyOUDFJE

function pushToS3()
{
  files_path=$1
  for file in $files_path*
  do
    fname=$(basename $file)
    logInfo "Start sending $fname to S3"
    resource="/${bucket}/${now_date}/${fname}_${now_time}"
    stringToSign="PUT\n\n${contentType}\n${dateValue}\n${resource}"
    signature=`echo -en ${stringToSign} | openssl sha1 -hmac ${s3Secret} -binary | base64`
    curl -X PUT -T "${file}" \
     -H "Host: ${bucket}.s3.amazonaws.com" \
     -H "Date: ${dateValue}" \
     -H "Content-Type: ${contentType}" \
     -H "Authorization: AWS ${s3Key}:${signature}" \
      https://${bucket}.s3.amazonaws.com/${now_date}/${fname}_${now_time}
     logInfo "$fname has been sent to S3 successfully."
  done
}
pushToS3 $files_location

Run It:

Provide execute permissions to the file:

Output
$chmod 777 SendToS3.sh
$./SendToS3.sh

Resources:

Happy Learning 🙂