Export flow logs to an Amazon S3 Bucket

I found the instructions on Amazon’s website to be useless.

This is what worked for me

Define some variables


Create the bucket

aws s3 mb s3://$bucket --region $region

Create the Policy

cat <<'EOF' > ./policy.json
    "Version": "2012-10-17",
    "Statement": [
            "Action": "s3:GetBucketAcl",
            "Effect": "Allow",
            "Resource": "arn:aws:s3:::replace-bucket",
            "Principal": { "Service": "logs.replace-region.amazonaws.com" }
            "Action": "s3:PutObject" ,
            "Effect": "Allow",
            "Resource": "arn:aws:s3:::replace-bucket/*",
            "Condition": { "StringEquals": { "s3:x-amz-acl": "bucket-owner-full-control" } },
            "Principal": { "Service": "logs.replace-region.amazonaws.com" }

sed -i "s/replace-bucket/$bucket/g" "./policy.json"
sed -i "s/replace-region/$region/g" "./policy.json"

Apply the policy

aws s3api put-bucket-policy --bucket $bucket  --policy file://policy.json

Create Export Task

aws logs create-export-task --task-name "$taskname" \
--log-group-name "$log-group-name" \
--from $start-epoch \
--to $end-epoch \
--destination "$bucket" \
--destination-prefix "$prefix"

Check on Task Status

You will need the response from the previous command. Put that in to variable taskid.
aws logs describe-export-tasks –task-id $taskid

Leave a Reply

To create code blocks or other preformatted text, indent by four spaces:

    This will be displayed in a monospaced font. The first four 
    spaces will be stripped off, but all other whitespace
    will be preserved.
    Markdown is turned off in code blocks:
     [This is not a link](http://example.com)

To create not a block, but an inline code span, use backticks:

Here is some inline `code`.

For more help see http://daringfireball.net/projects/markdown/syntax

You can use these HTML tags

<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>