Create a Lambda Function to take a Snapshot of all Ec2 Instances in your VPC
Note:
1) This is a Python Script and will Take a Snapshot of Ec2 Instances with Tags Backup
2) If there is no tag with "Backup", then this script will not create a Snapshot.
3) The Snapshot will be Stored for 7 days and will be deleted by another Script
4) If you want the Retain the Snapshots permanently, then add another Tag "Retention"
First Create an IAM User Role
1) Go to Roles- Create A Role- Select Lambda- Click Next Permissions (don't assign any permissions)- Click Next: Review- Give The appropriate name for Role
2) Select the Role you have just created - Click Add inline Policy and Copy the below code and save.
--------------------------------------------------------------------------------------------------------------------------
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"logs:*"
],
"Resource": "arn:aws:logs:*:*:*"
},
{
"Effect": "Allow",
"Action": "ec2:Describe*",
"Resource": "*"
},
{
"Effect": "Allow",
"Action": [
"ec2:CreateSnapshot",
"ec2:DeleteSnapshot",
"ec2:CreateTags",
"ec2:ModifySnapshotAttribute",
"ec2:ResetSnapshotAttribute"
],
"Resource": [
"*"
]
}
]
}
--------------------------------------------------------------------------------------------------------------------------
Now Open Aws Console- Services-Lambda
1) Select Create Function
2) Name of the Function: Give an appropriate name for your Function
3) Runtime: Python2.7
4) Role: Choose Existing Role
5) Existing Role: Select the Role created the in last step
5) Click Create Function
Copy the below Python Code to the editor and save it
--------------------------------------------------------------------------------------------------------------------------
import boto3
import collections
import datetime
ec = boto3.client('ec2')
def lambda_handler(event, context):
reservations = ec.describe_instances(
Filters=[
{'Name': 'tag-key', 'Values': ['backup', 'Backup']},
]
).get(
'Reservations', []
)
instances = sum(
[
[i for i in r['Instances']]
for r in reservations
], [])
print "Found %d instances that need backing up" % len(instances)
to_tag = collections.defaultdict(list)
for instance in instances:
try:
retention_days = [
int(t.get('Value')) for t in instance['Tags']
if t['Key'] == 'Retention'][0]
except IndexError:
retention_days = 1
for dev in instance['BlockDeviceMappings']:
if dev.get('Ebs', None) is None:
continue
vol_id = dev['Ebs']['VolumeId']
print "Found EBS volume %s on instance %s" % (
vol_id, instance['InstanceId'])
snap = ec.create_snapshot(
VolumeId=vol_id,
)
to_tag[retention_days].append(snap['SnapshotId'])
print "Retaining snapshot %s of volume %s from instance %s for %d days" % (
snap['SnapshotId'],
vol_id,
instance['InstanceId'],
retention_days,
)
for retention_days in to_tag.keys():
delete_date = datetime.date.today() + datetime.timedelta(days=retention_days)
delete_fmt = delete_date.strftime('%Y-%m-%d')
print "Will delete %d snapshots on %s" % (len(to_tag[retention_days]), delete_fmt)
ec.create_tags(
Resources=to_tag[retention_days],
Tags=[
{'Key': 'DeleteOn', 'Value': delete_fmt},
]
)
-------------------------------------------------------------------------------------------------------------------------
The below script will delete the snapshots which are 7 days old
Now Open Aws Console- Services-Lambda
1) Select Create Function
2) Name of the Function: Give an appropriate name for your Function
3) Runtime: Python2.7
4) Role: Choose Existing Role
5) Existing Role: Select the Role created the in the last step
5) Click Create Function
Copy the below Python Code to the editor and save it
--------------------------------------------------------------------------------------------------------------------------
import boto3
import re
import datetime
ec = boto3.client('ec2')
iam = boto3.client('iam')
"""
This function looks at *all* snapshots that have a "DeleteOn" tag containing
the current day formatted as YYYY-MM-DD. This function should be run at least
daily.
"""
def lambda_handler(event, context):
account_ids = list()
try:
"""
You can replace this try/except by filling in `account_ids` yourself.
Get your account ID with:
> import boto3
> iam = boto3.client('iam')
> print iam.get_user()['User']['Arn'].split(':')[4]
"""
iam.get_user()
except Exception as e:
# use the exception message to get the account ID the function executes under
account_ids.append(re.search(r'(arn:aws:sts::)([0-9]+)', str(e)).groups()[1])
delete_on = datetime.date.today().strftime('%Y-%m-%d')
filters = [
{'Name': 'tag-key', 'Values': ['DeleteOn']},
{'Name': 'tag-value', 'Values': [delete_on]},
]
snapshot_response = ec.describe_snapshots(OwnerIds=account_ids, Filters=filters)
for snap in snapshot_response['Snapshots']:
print "Deleting snapshot %s" % snap['SnapshotId']
ec.delete_snapshot(SnapshotId=snap['SnapshotId'])
--------------------------------------------------------------------------------------------------------------------------
The above scripts will Create Snapshots and will delete them after 7 days.
Now we need to Schedule these Functions to run at a specified Time
AWS- CloudWatch-Rules
Create Rule- Select Schedule- Select Cron Expression (if you want to run a specified time daily)
Add Target-Select Lambda Function- Configure Details- Save
Do the above step for both (Snapshot Creation and Deletion functions) Seperately.
Note:
1) This is a Python Script and will Take a Snapshot of Ec2 Instances with Tags Backup
2) If there is no tag with "Backup", then this script will not create a Snapshot.
3) The Snapshot will be Stored for 7 days and will be deleted by another Script
4) If you want the Retain the Snapshots permanently, then add another Tag "Retention"
First Create an IAM User Role
1) Go to Roles- Create A Role- Select Lambda- Click Next Permissions (don't assign any permissions)- Click Next: Review- Give The appropriate name for Role
2) Select the Role you have just created - Click Add inline Policy and Copy the below code and save.
--------------------------------------------------------------------------------------------------------------------------
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"logs:*"
],
"Resource": "arn:aws:logs:*:*:*"
},
{
"Effect": "Allow",
"Action": "ec2:Describe*",
"Resource": "*"
},
{
"Effect": "Allow",
"Action": [
"ec2:CreateSnapshot",
"ec2:DeleteSnapshot",
"ec2:CreateTags",
"ec2:ModifySnapshotAttribute",
"ec2:ResetSnapshotAttribute"
],
"Resource": [
"*"
]
}
]
}
--------------------------------------------------------------------------------------------------------------------------
Now Open Aws Console- Services-Lambda
1) Select Create Function
2) Name of the Function: Give an appropriate name for your Function
3) Runtime: Python2.7
4) Role: Choose Existing Role
5) Existing Role: Select the Role created the in last step
5) Click Create Function
Copy the below Python Code to the editor and save it
--------------------------------------------------------------------------------------------------------------------------
import boto3
import collections
import datetime
ec = boto3.client('ec2')
def lambda_handler(event, context):
reservations = ec.describe_instances(
Filters=[
{'Name': 'tag-key', 'Values': ['backup', 'Backup']},
]
).get(
'Reservations', []
)
instances = sum(
[
[i for i in r['Instances']]
for r in reservations
], [])
print "Found %d instances that need backing up" % len(instances)
to_tag = collections.defaultdict(list)
for instance in instances:
try:
retention_days = [
int(t.get('Value')) for t in instance['Tags']
if t['Key'] == 'Retention'][0]
except IndexError:
retention_days = 1
for dev in instance['BlockDeviceMappings']:
if dev.get('Ebs', None) is None:
continue
vol_id = dev['Ebs']['VolumeId']
print "Found EBS volume %s on instance %s" % (
vol_id, instance['InstanceId'])
snap = ec.create_snapshot(
VolumeId=vol_id,
)
to_tag[retention_days].append(snap['SnapshotId'])
print "Retaining snapshot %s of volume %s from instance %s for %d days" % (
snap['SnapshotId'],
vol_id,
instance['InstanceId'],
retention_days,
)
for retention_days in to_tag.keys():
delete_date = datetime.date.today() + datetime.timedelta(days=retention_days)
delete_fmt = delete_date.strftime('%Y-%m-%d')
print "Will delete %d snapshots on %s" % (len(to_tag[retention_days]), delete_fmt)
ec.create_tags(
Resources=to_tag[retention_days],
Tags=[
{'Key': 'DeleteOn', 'Value': delete_fmt},
]
)
-------------------------------------------------------------------------------------------------------------------------
The below script will delete the snapshots which are 7 days old
Now Open Aws Console- Services-Lambda
1) Select Create Function
2) Name of the Function: Give an appropriate name for your Function
3) Runtime: Python2.7
4) Role: Choose Existing Role
5) Existing Role: Select the Role created the in the last step
5) Click Create Function
Copy the below Python Code to the editor and save it
--------------------------------------------------------------------------------------------------------------------------
import boto3
import re
import datetime
ec = boto3.client('ec2')
iam = boto3.client('iam')
"""
This function looks at *all* snapshots that have a "DeleteOn" tag containing
the current day formatted as YYYY-MM-DD. This function should be run at least
daily.
"""
def lambda_handler(event, context):
account_ids = list()
try:
"""
You can replace this try/except by filling in `account_ids` yourself.
Get your account ID with:
> import boto3
> iam = boto3.client('iam')
> print iam.get_user()['User']['Arn'].split(':')[4]
"""
iam.get_user()
except Exception as e:
# use the exception message to get the account ID the function executes under
account_ids.append(re.search(r'(arn:aws:sts::)([0-9]+)', str(e)).groups()[1])
delete_on = datetime.date.today().strftime('%Y-%m-%d')
filters = [
{'Name': 'tag-key', 'Values': ['DeleteOn']},
{'Name': 'tag-value', 'Values': [delete_on]},
]
snapshot_response = ec.describe_snapshots(OwnerIds=account_ids, Filters=filters)
for snap in snapshot_response['Snapshots']:
print "Deleting snapshot %s" % snap['SnapshotId']
ec.delete_snapshot(SnapshotId=snap['SnapshotId'])
--------------------------------------------------------------------------------------------------------------------------
The above scripts will Create Snapshots and will delete them after 7 days.
Now we need to Schedule these Functions to run at a specified Time
AWS- CloudWatch-Rules
Create Rule- Select Schedule- Select Cron Expression (if you want to run a specified time daily)
Add Target-Select Lambda Function- Configure Details- Save
Do the above step for both (Snapshot Creation and Deletion functions) Seperately.
