Intermediate
Introduction: Why You Need Automatic Item Expiration in DynamoDB
If you’ve been running DynamoDB tables in production for any meaningful period, you’ve probably noticed a familiar problem: your tables keep growing. Session data from users who logged in six months ago, temporary cache entries, expired tokens, old audit logs, event records nobody will ever query again — they all sit there, quietly inflating your storage bill month after month.
The naive solution is to write a scheduled Lambda function that scans your table and deletes old items. I’ve seen this pattern dozens of times, and it’s almost always a bad idea. You’re burning read capacity units (RCUs) on the scan, write capacity units (WCUs) on the deletes, adding operational complexity, and paying for the compute to run it. There’s a better way.
DynamoDB Time-to-Live (TTL) is a built-in feature that automatically deletes expired items from your table at zero additional cost. No RCUs consumed for the scan. No WCUs consumed for the deletion. No Lambda functions to maintain. AWS handles everything in the background.
This article is for intermediate AWS developers and DevOps engineers who are already working with DynamoDB and want to implement TTL correctly. We’ll cover exactly how TTL works internally, how to enable it, real code examples for setting TTL values, how to capture deleted items via DynamoDB Streams, common mistakes that silently break TTL, and cost implications you should understand before and after implementation.
Prerequisites
- An AWS account with permissions to manage DynamoDB tables
- AWS CLI v2 installed and configured (
aws configure) - Python 3.8+ with
boto3installed (for code examples) - Basic familiarity with DynamoDB concepts: tables, items, attributes, partition keys, sort keys
- A DynamoDB table to experiment with (we’ll create one in the examples)
How DynamoDB TTL Works Internally
Understanding the internals will save you from making incorrect assumptions. Here’s what actually happens when you enable TTL:
The TTL Mechanism
When you enable TTL on a table, you designate a specific attribute name as the TTL attribute. This attribute must contain a value representing an expiration timestamp as a Unix epoch time in seconds (not milliseconds — this is the single most common mistake). The value must be stored as a Number data type.
A background process continuously scans your table, comparing each item’s TTL attribute value against the current time. If the TTL value is older than the current time, the item is marked for deletion.
Critical Timing Details
Here’s what the AWS documentation states clearly but many developers miss: TTL typically deletes expired items within 48 hours of expiration. Items are usually deleted much faster than that — often within minutes to a few hours — but AWS does not guarantee immediate deletion. During this window, expired items still appear in reads, queries, and scans unless you explicitly filter them out.
This has a significant architectural implication: TTL is not a real-time expiration mechanism. If your application logic requires that an item become invisible the instant it expires (like a session token), you must add a filter condition in your application code or query filter expression.
Zero Cost — But What Does That Actually Mean?
TTL deletions are performed by a system process that does not consume provisioned or on-demand WCUs. The deletions do not count against your table’s throughput capacity. However, if you have DynamoDB Streams enabled, the TTL deletions do generate stream records (with a specific userIdentity field indicating it was a system deletion), and processing those stream records with Lambda or Kinesis has its own costs.
Setting Up TTL: Step-by-Step with AWS CLI and Console
Let’s start by creating a test table and enabling TTL on it.
Step 1: Create a DynamoDB Table
# Create a table for storing user sessions
aws dynamodb create-table \
--table-name UserSessions \
--attribute-definitions \
AttributeName=SessionId,AttributeType=S \
--key-schema \
AttributeName=SessionId,KeyType=HASH \
--billing-mode PAY_PER_REQUEST \
--region us-east-1
# Wait for the table to become ACTIVE
aws dynamodb wait table-exists --table-name UserSessions --region us-east-1
echo "Table is ready."Step 2: Enable TTL on the Table
# Enable TTL using the 'ExpiresAt' attribute
aws dynamodb update-time-to-live \
--table-name UserSessions \
--time-to-live-specification "Enabled=true, AttributeName=ExpiresAt" \
--region us-east-1The response will look like this:
{
"TimeToLiveSpecification": {
"Enabled": true,
"AttributeName": "ExpiresAt"
}
}Step 3: Verify TTL Status
aws dynamodb describe-time-to-live \
--table-name UserSessions \
--region us-east-1Note: After enabling TTL, the status transitions through ENABLING before reaching ENABLED. This can take up to one hour. Similarly, if you disable TTL, it goes through DISABLING and you cannot re-enable it until it reaches DISABLED — which can also take up to one hour.
Step 4: Insert Items with TTL Values
# Calculate a TTL value 24 hours from now (Unix epoch in seconds)
EXPIRES_AT=$(date -d '+24 hours' +%s)
# On macOS: EXPIRES_AT=$(date -v+24H +%s)
echo "Item will expire at epoch: $EXPIRES_AT"
aws dynamodb put-item \
--table-name UserSessions \
--item '{
"SessionId": {"S": "sess-abc123"},
"UserId": {"S": "user-42"},
"CreatedAt": {"N": "'$(date +%s)'"},
"ExpiresAt": {"N": "'$EXPIRES_AT'"},
"Data": {"S": "session payload here"}
}' \
--region us-east-1Working with TTL in Python (Boto3): Real-World Patterns
Here’s a practical Python module that demonstrates the patterns you’ll actually use in production.
Writing Items with TTL
import boto3
import time
from datetime import datetime, timedelta
from typing import Optional
dynamodb = boto3.resource('dynamodb', region_name='us-east-1')
table = dynamodb.Table('UserSessions')
def create_session(session_id: str, user_id: str, ttl_hours: int = 24) -> dict:
"""Create a session with automatic expiration via TTL."""
now = int(time.time())
expires_at = now + (ttl_hours * 3600)
item = {
'SessionId': session_id,
'UserId': user_id,
'CreatedAt': now,
'ExpiresAt': expires_at, # This is the TTL attribute — must be int (epoch seconds)
'IsActive': True,
}
table.put_item(Item=item)
print(f"Session {session_id} created, expires at {datetime.fromtimestamp(expires_at)}")
return item
def get_valid_session(session_id: str) -> Optional[dict]:
"""
Get a session only if it hasn't expired.
IMPORTANT: Because TTL deletion can be delayed up to 48 hours,
expired items may still be returned by GetItem. Always filter
in your application code.
"""
response = table.get_item(Key={'SessionId': session_id})
item = response.get('Item')
if item is None:
return None
# Critical: Check expiration in application code
current_time = int(time.time())
if item.get('ExpiresAt', 0) < current_time:
print(f"Session {session_id} has expired but not yet deleted by TTL")
return None
return item
def extend_session(session_id: str, additional_hours: int = 1) -> None:
"""Extend a session's TTL by updating the ExpiresAt attribute."""
new_expires_at = int(time.time()) + (additional_hours * 3600)
table.update_item(
Key={'SessionId': session_id},
UpdateExpression='SET ExpiresAt = :new_ttl',
ExpressionAttributeValues={':new_ttl': new_expires_at},
ConditionExpression='attribute_exists(SessionId)',
)
print(f"Session {session_id} extended to {datetime.fromtimestamp(new_expires_at)}")
def revoke_session_immediately(session_id: str) -> None:
"""
Set TTL to a past timestamp for near-immediate background deletion.
Also mark as inactive for immediate application-level enforcement.
"""
table.update_item(
Key={'SessionId': session_id},
UpdateExpression='SET ExpiresAt = :past_ttl, IsActive = :false_val',
ExpressionAttributeValues={
':past_ttl': 0, # Epoch 0 = January 1, 1970 — definitely expired
':false_val': False,
},
)
print(f"Session {session_id} revoked and marked for TTL deletion")
# Usage
if __name__ == '__main__':
create_session('sess-001', 'user-42', ttl_hours=2)
session = get_valid_session('sess-001')
print(f"Retrieved: {session}")
extend_session('sess-001', additional_hours=4)
revoke_session_immediately('sess-001')Querying with Filter Expressions to Exclude Expired Items
import boto3
import time
from boto3.dynamodb.conditions import Key, Attr
dynamodb = boto3.resource('dynamodb', region_name='us-east-1')
# Assume a table with a GSI where UserId is the partition key
table = dynamodb.Table('UserSessions')
def get_active_sessions_for_user(user_id: str) -> list:
"""
Query all active (non-expired) sessions for a user.
Uses a FilterExpression to exclude items that TTL hasn't deleted yet.
"""
current_time = int(time.time())
response = table.query(
IndexName='UserId-index',
KeyConditionExpression=Key('UserId').eq(user_id),
FilterExpression=Attr('ExpiresAt').gte(current_time),
)
return response.get('Items', [])Capturing TTL Deletions with DynamoDB Streams
One of the most powerful patterns with TTL is reacting to item deletions. For example, you might want to archive expired items to S3, send a notification when a subscription expires, or update an analytics counter.
Enabling Streams to Capture TTL Deletions
# Enable DynamoDB Streams with both old and new images
aws dynamodb update-table \
--table-name UserSessions \
--stream-specification StreamEnabled=true,StreamViewType=NEW_AND_OLD_IMAGES \
--region us-east-1Lambda Function to Process TTL Deletions
import json
def lambda_handler(event, context):
"""
Process DynamoDB Stream records.
TTL-deleted items have a specific userIdentity field.
"""
for record in event['Records']:
# Only process REMOVE events
if record['eventName'] != 'REMOVE':
continue
# Distinguish TTL deletions from application-initiated deletes
user_identity = record.get('userIdentity')
is_ttl_deletion = (
user_identity is not None
and user_identity.get('type') == 'Service'
and user_identity.get('principalId') == 'dynamodb.amazonaws.com'
)
if is_ttl_deletion:
old_image = record['dynamodb'].get('OldImage', {})
session_id = old_image.get('SessionId', {}).get('S', 'unknown')
user_id = old_image.get('UserId', {}).get('S', 'unknown')
print(f"TTL expired session: {session_id} for user: ")
# Archive to S3, send SNS notification, update metrics, etc.
# archive_to_s3(old_image)
else:
print(f"Application-initiated delete: {record['dynamodb']['Keys']}")
return {'statusCode': 200}The key distinguishing factor is the userIdentity object in the stream record. When the TTL background process deletes an item, the record contains "principalId": "dynamodb.amazonaws.com". Application-initiated deletes do not have this field.
Common Mistakes and How to Avoid Them
After helping multiple teams implement TTL, these are the issues I see repeatedly:
| Mistake | Symptom | Fix |
|---|---|---|
| Using milliseconds instead of seconds | Items never get deleted (timestamp is interpreted as thousands of years in the future) | Always use int(time.time()) in Python, Math.floor(Date.now() / 1000) in JavaScript |
| Storing TTL as String type | TTL is silently ignored — DynamoDB requires Number type | Ensure the attribute is stored as {"N": "1700000000"}, not {"S": "1700000000"} |
| Assuming instant deletion | Expired items appear in query results | Always add a filter condition in your application logic checking expiration time against current time |
| TTL attribute name mismatch | Items never expire | The attribute name in your items must exactly match the name you specified when enabling TTL |
| Setting TTL more than 5 years in the past | Items are not deleted — DynamoDB ignores TTL values more than 5 years older than the current time | For immediate expiration, use a recent past
|