Intermediate
Introduction: The Silent Killer of DynamoDB Performance
You’ve provisioned what should be more than enough Read and Write Capacity Units (RCUs/WCUs) for your DynamoDB table, yet you’re still seeing ProvisionedThroughputExceededException errors in your application logs. Your CloudWatch metrics show that your overall consumed capacity is well below your provisioned limit. So what’s going on?
The answer is almost always a hot partition — a small number of partition keys are receiving a disproportionate share of traffic, overwhelming the throughput allocated to their specific partition while the rest of the table sits idle. The frustrating part? Standard CloudWatch metrics for DynamoDB (ConsumedReadCapacityUnits, ConsumedWriteCapacityUnits) only show you aggregate table-level numbers. They don’t tell you which partition keys are the culprits.
This is exactly the problem Amazon CloudWatch Contributor Insights for DynamoDB solves. It continuously analyzes your DynamoDB traffic patterns and surfaces the Top-N most accessed partition keys and sort keys in near real-time — no custom instrumentation, no application-level logging changes, and no sampling.
Who should read this: Backend engineers, DevOps engineers, and SREs who manage DynamoDB tables in production and need to diagnose throttling, identify hot keys, or proactively monitor access patterns. You should have basic familiarity with DynamoDB concepts (partition keys, sort keys, capacity modes) and be comfortable with the AWS CLI.
Prerequisites
- An AWS account with permissions to manage DynamoDB and CloudWatch
- AWS CLI v2 installed and configured (
aws --versionshould return 2.x) - At least one DynamoDB table (we’ll create a sample one if needed)
- IAM permissions including
dynamodb:EnableKinesisStreamingDestination,dynamodb:DescribeContributorInsights,dynamodb:UpdateContributorInsights,cloudwatch:GetInsightRuleReport, andcloudwatch:DescribeInsightRules - Python 3.8+ with
boto3installed (for the code examples)
Understanding How Contributor Insights for DynamoDB Works
Before diving into implementation, let’s understand the mechanics. CloudWatch Contributor Insights is a general-purpose CloudWatch feature that analyzes log group data using rules to find top contributors. For DynamoDB, AWS provides a built-in, managed integration that doesn’t require you to configure log groups or write custom rules manually.
When you enable Contributor Insights on a DynamoDB table or Global Secondary Index (GSI), AWS automatically creates two CloudWatch Insight Rules per table/index:
DynamoDBContributorInsights-PKC-[table-name]— Tracks the most accessed partition keys (by consumed capacity)DynamoDBContributorInsights-PKT-[table-name]— Tracks the most throttled partition keys
If your table has a sort key, two additional rules are created:
DynamoDBContributorInsights-SKC-[table-name]— Most accessed partition key + sort key combinationsDynamoDBContributorInsights-SKT-[table-name]— Most throttled partition key + sort key combinations
These rules analyze traffic continuously and report the top contributors over configurable time windows. The data is available with a ~1 minute delay and retains 24 hours of per-minute data and 14 days of per-hour data.
Key Concepts
| Concept | Description |
| Contributor | A unique partition key (or PK+SK combination) that appears in DynamoDB requests |
| Occurrence | The number of times a contributor appeared (i.e., number of requests for that key) |
| Top-N | The N contributors with the highest occurrence count in the selected time window (default N=10, max 500) |
| PKC Rule | Partition Key — most Consumed capacity |
| PKT Rule | Partition Key — most Throttled requests |
Step 1: Enable Contributor Insights on Your DynamoDB Table
Let’s start by enabling Contributor Insights. You can do this via the AWS Console, CLI, SDK, or Infrastructure as Code. Here’s the CLI approach:
# Enable Contributor Insights on a table
aws dynamodb update-contributor-insights \
--table-name Orders \
--contributor-insights-action ENABLE
# Verify it's enabled
aws dynamodb describe-contributor-insights \
--table-name OrdersThe response should show:
{
"TableName": "Orders",
"ContributorInsightsRuleList": [
"DynamoDBContributorInsights-PKC-Orders",
"DynamoDBContributorInsights-PKT-Orders",
"DynamoDBContributorInsights-SKC-Orders",
"DynamoDBContributorInsights-SKT-Orders"
],
"ContributorInsightsStatus": "ENABLED",
"LastUpdateDateTime": "2024-01-15T10:30:00+00:00"
}To enable it on a Global Secondary Index (GSI):
# Enable on a GSI
aws dynamodb update-contributor-insights \
--table-name Orders \
--index-name GSI-CustomerEmail \
--contributor-insights-action ENABLEFor CloudFormation / AWS SAM, add this to your table resource:
# In your CloudFormation template
Resources:
OrdersTable:
Type: AWS::DynamoDB::Table
Properties:
TableName: Orders
AttributeDefinitions:
- AttributeName: orderId
AttributeType: S
- AttributeName: customerId
AttributeType: S
KeySchema:
- AttributeName: orderId
KeyType: HASH
- AttributeName: customerId
KeyType: RANGE
BillingMode: PAY_PER_REQUEST
ContributorInsightsSpecification:
Enabled: trueStep 2: Query Contributor Insights to Find Hot Partition Keys
Once enabled, give it a few minutes to start collecting data. Then query the insight rules using the CloudWatch API. The key API call is cloudwatch:GetInsightRuleReport.
# Find the top 10 most THROTTLED partition keys in the last hour
aws cloudwatch get-insight-rule-report \
--rule-name "DynamoDBContributorInsights-PKT-Orders" \
--start-time $(date -u -d '1 hour ago' +%Y-%m-%dT%H:%M:%S) \
--end-time $(date -u +%Y-%m-%dT%H:%M:%S) \
--period 3600 \
--max-contributor-count 10For the top most accessed (by consumed capacity) partition keys:
# Find the top 10 most accessed partition keys in the last hour
aws cloudwatch get-insight-rule-report \
--rule-name "DynamoDBContributorInsights-PKC-Orders" \
--start-time $(date -u -d '1 hour ago' +%Y-%m-%dT%H:%M:%S) \
--end-time $(date -u +%Y-%m-%dT%H:%M:%S) \
--period 3600 \
--max-contributor-count 10The response includes a Contributors array. Each contributor has:
- Keys: The actual partition key value (and sort key if applicable)
- ApproximateAggregateValue: Total consumed/throttled capacity units for this key
- Datapoints: Time-series breakdown within the queried period
Python Script to Identify Hot Keys
Here’s a practical Python script that queries Contributor Insights and outputs a formatted report of your hottest partition keys:
import boto3
from datetime import datetime, timedelta, timezone
def get_top_contributors(table_name, rule_type="PKT", top_n=10, hours=1):
"""
Retrieve top-N contributors from DynamoDB Contributor Insights.
Args:
table_name: DynamoDB table name
rule_type: PKC (most consumed), PKT (most throttled),
SKC (sort key consumed), SKT (sort key throttled)
top_n: Number of top contributors to return (max 500)
hours: Lookback window in hours
"""
cw_client = boto3.client("cloudwatch")
rule_name = f"DynamoDBContributorInsights-{rule_type}-{table_name}"
end_time = datetime.now(timezone.utc)
start_time = end_time - timedelta(hours=hours)
try:
response = cw_client.get_insight_rule_report(
RuleName=rule_name,
StartTime=start_time,
EndTime=end_time,
Period=3600,
MaxContributorCount=top_n,
Metrics=["UniqueContributors", "MaxContributorValue", "SampleCount"],
OrderBy="Sum"
)
except cw_client.exceptions.ResourceNotFoundException:
print(f"Rule '{rule_name}' not found. Is Contributor Insights enabled?")
return
contributors = response.get("Contributors", [])
if not contributors:
print(f"No {rule_type} contributors found for '{table_name}' in the last {hours}h.")
print("This is good news if checking throttled keys (PKT/SKT)!")
return
label = "THROTTLED" if "T" in rule_type else "CONSUMED"
key_type = "Partition Key + Sort Key" if "SK" in rule_type else "Partition Key"
print(f"\n{'='*70}")
print(f"Top {top_n} Most {label} {key_type}s — Table: {table_name}")
print(f"Time window: {start_time.strftime('%H:%M')} - {end_time.strftime('%H:%M')} UTC")
print(f"{'='*70}")
print(f"{'Rank':<6}{'Key Value':<40}{'Total (CU)':<15}")
print(f"{'-'*6}{'-'*40}{'-'*15}")
for i, contributor in enumerate(contributors, 1):
key_values = " | ".join(contributor["Keys"])
approx_value = contributor["ApproximateAggregateValue"]
print(f"{i:<6}{key_values:<40}{approx_value:<15.1f}")
print(f"\nAggregate stats:")
print(f" Unique contributors: {response.get('ApproximateUniqueCount', 'N/A')}")
print(f" Total aggregate value: {response.get('AggregateValue', 'N/A')}")
if __name__ == "__main__":
TABLE = "Orders"
# Check for throttled partition keys (the most critical check)
get_top_contributors(TABLE, rule_type="PKT", top_n=10, hours=6)
# Check for most accessed partition keys (to find hot keys before they throttle)
get_top_contributors(TABLE, rule_type="PKC", top_n=10, hours=6)
# Check throttled PK+SK combinations
get_top_contributors(TABLE, rule_type="SKT", top_n=10, hours=6)
Example output when throttling is detected:
======================================================================
Top 10 Most THROTTLED Partition Keys — Table: Orders
Time window: 04:30 - 10:30 UTC
======================================================================
Rank Key Value Total (CU)
----------------------------------------------
1 customer-8832 45230.0
2 customer-1194 38102.0
3 customer-0051 12450.0
4 SYSTEM_BATCH_JOB 9870.0
5 customer-7721 3200.0
======================================================================In this example, customer-8832 is clearly your hottest key. The SYSTEM_BATCH_JOB key at rank 4 suggests a batch process is hammering a single partition — a common anti-pattern we'll discuss later.
Step 3: Set Up Automated Alarms for Hot Key Detection
Don't wait for users to report errors. You can create CloudWatch Alarms on Contributor Insights rules to get proactive notifications when a single key starts consuming disproportionate throughput.
Unfortunately, you cannot create alarms directly on Insight Rule reports via the CLI in one step. Instead, use CloudWatch Anomaly Detection on your throttling metrics combined with the Contributor Insights data for investigation. However, you can create a CloudWatch Alarm on the built-in Insight Rule metric using the MaxContributorValue metric:
import boto3
def create_hot_key_alarm(table_name, threshold_cu=1000):
"""
Create a CloudWatch alarm that fires when any single partition key
consumes more than threshold_cu capacity units in a 5-minute window.
"""
cw_client = boto3.client("cloudwatch")
rule_name = f"DynamoDBContributorInsights-PKC-{table_name}"
cw_client.put_metric_alarm(
AlarmName=f"DynamoDB-HotKey-{table_name}",
AlarmDescription=(
f"Alarm when a single partition key in {table_name} "
f"consumes more than {threshold_cu} CU in 5 minutes"
),
Namespace="AWS/DynamoDB",
MetricName="ConsumedWriteCapacityUnits