AWS Lambda Destinations: Route Asynchronous Invocation Results Without Extra Code

AWS Lambda / Leave a Comment

Intermediate

Introduction: The Problem with Async Lambda Error Handling

If you’ve built event-driven architectures on AWS, you’ve probably written this pattern before: a Lambda function that catches its own errors, formats the result, and then explicitly calls another service — maybe pushes a message to SQS on failure, or invokes another Lambda on success. You end up with boto3 calls littered through your handler, extra IAM permissions to manage, and error-handling code that’s longer than your actual business logic.

AWS Lambda Destinations, launched in November 2019, solve this problem cleanly. They let you configure where Lambda should automatically send the result of an asynchronous invocation — both on success and on failure — without writing a single line of routing code in your function.

This article is for intermediate AWS developers who are already comfortable writing Lambda functions and want to build cleaner, more resilient async workflows. You should have a working understanding of Lambda invocation types, IAM roles, and at least one of the supported destination services (SQS, SNS, EventBridge, or another Lambda function).

By the end, you’ll understand exactly how destinations work under the hood, how to configure them via CLI, CloudFormation, and CDK, and — critically — the common mistakes that trip people up.

Prerequisites

  • An AWS account with permissions to create Lambda functions, SQS queues, SNS topics, and IAM roles
  • AWS CLI v2 installed and configured (aws --version should return 2.x)
  • Python 3.12 runtime (examples use Python, but destinations are runtime-agnostic)
  • Basic familiarity with asynchronous Lambda invocations (InvocationType: Event)

How Lambda Destinations Work: The Mechanics

When you invoke a Lambda function asynchronously (using InvocationType: Event), Lambda places the event in an internal queue and returns a 202 immediately. Your function executes independently. Before destinations existed, if the function failed, Lambda would retry it twice (by default), and then the event was simply lost — unless you configured a Dead Letter Queue (DLQ).

Destinations extend this model. You can configure two separate routing targets:

  • OnSuccess — where Lambda sends the result when your function completes without error
  • OnFailure — where Lambda sends the result after all retry attempts are exhausted

The supported destination types are:

Destination Type ARN Format Typical Use Case
Amazon SQS arn:aws:sqs:region:account-id:queue-name Buffering results for downstream processing
Amazon SNS arn:aws:sns:region:account-id:topic-name Fan-out notifications on success/failure
AWS Lambda arn:aws:lambda:region:account-id:function:name Chaining functions in a pipeline
Amazon EventBridge arn:aws:events:region:account-id:event-bus/bus-name Complex routing via event rules

What Lambda Actually Sends to the Destination

This is critical to understand. Lambda doesn’t just forward your function’s return value. It wraps everything in an invocation record that looks like this:

{
  "version": "1.0",
  "timestamp": "2024-01-15T10:30:00.000Z",
  "requestContext": {
    "requestId": "a1b2c3d4-5678-90ab-cdef-EXAMPLE11111",
    "functionArn": "arn:aws:lambda:us-east-1:123456789012:function:MyFunction:$LATEST",
    "condition": "Success",
    "approximateInvokeCount": 1
  },
  "requestPayload": {
    "key1": "value1"
  },
  "responseContext": {
    "statusCode": 200,
    "executedVersion": "$LATEST"
  },
  "responsePayload": {
    "result": "processed successfully"
  }
}

For failures, the condition field will be "RetriesExhausted", and responsePayload will contain the error message and stack trace. The requestPayload is always included — this is a major advantage over DLQs, which only send limited metadata.

Setting Up Destinations: Three Ways

Method 1: AWS CLI

Let’s build a concrete example. We’ll create a Lambda function that processes orders, route successes to an SQS queue, and route failures to an SNS topic for alerting.

First, create the destination resources:

# Create the success destination queue
aws sqs create-queue \
  --queue-name order-processing-success

# Create the failure destination topic
aws sns create-topic \
  --name order-processing-failures

# Subscribe your email to the failure topic (for alerts)
aws sns subscribe \
  --topic-arn arn:aws:sns:us-east-1:123456789012:order-processing-failures \
  --protocol email \
  --notification-endpoint your-email@example.com

Now create the Lambda function. Here’s a simple handler:

import json
import random

def lambda_handler(event, context):
    order_id = event.get("order_id")
    
    if not order_id:
        raise ValueError("order_id is required")
    
    # Simulate processing that occasionally fails
    if random.random() < 0.3:
        raise RuntimeError(f"Failed to process order {order_id}: upstream timeout")
    
    return {
        "statusCode": 200,
        "order_id": order_id,
        "status": "processed"
    }

The function's execution role needs permissions to send to your destinations. Add this policy:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "sqs:SendMessage",
      "Resource": "arn:aws:sqs:us-east-1:123456789012:order-processing-success"
    },
    {
      "Effect": "Allow",
      "Action": "sns:Publish",
      "Resource": "arn:aws:sns:us-east-1:123456789012:order-processing-failures"
    }
  ]
}

Now configure the destinations using put-function-event-invoke-config:

# Configure both success and failure destinations
aws lambda put-function-event-invoke-config \
  --function-name order-processor \
  --destination-config '{
    "OnSuccess": {
      "Destination": "arn:aws:sqs:us-east-1:123456789012:order-processing-success"
    },
    "OnFailure": {
      "Destination": "arn:aws:sns:us-east-1:123456789012:order-processing-failures"
    }
  }'

You can also configure retry behavior in the same command:

aws lambda put-function-event-invoke-config \
  --function-name order-processor \
  --maximum-retry-attempts 1 \
  --maximum-event-age-in-seconds 3600 \
  --destination-config '{
    "OnSuccess": {
      "Destination": "arn:aws:sqs:us-east-1:123456789012:order-processing-success"
    },
    "OnFailure": {
      "Destination": "arn:aws:sns:us-east-1:123456789012:order-processing-failures"
    }
  }'

Test it with an async invocation:

# The --invocation-type Event flag is crucial — destinations only work with async
aws lambda invoke \
  --function-name order-processor \
  --invocation-type Event \
  --payload '{"order_id": "ORD-12345"}' \
  --cli-binary-format raw-in-base64-out \
  response.json

You should get a 202 status code. Check your SQS queue or SNS subscription for the result.

Method 2: AWS CloudFormation

Resources:
  OrderProcessor:
    Type: AWS::Lambda::Function
    Properties:
      FunctionName: order-processor
      Runtime: python3.12
      Handler: index.lambda_handler
      Role: !GetAtt OrderProcessorRole.Arn
      Code:
        ZipFile: |
          def lambda_handler(event, context):
              order_id = event.get("order_id")
              if not order_id:
                  raise ValueError("order_id is required")
              return {"statusCode": 200, "order_id": order_id, "status": "processed"}

  SuccessQueue:
    Type: AWS::SQS::Queue
    Properties:
      QueueName: order-processing-success

  FailureTopic:
    Type: AWS::SNS::Topic
    Properties:
      TopicName: order-processing-failures

  OrderProcessorEventInvokeConfig:
    Type: AWS::Lambda::EventInvokeConfig
    Properties:
      FunctionName: !Ref OrderProcessor
      Qualifier: $LATEST
      MaximumRetryAttempts: 1
      MaximumEventAgeInSeconds: 3600
      DestinationConfig:
        OnSuccess:
          Destination: !GetAtt SuccessQueue.Arn
        OnFailure:
          Destination: !Ref FailureTopic

Method 3: AWS CDK (TypeScript)

from aws_cdk import (
    Stack,
    Duration,
    aws_lambda as lambda_,
    aws_sqs as sqs,
    aws_sns as sns,
    aws_lambda_destinations as destinations,
)
from constructs import Construct

class OrderProcessingStack(Stack):
    def __init__(self, scope: Construct, id: str, **kwargs):
        super().__init__(scope, id, **kwargs)

        success_queue = sqs.Queue(self, "SuccessQueue",
            queue_name="order-processing-success"
        )

        failure_topic = sns.Topic(self, "FailureTopic",
            topic_name="order-processing-failures"
        )

        order_processor = lambda_.Function(self, "OrderProcessor",
            function_name="order-processor",
            runtime=lambda_.Runtime.PYTHON_3_12,
            handler="index.lambda_handler",
            code=lambda_.Code.from_inline(
                "def lambda_handler(event, context):\n"
                "    return {'statusCode': 200, 'order_id': event['order_id']}"
            ),
            on_success=destinations.SqsDestination(success_queue),
            on_failure=destinations.SnsDestination(failure_topic),
            retry_attempts=1,
            max_event_age=Duration.hours(1),
        )

Notice how CDK handles the IAM permissions automatically — it grants the Lambda function's role the necessary sqs:SendMessage and sns:Publish permissions. This is one of CDK's biggest wins for destinations.

Destinations vs. Dead Letter Queues: When to Use Which

This is the question everyone asks. Here's the definitive comparison:

Feature Destinations Dead Letter Queues (DLQ)
Supported conditions Success AND Failure Failure only
Payload content Full invocation record (request + response + metadata) Original event payload only
Supported targets SQS, SNS, Lambda, EventBridge SQS, SNS only
Works with Asynchronous invocations only Asynchronous invocations only
Can coexist Yes — both can be configured on the same function

My recommendation: Use destinations as your default. They're strictly more powerful than DLQs. The only reason to keep a DLQ is if you have existing tooling built around the DLQ message format, or if you specifically need the DLQ behavior where Lambda itself sends the original event without wrapping it.

When both are configured, Lambda sends to the OnFailure destination first, then to the DLQ. Both receive the message.

Common Mistakes and How to Avoid Them

Mistake #1: Using Synchronous Invocations and Wondering Why Destinations Don't Fire

This is the number one issue. Destinations only work with asynchronous invocations. If you're invoking Lambda from API Gateway (default synchronous), the SDK with default RequestResponse invocation type, or the console's "Test" button, destinations will never trigger.

Async invocation sources include: S3 event notifications, SNS, EventBridge, CloudWatch Events, InvocationType: Event via the API, and certain other services. Always verify your invocation type.

# WRONG — this is synchronous, destinations won't fire
aws lambda invoke \
  --function-name order-processor \
  --payload '{"order_id": "ORD-12345"}' \
  --cli-binary-format raw-in-base64-out \
  response.json

# CORRECT — async invocation
aws lambda invoke \
  --function-name order-processor \
  --invocation-type Event \
  --payload '{"order_id": "ORD-12345"}' \
  --cli-binary-format raw-in-base64-out \
  response.json

Mistake #2: Missing IAM Permissions on the Function's Execution Role

Lambda uses the function's execution role to send to the destination. If the role doesn't have the right permissions, the destination delivery silently fails. There's

Must Read

Leave a Comment

Your email address will not be published. Required fields are marked *