Why a /health that returns 200 is not enough in serverless

Almost every service has a /health. A monitor calls it every minute, gets a 200 and paints a green dot. On a traditional server that meant something: if the process answers, the process is alive. In serverless the health endpoint is usually just another function, sharing neither process nor memory with the ones doing the real work. It can answer 200 for hours while orders sit in a queue. In this article we go through the failures it misses, what to do instead and how AAA Sentinel handles it.

What a 200 actually checks

This is the most common /health in an API on AWS Lambda. If it answers, you know three things: API Gateway routes, that function starts, and its code does not throw. Nothing more. It has not touched the database, it does not know whether the queue is moving and it has no idea whether the function consuming that queue has been failing for an hour.

export const handler = async () => ({
  statusCode: 200,
  body: JSON.stringify({ status: 'ok' }),
})

Five failures a 200 cannot see

These are the ones that keep coming back in event-driven architectures, and none of them changes the /health response:

• The growing queue: the consumer is slower than what comes in, or it is held back by concurrency. The API accepts orders and answers 202; the work gets done half an hour late.
• Silent retries: a failed asynchronous Lambda invocation is retried up to two more times by default. An SQS message that is not deleted becomes visible again when its visibility timeout expires and is processed again. The system looks fine, doing double or triple the work and the cost.
• The filling dead-letter queue: when a message runs out of receives (maxReceiveCount), SQS moves it to the DLQ. Nobody notices until someone opens it, and every message in there is something that did not happen.
• The event nobody consumes: PutEvents on EventBridge succeeds even if no rule matches the event. Rename the source field or disable a rule, and events vanish without a single error.
• The dependency that is down: the payment provider or the database fails, but /health never calls them, so it stays green.

First piece: a health contract per component

If the health endpoint is going to exist, make it say something useful. There is a format for that: health+json, an IETF draft that uses the application/health+json media type. The response carries an overall status (pass, warn or fail), the service version and a checks object with a list per component, each with its componentType, observedValue, status and the time of the measurement. Each check key usually follows the component:measurement shape.

HTTP/1.1 200 OK
Content-Type: application/health+json
Cache-Control: no-store

{
  "status": "warn",
  "version": "1.42.0",
  "checks": {
    "orders-table:responseTime": [
      {
        "componentType": "datastore",
        "observedValue": 38,
        "observedUnit": "ms",
        "status": "pass",
        "time": "2026-10-18T09:30:00.000Z"
      }
    ],
    "orders-dlq:messages": [
      {
        "componentType": "component",
        "observedValue": 3,
        "status": "warn",
        "time": "2026-10-18T09:30:00.000Z"
      }
    ]
  }
}

• pass: the component works.
• warn: it works, but something is off. Here, three messages are sitting in the dead-letter queue. The service is degraded, not down, and answers 200.
• fail: the component does not work. The service answers 503 so that a load balancer or a basic HTTP monitor gets it too.

How to produce it without writing the format by hand

The @aaa-sentinel/health package on npm, with no dependencies, builds that response from your checks. Each check gets an AbortSignal that fires when it runs out of time; the overall status is the worst of them. This handler produces exactly the response above:

import { runHealthChecks } from '@aaa-sentinel/health'
import { DynamoDBClient, GetItemCommand } from '@aws-sdk/client-dynamodb'
import { SQSClient, GetQueueAttributesCommand } from '@aws-sdk/client-sqs'

const dynamo = new DynamoDBClient({})
const sqs = new SQSClient({})

// Lambda behind API Gateway or a function URL: the result is the HTTP response.
export const handler = () =>
  runHealthChecks(
    {
      'orders-table:responseTime': async (signal) => {
        const started = Date.now()
        await dynamo.send(
          new GetItemCommand({ TableName: process.env.ORDERS_TABLE, Key: { pk: { S: 'health' } } }),
          { abortSignal: signal },
        )
        return { componentType: 'datastore', observedValue: Date.now() - started, observedUnit: 'ms' }
      },
      'orders-dlq:messages': async (signal) => {
        const { Attributes } = await sqs.send(
          new GetQueueAttributesCommand({
            QueueUrl: process.env.ORDERS_DLQ_URL,
            AttributeNames: ['ApproximateNumberOfMessages'],
          }),
          { abortSignal: signal },
        )
        const messages = Number(Attributes?.ApproximateNumberOfMessages ?? 0)
        return { componentType: 'component', observedValue: messages, status: messages === 0 ? 'pass' : 'warn' }
      },
    },
    { version: process.env.APP_VERSION, timeoutMs: 2000 },
  )

• A check that throws or takes longer than timeoutMs (2 s by default, 60 s at most) is a fail for that component; the endpoint still answers.
• Error messages stay out of the response unless you pass exposeErrors: true. They often carry hosts or connection strings, and health endpoints are usually public.
• minimumStatus: 'warn' keeps the service at warn or worse, during maintenance for example.
• If you already use Spring Boot Actuator, ASP.NET Core Health Checks or the Kubernetes endpoints, there is nothing to change: the AAA Sentinel health contract monitor reads them as they are.

Second piece: watch the resources behind it

Some things an endpoint cannot measure well. The age of the oldest message in a queue, for instance, is not returned by GetQueueAttributes: it lives in the ApproximateAgeOfOldestMessage CloudWatch metric. And it is a better signal than message count, because a thousand messages can be a normal peak while a message from forty minutes ago is always a problem. That is why AAA Sentinel has cloud resource monitors: they are passive, call nothing of yours and read metrics through the cloud connection you already gave for costs.

• On Azure it watches Function Apps, Service Bus, Cosmos DB, API Management, Event Grid topics and Logic Apps; on Google Cloud, Cloud Run, Cloud Functions, Pub/Sub subscriptions, Eventarc triggers, Workflows, Firestore and API Gateway.
• The cost lands on your bill, not ours: on AWS, GetMetricData comes to up to about 0.20 USD per resource per month with reads every 10 minutes. On Azure it usually fits within the free million queries a month, and on Google Cloud within the free million time series read a month.

Third piece: heartbeats for crons

A scheduled job has no endpoint to probe. If the EventBridge Scheduler schedule gets disabled or the script dies halfway, there is no error to see: nothing happens at all. The fix is to turn it around. The heartbeat monitor gives you a URL; your job calls it when it finishes successfully, and if the call does not arrive within the window you set, the monitor goes down. It accepts GET and POST, because many schedulers only know how to request a URL.

#!/usr/bin/env bash
set -euo pipefail

node scripts/nightly-billing-export.js

# Only reached when the job succeeded: a failed or skipped run stays silent.
curl -fsS --max-time 10 --retry 3 "$HEARTBEAT_URL" > /dev/null

// Scheduled Lambda (EventBridge Scheduler): the ping is the last line, after the real work.
export const handler = async () => {
  await exportBilling()
  await fetch(process.env.HEARTBEAT_URL, { method: 'POST', signal: AbortSignal.timeout(5000) })
}

Fourth piece: flows that name the broken step

The three previous pieces look at single components. A flow looks at the whole chain: AAA Sentinel fires a test transaction (an SQS message, a Lambda invocation, an EventBridge event or an HTTP call) carrying a 32-hex traceId and a W3C traceparent, and checks it reaches every step within its deadline. A step can be a DynamoDB item, an S3 object, a line in CloudWatch Logs or a call from your code to a URL of ours. There are 1 to 5 steps, each with a deadline between 10 seconds and 15 minutes, and each hop is timed from the previous one. Flows run on AWS today, at most every 15 minutes, with a monthly cost cap per flow.

import { propagateTrace } from '@aaa-sentinel/health'
import { DynamoDBClient } from '@aws-sdk/client-dynamodb'
import { DynamoDBDocumentClient, PutCommand } from '@aws-sdk/lib-dynamodb'

const db = DynamoDBDocumentClient.from(new DynamoDBClient({}))

export const handler = async (event) => {
  for (const record of event.Records) {
    const trace = propagateTrace(record) // null for real traffic

    if (trace) {
      // Step 1 looks for ORDER#{traceId}; step 2 for the line "order stored {traceId}".
      await db.send(new PutCommand({ TableName: process.env.ORDERS_TABLE, Item: { pk: `ORDER#${trace.traceId}`, test: true } }))
      trace.log('order stored')
      continue // no charge, no email for the test transaction
    }

    const order = JSON.parse(record.body)
    await db.send(new PutCommand({ TableName: process.env.ORDERS_TABLE, Item: { pk: `ORDER#${order.id}`, ...order } }))
    await chargeCustomer(order)
  }
}

• Your code passes the traceId on to the next step, in the x-aaa-trace header or as a message attribute. propagateTrace finds it in SQS, SNS, EventBridge, DynamoDB Streams, Kinesis and API Gateway events or in the headers of an HTTP request, and never throws.
• Only the AAA Sentinel mark turns a request into a test: a traceparent from your own OpenTelemetry is real traffic.
• The flow uses a separate role, with its own CloudFormation template, that can only write to the resources you choose.
• The event nobody consumes shows up here: the trigger succeeds and the first step never arrives.

How AAA Sentinel decides something is down

A false alert at three in the morning teaches the team to ignore the next ones. So a failure does not alert on the first try. Each monitor moves through these states: up, suspect, confirming, down, recovering and back to up. Nothing is ever sent from suspect. A URL is checked again from a second AWS region (probes run from eu-west-1 and confirmation from eu-central-1); a resource or a flow has to fail several times in a row. Only down opens an incident and sends the alert, via email, Slack, Microsoft Teams or a signed webhook, with the same channels and policies as cost alerts.

• A health+json warn (or an ASP.NET Degraded) leaves the service degraded, not down. An Actuator UNKNOWN is not a failure.
• New monitors spend 24 hours learning: they observe, do not alert and propose thresholds.
• Incidents open and close on their own, with a timeline, notes, acknowledgement and a per-minute cost you set, in your organisation's currency.
• Every probe carries the X-AAA-Signature header, signed with a secret of your organisation, so your API can recognise it. We do not publish fixed IPs, so the signature is how you identify it.

What it does not do

So you know where the limits are:

• It is not an APM: it does not ingest logs, do distributed tracing, store custom metrics or offer custom dashboards.
• On Azure the flow wizard does not discover the chain: you tell it where the test enters. On Google Cloud the trigger enters through Pub/Sub or HTTP.
• The packages are for Node.js (@aaa-sentinel/health) and Python (aaa-sentinel-health). In other languages, your code reads the x-aaa-trace or aaaTraceId mark itself.
• It probes from two AWS regions, not from a network of locations around the world.

Checklist to get started

With these pieces in place, a green dot means again that orders reach their destination.

• Your /health answers health+json with one check per dependency, warn for degraded and 503 for down.
• Every queue has its DLQ and a monitor on the age of the oldest message.
• Functions that consume events have their errors and throttles watched.
• Every cron calls its heartbeat URL when it finishes successfully.
• The chain that matters most (the order, the payment, the invoice) has a flow that names the step that breaks.
• The 3 free http or heartbeat monitors on every plan cover /health and the most critical cron; the monitoring pack adds the rest.