Amazon CloudWatch

In brief: Connect CloudWatch with temporary STS or dedicated read credentials, then map an Application Signals service or API Gateway stage to an OpenAPI service.

The CloudWatch integration discovers routes and reads aggregate metrics from AWS Application Signals or Amazon API Gateway.

What TestChimp reads

Source	Route identity	Metrics
Application Signals	Service and Operation dimensions	Request count and p95 latency
API Gateway	ApiName, optional Stage, Method, and Resource dimensions	Request count, 5xx count, and p95 latency

The resources must expose route-level dimensions. Metrics aggregated only at load balancer, target group, function, or service level cannot map individual OpenAPI operations.

IAM permissions

Use temporary AWS STS credentials where possible. The connection needs:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "cloudwatch:ListMetrics",
        "cloudwatch:GetMetricData",
        "sts:GetCallerIdentity"
      ],
      "Resource": "*"
    }
  ]
}

Do not use root credentials or an administrator access key. If temporary credentials are not available, create a dedicated least-privilege IAM user and rotate its key regularly.

Configure TestChimp

In Project Settings → Integrations → Observability:

1. Select CloudWatch.
2. Enter the Access key ID.
3. Enter the Secret access key.
4. For temporary STS credentials, enter the Session token. Leave it blank for an IAM user access key.
5. Enter the AWS Region containing the metrics.
6. Choose Test Connection and save.
7. In APIs, map the OpenAPI service to an Application Signals service or API Gateway API/stage.

Credentials and the selected region must belong to the same AWS partition and account that contains the metrics.

Prepare Application Signals

Enable Application Signals for the workload and ensure the Operation dimension is a stable value in the form METHOD /route/{parameter}. Confirm the AWS/ApplicationSignals namespace contains Latency and Call metrics for the service.

Prepare API Gateway

Enable detailed CloudWatch metrics for the API stage. Confirm the AWS/ApiGateway namespace contains Count, 5XXError, and Latency with the Method and Resource dimensions.

Troubleshooting

• Connection succeeds but no resources appear — verify the region and confirm metrics exist in AWS/ApplicationSignals or AWS/ApiGateway.
• API Gateway appears without routes — enable detailed metrics on the stage and generate traffic.
• Routes do not map — compare the CloudWatch route template and HTTP method with the OpenAPI path.
• Temporary credentials stop working — replace the access key, secret, and session token before the STS session expires.
• Access denied — verify all three actions in the policy and check for organization SCP or permission-boundary restrictions.