AWS Canary Credentials: How to Set Up an IAM Honeytoken

1. What a canary credential is, and why it's the cheapest certainty

A honeypot is any decoy built to be touched only by an intruder. A honeytoken is a decoy piece of data, such as a credential, a document or a database row, rather than a whole decoy system. A canary token is a honeytoken that alerts you when it's used. An AWS canary credential is all three: a real IAM access key (AKIA…) attached to a user that is explicitly denied everything.

The point isn't the decoy, it's the quality of the signal. Most alerts measure a symptom: CPU, egress, error rate, cost. They fire after something has started to go wrong, against a baseline that legitimate work (and attackers) move around in. That's why a human has to triage them, and why automating a response to them is risky. If one alert in a thousand is an attack, an automatic kill switch is wrong 999 times.

A canary trip measures intent. Someone found a credential, decided it was worth stealing, and used it. There's no baseline to tune, and because each key lives in exactly one place, the key ID tells you where it was taken from. That's a signal you can act on immediately, including automatically, as long as the action is one you can undo.

Canaries aren't the only high-confidence signal. A GuardDuty finding for instance credentials used outside AWS, or a Falco rule for a shell spawned in a production container, can be just as trustworthy, and if you trust them you should automate on them too. The rule is to automate on certainty. Canaries are simply the cheapest way to manufacture certainty: no agents, they work on ECS Fargate and Lambda, and one takes an afternoon. They don't replace your alerts. Keep both.

2. Your options: canarytokens.org, Thinkst Canary, DIY, Compass

If you're unsure, start with one free token today and build the DIY version this week. The rest of this guide is the DIY version.

3. How detection works in AWS: the facts that matter

The whole design rests on a few AWS behaviours. Each one below is checked against AWS documentation, which is linked.

• CloudTrail records calls made with the key, including denied ones. A call refused by the deny-all policy is still an authenticated API call. It's logged with errorCode (for example AccessDenied), plus sourceIPAddress, userAgent and userIdentity.accessKeyId. (CloudTrail record contents.)
• sts:GetCallerIdentity can't be denied, and it is logged. AWS says no permissions are required and that an explicit deny doesn't stop it (API reference). CloudTrail captures all IAM and STS API calls (IAM user guide). This is the "whoami" call that attackers and secret scanners make first. Your canary succeeds at it, which helps it look real, and the call still lands in your logs.
• EventBridge ignores read-only calls unless you opt in. CloudTrail sends "AWS API Call via CloudTrail" events to the default event bus. A rule in the normal ENABLED state matches write events but not read-only management events such as Get*, List*, Describe* and GetCallerIdentity. Only a rule in the ENABLED_WITH_ALL_CLOUDTRAIL_MANAGEMENT_EVENTS state matches those (EventBridge: events via CloudTrail, read-only management events). In Terraform this is the state argument of aws_cloudwatch_event_rule, added in AWS provider 5.27.0 (it replaces the deprecated is_enabled). This is the single most common bug in DIY AWS canaries: a rule that only catches the attacker's second, mutating call, if they ever make one.
• You need a trail. EventBridge only receives CloudTrail-sourced events if a trail with logging is enabled. Event history alone isn't enough (same page).
• Events land in the region that was called. EventBridge rules are regional. A call to a regional endpoint such as sts.eu-west-1.amazonaws.com is logged in eu-west-1. Calls to the global STS endpoint are logged in us-east-1 (IAM user guide). The attacker chooses the region, not you.
• Forwarding read-only events needs the special state twice. If you forward events from one region's default bus to another bus, both the forwarding rule and the destination rule need ENABLED_WITH_ALL_CLOUDTRAIL_MANAGEMENT_EVENTS (same page).
• CloudTrail isn't instant. AWS says CloudTrail typically delivers events to S3 and CloudWatch Logs within an average of about 5 minutes, and that this isn't guaranteed (CloudTrail to CloudWatch Logs). One vendor's measurement of S3 delivery found an average around 2.5 minutes and a 99th percentile just over 5 (Tracebit). AWS doesn't publish a latency figure for the EventBridge path, so measure your own (section 9).

4. Step 1: the canary user and key

Use one IAM user per placement. When a key trips, the user name and key ID should map to exactly one location. Name it like your other service users. Anything visible in AWS (name, path, description, tags) can be read by whoever holds a credential with iam:Get* or iam:List*, so none of it should say "canary".

terraform {
  required_version = ">= 1.5"
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = ">= 5.27, < 6.0" # 5.27.0 added aws_cloudwatch_event_rule.state
    }
  }
}

# Home region. Global-endpoint IAM/STS calls are logged here.
provider "aws" {
  region = "us-east-1"
}

data "aws_caller_identity" "current" {}

locals {
  account_id  = data.aws_caller_identity.current.account_id
  canary_name = "svc-backup-reporting" # match your real naming style
  home_bus    = "arn:aws:events:us-east-1:${local.account_id}:event-bus/default"
}

resource "aws_iam_user" "canary" {
  name = local.canary_name
  path = "/service/"
  # No aws_iam_user_login_profile: no console password, API key only.
  # No tags or descriptions that give the game away.
}

resource "aws_iam_user_policy" "deny_all" {
  name = "backup-reporting"
  user = aws_iam_user.canary.name
  policy = jsonencode({
    Version   = "2012-10-17"
    Statement = [{ Effect = "Deny", Action = "*", Resource = "*" }]
  })
}

resource "aws_iam_access_key" "canary" {
  user = aws_iam_user.canary.name
}

output "canary_access_key_id" {
  value = aws_iam_access_key.canary.id
}

output "canary_secret_access_key" {
  value     = aws_iam_access_key.canary.secret
  sensitive = true
}

Two notes. First, the secret ends up in Terraform state in plain text. For a key that can do nothing that's usually acceptable, and anyone who steals it from state trips it. If your policy forbids it, use the pgp_key argument or create the key outside Terraform. Second, an explicit Deny wins over any Allow, so even if someone later attaches a policy to this user by mistake, the key stays useless.

5. Step 2a: fast alerts with EventBridge, in every region

Match on the user's ARN rather than the key ID. If an attacker exchanges the key for a session token, later calls carry a temporary ASIA… key ID but the same user ARN. The home-region rule sends matches to an SNS topic. Every other region gets a copy of the rule that forwards matches to the home region's default bus.

variable "alert_email" {
  type = string
}

resource "aws_sns_topic" "alerts" {
  name = "ops-notifications"
}

resource "aws_sns_topic_subscription" "email" {
  topic_arn = aws_sns_topic.alerts.arn
  protocol  = "email"
  endpoint  = var.alert_email # click the confirmation link AWS sends
}

# Let EventBridge rules and CloudWatch alarms in this account publish.
data "aws_iam_policy_document" "alerts" {
  statement {
    sid       = "EventBridgePublish"
    actions   = ["sns:Publish"]
    resources = [aws_sns_topic.alerts.arn]
    principals {
      type        = "Service"
      identifiers = ["events.amazonaws.com"]
    }
    condition {
      test     = "ArnLike"
      variable = "aws:SourceArn"
      values   = ["arn:aws:events:us-east-1:${local.account_id}:rule/*"]
    }
  }
  statement {
    sid       = "CloudWatchAlarmPublish"
    actions   = ["sns:Publish"]
    resources = [aws_sns_topic.alerts.arn]
    principals {
      type        = "Service"
      identifiers = ["cloudwatch.amazonaws.com"]
    }
    condition {
      test     = "ArnLike"
      variable = "aws:SourceArn"
      values   = ["arn:aws:cloudwatch:us-east-1:${local.account_id}:alarm:*"]
    }
  }
}

resource "aws_sns_topic_policy" "alerts" {
  arn    = aws_sns_topic.alerts.arn
  policy = data.aws_iam_policy_document.alerts.json
}

locals {
  canary_pattern = jsonencode({
    "detail-type" = ["AWS API Call via CloudTrail"]
    detail = {
      userIdentity = {
        arn = [aws_iam_user.canary.arn]
      }
    }
  })
}

# Home region rule. It also matches events forwarded from other regions.
resource "aws_cloudwatch_event_rule" "canary_home" {
  name          = "backup-reporting-activity"
  state         = "ENABLED_WITH_ALL_CLOUDTRAIL_MANAGEMENT_EVENTS"
  event_pattern = local.canary_pattern
}

resource "aws_cloudwatch_event_target" "canary_home_sns" {
  rule = aws_cloudwatch_event_rule.canary_home.name
  arn  = aws_sns_topic.alerts.arn
}

# Role EventBridge uses to forward events from other regions to us-east-1.
resource "aws_iam_role" "events_forwarder" {
  name = "events-forwarder"
  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Effect    = "Allow"
      Principal = { Service = "events.amazonaws.com" }
      Action    = "sts:AssumeRole"
    }]
  })
}

resource "aws_iam_role_policy" "events_forwarder" {
  name = "put-events-home-bus"
  role = aws_iam_role.events_forwarder.id
  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Effect   = "Allow"
      Action   = "events:PutEvents"
      Resource = local.home_bus
    }]
  })
}

# Repeat this block for EVERY region enabled in the account.
# eu-west-1 shown. List yours with: aws account list-regions --region-opt-status-contains ENABLED ENABLED_BY_DEFAULT
provider "aws" {
  alias  = "eu_west_1"
  region = "eu-west-1"
}

resource "aws_cloudwatch_event_rule" "canary_eu_west_1" {
  provider      = aws.eu_west_1
  name          = "backup-reporting-activity"
  state         = "ENABLED_WITH_ALL_CLOUDTRAIL_MANAGEMENT_EVENTS"
  event_pattern = local.canary_pattern
}

resource "aws_cloudwatch_event_target" "canary_eu_west_1" {
  provider = aws.eu_west_1
  rule     = aws_cloudwatch_event_rule.canary_eu_west_1.name
  arn      = local.home_bus
  role_arn = aws_iam_role.events_forwarder.arn
}

Wrap the per-region pair in a small module and instantiate it once per region with a provider alias. (AWS provider v6 adds a per-resource region argument that lets you for_each over regions instead. This guide sticks to v5.) Email is the simplest target to show. In practice, subscribe your paging tool to the topic, or point the rule's target at an SQS queue or Lambda that posts to Slack.

EventBridge still needs a trail that is logging (section 3). Most accounts already have one. If yours doesn't, the trail in step 2b covers it. Configure it for both read and write management events. We haven't verified whether a write-only trail still feeds read-only events to EventBridge, so don't rely on it.

6. Step 2b: the backstop, a multi-region trail with a metric filter

A multi-region trail sends events from every enabled region to one CloudWatch Logs log group (AWS docs). A metric filter on the canary's ARN, plus an alarm, catches calls in any region, including regions you forget to add rules for. It's slower, because it waits for CloudTrail's delivery to CloudWatch Logs. If you already have a multi-region trail, add the CloudWatch Logs settings to it instead of creating a second one. CloudTrail bills additional copies of management events (pricing).

locals {
  trail_name = "management-events"
  trail_arn  = "arn:aws:cloudtrail:us-east-1:${local.account_id}:trail/${local.trail_name}"
}

resource "aws_s3_bucket" "trail" {
  bucket_prefix = "audit-logs-"
}

resource "aws_s3_bucket_public_access_block" "trail" {
  bucket                  = aws_s3_bucket.trail.id
  block_public_acls       = true
  block_public_policy     = true
  ignore_public_acls      = true
  restrict_public_buckets = true
}

data "aws_iam_policy_document" "trail_bucket" {
  statement {
    sid       = "AWSCloudTrailAclCheck"
    actions   = ["s3:GetBucketAcl"]
    resources = [aws_s3_bucket.trail.arn]
    principals {
      type        = "Service"
      identifiers = ["cloudtrail.amazonaws.com"]
    }
    condition {
      test     = "StringEquals"
      variable = "aws:SourceArn"
      values   = [local.trail_arn]
    }
  }
  statement {
    sid       = "AWSCloudTrailWrite"
    actions   = ["s3:PutObject"]
    resources = ["${aws_s3_bucket.trail.arn}/AWSLogs/${local.account_id}/*"]
    principals {
      type        = "Service"
      identifiers = ["cloudtrail.amazonaws.com"]
    }
    condition {
      test     = "StringEquals"
      variable = "s3:x-amz-acl"
      values   = ["bucket-owner-full-control"]
    }
    condition {
      test     = "StringEquals"
      variable = "aws:SourceArn"
      values   = [local.trail_arn]
    }
  }
}

resource "aws_s3_bucket_policy" "trail" {
  bucket = aws_s3_bucket.trail.id
  policy = data.aws_iam_policy_document.trail_bucket.json
}

resource "aws_cloudwatch_log_group" "trail" {
  name              = "cloudtrail/management-events"
  retention_in_days = 90
}

resource "aws_iam_role" "trail_to_logs" {
  name = "cloudtrail-to-cloudwatch-logs"
  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Effect    = "Allow"
      Principal = { Service = "cloudtrail.amazonaws.com" }
      Action    = "sts:AssumeRole"
    }]
  })
}

resource "aws_iam_role_policy" "trail_to_logs" {
  name = "write-trail-log-group"
  role = aws_iam_role.trail_to_logs.id
  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Effect   = "Allow"
      Action   = ["logs:CreateLogStream", "logs:PutLogEvents"]
      Resource = "${aws_cloudwatch_log_group.trail.arn}:*"
    }]
  })
}

resource "aws_cloudtrail" "main" {
  name                          = local.trail_name
  s3_bucket_name                = aws_s3_bucket.trail.id
  is_multi_region_trail         = true
  include_global_service_events = true
  enable_log_file_validation    = true
  cloud_watch_logs_group_arn    = "${aws_cloudwatch_log_group.trail.arn}:*"
  cloud_watch_logs_role_arn     = aws_iam_role.trail_to_logs.arn

  event_selector {
    read_write_type           = "All" # read-only calls like GetCallerIdentity included
    include_management_events = true
  }

  depends_on = [aws_s3_bucket_policy.trail, aws_iam_role_policy.trail_to_logs]
}

resource "aws_cloudwatch_log_metric_filter" "canary" {
  name           = "backup-reporting-activity"
  log_group_name = aws_cloudwatch_log_group.trail.name
  pattern        = "{ $.userIdentity.arn = \"${aws_iam_user.canary.arn}\" }"

  metric_transformation {
    name      = "BackupReportingActivity"
    namespace = "Ops"
    value     = "1"
  }
}

resource "aws_cloudwatch_metric_alarm" "canary" {
  alarm_name          = "backup-reporting-activity"
  namespace           = "Ops"
  metric_name         = "BackupReportingActivity"
  statistic           = "Sum"
  period              = 60
  evaluation_periods  = 1
  threshold           = 1
  comparison_operator = "GreaterThanOrEqualToThreshold"
  treat_missing_data  = "notBreaching"
  alarm_actions       = [aws_sns_topic.alerts.arn]
}

An alarm notifies when it moves into ALARM, not on every matching event. A second trip while it's still in alarm won't page again, so use the EventBridge path for per-event detail and treat the alarm as the "something happened somewhere" backstop.

If you can afford both, run both: 2a for speed, 2b so a missed region can't hide a trip.

7. Account-ID fingerprinting: which account should own the canary?

Anyone holding an access key ID can find out which AWS account owns it, without ever using the key. sts:GetAccessKeyInfo returns the account ID for any key ID (API reference), and the call is logged only in the caller's account, not yours (Hacking the Cloud). The account ID can also be decoded offline from the key ID itself, a technique published by Aidan Steele and later refined by Tal Be'ery. That's how TruffleHog spots canarytokens.org keys without tripping them.

So the account a canary lives in is part of the disguise:

• In the same account as the workload. The key matches the account ID an attacker sees everywhere else in the environment (ARNs, task metadata), so it looks like it belongs. The cost is a deny-all IAM user in production. It's harmless, but it's one more principal to explain to auditors.
• In a dedicated canary account. This keeps production clean and makes the canary easy to manage centrally. The cost is that its account ID matches nothing else the attacker has found. Give the account a believable name in your organisation and don't reuse it for anything that would make it look empty.
• In a third party's account (canarytokens.org). This is the easiest option, and also the easiest for tools to recognise.

There's no universally right answer. Pick one deliberately, and write the choice down next to the Terraform.

8. Where to plant canaries, and where not to

Good places, where an attacker who got in would look:

• CI runners: a pipeline variable with a plausible but non-default name, such as BACKUP_AWS_ACCESS_KEY_ID and BACKUP_AWS_SECRET_ACCESS_KEY.
• Build boxes and bastions: a named profile in ~/.aws/credentials, for example [backup-prod]. Not [default].
• Containers (ECS, Fargate, Kubernetes): extra env vars in the task definition, or a decoy .env file in the image. Malware routinely dumps the environment, and on Fargate this is the canary with no agent at all.
• Secrets stores: a decoy secret named like your real ones, holding the canary key.
• Old internal repos, wikis and runbooks: places where leaked keys realistically sit for years.

Bad places, which cause false trips or outages:

• Standard variable names in a workload that uses an AWS SDK. The SDK default credential chain reads AWS_ACCESS_KEY_ID before the container or instance role. You'd silently replace the task's real credentials with a deny-all key, break the service, and trip the canary yourself.
• A [default] profile on any machine where the CLI, Terraform or an SDK runs.
• Anywhere your own secret scanner verifies keys. Scanners that validate AWS keys typically call GetCallerIdentity. Either exclude the canary locations or recognise your scanner's user agent in the alert.
• Public repos, unless you want to measure the internet. Public credentials are found and tried by bots, and AWS itself may react to keys exposed publicly. You'll get trips that say nothing about your own environment.
• Backups or images you share with vendors, where a legitimate third party may reasonably test what they find.

One key per location, always. When it trips, the key is the address.

9. Testing it safely

Tell whoever is on call first. Run the test from a machine whose IP you know, in a subshell so the credentials don't leak into your session:

(
  export AWS_ACCESS_KEY_ID="$(terraform output -raw canary_access_key_id)"
  export AWS_SECRET_ACCESS_KEY="$(terraform output -raw canary_secret_access_key)"
  unset AWS_SESSION_TOKEN AWS_PROFILE
  date -u +%H:%M:%S

  # 1. Read-only, can't be denied: succeeds. Proves the read-only state works.
  aws sts get-caller-identity --region us-east-1

  # 2. Denied call: fails with AccessDenied, and is still logged.
  aws s3api list-buckets --region us-east-1

  # 3. A regional endpoint in another region: proves multi-region coverage.
  aws sts get-caller-identity --region eu-west-1 \
    --endpoint-url https://sts.eu-west-1.amazonaws.com
)

You should get three alerts, or one alarm for path 2b. Note the time each arrives and compare it with the event's eventTime. That difference is your latency, which is the only number worth quoting. If test 1 produces nothing, the rule is missing ENABLED_WITH_ALL_CLOUDTRAIL_MANAGEMENT_EVENTS. If test 3 produces nothing, eu-west-1 has no rule. To confirm CloudTrail saw the calls regardless of alerting:

aws cloudtrail lookup-events --region us-east-1 \
  --lookup-attributes AttributeKey=AccessKeyId,AttributeValue=AKIA...

Event history is regional, so run the lookup with --region eu-west-1 as well to see the third call.

Repeat the test after every change to regions, trails or alert routing, and on a schedule. A canary that silently stopped alerting is worse than none, because you believe you're covered.

10. What to do when it trips

1. Read the event: eventName, sourceIPAddress, userAgent, awsRegion. Rule out your own test or your own scanner in under a minute. Everything else is real until proven otherwise.
2. Go to the source, not the key. The key tells you where it was planted: that runner, that task definition, that laptop. Whatever could read the canary could also read the real credentials stored next to it.
3. Contain that source. Stop or isolate the workload (on ECS, stopping the task lets the service replace it from a clean image). Revoke active sessions for the role it ran as, and rotate any real secrets it could reach.
4. Leave the canary key active for now. It can't do anything, and every further call it makes shows you what the attacker is trying. Deactivating it immediately also tells them they've been seen.
5. Afterwards, burn it and plant a fresh one. A used canary is a known canary.

Steps 2 and 3 are good candidates for automation, because the signal is certain and the action is reversible and contained. Our autonomous DevOps guide sets out that test (verifiable, reversible, contained) in more detail.

11. Limits and what we couldn't verify

• EventBridge latency. AWS publishes no figure for CloudTrail-to-EventBridge delivery. Measure it as in section 9.
• Write-only trails. We haven't confirmed whether a trail set to write-only management events still feeds read-only events to EventBridge. Use read_write_type = "All".
• Unlogged APIs. Security researchers have repeatedly found AWS API calls that weren't recorded by CloudTrail and could reveal the caller's identity in an error message. AWS has fixed reported cases, but a careful attacker may still validate a key without tripping it. A canary makes detection very likely, not guaranteed.
• Delivery delays. CloudTrail occasionally delivers late. Delayed events carry an addendum with reason DELIVERY_DELAY.
• Coverage. A canary only covers where it's planted. It tells you nothing about an attacker who never looks there. That's why you keep your other alerts.