Engineering

Kill Your Darlings: Why I Finally Deleted My AWS Access Keys

Kill Your Darlings: Why I Finally Deleted My AWS Access Keys

The confession

For the first five years of my career, I treated AWS Access Keys like loose change. Keys in local .env files, keys buried in CI/CD secrets, keys hardcoded in scripts named quick_deploy.sh, and occasionally committed to private Git repos because "who's going to see it anyway?"

It worked. Until it didn't.

The moment you receive an email from AWS Abuse about "irregular activity" in a region you didn't know existed, your heart drops. That panic is a rite of passage for many engineers. Triggered by an ISMS audit, I checked my IAM dashboard. A graveyard of long-lived credentials, some older than the frameworks I was using.

Time to stop patching the leak and fix the plumbing. Move from static Access Keys to Role-based authentication.

The rot of static keys

The "Swiss Army Knife" anti-pattern

The most common pattern: one IAM User for CI/CD. Generate a key pair (AKIA...), give it AdministratorAccess because you're tired of debugging permission errors at 2 AM.

This single key pushes Docker images to ECR, updates ECS services, invalidates CloudFront caches, reads SSM parameters. If it leaks, the attacker owns your entire delivery pipeline. Blast radius: everything.

The rotation paradox

Best practices say rotate every 90 days. How many of us automate this? Rotating a key used in five repos, three developer laptops, and a legacy Jenkins server is not a task. It's a project. Because the friction is high, we delay it. Because we delay, the risk compounds.

Identity as context, not a string

Role-based authentication (OIDC + sts:AssumeRole) changes the question from "who has the key?" to "in what context is this request being made?"

Assume Role: Your code doesn't hold a key. It asks AWS, "I'm running on this EC2 instance. Can I have the permissions for MyAppRole?" AWS validates the environment and hands over a temporary token.

OIDC: For external systems like GitHub Actions, we tell AWS to trust tokens signed by GitHub for a specific repository. GitHub signs a JWT, sends it to AWS, AWS exchanges it for temporary credentials.

No secret to steal. Token valid for an hour or less. Tied to the execution context.

Implementation with AWS CDK

Step 1: OIDC provider

Tell AWS that GitHub is a trusted identity provider. One-time setup per account:

import * as cdk from 'aws-cdk-lib';
import * as iam from 'aws-cdk-lib/aws-iam';
import { Construct } from 'constructs';
 
export class IdentityStack extends cdk.Stack {
  constructor(scope: Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);
 
    const githubProvider = new iam.OpenIdConnectProvider(this, 'GithubOidcProvider', {
      url: 'https://token.actions.githubusercontent.com',
      clientIds: ['sts.amazonaws.com'],
    });
 
    new cdk.CfnOutput(this, 'GithubProviderArn', {
      value: githubProvider.openIdConnectProviderArn,
    });
  }
}

Step 2: Scoped roles

Instead of one monolithic IAM User, create roles with specific duties. The conditions restrict the trust policy so only your specific repo can assume the role:

import * as cdk from 'aws-cdk-lib';
import * as iam from 'aws-cdk-lib/aws-iam';
import { Construct } from 'constructs';
 
interface AppDeployRoleProps extends cdk.StackProps {
  repoName: string;
}
 
export class PermissionsStack extends cdk.Stack {
  constructor(scope: Construct, id: string, props: AppDeployRoleProps) {
    super(scope, id, props);
 
    const providerArn = `arn:aws:iam::${this.account}:oidc-provider/token.actions.githubusercontent.com`;
    const githubProvider = iam.OpenIdConnectProvider.fromOpenIdConnectProviderArn(
      this, 'ImportedGithubProvider', providerArn
    );
 
    const deployRole = new iam.Role(this, 'GithubDeployRole', {
      assumedBy: new iam.WebIdentityPrincipal(githubProvider.openIdConnectProviderArn, {
        StringLike: {
          'token.actions.githubusercontent.com:sub': `repo:${props.repoName}:*`
        },
        StringEquals: {
          'token.actions.githubusercontent.com:aud': 'sts.amazonaws.com',
        },
      }),
      description: 'Role assumed by GitHub Actions for deployment',
      roleName: 'GitHubAction-DeployRole',
    });
 
    deployRole.addToPolicy(new iam.PolicyStatement({
      effect: iam.Effect.ALLOW,
      actions: [
        'ecr:GetAuthorizationToken',
        'ecr:BatchCheckLayerAvailability',
        'ecr:PutImage',
        'ecr:InitiateLayerUpload',
        'ecr:UploadLayerPart',
        'ecr:CompleteLayerUpload'
      ],
      resources: ['*'],
    }));
 
    deployRole.addToPolicy(new iam.PolicyStatement({
      effect: iam.Effect.ALLOW,
      actions: ['ecs:UpdateService', 'ecs:DescribeServices'],
      resources: ['*'],
    }));
  }
}

Step 3: GitHub Actions workflow

Remove AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY from repository secrets. They're gone. Forever.

name: Build and Deploy
on:
  push:
    branches: [ "main" ]
 
permissions:
  id-token: write
  contents: read
 
jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout Code
        uses: actions/checkout@v3
 
      - name: Configure AWS Credentials
        uses: aws-actions/configure-aws-credentials@v4
        with:
          role-to-assume: arn:aws:iam::123456789012:role/GitHubAction-DeployRole
          aws-region: us-east-1
 
      - name: Login to ECR
        id: login-ecr
        uses: aws-actions/amazon-ecr-login@v2
 
      - name: Build and Push
        run: |
          docker build -t my-app .
          docker tag my-app:latest ${{ steps.login-ecr.outputs.registry }}/my-app:latest
          docker push ${{ steps.login-ecr.outputs.registry }}/my-app:latest

The workflow requests credentials, AWS checks the OIDC token signed by GitHub, verifies the StringLike condition, grants access.

Local development

"What about scripts on my laptop?"

Use AWS SSO (IAM Identity Center). Log in via browser, the CLI manages temporary tokens in ~/.aws/config.

If you must assume a role from a script:

import { STSClient, AssumeRoleCommand } from "@aws-sdk/client-sts";
 
const stsClient = new STSClient({ region: "us-east-1" });
 
async function getTemporaryCredentials() {
  try {
    const command = new AssumeRoleCommand({
      RoleArn: "arn:aws:iam::123456789012:role/SpecificTaskRole",
      RoleSessionName: "LocalDebugSession",
      DurationSeconds: 900,
    });
 
    const response = await stsClient.send(command);
 
    if (!response.Credentials) {
      throw new Error("Failed to retrieve credentials");
    }
 
    console.log("Temporary Access Key ID:", response.Credentials.AccessKeyId);
    return response.Credentials;
  } catch (error) {
    console.error("Error assuming role:", error);
  }
}
 
getTemporaryCredentials();

Migration strategy

  1. Audit: Use the IAM Credential Report to list active keys. Map to owners.
  2. Create roles: Implement OIDC/Role infrastructure alongside existing keys.
  3. Dual run: Update CI/CD to use the new Role ARN. Leave old secrets unused but in place.
  4. Disable: Set Access Key to "Inactive" in IAM. Wait for the scream test.
  5. Delete: After a week of silence, delete the key.

Switching from Access Keys to IAM Roles shifts security from a static secret that must be guarded to a dynamic state that is negotiated. The CDK code takes more effort than clicking "Create Access Key." But the peace of mind from having zero credentials in your codebase is worth every line.

Stop hoarding keys. Let the system manage the trust.