A CTO can buy a CI/CD platform and still own every difficult part of releasing software. The expensive mistakes happen at the boundary: a vendor runs the jobs, while nobody on the team owns credentials, evidence, or recovery when those jobs fail. I would buy the routine execution layer and keep release authority in-house, because outsourcing the control plane makes a vendor outage a production decision.
Teams buy pipeline automation and forget to assign release authority
“The pipeline passed” is often treated as permission to deploy. That is a mistake because a green check says what ran, not who was authorized to change production. A CTO should be able to name the owner of each decision: who approves a release, who can issue a production credential, who can stop a deployment, and who can override a failed check. If those answers point to a vendor dashboard and an undocumented administrator account, the apparent savings have purchased an incident-response problem.
The common build-or-buy comparison asks whether a team can build runners more cheaply than it can rent them. That question is too narrow because runners are replaceable; authority is harder to move. GitHub Actions or GitLab CI can execute a job, while AWS IAM can decide whether that job may assume a production role. Keeping that policy in IAM lets the team change CI providers without also redesigning its production access rules. The cost is additional policy maintenance and a real need to test those policies, rather than assuming a vendor’s permissions screen expresses the whole rule.
I would not build a general-purpose CI/CD platform in-house, because maintaining schedulers, runner images, logs, upgrades, and isolation would consume engineering time without giving most teams a distinctive release advantage. I would also not let a purchased platform hold long-lived production keys, because a leaked key can remain useful after the job that exposed it has finished. OpenID Connect (OIDC) federation to AWS Security Token Service (STS) is a better boundary for that case: the job requests a short-lived session, and the role trust policy limits which repository and branch may request it.
The hard part is not enabling OIDC; it is proving that the trust policy matches the release rule. A workflow restricted to a protected branch can still be unsafe if the AWS role trusts every branch in the repository. Conversely, a correctly scoped role can block an urgent fix if nobody has rehearsed how to restore access after a configuration error. For a new setup, use a 30-day pilot as a proposed decision window, not as a promise of reliability. Before that window closes, have a person outside the implementation team demonstrate both a permitted deployment and a denied one.
The framing in Build or Buy? Surface Security and Privacy Trade-Offs usefully puts exposure on the decision table, but I would make revocation an explicit purchasing test. A vendor’s answers about encryption and access controls do not tell the CTO how quickly the team can disable its own compromised release path.
A cheap run becomes expensive when nobody can reproduce its artifact
Many teams compare CI prices per minute and omit the cost of explaining a production binary. The omission matters during an incident: if the team cannot identify the source commit, dependencies, builder, and approval associated with an artifact, it may have to stop more releases than the affected one. Buying more runner capacity does not repair that gap; it only produces ambiguous artifacts faster.
Set an evidence contract before choosing a runner. Require a commit identifier, an immutable artifact digest, a Software Bill of Materials in SPDX 2.3 or CycloneDX 1.6 format, and provenance whose claims the team can verify. SLSA v1.0 provides vocabulary for provenance expectations, while Sigstore cosign can sign and verify artifact attestations. These are useful only if deployment checks the digest and signature against an approved identity; storing an SBOM beside an unchecked image adds paperwork without preventing a substitution.
Docker BuildKit can build the image and emit metadata, but “reproducible” needs a test rather than a label. A base image referenced by a mutable tag, such as latest, may change between builds, so pin it by digest when investigating whether two builds should match. The same discipline applies to actions and plugins: a major tag such as actions/checkout@v4 is convenient to maintain, while pinning a full commit SHA gives tighter control over exactly what executed. Each choice has a cost. Pinning increases review and update work; relying on moving references increases the number of changes an incident investigation must rule out.
Compare GitHub Actions with AWS OIDC against self-managed GitLab CI runners at this boundary. GitHub Actions wins when the team wants managed job orchestration and can express release authority in IAM and repository protections; it costs hosted-runner fees and some dependence on GitHub’s job environment. Self-managed GitLab CI runners win when workloads need specific machines or network placement that the team already operates; they cost capacity planning, patching, isolation work, and on-call ownership. Neither option wins merely because its invoice is smaller. The relevant comparison is the total effort required to produce, verify, and recover a trusted artifact.
The title Cheap CI/CD vs Doing It Right What a CTO Should Buy or Build captures the tempting price comparison; I would resist treating “doing it right” as a larger tool purchase. A small pipeline with a verifiable artifact and a named release owner can be safer than an elaborate platform whose outputs nobody checks.
Credential settings fail when teams never test the refusal path
A successful deployment proves that credentials worked once. It does not prove that a pull request, an unprotected branch, or a copied workflow cannot use them. Teams get this wrong when they test only the path that ships software, because a permissive trust policy looks identical to a narrow one while the approved job is running.
This GitHub Actions workflow checks whether a configured AWS role can be assumed through OIDC. It runs after DEPLOY_ROLE_ARN is set as a repository variable and the role’s AWS trust policy permits this repository’s intended workflow identity. The check is deliberately small so its success can be distinguished from a deploy’s other failures:
name: Verify deploy identity
on: [workflow_dispatch]
permissions:
id-token: write
contents: read
jobs:
verify:
runs-on: ubuntu-latest
steps:
- uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: ${{ vars.DEPLOY_ROLE_ARN }}
aws-region: us-east-1
- run: aws sts get-caller-identity
A successful STS response identifies the assumed role; it does not establish that the role is scoped correctly. Repeat the test from an identity that must be refused, and inspect the IAM trust policy’s OIDC aud and sub conditions if it succeeds. Set a 15-minute session duration as an initial value to tune against deployment length and recovery needs, rather than assuming that the shortest possible session is operationally safe. Also rehearse revocation: disabling role assumption should stop new sessions, while already issued sessions require a separate assessment of their remaining lifetime.
Vendor limits belong in this test plan, not in the purchasing slide alone. GitHub’s published limit for a GitHub-hosted job is 6 hours; a team with longer builds must split the work, choose another execution arrangement, or accept failed jobs. That number does not measure the team’s typical build time. Record its own p95 build duration and queue delay before deciding whether hosted jobs are actually the bottleneck, because buying self-managed runners to fix slow builds may merely move a slow dependency download onto machines the team now has to maintain.
Finally, test the human path. Give one engineer the ability to stop a release without giving that person unrestricted power to change the approval rule. If the same account can edit the workflow, relax branch protection, and assume the production role, the nominal checks are easier to bypass because no independent boundary remains.
Evidence retention is a product of ownership, not a storage setting
Buying a service often creates the impression that audit evidence will be there when needed. That assumption fails when job logs expire, a repository is deleted, or an artifact registry retains an image but not the approval and provenance that explain it. The resulting cost is investigation time: people reconstruct a release from chat messages and local terminals instead of inspecting a durable record.
Assign an owner for the full chain from commit to running digest. GitHub Actions logs, an OCI registry, cosign attestations, AWS CloudTrail events, and deployment records can each hold part of that chain, but none automatically guarantees the others will be retained. A team might choose 90 days as a provisional evidence-retention target, then adjust it to its actual investigation and contractual needs. The useful question is whether an incident responder can retrieve the complete chain for a deployment near the end of that period, not whether a settings page displays “90.”
Measure that retrieval as an exercise. Select an old production deployment, find its running digest, identify the source commit and approver, verify its attestation, and locate the credential-assumption event in CloudTrail. Record the elapsed time as the team’s own measured baseline; a missing record is a failure even if the search is quick. OpenTelemetry traces and Prometheus metrics can help connect a deployment to a change in application behavior, but they cannot replace the artifact and authorization evidence that establishes what was released.
This is where an in-house platform can seem attractive for the wrong reason. If records are scattered, building one dashboard feels like ownership, but a dashboard cannot recreate evidence that was never emitted or was already deleted. Buy storage and execution where they reduce routine work; build the small internal contract that defines required evidence, retention, verification, and who responds when a link in the chain is missing. That division has a maintenance cost, but it is narrower than owning an entire CI/CD service.
The next buying decision should begin with a failed deployment
Before renewing a platform contract or funding an internal replacement, schedule one refusal-and-recovery drill. Try to assume the production role from an unauthorized branch, stop a legitimate release, revoke access, and reconstruct the last deployed artifact. Time each step and name the person who can fix a failure. Buy the runner that supports that boundary at an acceptable cost; keep the authority and the proof under your team’s control.
