Repository navigation
Stop trusting compromised eMudhra roots in CI images - #897
eliottness wants to merge 2 commits into
Conversation
Upgrade the default CI image to alpine 3.24.3, use a plain alpine image for the no-op reliability job, and patch the async-profiler-build base Dockerfiles (CA bundle, Node 22.23.4, botocore bundle, JDK cacerts) so the next REBUILD_IMAGES run publishes clean images (IR-62343). Jira: APPSEC-71294
|
@copilot resolve the merge conflicts in this pull request |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![datadog-prod-us1-6[bot]](https://avatars.githubusercontent.com/u/365230?s=60&v=4){kind=small}
There was a problem hiding this comment.
More details
Image-level verification remains incomplete: full builds and certificate scans were not independently confirmed, so this review does not establish that rebuilt CI images contain none of the compromised roots.
🤖 Bits Code Review · Commit 8715e05
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 8715e053d4
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| ARG CI_JOB_TOKEN | ||
| WORKDIR /root | ||
|
|
||
| RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y ca-certificates \ |
There was a problem hiding this comment.
The repository requires a copyright line whenever an existing file without one is modified. This Dockerfile still has no header, and the same omission is present in .gitlab/base/Dockerfile.musl, .gitlab/base/el7/Dockerfile, .gitlab/base/el8/Dockerfile, .gitlab/benchmarks/.gitlab-ci.yml, .gitlab/benchmarks/child.gitlab-ci.yml, and .gitlab/build-deploy/.gitlab-ci.yml; add the 2026 Datadog line to these seven changed files, as was already done for .gitlab-ci.yml.
AGENTS.md reference: AGENTS.md:L334-L336
Useful? React with 👍 / 👎.
What
Images used by this repository's CI still ship CA stores that trust emSign ECC Root CA - G3 and/or AAA Certificate Services, the roots behind the compromised eMudhra intermediates (incident IR-62343). The APPSEC-71273 ledger lists 16 impacted image references here. Images that CI never builds or pulls are left alone.
Changes
.gitlab-ci.ymlalpine:3.22.1@sha256:4bcff63911fcb4448bd4fdacec207030997caf25e9bea4045fa6c8c44de311d1->alpine:3.24.3@sha256:be2656061b536c1883c4034790e21d0a09c5aaaf51968b89368a3c0371ea2c9a: Pipeline default image upgraded to current stable Alpine 3.24.3; exact digest scans clean..gitlab/base/Dockerfile.gitlab/base/Dockerfile.musl.gitlab/base/el7/Dockerfile.gitlab/base/el8/Dockerfile.gitlab/benchmarks/.gitlab-ci.ymlregistry.ddbuild.io/images/benchmarking-platform-tools-ubuntu:newest->alpine:3.24.3@sha256:be2656061b536c1883c4034790e21d0a09c5aaaf51968b89368a3c0371ea2c9a: Generated skip-benchmarks child job only echoes a skip message; current stable Alpine scans clean..gitlab/benchmarks/child.gitlab-ci.ymlregistry.ddbuild.io/images/benchmarking-platform-tools-ubuntu:newest->alpine:3.24.3@sha256:be2656061b536c1883c4034790e21d0a09c5aaaf51968b89368a3c0371ea2c9a: benchmarks-inputs-guard only checks two variables and echoes values; Alpine sh runs the same guard..gitlab/build-deploy/.gitlab-ci.ymlalpine:3.22.1@sha256:4bcff63911fcb4448bd4fdacec207030997caf25e9bea4045fa6c8c44de311d1->alpine:3.24.3@sha256:be2656061b536c1883c4034790e21d0a09c5aaaf51968b89368a3c0371ea2c9a: Pipeline default image upgraded to current stable Alpine 3.24.3; exact digest scans clean.Verification
Every new image reference and every changed Dockerfile was pulled or built and scanned with the eMudhra scanner (checks every file in the image for the roots' public keys); all return empty hits. Of the 16 impacted references: 1 fixed here, 7 fixed at their in-repo image source (clean once the image job republishes), 8 follow-ups below.
Follow-ups (not in this PR)
registry.ddbuild.io/images/benchmarking-platform-tools-ubuntu:newest: DataDog/images/benchmarking-platform-tools-ubuntu:newest; newest/latest resolve to bad digest 1e27d877; active PREPARE_IMAGE or benchmark-image base needs a republished clean tool image.registry.ddbuild.io/ci/async-profiler-build:v137281788-datadog-ci@sha256:84b7902bec3b4cef3e5dbeb0a8a2c2542f388: java-profiler/.gitlab/Dockerfile.datadog-ciregistry.ddbuild.io/slack-notifier:v91289620-4ec922a-latest@sha256:06b24f392ccc383d371c72001520a254edef523bc0b: DataDog/slack-notifier:latest; latest resolves to the pinned bad digest 06b24f39 (10 trust-store hits).eclipse-temurin:21-jdk-alpine@sha256:c98f0d2e171c898bf896dc4166815d28a56d428e218190a1f35cdc7d82efd61f: java-profiler/.gitlab/base/Dockerfile.musl (lane src-java-profiler), then update pinned digests; JDK 21 is deliberately pinned by .gitlab/build-deploy/images.yml comments; the current 21-jdk-alpine digest remains bad.almalinux:8@sha256:9f355ae942d6a6c0561f0771dc053a2cfae9580fc45fa4252756db7c7e80c09f: java-profiler/.gitlab/base/el8/Dockerfile (lane src-java-profiler), then update pinned digests; AlmaLinux 8 pins the shipped binary glibc 2.28 floor; scanned AlmaLinux 9/10 also remain bad.registry.ddbuild.io/ci/async-profiler-build-amd64:v137281788-amd64-benchmarks@sha256:28e10be33f32e73ba3850796b: java-profiler/.gitlab/benchmarks/docker/Dockerfileregistry.ddbuild.io/ci/async-profiler-build-arm64:v137281788-arm64-benchmarks@sha256:c03d7ac565f4b36fc3086a00f: java-profiler/.gitlab/benchmarks/docker/Dockerfile486234852809.dkr.ecr.us-east-1.amazonaws.com/images/docker:24.0.4-gbi-focal: pull: Error response from daemon: failed to resolve reference "486234852809.dkr.ecr.us-east-1.amazonaws.com/images/docker:24.0.4-gbi-focal": pull access denied, repository does not exist or may require authorization: autJira: APPSEC-71294 (parent APPSEC-71273)