Hermes Wiki
Developer/Containers/ContainerRegistry/Fundamentals/container-image-tagging-and-vulnerability-scanning

Container Image Tagging and Vulnerability Scanning

Concept

A container registry (Docker Hub, Amazon ECR, GitHub Container Registry, Google Artifact Registry) stores built images and serves them to whatever pulls them — a Kubernetes node, a CI runner, a developer's laptop. Two registry-level decisions determine whether that pull is trustworthy: what a tag actually points to, and whether anything checks the image for known vulnerabilities before it's allowed to run.

Tags are mutable pointers, not identities. myapp:latest (or even myapp:1.4.2, depending on registry configuration) is a label that can be repointed to a different underlying image digest at any time — pushing a new image with the same tag silently replaces what that tag resolves to. The image's true, immutable identity is its content-addressable digest (a SHA-256 hash of its manifest, e.g. myapp@sha256:abcd...), which changes if and only if the image's actual contents change. Every registry supports pulling by digest; most default to letting humans and CI pipelines pull by tag instead, because tags are readable and digests aren't.

Vulnerability scanning checks an image's installed packages and layers against a database of known CVEs (from the OS package manager's advisories, language-ecosystem advisory databases, or both) and reports matches by severity. Registries increasingly run this automatically — ECR's scan-on-push, GHCR's Dependabot-backed scanning — but a scan finding a vulnerability doesn't by itself stop a deployment; that requires a separate policy (a CI gate, an admission controller) that actually reads the scan result and blocks on it.

Tradeoffs

Tagging approach What it gives you What it costs
latest / mutable tags only Simple — one tag always means "current" No reproducibility: the same tag pulled twice can silently return different images; can't roll back to "what was running last Tuesday" by tag alone
Semantic version tags (1.4.2), mutable Human-readable version history Still mutable in most registries by default — a compromised or mistaken push can silently repoint 1.4.2 to different content unless immutability is explicitly enabled
Semantic version or git-SHA tags + registry-enforced immutability Reproducible: a given tag is guaranteed to always resolve to the same content Requires deleting and re-tagging (not overwriting) to fix a bad release; adds a small amount of registry configuration
Pin by digest in deploy manifests Fully reproducible regardless of tag mutability elsewhere Not human-readable; requires tooling to resolve/update digests in CI rather than hand-editing a tag

The tagging tradeoff isn't really about which scheme to use for human-facing version numbers — it's whether the registry is configured to make a tag's meaning immutable once pushed. A team can use perfectly sensible semantic-version tags and still have no reproducibility guarantee if the registry allows overwriting 1.4.2 with different content later; immutable tags (or pinning deploy configs to digests) is what actually closes that gap.

When to use / when not to

  • Tag every image traceably (git SHA, semantic version, or both) as soon as it's deployed anywhere beyond a developer's local docker compose uplatest alone is fine for pure local iteration, never for anything that needs to be rolled back or audited.
  • Enable registry-level tag immutability (ECR, GHCR, and most managed registries support it) once images are used in any environment where "what's actually running" needs to be a reliable, auditable fact — production, and usually staging too.
  • Turn on scan-on-push as the default for any registry serving production images; treat it the same as a required CI check, not an optional dashboard nobody reads.
  • Don't rely on scanning alone as a gate without also pinning what actually gets deployed by digest — a scan result for a tag is only meaningful if that tag can't be silently repointed to unscanned content after the scan ran.
  • Don't over-invest in exotic tagging schemes (semver + build metadata + environment suffixes all at once) before the basics — traceable tag + immutability + scan-on-push — are in place; the basics close most of the actual risk.

Common pitfall

Treating "we have vulnerability scanning enabled" as equivalent to "we don't deploy vulnerable images" — the two are only the same if a scan result actually blocks something. A registry that scans on push but has no CI gate or admission controller reading those results just produces a dashboard of known vulnerabilities that nobody is forced to act on; a critical CVE can sit unaddressed in a production image indefinitely because the scan generated a report, not a stop. The scan is necessary but not sufficient — the enforcement point (fail the build, block the deploy, alert and require sign-off) is the part that actually changes outcomes, and it's the part teams most often skip because it's the harder, more disruptive half of the setup.

A related trap on the tagging side: assuming a semantic-version tag is inherently trustworthy because it looks like a stable release identifier. Without registry-enforced immutability, myapp:2.1.0 is exactly as mutable as myapp:latest — the only thing that actually pins content is a digest, or a registry configuration that refuses a second push to an existing tag. Teams that skip this step have discovered the hard way that "we deployed 2.1.0" doesn't answer "which build" if 2.1.0 was pushed to twice.

Engineering Lens

The pattern that generalizes here: a security or reliability control that produces information (a scan report, a log line, a metric) is not the same as a control that produces an outcome (a blocked deploy, a rejected pull, a paged on-call). It's easy to check the box on "we scan images" and stop there, because enabling scanning is a one-time registry setting while wiring an enforcement gate into CI/CD touches build pipelines, deploy tooling, and often a negotiation with other teams about what severity threshold actually blocks a release. The strong review answer isn't "our registry scans on push" — it's naming the specific gate that reads that scan result and what happens when it finds a critical CVE, plus confirming that whatever gets deployed is pinned (by digest, or by an immutable tag) tightly enough that the thing that was scanned is provably the thing that runs. This is the identical shape of gap that shows up in thread pool starvation (a pool that queues instead of failing loudly) and in isolation-level defaults (a level that permits a race silently instead of erroring) — a control that's present but not wired to actually stop the bad outcome gives a false sense of coverage.

  • Docker Multi-Stage Builds and Layer Caching — smaller, leaner images built via multi-stage builds also shrink the attack surface a registry scan has to check, since fewer installed packages means fewer potential CVE matches.

Sources

Hermes Wiki