Compile an Automated Audit Evidence Package

Objective

When an auditor asks "show me that your cluster enforces least-privilege RBAC", you need to produce evidence quickly and consistently. Manual collection takes hours and produces inconsistent snapshots. This exercise automates evidence collection for 10 controls that commonly appear in SOC 2, ISO 27001, and PCI-DSS audits. The output is a ZIP archive with a manifest.json that records collection timestamps and command provenance.

The 10 Controls and Their Evidence

#	Control	Evidence Collected
C01	Network segmentation	All NetworkPolicy objects across all namespaces
C02	Encryption at rest	API server encryption provider config, etcd storage check
C03	RBAC least privilege	All ClusterRoleBindings + RoleBindings (not system:)
C04	Image scanning	Latest Trivy scan results per running image
C05	Pod Security Standards	Namespace labels showing PSS enforcement level
C06	Secret management	ExternalSecret resources, no plaintext secrets in manifests
C07	Audit logging enabled	API server flags confirming --audit-log-path is set
C08	Node OS hardening	kube-bench results for worker node sections
C09	Admission control policies	All Kyverno/OPA ClusterPolicies and their enforcement actions
C10	TLS certificate validity	Certificate expiry dates from cert-manager CertificateRequests

Prerequisites

kubectl with cluster-admin access
Python 3.9+ with the kubernetes client library installed
Trivy installed locally or accessible in-cluster
jq for ad-hoc log parsing

Steps

01

Write the evidence collection script

collect_evidence.py

import json, subprocess, zipfile, hashlib, sys
from datetime import datetime, timezone
from pathlib import Path

AUDIT_DATE = datetime.now(timezone.utc).strftime("%Y-%m-%d")
EVIDENCE_DIR = Path(f"audit-evidence-{AUDIT_DATE}")
EVIDENCE_DIR.mkdir(exist_ok=True)

manifest = {
    "audit_date": AUDIT_DATE,
    "collector_version": "1.0.0",
    "controls": [],
}

def collect(control_id: str, description: str, cmd: list, filename: str):
    start = datetime.now(timezone.utc)
    print(f"  Collecting {control_id}: {description}", end="", flush=True)

    result = subprocess.run(cmd, capture_output=True, text=True)
    output = result.stdout if result.returncode == 0 else result.stderr

    path = EVIDENCE_DIR / filename
    path.write_text(output)

    sha256 = hashlib.sha256(output.encode()).hexdigest()
    elapsed_ms = int((datetime.now(timezone.utc) - start).total_seconds() * 1000)

    manifest["controls"].append({
        "id":           control_id,
        "description":  description,
        "file":         filename,
        "collected_at": start.isoformat(),
        "elapsed_ms":   elapsed_ms,
        "sha256":       sha256,
        "status":       "ok" if result.returncode == 0 else "error",
        "command":      " ".join(cmd),
    })
    print(f" ({elapsed_ms}ms)")
    return result.returncode

# ── Evidence collection per control ───────────────────────────────────────────

print("\nKubernetes Audit Evidence Collector")
print(f"Date: {AUDIT_DATE}\n")

# C01 — Network segmentation
collect("C01", "Network Policies",
    ["kubectl", "get", "networkpolicies", "-A", "-o", "json"],
    "C01-network-policies.json")

# C02 — Encryption at rest (check API server pod for flag)
collect("C02", "Encryption Provider Config",
    ["kubectl", "-n", "kube-system", "get", "pod",
     "-l", "component=kube-apiserver", "-o", "json"],
    "C02-apiserver-config.json")

# C03 — RBAC bindings (exclude system: accounts)
collect("C03a", "ClusterRoleBindings",
    ["kubectl", "get", "clusterrolebindings", "-o", "json"],
    "C03a-clusterrolebindings.json")
collect("C03b", "RoleBindings (all namespaces)",
    ["kubectl", "get", "rolebindings", "-A", "-o", "json"],
    "C03b-rolebindings.json")

# C05 — Pod Security Standards labels
collect("C05", "Namespace PSS Labels",
    ["kubectl", "get", "namespaces", "-o",
     "jsonpath={range .items[*]}{.metadata.name}{'\\t'}{.metadata.labels}{'\\n'}{end}"],
    "C05-namespace-pss-labels.txt")

# C06 — ExternalSecrets (secrets from external stores)
collect("C06", "ExternalSecret Resources",
    ["kubectl", "get", "externalsecrets", "-A", "-o", "json"],
    "C06-external-secrets.json")

# C09 — Admission control policies
collect("C09a", "Kyverno ClusterPolicies",
    ["kubectl", "get", "clusterpolicies", "-o", "json"],
    "C09a-kyverno-policies.json")
collect("C09b", "Validating Webhook Configurations",
    ["kubectl", "get", "validatingwebhookconfigurations", "-o", "json"],
    "C09b-validating-webhooks.json")

# C10 — TLS certificates
collect("C10", "cert-manager Certificates",
    ["kubectl", "get", "certificates", "-A", "-o", "json"],
    "C10-certificates.json")

# ── Write manifest and package ────────────────────────────────────────────────
manifest_path = EVIDENCE_DIR / "manifest.json"
manifest_path.write_text(json.dumps(manifest, indent=2))

zip_path = f"audit-evidence-{AUDIT_DATE}.zip"
with zipfile.ZipFile(zip_path, "w", zipfile.ZIP_DEFLATED) as zf:
    for file in EVIDENCE_DIR.iterdir():
        zf.write(file, file.name)

total_ms = sum(c["elapsed_ms"] for c in manifest["controls"])
errors    = [c for c in manifest["controls"] if c["status"] == "error"]

print(f"\n✓ Package: {zip_path}")
print(f"  Controls collected : {len(manifest['controls'])}")
print(f"  Total time         : {total_ms/1000:.1f}s")
print(f"  Errors             : {len(errors)}")
if errors:
    print("  Missing:", [e["id"] for e in errors])
sys.exit(1 if errors else 0)

02

Run the collector and inspect the output

pip install kubernetes
python3 collect_evidence.py

## Kubernetes Audit Evidence Collector
## Date: 2026-03-19
##
##   Collecting C01: Network Policies (312ms)
##   Collecting C02: Encryption Provider Config (287ms)
##   Collecting C03a: ClusterRoleBindings (445ms)
##   Collecting C03b: RoleBindings (all namespaces) (891ms)
##   Collecting C05: Namespace PSS Labels (201ms)
##   Collecting C06: ExternalSecret Resources (error — CRD not installed)
##   Collecting C09a: Kyverno ClusterPolicies (330ms)
##   Collecting C09b: Validating Webhook Configurations (278ms)
##   Collecting C10: TLS Certificates (188ms)
##
## ✓ Package: audit-evidence-2026-03-19.zip
##   Controls collected : 9
##   Total time         : 3.9s
##   Errors             : 1
##   Missing: ['C06']

# Inspect the ZIP
unzip -l audit-evidence-2026-03-19.zip

# Verify manifest integrity
python3 -c "
import json, hashlib, zipfile
with zipfile.ZipFile('audit-evidence-2026-03-19.zip') as zf:
    manifest = json.loads(zf.read('manifest.json'))
    for c in manifest['controls']:
        content = zf.read(c['file']).decode()
        actual = hashlib.sha256(content.encode()).hexdigest()
        match = '✓' if actual == c['sha256'] else '✗ TAMPERED'
        print(f\"{match} {c['id']} {c['file']}\")
"

03

Analyse gaps and build a remediation backlog

# Review manifest for errors and slow collections
python3 -c "
import json, zipfile
with zipfile.ZipFile('audit-evidence-2026-03-19.zip') as zf:
    m = json.loads(zf.read('manifest.json'))
    print(f'{'ID':<8} {'Status':<8} {'ms':>6}  Description')
    for c in sorted(m['controls'], key=lambda x: -x['elapsed_ms']):
        flag = '⚠' if c['status'] == 'error' else ' '
        print(f\"{flag}{c['id']:<7} {c['status']:<8} {c['elapsed_ms']:>6}ms  {c['description']}\")
"

## ⚠C06     error       178ms  ExternalSecret Resources    ← CRD missing: collect via Vault API
##  C03b    ok          891ms  RoleBindings (all namespaces) ← slowest: cache result
##  C03a    ok          445ms  ClusterRoleBindings
##  C01     ok          312ms  Network Policies

The manifest's sha256 hashes provide tamper-evidence. If evidence files are modified after collection, the integrity check fails. For stronger guarantees, GPG-sign the manifest or upload to an immutable object store (S3 with Object Lock, Azure Blob with immutability policy).

04

Add missing controls C04, C07, C08 to complete the set

# C04 — Image scanning results (run Trivy, save JSON)
collect("C04", "Image Vulnerability Scan",
    ["trivy", "image", "--format", "json", "--severity", "CRITICAL,HIGH",
     "nginx:1.25"],
    "C04-image-scan.json")

# C07 — Audit logging (grep API server flags)
collect("C07", "Audit Logging Config",
    ["kubectl", "-n", "kube-system", "get", "pods",
     "-l", "component=kube-apiserver",
     "-o", "jsonpath={.items[0].spec.containers[0].command}"],
    "C07-apiserver-flags.txt")

# C08 — kube-bench results (run as Job and capture output)
collect("C08", "CIS Benchmark (kube-bench)",
    ["kubectl", "logs", "job/kube-bench"],    # pre-run the job
    "C08-kube-bench.txt")

# After adding all 10, verify total collection time
## ✓ Package: audit-evidence-2026-03-19.zip
##   Controls collected : 12  (10 controls + 2 sub-items for C03)
##   Total time         : 48.2s  (kube-bench job dominates)
##   Errors             : 0

05

Measure time-to-evidence and identify automation gaps

# Time-to-evidence analysis template
# ┌─────────────────────────────────────────────────────┐
# │ Control  │ Collection Time │ Manual Time  │ Status   │
# ├──────────┼─────────────────┼──────────────┼──────────┤
# │ C01      │ 312ms           │ ~10 min      │ Automated│
# │ C02      │ 287ms           │ ~30 min      │ Automated│
# │ C03      │ 1.3s            │ ~60 min      │ Automated│
# │ C04      │ 45s             │ ~2 hrs       │ Automated│
# │ C05      │ 201ms           │ ~5 min       │ Automated│
# │ C06      │ ERROR           │ ~45 min      │ MANUAL   │ ← gap
# │ C07      │ 189ms           │ ~20 min      │ Automated│
# │ C08      │ 42s             │ ~4 hrs       │ Automated│
# │ C09      │ 608ms           │ ~30 min      │ Automated│
# │ C10      │ 188ms           │ ~30 min      │ Automated│
# ├──────────┼─────────────────┼──────────────┼──────────┤
# │ TOTAL    │ ~90s automated  │ ~7.5 hrs     │ 90% auto │
# └─────────────────────────────────────────────────────┘

# Backlog item for C06 gap:
# ACTION: Migrate secret management from inline kubectl secrets to
#         External Secrets Operator + AWS Secrets Manager.
#         Install ESO CRDs so automated collection works.
#         Owner: platform-team | Priority: HIGH | Due: 2026-04-15

Success Criteria

Script runs end-to-end and produces a ZIP archive containing evidence files and manifest.json manifest.json contains sha256 hash for each file — integrity check script passes Controls with missing CRDs or unavailable resources are recorded as "error" (not silent failure) Total automated collection takes under 2 minutes for all 10 controls Time-to-evidence comparison table is completed showing manual vs automated time Automation gap analysis produces at least one backlog item with owner and due date ZIP can be handed directly to an auditor as a complete evidence package