Multi-Tenant Namespace Model with RBAC — K8s Platform Engineering

Objective

Create two isolated tenant namespaces — team-alpha and team-beta — each with their own resource quotas, default limit ranges, and RBAC bindings that prevent cross-namespace access. Then simulate a tenant attempting to escape their namespace by creating a privileged pod or reading another tenant's secrets. Validate that admission control and RBAC prevent the breach.

Prerequisites

Running Kubernetes cluster with cluster-admin access (any provider)
kubectl configured and pointing to your test cluster
Pod Security Standards enabled (Kubernetes >= 1.25)
Basic understanding of Kubernetes RBAC (Roles, RoleBindings, ServiceAccounts)

Steps

Create the tenant namespaces with PSS labels

Apply the restricted Pod Security Standard to both namespaces at enforce level. This is the first line of defence against privilege escalation.

# team-alpha-ns.yaml
apiVersion: v1
kind: Namespace
metadata:
  name: team-alpha
  labels:
    pod-security.kubernetes.io/enforce: restricted
    pod-security.kubernetes.io/enforce-version: latest
    pod-security.kubernetes.io/warn: restricted
    pod-security.kubernetes.io/audit: restricted
    team: alpha
---
apiVersion: v1
kind: Namespace
metadata:
  name: team-beta
  labels:
    pod-security.kubernetes.io/enforce: restricted
    pod-security.kubernetes.io/enforce-version: latest
    team: beta

kubectl apply -f team-alpha-ns.yaml

Apply ResourceQuotas per namespace

ResourceQuotas limit the total resource consumption per namespace. Without quotas, a noisy neighbour tenant can starve others.

# resource-quota-alpha.yaml
apiVersion: v1
kind: ResourceQuota
metadata:
  name: team-alpha-quota
  namespace: team-alpha
spec:
  hard:
    requests.cpu: "8"
    requests.memory: 16Gi
    limits.cpu: "16"
    limits.memory: 32Gi
    pods: "50"
    services: "10"
    persistentvolumeclaims: "20"
    count/deployments.apps: "20"
    count/secrets: "50"

kubectl apply -f resource-quota-alpha.yaml

# Verify quota is active
kubectl describe resourcequota team-alpha-quota -n team-alpha

Apply LimitRanges for default container limits

LimitRanges set default requests/limits for containers that don't specify them. Without this, containers with no limits bypass quota enforcement and can consume unbounded CPU.

# limitrange-alpha.yaml
apiVersion: v1
kind: LimitRange
metadata:
  name: team-alpha-limits
  namespace: team-alpha
spec:
  limits:
  - type: Container
    default:
      cpu: "500m"
      memory: 512Mi
    defaultRequest:
      cpu: "100m"
      memory: 128Mi
    max:
      cpu: "4"
      memory: 8Gi
    min:
      cpu: "10m"
      memory: 16Mi
  - type: PersistentVolumeClaim
    max:
      storage: 50Gi
    min:
      storage: 1Gi

kubectl apply -f limitrange-alpha.yaml

Create ServiceAccounts and RBAC bindings

Each team gets a ServiceAccount and a RoleBinding granting edit rights only within their own namespace. No ClusterRoleBindings are granted — teams cannot see cluster-wide resources.

# rbac-alpha.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  name: team-alpha-sa
  namespace: team-alpha
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: team-alpha-edit
  namespace: team-alpha
subjects:
- kind: ServiceAccount
  name: team-alpha-sa
  namespace: team-alpha
roleRef:
  kind: ClusterRole
  name: edit          # built-in: full ns access, no RBAC writes
  apiGroup: rbac.authorization.k8s.io

kubectl apply -f rbac-alpha.yaml
# Repeat for team-beta with namespace: team-beta

Validate isolation with kubectl auth can-i

Test what each ServiceAccount can and cannot do. These checks confirm your RBAC model works as intended before any breach simulation.

# Verify alpha-sa can deploy in team-alpha
kubectl auth can-i create deployments \
  --namespace=team-alpha \
  --as=system:serviceaccount:team-alpha:team-alpha-sa
# Expected: yes

# Verify alpha-sa CANNOT deploy in team-beta
kubectl auth can-i create deployments \
  --namespace=team-beta \
  --as=system:serviceaccount:team-alpha:team-alpha-sa
# Expected: no

# Verify alpha-sa cannot read beta's secrets
kubectl auth can-i get secrets \
  --namespace=team-beta \
  --as=system:serviceaccount:team-alpha:team-alpha-sa
# Expected: no

# Verify alpha-sa cannot list nodes (cluster resource)
kubectl auth can-i list nodes \
  --as=system:serviceaccount:team-alpha:team-alpha-sa
# Expected: no

Simulate a privileged pod breach attempt

Try to deploy a pod that requests host networking, runs as root, and mounts the host filesystem — the classic container escape primitives. The restricted PSS should reject this at admission.

# breach-attempt.yaml — this should be REJECTED
apiVersion: v1
kind: Pod
metadata:
  name: breach-attempt
  namespace: team-alpha
spec:
  hostNetwork: true        # PSS restricted: DENIED
  hostPID: true            # PSS restricted: DENIED
  containers:
  - name: breach
    image: ubuntu:22.04
    command: ["sleep", "3600"]
    securityContext:
      privileged: true      # PSS restricted: DENIED
      runAsUser: 0          # PSS restricted: DENIED
    volumeMounts:
    - mountPath: /host
      name: host-root
  volumes:
  - name: host-root
    hostPath:
      path: /             # PSS restricted: DENIED

kubectl apply -f breach-attempt.yaml
# Expected error output:
# Error from server (Forbidden): pods "breach-attempt" is forbidden:
# violates PodSecurity "restricted:latest": host namespaces
# (hostNetwork=true, hostPID=true), privileged (containers
# "breach" must not set securityContext.privileged=true),
# runAsNonRoot (containers "breach" must not set runAsUser=0)

The admission controller rejects this before any container is created. The rejection includes a list of all violated policies, making it easier to remediate legitimate workloads that accidentally trigger restrictions.

Verify a compliant workload still deploys

Confirm that the restrictions don't break normal workloads. A properly configured pod should deploy without issues.

# compliant-pod.yaml
apiVersion: v1
kind: Pod
metadata:
  name: compliant-workload
  namespace: team-alpha
spec:
  securityContext:
    runAsNonRoot: true
    runAsUser: 1000
    seccompProfile:
      type: RuntimeDefault
  containers:
  - name: app
    image: nginx:1.25-unprivileged
    securityContext:
      allowPrivilegeEscalation: false
      readOnlyRootFilesystem: true
      capabilities:
        drop: ["ALL"]

kubectl apply -f compliant-pod.yaml
# Expected: pod/compliant-workload created
kubectl get pod compliant-workload -n team-alpha

Verify resource quota enforcement

Try to create a pod exceeding the quota and observe the rejection. Then view the current quota usage.

# Check current quota utilisation
kubectl describe resourcequota team-alpha-quota -n team-alpha

# Output shows used vs hard limits:
# Name:                   team-alpha-quota
# Namespace:              team-alpha
# Resource                Used  Hard
# --------                ----  ----
# count/deployments.apps  0     20
# limits.cpu              0     16
# pods                    1     50
# requests.memory         128Mi 16Gi

# Attempt to exceed limit (create 51 pods script)
for i in $(seq 1 51); do
  kubectl run test-pod-$i \
    --image=nginx:alpine \
    --namespace=team-alpha \
    --restart=Never \
    2>&1 | tail -1
done
# After pod 50: "Error from server (Forbidden): pods
#   "test-pod-51" is forbidden: exceeded quota: team-alpha-quota"

# Cleanup test pods
kubectl delete pods -l run -n team-alpha

Success Criteria

Both namespaces created with correct PSS labels (verify with kubectl get ns --show-labels) ResourceQuota and LimitRange active in both namespaces All 4 kubectl auth can-i checks return the expected yes/no results Privileged pod breach attempt is rejected with a clear PSS violation message Compliant pod deploys successfully in team-alpha namespace Resource quota enforces the pod count limit team-alpha ServiceAccount cannot read secrets from team-beta namespace

Key Concepts

PSS enforce vs warn vs audit — enforce blocks creation; warn sends a warning but allows; audit logs violations to the audit log without blocking
ResourceQuota + LimitRange interaction — quota counts resource usage; LimitRange injects defaults so containers without explicit limits still consume against the quota
edit ClusterRole — allows full CRUD on most namespaced resources but cannot modify RBAC, ResourceQuotas, or LimitRanges — safe for developer grants
Namespace-scoped vs cluster-scoped resources — RBAC Roles only cover namespace resources; nodes, PersistentVolumes, and ClusterRoles require ClusterRoleBindings