Deployment

How to run MicroDCS on Kubernetes, which limits apply, and how islands and tenants are separated. Security settings and the trust model are in Security.

Shape of a deployment

One island is one independent deployment with its own Valkey (D-12): a plant area, a line, a site. Inside an island the same image runs in two roles (D-51):

Roles=All (the default) runs both in one process, for development and small sites.

The manifest

deploy/ holds an example to adapt, not a chart:

• config.yaml: the ConfigMap with the settings both Deployments share.
• k8s.yaml: ServiceAccount, the two Deployments, a PodDisruptionBudget for the processors and three NetworkPolicies (default deny, egress, management ingress).
• kustomization.yaml: pins the image. Set the digest of an image you verified (Verifying the image): kustomize edit set image ghcr.io/aschamberger/microdcs=ghcr.io/aschamberger/microdcs@sha256:<digest>. Until then it names the moving main tag and sets imagePullPolicy: Always, so no node keeps an older main build; remove that patch with the digest.
• valkey-island-acl.conf: the Valkey ACL user of an island (Tenancy).

kubectl create secret generic microdcs-management --from-file=token=./management-token -n <ns>
kubectl create secret generic microdcs-valkey --from-file=password=./valkey-password --from-file=ca.crt=./valkey-ca.crt -n <ns>
kubectl create secret generic microdcs-mqtt --from-file=ca.crt=./broker-ca.crt -n <ns>
kubectl apply -k deploy/ -n <ns>

Settings are environment variables with the prefix APP_ and __ for : (D-39); the shared ones sit in the ConfigMap, the role, the instance id (POD_ID from the pod name) and the paths of the mounted secrets in each Deployment. The NetworkPolicy expects the labels app.kubernetes.io/name=valkey, =mqtt-broker and =otel-collector on the dependencies; change the selectors to match yours, and add NATS (4222) when a binding uses it.

Probes and startup

The management listener binds the pod address (Url=http://0.0.0.0:8081), which needs BearerTokenFile; the probes stay open.

A pod that starts before Valkey or the broker accepts connections ends, and Kubernetes starts it again with an increasing back-off (up to five minutes). That is the fail-fast rule (D-27), not an error, but a long back-off delays the start after the dependency is up. Start the dependencies first, or delete the waiting pods once they are ready.

terminationGracePeriodSeconds: 45 is longer than ShutdownGracePeriod (30 s), so the drain finishes before the kubelet kills the process (D-37).

Resources

The requests and limits in the example (processor 100m to 1 CPU, 128 to 256 MiB; publisher 50m to 500m, 96 to 192 MiB; 16 to 64 MiB ephemeral storage) are starting points from the kind test, not measurements under load. Size them with the limits below: the memory a processor needs grows with MessageWorkers and OutgoingQueueMaxSize times the largest message.

Testing the manifest

scripts/k8s-test runs deploy/ in a kind cluster with the overlay in tests/k8s (no TLS, no collector, the image built locally) and checks:

• both Deployments become ready through their probes in a namespace that enforces the Pod Security Standard restricted;
• nothing restarts after the rollout;
• the egress policy lets a MicroDCS pod reach DNS, Valkey and the broker, and nothing else in or outside the cluster;
• the management port answers only pods with the client label, and /api/v1/jobs needs the token;
• Valkey runs with the default user off and the restricted ACL user from Tenancy, and denies nothing.

scripts/k8s-test                 # build, create the cluster, test, delete it
KEEP_CLUSTER=1 scripts/k8s-test  # keep the cluster for debugging; a rerun reuses it

It needs Docker, kind and kubectl; scripts/k8s-test install <dir> installs the pinned versions with checksums. CI runs it in the kubernetes job on main, daily, and on pull requests that change the manifest or the image inputs (D-56).

Not verified yet

These need a real cluster or a long run and are open:

• TLS to the broker and to Valkey inside a cluster, and the Kubernetes service account token (K8S-SAT) with a broker that supports it.
• An admission policy (Sigstore policy-controller, Kyverno) enforcing the signature identity.
• NetworkPolicy behaviour on other network plugins (Calico, Cilium), including whether kubelet probes are exempt from the ingress policy as they are with kindnet.
• Several nodes: the PodDisruptionBudget during a drain, scheduling spread, the publisher lease hand-over in a rolling update under load.
• Resource sizing under load, and the arm64 image on a cluster.

Operational limits

MicroDCS sets no message size limit of its own: custommetadata is unbounded (D-29), and the OPC UA JSON decoder of the job models has no string, array or message limit (only its nesting limit), because recipes are opaque and can be large. The transport bounds a message, so set its limit deliberately.

Memory bound per processor, roughly: MessageWorkers × largest inbound message + outgoing queue entries × largest outbound message.

Tenancy

The recommended separation is one island per tenant: own namespace, own Valkey, own broker credentials. Inside shared infrastructure:

Valkey ACL user for the island with KeyPrefix=site1. The Valkey integration tests run the production code as such a user (valkey-acl in the compose harness), and the kind test runs the application with it:

# Valkey ACL user of one MicroDCS island (docs/deployment.md#tenancy), as a valkey.conf directive. Replace `site1` with the island's
# MicroDcs:Valkey:KeyPrefix and PASSWORD with its password (or use ACL SETUSER with the same rules). The Valkey integration tests run
# the production code as this user, and scripts/k8s-test checks that tests/k8s uses the same line.
#
# Granted: the island's keys and channels, the data, stream, scripting and pub/sub commands, TIME (server clock of the SFC host).
# Taken away again: everything server-wide that would let one island harm or watch another on a shared Valkey: @dangerous
# (FLUSHALL, SWAPDB, KEYS, CLIENT KILL/LIST/PAUSE, CONFIG, INFO, ...), FLUSHDB, SELECT, MOVE, COPY, RANDOMKEY, DBSIZE, functions,
# SCRIPT FLUSH/KILL, PUBSUB introspection, client tracking. SCAN stays for the index and maintenance tools; it lists key names of
# every island, which is one reason to give each island its own Valkey.
user microdcs on >PASSWORD resetkeys ~site1:* resetchannels &site1:* -@all +@read +@write +@keyspace +@stream +@hash +@sortedset +@set +@string +@list +@scripting +@pubsub +@connection +@transaction +time -@dangerous -flushdb -select -move -copy -randomkey -dbsize -function -fcall -fcall_ro -script|flush -script|kill -pubsub -client|tracking

All keys, including the dedupe claims, leases and the scope registry, and the nudge channels sit under {KeyPrefix}:. MicroDCS turns off the client's housekeeping that would reach outside the prefix (tiebreaker key, configuration channel) or read server-wide data (INFO, CONFIG, CLUSTER). The rule takes away every server-wide command that would let one island flush, kill or watch another; a test checks that the user is refused them. SCAN stays because the index and maintenance tools need it, and it lists the key names of every island: on a shared Valkey the islands see each other's job and scope ids, not their contents.

Support

The project has one maintainer and no service-level agreement. Operators own the deployment, the dependencies (broker, Valkey, Kubernetes) and their hardening; the split is in the IEC 62443 table.