Values Reference
Deploy the Hades build system (API, Scheduler, Operator, and NATS broker) into any Kubernetes cluster using Helm.
This file is generated from
Readme.md.gotmpl; the Values Reference table is produced by helm-docs from the comments invalues.yaml. Runmake docs-helmfrom the repository root to regenerate it. Edit the.gotmpl(notReadme.md) for prose changes.
Contents
Overview
This chart bundles the core services of Hades:
| Component | Description |
|---|---|
| hades-api | Processes and validates the request and produces the build request as NATS events |
| hades-scheduler | Consumes NATS events and translates each into a BuildJob custom resource |
| hades-operator | Watches BuildJob CRs and reconciles them into Kubernetes batchv1.Jobs with one container per step |
| hades-log-manager | Aggregates per-job build logs from NATS and forwards them to the Artemis adapter |
| hades-nats | Embedded NATS JetStream message broker (sub-chart) |
The Scheduler does not create Pods directly; instead it creates BuildJob CRs, which the operator reconciles into Kubernetes Jobs.
Prerequisites
- Kubernetes v1.25+
- Helm v3.12+
Quick Start
-
Install the NATS sub-chart (if not already installed)
helm repo add nats https://nats-io.github.io/k8s/helm/charts
helm dependency build ./helm/hades/ -
Adjust the values in
values.yamlas needed. (e.g., the hostname)cat ./helm/hades/values.yaml -
Install the chart (default: operator mode)
helm upgrade --install hades ./helm/hades -n hades --create-namespaceor if you prefer to use the
--setflag to override values directly in the command line, you can do so like this:helm upgrade --install hades ./helm/hades -n hades --create-namespace \
--set ingress.host=hades.example.com \
--set ingress.tls.secretName=my-secret
In the above command:
The first "hades" is the Helm release name, i.e., the name Helm will use to track this deployment. You can change this to any name (e.g., hades-dev, ci-release).
The second "hades" after -n is the Kubernetes namespace where the resources will be deployed. This namespace will be created automatically if it does not exist using --create-namespace
- Tail the Scheduler logs to verify connectivity
kubectl -n hades logs deploy/hades-scheduler -f
You maybe have to wait a few seconds until the NATS broker is set up.
Expected healthy log lines:
INFO Connected to NATS server url=nats://hades-nats.hades.svc:4222
INFO Started HadesScheduler in Kubernetes mode
INFO Using operator mode (dynamic client)
Configuration
All user-configurable options live in values.yaml. The default mode is operator.
Values Reference
The table below is generated by helm-docs from the comments in values.yaml. Regenerate it with make docs-helm. Values without a description (image repository/tag, internal ports) are omitted.
| Key | Type | Default | Description |
|---|---|---|---|
| debug | bool | false | Global debug switch. When true, every Hades container gets DEBUG=true (and the operator additionally gets DEV_MODE=true) for verbose logging. |
| hadesApi.authKeySecret | string | "" | Name of an existing Secret holding key AUTH_KEY to protect the /build endpoint. Empty = no auth (endpoint is open). |
| hadesApi.dashboard | object | {"cookieInsecure":false,"jobRetention":"1h","logManagerURL":"http://hades-log-manager-service:8081","redactMode":"","secretKeyPatterns":"","secretName":"","sessionTTL":"12h","trustedProxies":[]} | Operator dashboard (web UI) served by the API. |
| hadesApi.dashboard.cookieInsecure | bool | false | Drop the Secure flag on the session cookie. Only for local HTTP dev; never enable in a real deployment (requires TLS otherwise login breaks). |
| hadesApi.dashboard.jobRetention | string | "1h" | How long finished jobs stay in the dashboard's in-memory list (Go duration). |
| hadesApi.dashboard.logManagerURL | string | "http://hades-log-manager-service:8081" | Internal URL of HadesLogManager used by the dashboard's logs proxy. |
| hadesApi.dashboard.redactMode | string | "" | Secret redaction mode for job metadata: "smart" (default, key + value heuristics) or "all" (mask every value). Empty uses the smart default. |
| hadesApi.dashboard.secretKeyPatterns | string | "" | Optional regex overriding which metadata keys are treated as secret. Empty uses the built-in pattern (token/password/secret/key/credential/...). |
| hadesApi.dashboard.secretName | string | "" | Name of an existing Secret with keys DASHBOARD_USERNAME, DASHBOARD_PASSWORD_HASH (a bcrypt hash), and DASHBOARD_SESSION_SECRET (>=32 chars). Empty = dashboard disabled (its /api routes return 503 and the SPA is not served). Requires TLS (the session cookie is Secure). |
| hadesApi.dashboard.sessionTTL | string | "12h" | Session cookie lifetime (Go duration). |
| hadesApi.dashboard.trustedProxies | list | [] | CIDRs/IPs of front proxies (e.g. the ingress) allowed to set X-Forwarded-For. Empty trusts none, so the login lockout keys on the un-spoofable direct address. Set to your ingress range behind a proxy. |
| hadesApi.image.pullPolicy | string | "Always" | Image pull policy for the API. |
| hadesApi.replicaCount | int | 1 | Number of API replicas. |
| hadesApi.resources | object | {"limits":{"cpu":"500m","memory":"512Mi"},"requests":{"cpu":"200m","memory":"256Mi"}} | Resource requests and limits for the API pod. |
| hadesApi.service.port | int | 8080 | Service port for the API. |
| hadesApi.service.targetPort | int | 8080 | Container port the API listens on. |
| hadesLogManager.image.pullPolicy | string | "Always" | Image pull policy for the log manager. |
| hadesLogManager.replicaCount | int | 1 | Keep at 1: the log manager aggregates per-job logs in memory and uses plain (non-queue, non-durable) NATS subscriptions, so multiple replicas would independently forward duplicate/split logs to the Artemis adapter. |
| hadesLogManager.resources | object | {"limits":{"cpu":"500m","memory":"512Mi"},"requests":{"cpu":"200m","memory":"256Mi"}} | Resource requests and limits for the log manager pod. |
| hadesLogManager.service.port | int | 8081 | Service port for the log manager. |
| hadesLogManager.statusWebhook.concurrency | int | 16 | Deliveries in flight at once; keeps one dead receiver from delaying other jobs. |
| hadesLogManager.statusWebhook.enabled | bool | true | Deliver the job-status webhook to each job's "status_callback_url". The feature is inert for jobs that do not set that field, so it is safe to leave enabled. The destination is per job; there is no global webhook URL. |
| hadesLogManager.statusWebhook.initialBackoff | string | "5s" | Delay before the second attempt; doubles per attempt. |
| hadesLogManager.statusWebhook.maxAttempts | int | 6 | Delivery attempts per job (first try plus retries) before the status event is dropped. |
| hadesLogManager.statusWebhook.maxBackoff | string | "5m" | Ceiling for the retry delay. |
| hadesLogManager.statusWebhook.maxPending | int | 1000 | Status events awaiting acknowledgement (in flight or waiting out a retry backoff). |
| hadesLogManager.statusWebhook.timeout | string | "10s" | Bound on a single delivery, including the callback-URL lookup. |
| hadesOperator.DeleteOnComplete | bool | false | Delete BuildJob CRs (and their Jobs) once they finish. Defaults to false so completed BuildJobs/pods linger (they self-clean via the Job TTL, ~1h) and the operator can capture every step's logs before the pod is removed; Kubernetes only serves container logs while the pod exists, so immediate deletion can lose logs for fast jobs. Set to true only once the operator captures all logs before deletion (the companion barrier fix). |
| hadesOperator.clusterWide | bool | false | Grant cluster-wide RBAC instead of namespace-scoped. |
| hadesOperator.image.pullPolicy | string | "Always" | Image pull policy for the operator. |
| hadesOperator.leaderElection.enabled | bool | true | Enable leader election for the operator (recommended with >1 replica). |
| hadesOperator.logDrainTimeout | string | "45s" | How long a completed BuildJob is kept (undeleted) while its container logs are still being published, before it is deleted anyway (Go duration). Kubernetes only serves container logs while the pod exists, so this is the safety valve on the log-drain gate. Only relevant when DeleteOnComplete is true. A zero or negative duration falls back to the default; a value that is not a valid Go duration fails operator startup. |
| hadesOperator.maxParallelism | string | "100" | Maximum number of Jobs the operator admits concurrently; excess are suspended. |
| hadesOperator.replicaCount | int | 1 | Number of operator replicas. |
| hadesOperator.requeueDelay | string | "2s" | How often the operator re-reconciles a BuildJob whose Job is still running (Go duration). The operator watches BuildJobs and the Jobs it owns but not Pods, so container transitions and job completion are only observed on these requeues: completion is detected up to this long after it happens. Lower it for faster completion signalling (or for latency measurements) at the cost of more API-server traffic. A zero or negative duration falls back to the default; a value that is not a valid Go duration fails operator startup. |
| hadesOperator.resources | object | {"limits":{"cpu":"500m","memory":"512Mi"},"requests":{"cpu":"200m","memory":"256Mi"}} | Resource requests and limits for the operator pod. |
| hadesOperator.targetNamespace | string | "" | Namespace the operator watches for BuildJob CRs (empty = release namespace). |
| hadesScheduler.concurrency | string | "4" | Number of NATS consumer workers each scheduler pod runs. This is a per-replica setting, so the total number of workers consuming the queue is concurrency * hadesScheduler.replicaCount. The meaning depends on the executor: with k8s a worker only creates a BuildJob CR and returns, so this caps how fast queued jobs are turned into CRs (the operator's hadesOperator.maxParallelism remains the real limit on running Jobs); with docker a worker blocks for the entire job, so this is the maximum number of jobs running concurrently on the Docker host. Raise it for k8s to keep the scheduler from becoming the bottleneck under a burst; raise it for docker only if the host can take the extra parallel containers. |
| hadesScheduler.executor | string | "k8s" | Executor backend (k8s for in-cluster execution). |
| hadesScheduler.image.pullPolicy | string | "Always" | Image pull policy for the scheduler. |
| hadesScheduler.replicaCount | int | 1 | Number of scheduler replicas. |
| hadesScheduler.resources | object | {"limits":{"cpu":"500m","memory":"512Mi"},"requests":{"cpu":"200m","memory":"256Mi"}} | Resource requests and limits for the scheduler pod. |
| ingress.annotations | object | {"cert-manager.io/cluster-issuer":"letsencrypt-prod"} | Extra annotations for the Ingress (e.g. cert-manager issuer). |
| ingress.className | string | "nginx" | Ingress class name. |
| ingress.enabled | bool | true | Enable the Ingress for the API. |
| ingress.host | string | "hades.student.k8s.aet.cit.tum.de" | Public hostname for the API. |
| ingress.tls.enabled | bool | true | Enable TLS on the Ingress. |
| ingress.tls.secretName | string | "hades-tls" | Secret name holding the TLS certificate. |
| monitoring | object | {"enabled":false,"port":8082,"serviceMonitor":{"interval":"30s","labels":{},"scrapeTimeout":"10s"}} | Prometheus monitoring. Every Hades service exposes a Prometheus /metrics endpoint on a dedicated, cluster-internal port (never via the public ingress); a ServiceMonitor registers them with a cluster Prometheus Operator. |
| monitoring.enabled | bool | false | Render a ServiceMonitor for the Hades services. Requires the Prometheus Operator CRDs (monitoring.coreos.com) on the cluster. The /metrics endpoints are always exposed; this flag only controls the ServiceMonitor, so the chart installs cleanly on clusters without the Prometheus Operator. |
| monitoring.port | int | 8082 | Container/Service port each service serves /metrics on. |
| monitoring.serviceMonitor.interval | string | "30s" | Prometheus scrape interval. |
| monitoring.serviceMonitor.labels | object | {} | Extra labels on the ServiceMonitor so a Prometheus instance selects it (e.g. { release: kube-prometheus-stack }). |
| monitoring.serviceMonitor.scrapeTimeout | string | "10s" | Per-scrape timeout. |
| nats.config.jetstream.enabled | bool | true | Enable JetStream (required by Hades). |
| nats.config.jetstream.memoryStore.maxSize | string | "1Gi" | Maximum in-memory JetStream store size. |
| nats.host | string | "hades-nats.hades.svc.cluster.local" | In-cluster NATS service hostname. |
| nats.port | int | 4222 | NATS client port. |
| rbac.hadesOperator.serviceAccountName | string | "hades-operator" | ServiceAccount name for the operator. |
| rbac.hadesScheduler.serviceAccountName | string | "hades-scheduler" | ServiceAccount name for the scheduler. |
| tracing | object | {"deployJaeger":false,"enabled":false,"endpoint":"","jaeger":{"image":"jaegertracing/all-in-one:1.62.0","ui":{"auth":{"existingSecret":"","password":"","username":"admin"},"ingress":{"annotations":{},"className":"nginx","enabled":false,"host":"","tls":{"enabled":false,"secretName":""}}}}} | OpenTelemetry tracing. When enabled, every Hades service exports spans to an OTLP endpoint so a job's overhead/runtime breakdown is visible as a per-job trace waterfall (API -> scheduler -> operator). Disabled by default: with no endpoint the services run a noop tracer, so the chart installs cleanly without a tracing backend. |
| tracing.deployJaeger | bool | false | Deploy an in-cluster Jaeger all-in-one as the tracing backend and UI. It stores traces in memory, so it is for dev/test clusters, not production; point endpoint at a real collector there instead. |
| tracing.enabled | bool | false | Export OTLP spans from every service. Requires a resolvable endpoint (tracing.endpoint, or the bundled Jaeger via tracing.deployJaeger). |
| tracing.endpoint | string | "" | OTLP gRPC endpoint spans are exported to (an OpenTelemetry Collector or a Jaeger with OTLP enabled), e.g. http://otel-collector.observability:4317. When empty and deployJaeger is true, the bundled Jaeger is used. |
| tracing.jaeger.image | string | "jaegertracing/all-in-one:1.62.0" | Jaeger all-in-one image used when deployJaeger is true. |
| tracing.jaeger.ui | object | {"auth":{"existingSecret":"","password":"","username":"admin"},"ingress":{"annotations":{},"className":"nginx","enabled":false,"host":"","tls":{"enabled":false,"secretName":""}}} | Bundled-Jaeger UI exposure. The UI Service is ClusterIP (in-cluster / port-forward only) by default; enable this Ingress to reach it from outside. It is protected by HTTP basic auth - rendering fails if enabled without a password or an existing secret. IMPORTANT: the basic auth is enforced with ingress-nginx annotations (nginx.ingress.kubernetes.io/auth-*), so the login only takes effect when className routes to an ingress-nginx controller. If you point it at a different controller (Traefik, HAProxy, ...), those annotations are ignored and the UI would be exposed WITHOUT auth - provide equivalent auth for that controller instead, or keep it ClusterIP + port-forward. |
| tracing.jaeger.ui.auth.existingSecret | string | "" | Use a pre-created basic-auth Secret (an auth key holding htpasswd lines). When set, username/password are ignored and no Secret is generated. |
| tracing.jaeger.ui.auth.password | string | "" | Basic-auth password. Required when the UI Ingress is enabled and no existingSecret is given; the chart hashes it into an htpasswd Secret. |
| tracing.jaeger.ui.auth.username | string | "admin" | Basic-auth username for the generated htpasswd Secret. |
| tracing.jaeger.ui.ingress.annotations | object | {} | Extra annotations merged onto the UI Ingress. |
| tracing.jaeger.ui.ingress.className | string | "nginx" | Ingress class for the UI. Basic auth requires an ingress-nginx controller (see the note above); other controllers ignore the auth annotations. |
| tracing.jaeger.ui.ingress.enabled | bool | false | Expose the Jaeger UI via an Ingress (behind ingress-nginx basic auth). |
| tracing.jaeger.ui.ingress.host | string | "" | Hostname for the Jaeger UI. Required when enabled. |
| tracing.jaeger.ui.ingress.tls.enabled | bool | false | Enable TLS on the UI Ingress. |
| tracing.jaeger.ui.ingress.tls.secretName | string | "" | TLS secret for the UI host. |
Upgrade & Rollback
# Upgrade in place
helm upgrade hades ./helm/hades -n hades
CRDs are not upgraded by Helm. When a release changes the
BuildJobCRD, apply it manually from the matching chart version:helm show crds oci://ghcr.io/hades-scheduler/charts/hades --version <version> | kubectl apply -f -
Roll back to a previous release revision:
# List revision history, then roll back to a specific revision
helm history hades -n hades
helm rollback hades <revision> -n hades
Uninstall
helm uninstall hades -n hades
# Optional: delete namespace and any leftover ConfigMaps or Secrets
kubectl delete namespace hades
Development
# Lint the chart
helm lint ./helm/hades
# Render templates without deploying
helm template hades ./helm/hades -n hades