Engine Configuration

engine-config.yaml is the sole configuration surface for an engine — no code changes needed to add a new variant or adjust parameter handling. For the crate structure and how to build a custom engine binary, see Engine Internals.

Starting an Engine

ENGINE_CONFIG_PATH=/etc/metis/engine-config.yaml \
  metis-engine-nextflow

Required environment variables:

DATABASE_URL=postgres://user:pass@host:5432/metis
REDIS_URL=redis://:pass@host:6379
NATS_URL=nats://user:pass@host:4222
KEYCLOAK_SERVER=http://keycloak:8080
KEYCLOAK_REALM=metis
KEYCLOAK_AUDIENCE=metis-engine
SERVICE_CLIENT_ID=metis-engine
SERVICE_CLIENT_SECRET=<secret>

Environment Variable Substitution

Config values support ${VAR:default} placeholders that resolve from environment variables at startup. If the variable is unset and a default is provided, the default is used. If no default is provided and the variable is unset, the placeholder is left as-is.

runs:
  workdir:
    workdir: "${METIS_WORKDIR_BASE:/nextflow/runs}/{user_id}/{run_id}"

This allows baking engine behavior into the Docker image while letting infra control paths and storage via Helm env vars.

Workdir Configuration

The workdir field is a template string that resolves per-run. Supported template variables: {user_id}, {run_id}, {date}, {time}, {timestamp}.

runs:
  workdir:
    workdir: "/var/wes/runs/{user_id}/{run_id}"
    subdirs:
      logs: "logs"
      outputs: "outputs"
      work: "work"
      tmp: "tmp"

Per-Run Config Template

Engines can generate a config file per run. The content field supports both ${ENV:default} (resolved at startup) and {template_var} (resolved per-run).

runs:
  config:
    filepath: "nextflow.config"
    content: |
      workDir = "${METIS_RUN_WORK_DIR:{workdir}/work"
      params {
        outdir = "${METIS_RUN_OUTPUT_DIR:s3://bucket/{user_id}/{run_id}/outputs"
      }

Parameter Fields Reference

Field	Type	Required	Description
names	[string]	yes	Accepted aliases. First is canonical
cliFlag	string	yes	Flag written to the CLI command
type	string	yes	string, int, bool, list
default	any	no	Value when client omits the param
strictDefault	bool	no	Always use default, ignore client
required	bool	no	Reject run if missing, no default
listSeparator	string	no	Separator for list type
booleanStyle	string	no	flag or value (-flag true)
sensitive	bool	no	Redact value from logs
envVar	string	no	Set as env var instead of CLI flag
validate	[Spec]	no	List of validation rules

Validation Types

Type	Fields	Description
enum	allowed, message	Value must be in list
regex	pattern, message	Value must match regex
range	min, max	Numeric value in range
file_exists	—	File must exist on host
file_extension	allowed	Must end with extension

Denied Parameters

Denied params are evaluated against raw keys before alias resolution.

Field	Values	Description
pattern	string	Pattern to match against
type	exact, regex	Match mode
reason	string	Error message to client

workflowParams Methods

Method	Behaviour
inline	Params appended as CLI flags
file	Params written to file, path in template

Registering a New Engine

The engine registers its heartbeat in Valkey and appears in GET /service-info. Submit runs using a workflow_engine that matches engine.name. For custom parsing or hooks, implement the Engine trait and build a custom binary. See Engine Internals.