Install the Operator

Prerequisites and the Helm install of the Kubemoot operator; other components install separately.

This page installs the Kubemoot operator: what a cluster needs before it runs, and how to install it on your own cluster - the accelerator-backed deployment Kubemoot is built for. If you want to try the protocol first without an accelerator, the Quickstart runs a smaller CPU-only trial with its own script and its own limits, described below. To see where the operator, the model server, and your machines sit in a cluster before you start, read How the Pieces Fit Together.

The operator is one part of the ecosystem. The others install separately, each from its own page:

  • kmctl, the command-line tool: install it from the release (the README holds the commands, with checksum and signature verification for Linux, macOS, and Windows), then see the kmctl User Guide.
  • Crews, packaged as Helm charts, including the reference crew: Crews and Pilot.
  • CrewForge, the VS Code extension: CrewForge.
  • Integrations that reach a crew from elsewhere, such as Claude Code: Integrations.

Prerequisites

  • amd64 (x86-64) nodes. amd64 is the processor family of Intel and AMD PCs; arm64 is the family of the Raspberry Pi and Apple silicon. Kubemoot’s release images are amd64 only today, so on an arm64 node the image pulls but the container does not start: the pod shows Error or CrashLoopBackOff, and its log (kubectl logs) says exec format error. The Roadmap tracks arm64 images.
  • A Kubernetes cluster (v1.30+) and kubectl configured against it. Cluster-admin is required for the initial install, since Kubemoot installs CRDs and cluster-scoped RBAC.
  • Helm (v3.8+, for OCI charts). The operator ships only as a Helm chart.
  • A model server - at least one inference server on an accelerator that serves the models your crew will use. vLLM is the default engine: a moot sends overlapping calls to the same model, and vLLM served those faster than Ollama in the measured results. Install it with the kubemoot-vllm chart, as Install a model server below shows. Ollama stays supported, and is the engine of the Quickstart. A crew composes capability from several small models, so a single modest accelerator is enough to start. The accelerator must be visible to the worker node, on bare metal or passed through to a VM (see GPU Worker Nodes).
  • NATS JetStream - the message bus crews deliberate over. It can run in-cluster; the operator publishes discussion and event traffic to it. Install it with the official NATS Helm chart with JetStream enabled.
  • A vector store (pgvector) - only if a crew uses RAG knowledge sources. Not required for a discussion-only crew. See pgvector for PostgreSQL with vector search.

CPU trial vs. accelerator deployment

Kubemoot does not require an accelerator to run. The Quickstart runs on a node with 2 CPUs and 8 GB free and no accelerator, on Ollama, and its CPU trial profile table lists what differs from an accelerator deployment: smaller models, a smaller crew, and slower answers. The rest of this page installs the accelerator-backed profile, on vLLM.

Install a model server

Install the model server before the operator, because the operator is pointed at it. vLLM is the default engine. The kubemoot-vllm chart installs it as plain Kubernetes objects: one pod on one accelerator that runs a vLLM server for each model you list, a Service with a port per model, and a volume for the weights. Describe the pod, the volume, and the models in a values file (the full set of values, with the node and memory requirements, is in Run vLLM on a GPU node):

helm install kubemoot-vllm oci://ghcr.io/kubemoot/charts/kubemoot-vllm \
  --namespace vllm --create-namespace -f values.yaml

Each model’s parameters and capabilities go in those values, because vLLM cannot report them. The chart publishes them on its Service, and the operator reads them into the cluster’s model catalog, so no Model object is needed. The first start downloads the weights and compiles each model, which takes minutes.

Ollama is the alternative, and the engine the quickstart uses. It suits CPU-only and small setups, and hosts without the memory to park vLLM weights. Deploy it as an ordinary workload, for example with the community Ollama Helm chart, and point the operator at its Service as shown next.

Install the operator

The operator ships as the kubemoot-operator Helm chart. The chart installs the CRDs, the controller deployment, and its RBAC.

helm upgrade --install kubemoot-operator \
  <chart-reference> \
  --namespace kubemoot --create-namespace

Replace <chart-reference> with the chart source. From a checkout of the repository you can install the bundled chart directly:

helm upgrade --install kubemoot-operator \
  operator/chart/kubemoot-operator \
  --namespace kubemoot --create-namespace

The published chart is oci://ghcr.io/kubemoot/charts/kubemoot-operator; pass it as the chart reference with --version for a specific release.

Point the operator at your model server

The chart’s modelServers value is empty, because a chart cannot point at a server that does not exist. Once your model server is up, list it in a values file. For the vLLM release installed above:

# values.yaml for kubemoot-operator
modelServers:
  - name: vllm-gpu
    type: vllm
    sleepLevel: 1            # the same level as the kubemoot-vllm release
    serviceRef: {name: kubemoot-vllm, namespace: vllm}

For Ollama, name the endpoint and the models the cluster should have; Kubemoot downloads each one the server lacks:

modelServers:
  - name: ollama-gpu
    type: ollama
    endpoint: http://ollama.ollama:11434
    models:
      - model: qwen3:14b

Pass the file with --values values.yaml to the helm upgrade --install above. The chart creates a ModelServer for each entry, and a Model in the operator’s namespace for each models entry. Crews ship no Models: they select from this catalog by label, so changing the engine or adding a server changes the catalog and no crew.

Optional: a placement service

The placement service is the one writer that decides where each model call runs. It picks the model and the accelerator at the moment of the call and owns the call tickets. Agents that run a harness behind a Moot Mesh sidecar ask it where each call runs, and an agent runtime can use it in place of placing calls in its own process. The chart ships it off by default. Turn it on with two values:

placementService:
  enabled: true      # deploy the service: two replicas, only the leader is Ready
  agentsUse: true    # point agents at it

placementService.url overrides the address agents call; empty means the in-cluster Service on port 8080. The value reaches each sidecar as placement.serviceURL in the agent’s contract. With the service off, every sidecar uses the model server its Agent was bound to and no call is placed afresh, and an agent runtime places calls in its own process. Agents without a harness block are not affected by sidecar placement.

Optional: the coordinator log, and how long discussions are kept

Two sets of chart values concern what the coordinators and the discussion stream keep. The coordinator log and resume are on by default; leases and the stream limits keep the behavior Kubemoot has always had.

coordinator:
  eventLog: true       # each coordinator writes its decisions to a log before acting on them
  resume: true         # a restarted coordinator picks up its threads from that log (needs eventLog; one replica)
  resumeMaxAge: 30m    # a thread idle longer than this is closed, not resumed; 0 for no limit
  lease: false         # opt in: true lets more than one coordinator replica run (needs eventLog and resume)

nats:
  streams:
    discuss:           # the stream every crew's discussion messages travel on
      maxMessages: 5000   # messages kept across all crews; -1 for no limit
      maxAge: 24h         # how long a message is kept
      maxBytes: -1        # for example 256MiB; -1 for no limit

coordinator.eventLog and coordinator.resume are on by default: a restarted coordinator picks up its threads where it left them. To keep the behavior of earlier releases, set both to false; the chart refuses resume with the log off, so a release that turns the log off turns resume off too. They are described in The coordinator log, which also says how to size the stream values. coordinator.lease is described in Leases and more than one replica: turn it on, then set spec.deployment.replicas: 2 on a crew’s coordinator Agent for a coordinator that survives a node loss or a rolling update without losing the discussions in flight. With it off, the operator runs every coordinator Agent with exactly one replica and the Recreate strategy, whatever spec.deployment.replicas asks for, and says so in the Agent’s ReplicasLimited status condition. The job that creates the NATS streams applies nats.streams.discuss.* on every install and upgrade, to an existing stream too.

Optional: the dashboard

The operator chart carries the dashboard as an optional subchart. It is off by default because the dashboard has no login and can purge NATS streams and delete models. To install it with the operator:

helm upgrade --install kubemoot-operator \
  oci://ghcr.io/kubemoot/charts/kubemoot-operator \
  --namespace kubemoot --create-namespace \
  --set dashboard.enabled=true

It has no route; reach it with kubectl port-forward, as the dashboard page shows. The dashboard chart is also published by itself if you prefer a separate release.

Optional: metrics and Grafana dashboards

Prometheus and Grafana are a recommended add-on, not a prerequisite. If your cluster runs the Prometheus Operator, the operator chart creates PodMonitors for Kubemoot’s metrics, and it installs Grafana dashboards for them. Both are values of the same chart, and Observability covers the labels your Prometheus selects on, how to see the live list of metrics, and how to load the dashboards into your Grafana.

Optional: GitOps with Flux

If you already manage your cluster with GitOps, you can skip the helm command above. In a GitOps setup the operator is reconciled from the chart by a Flux HelmRelease pointed at an OCIRepository, rather than installed imperatively. Set install.crds: Create and upgrade.crds: CreateReplace so CRDs are managed with the release. This is how the reference deployment runs Kubemoot.

Upgrading

Kubernetes MCP servers get their access from the crew chart

Earlier releases bound the operator chart’s Kubernetes MCP server to a ClusterRole in every namespace (kubemoot-kubernetes-mcp-binding), and a later release bound crews’ Kubernetes MCP servers from namespace labels (kubemoot.ai/crew-read, crew-read-secrets, crew-write). The operator now grants neither. A crew grants its own tool servers’ access with a ServiceAccount, roles and bindings in its chart, so upgrade in this order:

  1. Upgrade each crew chart, listing the namespaces its Kubernetes servers should read (for the reference crew, kubernetesAccess.namespaces; see its cluster permissions).
  2. Upgrade the operator. When it starts it deletes the cluster-wide kubemoot-kubernetes-mcp-binding, the RoleBindings the label controller made (labels app.kubernetes.io/managed-by=kubemoot-operator and app.kubernetes.io/component=kubernetes-mcp-read-consent) and the kubemoot-kubernetes-mcp-namespaces ClusterRoleBinding. You do not delete them by hand. Upgrading the operator before the crew chart leaves the crew without read access until the chart is upgraded.
  3. Remove the namespace labels. They do nothing now.

See Namespaces a Kubernetes tool may read.

The MCPServer create and update rights the chart gave that server cluster-wide now apply in the release namespace only.

Verify

kubectl get pods -n kubemoot
kubectl get crds | grep kubemoot.ai

You should see the operator pod Running and the Kubemoot CRDs registered (crews, agents, models, modelservers, mcpservers, mcpgateways, ragsources, promptmodules, and the cluster-scoped kubemootconfig).

Put a model server on your accelerator nodes

Kubemoot does not install model servers, and it does not pick accelerator nodes for you. The seam is deliberate: placement is yours, capacity is discovered. Getting an accelerator into a worker node in the first place, including VM passthrough, is covered in GPU Worker Nodes.

  1. You run a model server per accelerator worker node. A common topology is a worker node per accelerator, and a cluster scales by adding such nodes. For every accelerator node, deploy its own Ollama (or other server) as an ordinary workload pinned to that node by node affinity, requesting the accelerator resource your device plugin exposes (for NVIDIA GPUs, nvidia.com/gpu with the runtime class your cluster uses). Each such server becomes one ModelServer with its accelerator’s memory, and the scheduler places models across the servers, loading and evicting them on demand. The CPU trial runs a single Ollama Deployment with no accelerator at all.
  2. You declare a ModelServer per model server. One manifest for each: the type, that server’s Service URL, a weight, and, when the host cannot be measured, a memory budget. Two accelerator nodes, two model servers, two ModelServers. This is the only static declaration the operator needs.
  3. The operator discovers the rest. It probes the server for available and loaded models, follows the Service to its backing pod, records that pod’s node, reads OLLAMA_NUM_PARALLEL, and, when a Prometheus endpoint is configured, queries the DCGM metrics for that node to learn the GPU model and its total and used VRAM (NVIDIA GPUs; other accelerators use the memory budget). Without DCGM, spec.scheduling.memoryMiB is the budget.
  4. Agents choose a server per call. At each inference call the runtime fits the model it needs against every ready server’s discovered headroom and picks one. So “sized to accelerator memory” in the table above means sized to the memory discovered on the server that serves the call, not to the largest accelerator in the cluster: a 32B model needs a server whose accelerator holds it, and smaller roles land on the smaller one.

The model server comes first, then the declaration. Besides the chart value shown above, a ModelServer is an ordinary manifest. The example below declares two Ollama servers; for a vLLM server, set type: vllm and serviceRef instead of endpoint, as Run vLLM on a GPU node shows.

apiVersion: kubemoot.ai/v1alpha1
kind: ModelServer
metadata:
  name: ollama-gpu-a          # the Ollama pinned to accelerator worker node A
  namespace: kubemoot
spec:
  type: ollama
  endpoint: http://ollama.ollama-a:11434
  scheduling:
    weight: 100
---
apiVersion: kubemoot.ai/v1alpha1
kind: ModelServer
metadata:
  name: ollama-gpu-b          # the Ollama pinned to accelerator worker node B
  namespace: kubemoot
spec:
  type: ollama
  endpoint: http://ollama.ollama-b:11434
  scheduling:
    weight: 100

The full field reference, discovery sources, and the budget rules are in Models.

What you’ll see before a model server is ready

The operator, a crew, and its agents can all exist as Kubernetes resources before a ModelServer is reachable. Nothing crashes; nothing answers either. Knowing what that state looks like saves you from wondering whether the install is broken:

  • ModelServer reports status.phase: Failed and status.ready: false with a message such as Failed to connect to Ollama: ... until the operator can reach the endpoint and parse a version response. Once it can, status.phase becomes Ready.
  • Agent reports status.phase: Unschedulable and status.ready: false with a message such as no model in the cluster catalog ... satisfies the require of phase=mulling (or a more specific scheduling reason) when the model catalog holds no model that satisfies its declared capabilities; an empty catalog says that no server reports a model. An unschedulable agent gets no Deployment at all - there is no crash-looping pod to find, because none was created. The operator keeps retrying on a backoff until a server appears.
  • Crew follows its agents. It reports status.phase: Pending until its coordinator Agent is Running, and Degraded with status.ready: false when the coordinator is Unschedulable or when every specialist is, with a message naming the agents concerned (coordinator coordinator is unschedulable: no feasible Model for mulling phase, or no specialist can be scheduled: k8s-advisor, ...). A crew with a running coordinator and at least one schedulable specialist is Ready, and its message lists any specialists still unschedulable. kubectl get crews -A therefore tells you whether a crew can answer before you ask it.

Once a ModelServer becomes Ready and the operator reconciles, agents move to Running and pick up their Deployments without you having to do anything.

Next