Agent CRD

Overview

The Agent CRD declares an LLM-powered agent as a thin, capability-only resource and deploys it as a Kubernetes-native service. The Agent spec carries no model name, no server, and no accelerator hint. Agents declare what they can do; the scheduler matches them to a (model, server, endpoint) via CrewSchedulingPolicy rules over labeled Model CRs.

Agents orchestrate:

  • Models: LLM inference resolved by the scheduler from spec.capabilities + CrewSchedulingPolicy (see scheduler.md)
  • Knowledge: RAG retrieval via spec.ragSources
  • Tools: MCP tool execution via the MCPGateway, filtered by spec.enabledTools / spec.disabledTools
  • Collaboration: Multi-agent discussions via NATS JetStream, configured by spec.discussRole / spec.discussChannels / spec.discussKeywords
  • Prompts: System prompts composed from PromptModule CRs referenced by spec.promptRefs

Supported Types

TypeDescription
chatConversational agent with HTTP API
taskTask-oriented agent (planned)
workflowMulti-step workflow orchestration (planned)

Spec Fields

spec.type

Agent type. Default: chat. Only chat is implemented today.

spec.description

Human-readable description of the agent’s purpose. Shown in the dashboard. Keep it factual and short - one sentence.

spec.capabilities[]

Required. Open-string label set declaring what this agent can do. Matched by CrewSchedulingPolicy.spec.rules[].require.matchLabels (as capability/<name>: "true") against the labels on candidate Model CRs.

Common values: tool-calling, reasoning, kubernetes, proxmox, observability, harbor, git, helm, web-search.

spec.discussRole

Role this agent plays in a discussion thread.

ValueBehavior
generic (default)General contributor. Participates in discussions but does not hold the Tooler raw-output contract. Use when an agent is not a domain Tooler, Analyst, coordinator, or researcher.
toolerTooler role: subscribes to channels, calls MCP tools in the EVALUATING phase, participates with signals (agree, concern, stand_aside, failure, block). Thinking is OFF for deterministic tool selection.
analystAnalyst role: participates in the REVIEW phase with thinking ON. Carries RAG sources and reasons over the data Toolers gathered. No live MCP tools.
coordinatorReceives user queries, generates advisory, runs triage, synthesizes responses. KUBEMOOT_DISCUSS_COORDINATOR=true, KUBEMOOT_DISCUSS_TOOLER=false
researcherContributes to synthesis but excluded from settle triggers, fast path, and gap detection. Used for non-settle-gating agents such as internet search that augment the discussion without owning a definitive answer.

spec.discussChannels[]

NATS discussion channels the agent subscribes to. The coordinator should list every channel the crew uses (e.g., [kubernetes, observability, proxmox, general]); a Tooler or Analyst lists only its domain (e.g., [kubernetes]). Channels are metadata, not routing gates - subcommittee selection is the coordinator’s job (via the resume model), not channel routing.

spec.discussKeywords[]

Optional. Domain keywords that become part of this agent’s resume (alongside its description, tools, and role). The coordinator’s resume model embeds the resume and matches it semantically against each question to pick the subcommittee, so keywords inform selection without being a literal gate. Set to ["*"] to opt into every thread (used by researcher agents like internet-search).

spec.domains[]

Optional. The subject areas this agent works in, for example [proxmox, hypervisor]. The coordinator uses them to decide which Analysts review a discussion: an Analyst reviews when one of its domains is a domain of the question (the coordinator names them when it selects the subcommittee) or of a Tooler that gathered data. See Who reviews. Declare domains on every Tooler and Analyst of a crew; each value is compared as lower-case words (Proxmox-VE is proxmox and ve). An analyst that declares none is a generalist: in domain mode it reviews every discussion (the arithmetic analyst, for one). discussKeywords do not stand in for domains, because they are broad words for selection, not subject areas.

spec.promptRefs[]

Required. Ordered list of PromptModule names whose content is concatenated by order to form the agent’s system prompt. The composed text is written to a ConfigMap and mounted at /etc/kubemoot/policy/system.txt. All prompt text MUST live in PromptModules; inline system prompts are not supported.

See the PromptModule section below for the order-range conventions.

spec.triageSummary

Optional short description (≤200 chars) the coordinator’s triage LLM call sees when selecting participants. Distinct from spec.description (for humans); this string is in the triage prompt and should be optimized for the LLM’s pattern matching.

spec.enabledTools[] / spec.disabledTools[]

Optional. Whitelist / blacklist of MCP tool names. Resolved by the agent runtime after MCPGateway tool discovery. If both set, enabledTools takes precedence.

Set deliberately small. qwen3:32b reliably handles ~10-15 tools; beyond that, tool-call reliability drops sharply. The operator injects the resolved list as KUBEMOOT_ENABLED_TOOLS / KUBEMOOT_DISABLED_TOOLS.

A non-empty enabledTools is also a requirement. When a discussion evaluation starts, the agent checks the tools it declared (less any in disabledTools) against the tool list it fetched from the gateway. When the declared tools are missing, the agent publishes a failure signal instead of calling the model. A tooler needs every declared tool. Any other role needs at least one, and runs with the rest. An agent that declares no tools is not checked. See Bounded Tool Retries and Failure Signals.

A RAGSource’s search tool is listed here as <source>_search, for example kubectl-reference_search. It is generated by the operator and reaches the agent through the crew’s MCPGateway like any other tool.

Scheduling tools (set_reminder, schedule_followup, list_scheduled, cancel_scheduled) are provided by the scheduling-mcp MCP server, see Agent Self-Scheduling. Add a dedicated Tooler Agent CR for them rather than enabling on every agent.

spec.workspace.access

Optional. What the agent may do with the discussion’s workspace, the scratch data the agents of one discussion share (see Discussion artifact store). One of:

ValueReads the workspaceStores its tool results
nonenono: a large result returns inline under the runtime’s budget
readyesno
writenoyes: for a tooler that fetches and hands off
readWriteyesyes

Left out, the Crew’s spec.workspace.defaultAccess applies, then the KubemootConfig’s workspace.defaultAccess. With none of the three declared the access is implicit: the agent behaves as it always has. It stores every tool result, and it reads the workspace only through the tools its enabledTools names.

A declared access is authoritative. The operator disables every workspace tool the access does not grant (the workspace_* tools and the older artifact_* names), even when enabledTools names it, and adds the tools a read or readWrite access grants to a non-empty enabledTools. A tool named in disabledTools stays denied. The operator sets KUBEMOOT_WORKSPACE_ACCESS on the agent runtime.

Read is enforced through the agent’s tool list and write by the agent-runtime, which stores nothing without write access. NATS does not enforce either yet: nothing provisions per-agent NATS users, and that enforcement lands with user provisioning. The workspace server scopes reads by discussion, not by agent, so access does not isolate two agents of one crew from each other. Other routes read the workspace whatever the tool list says: the code sandbox’s /kubemoot/workspace mount (also mounted at /artifacts, a second mount of the same volume, for one release; the materializer copies every object of the namespace), the coordinator’s catalog of stored keys (an agent with none or read still receives it in the discussion), and direct MCP mode, where tool names carry a server prefix (server.tool) that the workspace tool names do not match.

The NoWorkspaceServer check recognizes only an MCPServer that selects the image with spec.kubemootImage; a server that names its image directly or an external endpoint is not counted. In a KubemootConfig workspace.defaultAccess, omit the field to leave agents implicit; an empty string is refused.

spec.ragSources[]

Optional. RAGSource references the runtime searches up front: before every model call it queries each listed source with the question and puts the top results in the system prompt.

To get reference text on demand instead, leave the source out of ragSources and list its search tool in spec.enabledTools. The operator generates the tool, named <source>_search, for each RAGSource that runs a query service; see Agent Context. Naming a source in both places searches it up front and also offers the tool.

FieldTypeDescription
namestringRAGSource resource name
topKint32Results to retrieve. Default: 5 - keep low; large RAG payloads degrade tool-calling
priorityint32Higher = retrieved first. Default: 0

spec.temperature

String representation of a float (0.0-2.0). Default: "0.3" - required for reliable tool selection. The operator writes this to the agent’s application.properties (Quarkus BUILD_AND_RUN_TIME_FIXED constraint; not overridable via env).

spec.maxTokens

Max generation length. Default: 4096. Coordinator typically uses 8192 to fit synthesis output.

spec.deployment

Standard Kubernetes deployment overrides.

FieldTypeDescription
replicasint32Default: 1. A coordinator runs more than one only with the chart value coordinator.lease on; with it off the operator runs one replica whatever this says and sets the ReplicasLimited condition on the Agent. See Leases and more than one replica
imagestringOverride agent-runtime image. Defaults to KubemootConfig.spec.images.agentRuntime. In sidecar mode it names the harness when spec.harness.image is unset
portint32HTTP API port. Default: 8080. In sidecar mode it is the sidecar’s port, which the Service targets
resourcesResourceRequirementsCPU and memory for the agent container. See Resources
env[]EnvVarAdditional env vars - useful for KUBEMOOT_GATEWAY_ENABLED=false on coordinator, longer timeouts, etc.
runtimeClassNamestringKubernetes RuntimeClass for the pod, for example gvisor. Unset, the pod takes KubemootConfig.spec.defaults.agentRuntimeClassName; set to "", the pod uses the node’s default runtime whatever the realm default is. While the class does not exist the operator holds the Deployment and reports RuntimeClassAvailable=False. See Isolation
serviceAccountNamestringPod service account; needs the requester’s use grant when security.secretUseGrants.enabled is on. An Agent that names none gets no mounted token, see Secrets and tools

The operator sets imagePullSecrets from KubemootConfig.spec.defaults.imagePullSecrets - no per-agent override needed. That field is empty by default (public images need no pull secret); set it on KubemootConfig if you point global.imageRegistry at a private mirror.

spec.egress

Optional extra destinations the agent may reach, for a harness that browses or calls an API. It has an effect only when the operator generates network policies with security.networkPolicies.egress.mode: Declared. An agent that sets it is restricted to DNS, the message bus, the crew’s tool gateway, the model servers it can be placed on, the query service of its RAG sources, the placement service, and the telemetry collector (the operator derives these), plus what egress lists. An agent without egress is not restricted.

The fields are the same as an MCPServer’s egress: allowedHosts, allowedServices, allowedCIDRs, ports (default 443). bus has no effect on an agent, which always reaches the bus. The query service and the placement service are read from their Services; the placement service gets no allow when no Service exists behind its configured URL, and the condition says so. See EgressRules. The agent reports EgressEnforced in status.conditions; it is False with reason FQDNNotSupported when the CNI cannot enforce host names, and with reason ServiceNotResolved when a Service it needs does not exist yet. What each CNI enforces is in Network policies.

spec.harness

Names the program that does the agent’s reasoning and how its pod joins the table. Without the block the agent is a legacy agent-runtime pod, as it has always been. With mode: sidecar the pod runs the harness beside a Moot Mesh sidecar that owns the bus, model placement, and tool access. To decide which mode fits, see Choosing a harness mode.

FieldTypeDefaultDescription
namestringThe harness’s name. Shown on the dashboard and carried on every signal as metadata.harness.
modeenumlegacylegacy runs agent-runtime alone. sidecar runs the harness beside the Moot Mesh sidecar.
imagestringThe harness image. Pin anything Kubemoot did not build by digest. In sidecar mode it is required unless spec.deployment.image names the harness or name is goose, which defaults to KubemootConfig.spec.images.harnessGoose.
portint329000Where the harness serves the turn (A2A) inside the pod. It must differ from the sidecar’s port, spec.deployment.port.
protocolenuma2aHow the sidecar hands the harness a turn. Only a2a exists.
profileenumv1The Kubemoot turn profile the harness implements.
concurrencyint321How many turns the harness runs at once. Above 1 is allowed in sidecar mode only, and the harness must send the per-turn key on every model and tool call.
nativeToolsenumnoneWhat the harness’s own built-in tools may do: none, declared, or all. See below.
command[]stringOverrides the image’s entrypoint.
args[]stringArguments for the harness container.
env[]EnvVarExtra environment for the harness container, set after the variables the mesh sets.
resourcesResourceRequirementsCPU and memory for the harness container. See Resources.

nativeTools. none turns the harness’s own tools off, so the only tools it can use are the ones at the tools door. declared allows the built-ins the crew author reviewed and declared. all leaves every built-in on. Anything but none is a deliberate choice: it is shown on the dashboard and recorded on the Deployment in the kubemoot.ai/native-tools annotation. The sidecar hands the value to the harness image in KUBEMOOT_NATIVE_TOOLS; the image applies it. The sidecar cannot see what a harness does with its own shell or file access, so the setting is a statement about the harness, not an enforcement by the mesh.

Validation. The CRD rejects a sidecar-mode Agent without an image (harness.image, deployment.image, or the name goose), concurrency above 1 outside sidecar mode, and nativeTools other than none outside sidecar mode. The operator checks two more things when it builds the pod and reports them in the Agent’s status (phase: Error, with the reason in message):

  • The Agent carries the kubemoot.ai/crew label. Every bus subject is scoped by crew.
  • harness.port and deployment.port differ. The sidecar listens on deployment.port (default 8080), the harness on harness.port.
apiVersion: kubemoot.ai/v1alpha1
kind: Agent
metadata:
  name: reviewer
  namespace: crew-my-crew
  labels:
    kubemoot.ai/crew: my-crew
spec:
  capabilities: [reasoning]
  discussRole: tooler
  promptRefs: [reviewer-system]
  harness:
    name: my-harness
    mode: sidecar
    image: registry.example.com/my-harness@sha256:...
    nativeTools: none

What the operator builds in sidecar mode

  • A contract ConfigMap, <agent>-contract, holding contract.yaml (fields below). The sidecar reads it at start. A change to the contract rolls the pod.
  • The pod. The harness is the one regular container, named agent. The sidecar is added as a native sidecar (an init container with restartPolicy: Always). The pod carries the label kubemoot.ai/harness-mode: sidecar, and the Deployment carries that label and the kubemoot.ai/native-tools annotation. The pod has no service account token.
  • The Service. It targets the sidecar port, spec.deployment.port.
  • A durable JetStream consumer for the agent, named sidecar-<namespace>-<crew>-<agent>, filtered to the broadcast channel and the channels the agent lists in spec.discussChannels. The sidecar binds it by name.

The harness container receives these environment variables. It learns two loopback URLs and its own port, and nothing about NATS, the gateway, or the cluster.

VariableValue
KUBEMOOT_MODEL_URLhttp://127.0.0.1:8081/v1, the OpenAI-compatible model door
KUBEMOOT_TOOLS_URLhttp://127.0.0.1:8082/mcp, the MCP tools door
KUBEMOOT_TURN_PORTThe port the harness serves the turn on (harness.port)
KUBEMOOT_NATIVE_TOOLSnone, declared, or all

contract.yaml

The contract is the one file the sidecar needs. The operator writes it and the sidecar refuses a file it does not fully understand: an unknown field or a version it does not speak is an error, not a guess. contractVersion follows the major version of the bus contract the sidecar serves.

FieldMeaning
contractVersionThe contract file’s version.
identity.namespace, identity.crew, identity.agentWho the agent is on the bus. Each is a single NATS subject token (no dot, wildcard, or space) and scopes every subject and key the sidecar builds.
identity.roleThe agent’s discussion role.
identity.channelsThe channels the agent listens on besides broadcast, which every agent has.
identity.triageSummaryThe agent’s one-line summary used in triage.
identity.harnessThe harness name, carried on every signal.
bus.natsURLWhere the sidecar reaches NATS.
bus.stream, bus.consumerThe durable consumer the operator created. Set together or not at all. With none, the sidecar subscribes on core NATS.
bus.progressAckSecondsHow often the sidecar tells JetStream a message is still being worked. A quarter of the consumer’s acknowledgement wait when unset.
bus.credentialsDirWhere the sidecar finds its own NATS credentials. Absent means an anonymous connection.
turn.urlThe harness’s A2A root, for example http://127.0.0.1:9000.
turn.concurrencyHow many turns may run at once. At least 1.
turn.deadlineSecondsThe safety net for one turn. The normal path ends on state.
models.aliases.<alias>.candidatesThe acceptable models for the alias, preferred first. A default alias is required; the operator also writes triage.
models.aliases.<alias>.fallbackThe server, endpoint, and model the operator bound, used when the placement service cannot be reached.
placement.policyThe placement policy the crew resolved.
placement.serviceURLThe placement service. Empty means every call uses the alias’s fallback. See the operator chart value placementService in Install the Operator.
tools.gatewayThe crew’s MCP gateway. Empty means the agent has no tools.
tools.enabled, tools.disabledThe agent’s enabledTools and disabledTools.
tools.nativeToolsRecorded for the dashboard and the receipt.
policy.systemPromptFile, policy.skillsDirWhere the assembled instructions and the crew’s skills are mounted.
observability.otlpEndpointAn OTLP/HTTP collector for spans. Empty turns export off; trace context still flows.

Resources

Every agent pod sets resources on its agent container, so the pod is never BestEffort (first to be evicted under node pressure). With the Moot Mesh sidecar, which keeps its own requests of 25m CPU and 32Mi and limits of 250m CPU and 64Mi, the pod is Burstable.

The container’s resources come from three layers, highest first:

  1. The Agent: spec.deployment.resources (legacy mode) or spec.harness.resources (sidecar mode).
  2. KubemootConfig spec.defaults.agentResources.
  3. The operator’s built-in default.
HarnessRequestsLimits
Agent runtime and any harness but goosecpu 25m, memory 96Mimemory 256Mi
goosecpu 100m, memory 128Mimemory 512Mi

The goose default comes from a measurement: the container’s cgroup memory peak over answered turns was 59 to 64 MiB, so the request is twice the peak and the limit leaves room for extensions and larger tool results. A goose seat that needs more sets spec.harness.resources on its Agent (for example limits.memory: 1Gi), or a cluster-wide value in KubemootConfig spec.defaults.agentResources. The default is sized for one turn at a time: when spec.harness.concurrency is above 1, raise spec.harness.resources by about 100 MB of memory per additional parallel goose turn.

The layers merge per resource name, not as a whole. A layer replaces only the requests and limits it names, so an Agent that sets only limits.memory: 512Mi keeps the default CPU and memory requests. KubemootConfig spec.defaults.agentResources applies on top of every harness’s built-in default, goose included, so a default set there replaces the goose built-in for the resources it names. Two rules keep the pod valid without lowering what you asked for:

  • A layer that sets a request above the inherited limit, without setting that limit, raises the limit to the request. An Agent with requests.memory: 512Mi ends up with a 512Mi limit, not 256Mi.
  • A layer that sets a limit below an inherited request lowers that request to the limit. A layer that sets both, with the request above the limit, has its request lowered to its own limit.

A limit of 0 is ignored, and resource claims merge by name with the later layer winning. Because layers only add or replace, you cannot remove a default limit (such as the memory limit) by leaving it out; set a larger one instead.

The defaults carry no CPU limit on purpose. A CPU limit pauses the container whenever it uses its quota within a scheduling window, and an agent mostly orchestrates inference calls and waits, so the pauses add latency to short bursts. The CPU request still protects the agent’s share on a busy node. Memory keeps a limit because it cannot be reclaimed. Set limits.cpu on an Agent or in the KubemootConfig default if your cluster requires one.

The goose harness runs goose run for each request, so its default is larger than the agent runtime’s; raise it for a goose seat that loads many extensions.

Status Fields

FieldTypeDescription
phasestringRunning once the Deployment and Service are applied (not a pod-readiness signal); Unschedulable when no feasible Model or Ready ModelServer exists; Pending while the requested RuntimeClass does not exist
readyboolTrue when the scheduler bound a Model and the agent is deployed
endpointstringInternal service URL (http://<name>.<namespace>.svc.cluster.local:<port>, following spec.deployment.port)
availableReplicasint32Declared, not populated by the controller today
conditions[]ConditionRuntimeClassAvailable is present only when the pod asks for a RuntimeClass: True when the class exists, False (reason RuntimeClassMissing) when it does not. See Isolation. ReplicasLimited (True, reason CoordinatorLeasesOff) is present only on a coordinator whose spec.deployment.replicas asks for more than 1 while coordinator.lease is off: the operator ran one replica. WorkspaceAccessResolved is present only when a workspace access is declared: True (reason Declared) with the access and where it came from, reason ToolsOverruled when enabledTools names a workspace tool the access does not allow, and False (reason NoWorkspaceServer) when the access reads but no MCPServer in the namespace selects spec.kubemootImage: workspaceAccess (artifactAccess is its old name and counts the same for one release)
workspaceobjectThe resolved workspace access (access: none, read, write, readWrite or implicit) and where it was declared (source: agent, crew, config or implicit)
messagestringThe bound models (mulling=<model>@<server>; triage=<model>@<server>), or the reason the agent is unschedulable
ragSourceStatus[]RAGSourceRefStatusDeclared, not populated by the controller today
mcpServerStatus[]MCPServerRefStatusDeclared, not populated by the controller today

Examples

Coordinator

apiVersion: kubemoot.ai/v1alpha1
kind: Agent
metadata:
  name: homelab-coordinator
  namespace: crew-homelab-pilot
  labels:
    kubemoot.ai/crew: homelab-pilot
    kubemoot.ai/role: coordinator
spec:
  type: chat
  description: "Homelab coordinator: orchestrates Tooler and Analyst agents via async NATS discussions with vector-based triage pre-filtering"
  discussRole: coordinator
  capabilities:
    - tool-calling
    - reasoning
  discussChannels:
    - kubernetes
    - observability
    - proxmox
    - general
  promptRefs:
    - advisory-prompt
    - coordinator-decision-logic
    - synthesis-prompt
  temperature: "0.1"
  maxTokens: 4096
  deployment:
    replicas: 1
    port: 8080
    env:
      - name: KUBEMOOT_GATEWAY_ENABLED   # coordinator has no MCP tools - only A2A delegation
        value: "false"
      - name: KUBEMOOT_DISCUSS_TOOLER # coordinator facilitates, doesn't participate
        value: "false"
      - name: KUBEMOOT_DISCUSS_SYNTHESIS_TIMEOUT_SECONDS
        value: "180"

Kubernetes Tooler

apiVersion: kubemoot.ai/v1alpha1
kind: Agent
metadata:
  name: k8s-workloads
  namespace: crew-homelab-pilot
  labels:
    kubemoot.ai/crew: homelab-pilot
spec:
  type: chat
  description: "Kubernetes workload Tooler: pods, deployments, services, replicasets, statefulsets, daemonsets, jobs, cronjobs"
  triageSummary: "Pod, deployment, service, replicaset, statefulset, daemonset, job, cronjob, namespace management via kubectl"
  discussRole: tooler
  capabilities:
    - tool-calling
    - kubernetes
  discussChannels:
    - kubernetes
  discussKeywords:
    - pod
    - deployment
    - service
    - replicaset
    - statefulset
    - daemonset
    - job
    - cronjob
    - namespace
  promptRefs:
    - discussion-protocol
    - response-style
    - k8s-workloads-system
  enabledTools:
    - kubectl_get
    - kubectl_describe
    - kubectl_logs
    - list_api_resources
    - explain_resource
    - kubectl_scale
  ragSources:
    - name: kubernetes-concepts
      topK: 3
      priority: 1
  temperature: "0.3"
  maxTokens: 4096
apiVersion: kubemoot.ai/v1alpha1
kind: Agent
metadata:
  name: web-search
  namespace: crew-homelab-pilot
spec:
  type: chat
  description: "Internet search researcher - augments discussion with current information"
  triageSummary: "Web search for current events, recent docs, external context"
  discussRole: researcher
  capabilities:
    - tool-calling
    - web-search
  discussChannels:
    - general
    - kubernetes
    - observability
    - proxmox
  discussKeywords:
    - "*"   # opt into every thread
  promptRefs:
    - discussion-protocol
    - response-style
    - web-search-system
  enabledTools:
    - web_search
    - fetch_url
  temperature: "0.3"

Multi-Agent Collaboration

Kubemoot agents collaborate asynchronously via NATS JetStream discussions.

Agent Topology

A typical deployment uses a coordinator + Tooler (+ optional Analyst) pattern:

                    +--------------+
                    |  coordinator |  <- Receives user queries
                    +------+-------+
                           | NATS discussions
            +--------------+--------------+
            |              |              |
    +-------v------+ +----v-----+ +------v-------+
    | k8s-workloads| | k8s-nodes| |  k8s-helm    |  <- Toolers, kubernetes channel
    +--------------+ +----------+ +--------------+
    +--------------+ +--------------+
    |  obs-metrics | |proxmox-nodes |  <- Toolers, observability / proxmox channels
    +--------------+ +--------------+
    +--------------+
    |  k8s-analyst |  <- Analyst, REVIEW phase, kubernetes channel
    +--------------+

Discussion Channels

ChannelPurposeSubscribing Agents
kubernetesK8s workload, node, and Helm queriesk8s-workloads, k8s-nodes, k8s-helm (Toolers); k8s-analyst (Analyst)
observabilityMetrics and monitoring queriesobs-metrics (Tooler)
proxmoxHypervisor and VM queriesproxmox-nodes (Tooler)
generalCatch-all for unclassified queriesAll Toolers and Analysts

Each agent declares its channels via Agent.spec.discussChannels. Coordinators typically list every channel the crew uses; Toolers and Analysts list only their domain. See agentic-consensus.md for the full discussion protocol.

Discussion Flow

  1. User sends query to coordinator agent
  2. Coordinator queries the crew’s resume model to select the subcommittee, then publishes thread_start (with that innerCircle) to kubemoot.discuss.<namespace>.<crew>.<channel>.<threadId>
  3. Only the selected Toolers are woken; the rest are silently excluded (no keyword self-selection)
  4. Each selected Tooler runs a per-agent LLM triage (CONTRIBUTE / NOTHING_TO_ADD) for this specific question
  5. Contributing Toolers call directChat() to generate a contribution (with tool calling); Analysts self-select in the REVIEW phase and reason over the gathered data
  6. Agents publish signals (triaging, evaluating, waiting, agree, concern, stand_aside, failure, block) back to the thread
  7. Coordinator settles the discussion based on signals (not a fixed timeout) and synthesizes a response

Bounded Tool Retries and Failure Signals

The tool-calling loop is bounded along three axes so a misbehaving tool can’t hang the discussion forever:

  • Same-tool failure caps: a tool that returns an error twice in the same evaluation, the second being a repeat (the same call, with whitespace and key order ignored, or the same error text again), aborts the loop: the agent is stuck. An error that came with new arguments and a new error text is a correction, such as a query fixed after an error that named the right columns. Corrections have their own budget: a tool that fails 4 times, each with new arguments and new error text, aborts the loop, so a model that keeps guessing still stops.
  • Total-failure cap (4): if 4 tool calls across all tools error, the loop aborts - this catches a model spreading its retries across many broken tools rather than one.
  • Iteration cap (properties.model().maxToolIterations(), default 15): if the LLM keeps requesting tools without producing a final answer, the loop aborts.

A fourth rule keeps a model from asking for the same thing again: a call with the same tool and the same arguments (key order and spacing do not count) runs at most twice in one evaluation. A third identical call is not run; the model gets an error telling it to change the arguments or answer from what it has, and that error counts toward the same-tool failure cap. A tooler that already holds output stops gathering and posts it instead.

Any of these aborts publishes a first-class failure consensus signal with structured cause metadata (failureType, failedTool, lastError) rather than a generic stand_aside. Other failures during mulling (a server timeout, an out-of-memory error, a messaging hiccup, a runtime error) route through the same failure path with heuristic classification (model_timeout, model_oom, server_unreachable, context_exceeded when the engine rejects a prompt larger than its window, internal_exception).

Before the model is called at all, the agent checks that the tools its spec declares are on the tool list it fetched. If the gateway was unreachable, the MCP server was not registered, or the list lacks the declared tools, the agent publishes a failure signal with failureType tools_unavailable, failedTool listing the missing tool names, and lastError naming the gateway address and the error it returned. The board message reads “Tools unavailable: agent library-search declares search_oreilly_content but the MCP tool source does not offer it. The model was not called.” and carries no address; the gateway address and its error are in lastError, the logs, and the dashboard timeline. The model is never asked to answer without the tools its prompt tells it to use, because it would invent the result. The check reads the tool list the runtime already fetched (re-read when it is empty or stale), so it adds no timeout.

Before every turn the loop also checks that the next prompt fits the context window of the server the call runs on. It uses the prompt size the engine reported for the previous turn plus the tool results added since. When the prompt would not fit, the loop fails with failureType CONTEXT_EXCEEDED rather than send a prompt the engine would cut. See Scheduler.

A single tool result can also be too large for the room left in the window. The room is the context window, minus the prompt so far, minus a reserve for the reply (an eighth of the window, at least 1,024 tokens). Results of one turn share the room in the order they arrive, and a result that does not fit is never shown in part. What the model sees instead depends on who needs the data:

  • A tooler with raw output (discussRole: tooler): every result is stored as its own file in the discussion workspace and its contribution is the description of those files (see Discussion artifact store). A stand-in ends its head with Stored as workspace artifact <key>.. Its model only decides whether to stop or call another tool, so it sees a short stand-in: the tool name, that the full result is kept for the crew, its size in bytes and lines (and items, for a JSON list), the first 20 lines (at most 1,500 characters), and a note to stop if this is the data it asked for. The stand-in is built in code, the same every time. Once less than twice the reply reserve is left in the window, the stand-in keeps only the tool name, the size and the note, without the first lines. Stand-ins add up, so a tooler that already holds output stops as soon as less than the reply reserve is left for the next turn, and posts the descriptions of the files it stored as its contribution (when every call failed, the gather fails instead). If the model server still rejects a prompt as longer than its window (the estimate counted stand-ins of numeric data too low), the tooler posts the descriptions of its files the same way, and the agent learns its characters-per-token ratio from the prompt size the server reports; other agents keep the context_exceeded failure. Stand-in text is counted denser than prose for this estimate: each digit counts three characters and each punctuation or quote character two.

  • Any other agent, which reads results to reason over them: the model gets a tool error naming the tool, the result size and the room left, and asking for a filter (a namespace, a label selector, a field selection, a limit, or a shorter time range), so it can call again with less.

    For an artifact or workspace read tool the error also says to ask for fewer lines or rows, or to aggregate (count, select, jq, or SQL in workspace_query) and read only the answer.

Result sizes are counted by content, not only by length. Numeric JSON (telemetry, RRD series, resource lists) tokenizes near 1.4 characters per token, against 3 or more for prose, so a flat characters-per-token estimate admitted results that overflowed the window. Every tool result and user message (the thread, which can carry inline contributions) counts each digit as three characters and each punctuation or quote character as two, the same rule stand-ins already used.

An agent that reads its own results (an analyst such as compute) also recovers when its estimate is still short:

  • When the next prompt would not fit the window, the largest earlier tool results are replaced with a short note (the model’s call stays in the prompt, so it can make the call again with less) until the prompt fits with the reply reserve free; only when nothing can be replaced does the loop fail with CONTEXT_EXCEEDED.
  • When the server rejects a prompt as longer than its window, the agent learns its ratio, replaces the largest earlier results the same way (cut by the share the estimate was short by), and tries the turn again, at most twice per loop. A tooler does not do this; it posts what it gathered, as above.

Every model call logs one PROMPT-TOKENS line with the prompt tokens by component (system, tools, thread, assistant, toolResults, estimated), the total the engine reported and the window, and records each component in the kubemoot_agent_prompt_tokens summary (tag component; reported is the engine’s total). A window overflow is read from the last line before the failure.

A window that is not published applies no rule.

A bounded retry policy on the model call itself keeps a slow server from silently multiplying its own timeout into a multi-minute hang: one retry, then the call surfaces as failed and the failure path takes over. Heartbeats stop the moment an agent publishes any terminal signal, so a finished agent never keeps looking alive after it’s done.

The consensus-protocol semantics of failure vs stand_aside and the three-tier gap classification (TOOL_GAP / INFRASTRUCTURE_GAP / SPECIALIST_GAP) are documented in agentic-consensus.md.

JIT Server Selection (Per-Call)

The agent runtime does not use a single, static endpoint frozen into env vars at reconcile time. Each mulling inference call picks the freest server at the call boundary from live server state (read from a shared NATS KV bucket), and falls back to the static reconcile-time endpoint only when that live state is unavailable. The picked server’s name flows through to the agent’s signal metadata as metadata.server, surfacing in the dashboard’s Agent Summary GPU column as the call’s real placement. See scheduler.md for the architectural rationale and the kube-scheduler analogy.

The call’s model is chosen per call too. The operator publishes the phase’s ranked candidate models (KUBEMOOT_MODEL_CANDIDATES_MULLING); the runtime uses the preferred model when it is loaded with a free slot, otherwise a loaded candidate within the quality tolerance, otherwise the preferred model under the normal cost ranking. The model the call ran is recorded as metadata.model.

When no accelerator has room, the agent publishes waiting once and retries on every server-state or ticket change until an accelerator frees up. It stands aside with metadata.reason = "gpu-busy" if the discussion ends or the wait’s safety limit passes first, and with "model-too-large" at once when no accelerator can ever hold the model. It stands aside with "prompt-too-large" at once when no accelerator gives an acceptable model a context that holds the prompt. When no accelerator has room, the scheduler compares waiting for a loaded copy with unloading idle models that no crew is using or about to use; a call that unloaded models records them as metadata.evicted. An agent’s first call is planned when the coordinator selects it, so a needed load overlaps triage. See Models & Scheduling.

Rate Limiting

  • Max 1 contribution per thread per agent, plus a sliding-window per-minute cap
  • Prevents runaway inference costs from chatty discussion threads
  • Configured via KUBEMOOT_DISCUSS_MAX_INFERENCES_PER_MINUTE env var

Liveness Heartbeat

Each agent writes a liveness record to the kubemoot_agent_state NATS KV bucket, keyed <namespace>.<agent>, every intervalSeconds (default 60). The entry expires from the bucket after 300 seconds, at which point there is no heartbeat at all.

FieldMeaning
agentAgent name
timestampWhen the record was written (ISO 8601)
nats, ollamaWhether the agent reached NATS and its model server
modelModel the agent is configured with
lastInferenceTime of the last successful inference (omitted until the first one)
intervalSecondsSeconds between this agent’s heartbeats

Consumers judge staleness from the record itself: a heartbeat is stale once two of its own intervalSeconds pass without a newer one, so changing the interval needs no change in any consumer. A record without intervalSeconds (written by an older agent), or one whose value is not a positive number or exceeds one hour, is judged against a 60 second interval. An agent configured with a zero or negative interval publishes 60.

Operator-Injected Env Vars

Env VarApplies ToSource
NATS_URLAll agentsOperator from NATS service
KUBEMOOT_WORKSPACE_ACCESSAgents with a declared workspace accessThe resolved spec.workspace.access (Agent, then Crew, then KubemootConfig); absent when none is declared. With none or read the runtime stores no tool results
KUBEMOOT_DISCUSS_CHANNELSAll agentsAgent.spec.discussChannels (comma-separated)
KUBEMOOT_DISCUSS_COORDINATORCoordinatorsSet true when discussRole=coordinator
KUBEMOOT_DISCUSS_TOOLERAll agentsfalse for coordinators and researcher-role agents
KUBEMOOT_DISCUSS_TIMEOUT_SECONDSCoordinatorOptional override via spec.deployment.env
KUBEMOOT_MODEL_CANDIDATES_MULLINGAll agentsRanked candidate models for the mulling phase, JSON [{"model", "score"}], preferred model first
KUBEMOOT_MODEL_CANDIDATES_TRIAGEAll agentsRanked candidate models for the triage phase, same shape
KUBEMOOT_MODEL_CANDIDATE_TOLERANCEAll agentsOptional override via spec.deployment.env; quality-score points a warm candidate may trail the preferred model (default 10)
KUBEMOOT_DISCUSS_GPU_WAIT_LIMIT_SECONDSAll agentsOptional override via spec.deployment.env; safety limit on one accelerator capacity wait (default: the synthesis timeout, 90 seconds)
KUBEMOOT_COORDINATOR_RESUME_MAX_AGECoordinatorOptional, with KUBEMOOT_COORDINATOR_RESUME: how old a thread’s last logged event may be for the coordinator to resume it, as a Go duration such as 30m or 1h30m. An older thread is closed with reason expired. 0 or empty turns the limit off. Default 30m; a value that is not a duration logs a warning and uses the default. The operator sets it from the chart value coordinator.resumeMaxAge; set here in spec.deployment.env it wins
KUBEMOOT_COORDINATOR_LEASECoordinatorOptional; true lets more than one coordinator replica run, each owning the threads it holds a lease for in the kubemoot_coordinator_leases bucket. Needs KUBEMOOT_COORDINATOR_RESUME=true; without it leases stay off. The operator sets it on every coordinator when the chart value coordinator.lease is true (with resume and the log on); set here in spec.deployment.env it wins. Default false (leases are opt in), and then it changes nothing
KUBEMOOT_COORDINATOR_RESUMECoordinatorOptional; true makes a restarted coordinator resume the threads it had in flight, from its coordinator log. Needs KUBEMOOT_COORDINATOR_EVENT_LOG=true; without it resume stays off. The operator sets it on every coordinator unless the chart value coordinator.resume is false (on by default); set here in spec.deployment.env it wins. Run one coordinator replica unless KUBEMOOT_COORDINATOR_LEASE is on. The runtime’s own default is false, which applies to an Agent no operator configures; the operator always sets the variable when resume is on
KUBEMOOT_COORDINATOR_EVENT_LOGCoordinatorOptional; true makes the coordinator write each decision to its coordinator log before it acts on it. The operator sets it on every coordinator unless the chart value coordinator.eventLog is false (on by default); set here in spec.deployment.env it wins. The runtime’s own default is false, which applies to an Agent no operator configures; the operator always sets the variable when the log is on

Tool Filtering

Agents connect to MCP tools via the MCPGateway. The number of registered tools significantly affects model behavior - qwen3:32b works reliably with ~10-15 tools but produces empty responses with >20 tools.

Spec Fields

Agent.spec.enabledTools[] is a whitelist; Agent.spec.disabledTools[] is a blacklist. The operator injects the resolved set as KUBEMOOT_ENABLED_TOOLS / KUBEMOOT_DISABLED_TOOLS. If both are set, enabledTools takes precedence.

Tool Categories

CategoryToolsUse Case
Readkubectl_get, kubectl_describe, kubectl_logs, list_api_resources, explain_resourceCluster inspection
Updatekubectl_scale, kubectl_patch, kubectl_rolloutWorkload management
Helmhelm_install, helm_upgrade, helm_uninstallChart management

Guidance

  • Temperature 0.3 (set via Agent.spec.temperature) is recommended for reliable tool selection
  • Keep total tool count under 15 per agent for reliable behavior
  • Specialists should only register tools relevant to their domain (e.g., k8s-helm agent only needs Helm tools + basic read tools)

Agent Design - Cohesion, Coupling, and Question Shape

The mechanics above (tool whitelists, count caps, temperature) are tactical. The strategic question is harder: how should a crew be decomposed into agents in the first place? This is the same problem as monolithic-app vs microservices decomposition - and the same heuristics apply. High cohesion within an agent. Loose coupling across agents. The “right” answer is rarely “all in one agent” or “one tool per agent.”

Question shape is the unit of cohesion

A well-designed Tooler answers ONE shape of question:

  • “What is X right now?” (real-time scalar / instant vector)
  • “How did X trend over the last hour?” (time series, regressions, spikes)
  • “What does X mean?” (semantics, units, definitions)
  • “Is the infrastructure itself healthy?” (meta-observability)
  • “Discover what’s available in domain Y.” (introspection)

Each shape pulls a different subset of tools, a different prompt voice (“be concise / show numbers” vs “analyze trend / explain”), and a different per-call latency profile. Mixing shapes inside one agent forces every iteration to carry the union of all tools and all guidance - the LLM pays the cost on every question regardless of which shape it’s actually answering.

When you have a “god agent”

Symptoms:

  • 5+ tools spanning several question shapes.
  • One-sentence purpose statement is hard to write (“it answers GPU-related questions” - too broad).
  • Resume-match scores hover around 0.5-0.7 across many topics - neither clearly relevant nor clearly out-of-scope.
  • Latency varies wildly call-to-call because some questions use 1 tool and others use 4 in a loop.
  • Per-iteration prompt is large; the model spends tokens choosing among tools that have nothing to do with the current question.

Costs:

  • Slow iterations - every tool definition is in the prompt, every iteration.
  • Tool-selection accuracy degrades with too many options (Qwen2.5:32b empirically degrades above ~20 tools - see top of section). qwen3:32b is more forgiving but still benefits from narrowness.
  • Triage ambiguity - the relevance filter can’t cleanly say yes/no.
  • Single failure mode - when this agent wedges or its model is busy, ALL its question shapes are blocked.
  • Hard to evolve - adding a tool affects every other capability the agent already covers.

When you have “atomic” agents

Symptoms:

  • 0-1 tools per agent, dozens of agents.
  • Many agents triage relevant to the same question; multiple agree on the same answer with the same data.
  • Composite questions (need 2+ tools that live in different agents) go unanswered or require multi-agent handoff.
  • Triage relevance is precise per agent but the crew listing is large and hard to reason about.
  • Each agent adds Kubernetes resources (Deployment, Service, ConfigMap) and an always-on pod.

Costs:

  • Orchestration overhead dominates per-discussion time.
  • Overlapping capabilities - when two agents have the same answer, the synthesis layer has to choose; if they disagree, the user has to.
  • Capability gaps - when a question requires N tools spanning N agents, no single agent can answer.
  • Cost of coordination vs. cost of within-agent inference. Below a threshold, splitting hurts more than it helps.

The Goldilocks zone

A well-sized Tooler tends to have:

DimensionTarget
Tools1-5, all serving the same question shape
Description (for triage)One sentence, clear yes/no for the question types it owns
Per-call iterationsPredictable (usually 1-3) because tool selection is unambiguous
Relevance score on its own questionsConsistently > 0.85
Relevance score on out-of-scope questionsConsistently < 0.3
Overlap with other agentsMinimal - at most a shared “read” tool family

Heuristics for deciding to split

  1. One-sentence test. If you can’t write “I answer X-shaped questions” without using “and” or “or” listing different shapes, split.
  2. Triage ambiguity test. If the relevance filter returns scores in the 0.4-0.7 band consistently, the agent’s scope is too fuzzy.
  3. Tool-shape mapping. Group the agent’s tools by question shape. If a group has only 1-2 tools that nobody else uses, that’s a candidate split.
  4. Latency variance. If the agent’s P50 and P90 differ by more than ~3×, different questions are doing very different amounts of work - likely different shapes.
  5. Failure blast radius. If one tool failing renders the whole agent useless across all question shapes, consider whether the tool deserves its own narrow agent.

Heuristics for deciding to merge

  1. Always-co-agreeing pair. If two agents always end up agreeing in the same discussions with substantively identical answers, they have overlapping cohesion - merge them or differentiate their roles.
  2. Sequential dependency. If question type Z always requires Agent A to answer first and Agent B to follow up, consider a single agent that owns both steps (or a coordinator-level orchestration if it’s really common).
  3. Underused capability. If a tool sits in its own agent but is invoked < 5% of discussions, it’s adding triage overhead without earning it - fold it into a related agent.

Real-world example: nvidia-gpu

Original (god agent, 5 tools, all Prometheus, but four question shapes):

name: nvidia-gpu
enabledTools: [execute_query, execute_range_query, list_metrics, get_metric_metadata, get_targets]

For “What’s GPU utilization on gpu-a?” the agent reasoned through all 5 tool definitions to choose execute_query, and a multi-iteration tool loop took ~3 minutes. For “Is the DCGM scrape target up?” it would similarly reason through irrelevant query tools to choose get_targets.

Decomposition (three single-shape Toolers):

name: nvidia-gpu-now
description: Real-time GPU state queries: current utilization, temperature, VRAM use.
enabledTools: [execute_query]

name: nvidia-gpu-history
description: GPU trend / time-series analysis: how a metric moved over a window.
enabledTools: [execute_range_query, get_metric_metadata]

name: nvidia-gpu-meta
description: Discovery and Prometheus-scrape-health for GPU monitoring.
enabledTools: [list_metrics, get_targets]

Each is fast within its shape because its prompt is small and its tool choice is obvious. The coordinator’s triage relevance filter picks the right one based on the question; the others stand aside cleanly. The crew gains all the same capabilities, distributed across narrower Toolers.

Right-sized models are a paired benefit of cohesion

Decomposing a god-agent into single-shape Toolers doesn’t only sharpen relevance and shorten tool loops - it also unlocks right-sized model selection. A Tooler whose job is one tool call plus a paragraph of formatting doesn’t need the same reasoning model as a coordinator synthesizing across multiple agent contributions. When every agent in a crew defaults to a single large model, small-shape Toolers pay for capability they never use, and under per-server num_parallel=1 they can starve each other on that one model’s slot.

The mechanism is Agent.spec.capabilities + CrewSchedulingPolicy.spec.qualityBias (see scheduler.md - Quality bias). The agent declares an abstract capability (tool-calling for a simple Tooler; +reasoning for a coordinator or Analyst); the crew author declares what each capability is worth in quality terms; the scheduler resolves to a concrete Model at reconcile time. No agent names a model - loose coupling preserved across cluster boundaries.

A decomposed nvidia-gpu cohort, for example, can have its three single-shape Toolers each declare [tool-calling, observability] and resolve to a small fast model, while the coordinator that synthesizes their findings declares [tool-calling, reasoning] and resolves to a larger one. One heavy model and many light ones co-exist on the same GPU pool without slot starvation.

Caveat

These heuristics are guidance, not law. A small single-cluster crew has different optimal granularity than a 100-agent enterprise crew. The right number of agents for your crew is the number where the triage layer can confidently route questions to a clear Tooler or Analyst without the per-discussion orchestration cost overwhelming the per-call inference cost.


MCPGateway Integration

Agents access MCP tools through a centralized MCPGateway rather than connecting to MCPServer pods directly.

Tool Chain

Agent Pod                    MCPGateway Pod               MCPServer Pod
┌──────────────┐            ┌──────────────┐            ┌──────────────────┐
│ Agent Runtime │──HTTP/──→ │  Gateway     │──HTTP/──→ │ MCP Bridge       │
│              │   SSE      │  (discovers  │   SSE      │ (sidecar)        │
│ LangChain4j   │            │   servers    │            │    │              │
│ tool calling │            │   via labels)│            │    ▼ stdio        │
│              │            │              │            │ MCP Server        │
│              │ ←─tools──  │              │ ←─tools──  │ (kubectl, helm..) │
└──────────────┘  response  └──────────────┘  response  └──────────────────┘

How It Works

  1. MCPGateway discovers MCPServer pods via label selectors
  2. Gateway connects to each server’s mcp-bridge sidecar HTTP/SSE endpoint
  3. Gateway caches tool definitions from all registered servers
  4. Agent connects to gateway via KUBEMOOT_GATEWAY_ENDPOINT env var
  5. Agent runtime registers available tools with LangChain4j’s tool calling framework
  6. When the LLM generates a tool_calls response, LangChain4j executes the tool via the gateway
  7. Gateway routes the tool call to the appropriate MCPServer

Key Details

  • Gateway caches failed connections - if a server is unavailable on first registration, the gateway must be restarted to retry
  • Agent gets gateway endpoint via KUBEMOOT_GATEWAY_ENDPOINT env var (injected by operator)
  • Coordinators set KUBEMOOT_GATEWAY_ENABLED=false because they have no MCP tools - only A2A delegation via NATS

System Prompt Composition

System prompts are composed from PromptModule CRs referenced by Agent.spec.promptRefs. The operator assembles their content in order and writes the result to a ConfigMap mounted at /etc/kubemoot/policy/system.txt. The agent runtime reads system.txt at startup (KUBEMOOT_SYSTEM_PROMPT_FILE env var).

spec:
  promptRefs:
    - discussion-protocol    # order 10
    - response-style         # order 20
    - my-agent-system        # order 30

NATS Events

Agents publish and subscribe to NATS subjects for real-time event streaming.

Chat Events

Each agent publishes to kubemoot.chat.<namespace>.<agent_name> on every chat response:

{
  "agentName": "k8s-workloads",
  "conversationId": "abc123",
  "userMessage": "List all pods in default namespace",
  "assistantMessage": "Here are the pods...",
  "toolCalls": [{"name": "kubectl_get", "status": "success", "duration": 1200}],
  "timestamp": "2026-02-07T10:30:00Z"
}

The dashboard subscribes to kubemoot.chat.> for real-time topology node pulsing and the Messages page.

Discussion Events

Each agent subscribes to kubemoot.discuss.<namespace>.<crew>.<channel>.> for inter-agent collaboration (see agentic-consensus.md).

Connection Management

The agent runtime shares one lazy NATS connection across every component in the pod. If NATS_URL is not set, all NATS operations are no-ops - the agent still answers direct chat requests, it simply doesn’t participate in discussions.


PromptModule

PromptModule is a CRD for reusable prompt fragments composed into agent system prompts via Agent.spec.promptRefs. Multiple agents can reference the same module. When a module changes, the operator re-reconciles all agents that reference it.

Short name: pm

Spec

FieldTypeDefaultDescription
contentstring(required)The prompt text (ADL format recommended)
orderint32100Ordering when composed (lower = earlier in system prompt)

Placeholders in content

The text ${agent.tools} in content is replaced by the agent runtime with the names of the tools that agent registered, so a shared module can list an agent’s tools without naming tools other agents lack. See Placeholders. The operator does not resolve it: the registered set is known only at runtime.

Status

FieldTypeDescription
readyboolModule is valid and available
referencedBy[]stringAgent names using this module

Order Conventions

Order RangePurposeExamples
1-9Coordinator advisoryadvisory-prompt (5)
10-19Shared protocoldiscussion-protocol (10)
20-29Shared styleresponse-style (20)
30-39Per-agent identityk8s-workloads-system (30)
40-59Coordinator logiccoordinator-decision-logic (45), synthesis-prompt (50)

Example

apiVersion: kubemoot.ai/v1alpha1
kind: PromptModule
metadata:
  name: discussion-protocol
spec:
  order: 10
  content: |
    DEFINE COMPONENT discussion-protocol
    DESCRIPTION Rules for participating in multi-agent discussions

    WHEN the question is NOT in your domain THEN respond NOTHING_TO_ADD immediately
    WHEN the question IS in your domain THEN USE YOUR TOOLS to gather real data
    ALWAYS use tools for verifiable facts
    NEVER give generic instructions without tool-backed evidence

HTTP API

Agents expose an HTTP API at the configured port (Agent.spec.deployment.port, default 8080):

EndpointMethodDescription
/chatPOSTSend message, get response
/healthGETHealth check (with dependencies)
/readyGETReadiness check (local only)
/toolsGETList the agent’s tools
/conversations/{id}DELETEClear a conversation

Chat Request

curl -X POST http://k8s-workloads.crew-homelab-pilot:8080/chat \
  -H "Content-Type: application/json" \
  -d '{"message": "What pods are running in the default namespace?"}'

Chat Response

{
  "conversationId": "abc123",
  "message": "Here are the pods running in the default namespace:\n\n| Name | Status | ...",
  "toolCalls": [
    {"name": "kubectl_get", "arguments": {"resource": "pods", "namespace": "default"}}
  ]
}

  • scheduler.md - Scheduler design (CRD surface, filter/score/bind)
  • models.md - Model catalog and Model CRD reference
  • agentic-consensus.md - Discussion protocol semantics
  • The Moot Mesh - The sidecar behind spec.harness
  • skill.md - Skill CRD reference: crew-level procedures the coordinator selects on demand
  • chart/kubemoot-operator/templates/mootarchetypes.yaml - built-in consent-3 MootArchetype
  • homelab-pilot/charts/homelab-pilot-crew/templates/ - example Agent CRs for the Homelab Pilot crew