Cognitive SLM Architecture (Heimdall)¶

This document is the architectural overview for Heimdall: NornicDB's embedded Small Language Model subsystem. Code lives in pkg/heimdall/ and pkg/inference/. For operator setup and YAML/env configuration, see Heimdall AI Assistant.

What's in scope¶

• An embedded reasoning model that runs alongside the BGE-M3 embedding model.
• An OpenAI-compatible chat API (Bifrost) for in-app and IDE-tool use.
• A plugin system that lets the model invoke registered actions, observe database events, and (optionally) trigger autonomous remediation.
• Inference-engine integration for Auto-TLP edge quality control (pkg/inference/HeimdallQC).

Architecture¶

┌─────────────────────────────────────────────────────────────────┐
│                    NornicDB Cognitive Engine                     │
├─────────────────────────────────────────────────────────────────┤
│  ┌──────────────┐    ┌──────────────┐    ┌──────────────────┐  │
│  │  Embedding   │    │   Reasoning  │    │   Graph Engine   │  │
│  │    Model     │    │     SLM      │    │   (Storage +     │  │
│  │  (BGE-M3)    │    │  (Qwen3 0.6B)│    │    Cypher)       │  │
│  │  1024 dims   │    │   default    │    │                  │  │
│  └──────┬───────┘    └──────┬───────┘    └────────┬─────────┘  │
│         │                   │                      │            │
│         └───────────────────┴──────────────────────┘            │
│                             │                                    │
│                    ┌────────▼────────┐                          │
│                    │ heimdall.Manager│                          │
│                    │ (single-model   │                          │
│                    │  per process)   │                          │
│                    └─────────────────┘                          │
└─────────────────────────────────────────────────────────────────┘

The reasoning SLM is a separate model from the embedding model. Both run inside the NornicDB process; the embedding model serves all vector-search paths, and the reasoning SLM serves the Bifrost chat endpoint and any registered Heimdall plugins.

Provider abstraction¶

heimdall.Manager is the entry point. It hides three provider backends behind a Generator interface:

The Manager is constructed once at startup with NewManager(cfg). Provider selection cannot change at runtime; restart the server to switch. Each provider implements the same Generate, GenerateStream, Chat, and GenerateWithTools methods.

CPU fallback for the local provider is implemented in loadLocalGenerator: if GPU loading fails, the manager retries with gpu_layers=0.

Model selection¶

Default model: qwen3-0.6b-instruct (Apache 2.0). Override via NORNICDB_HEIMDALL_MODEL. The model name is the GGUF filename without the .gguf extension for the local provider; for Ollama and OpenAI it's the provider-specific model identifier.

All recommended models are commercial-use compatible.

Bifrost (chat surface)¶

The Bifrost endpoint is an OpenAI-compatible chat API. It shares the regular HTTP listener and uses standard NornicDB authentication.

Streaming uses Server-Sent Events when the request sets stream: true, matching OpenAI's wire format. The model field in chat requests is accepted but ignored — Heimdall always uses the configured backend model.

Plugin system¶

Heimdall plugins implement heimdall.HeimdallPlugin (pkg/heimdall/plugin.go). Plugins register actions that the SLM can invoke, plus optional lifecycle hooks:

Plugins receive a SubsystemContext with handles to:

• The SLM (HeimdallInvoker) — for autonomous action invocation or natural-language prompts.
• The graph (DatabaseRouter) — for read/write Cypher.
• The Bifrost bridge (BifrostBridge) — for sending notifications to active chat sessions.
• Per-database RBAC context (PrincipalRoles, DatabaseAccessMode, ResolvedAccess) when the request is authenticated.

For the full plugin contract see Heimdall Plugins. For event-driven remediation patterns see Event Triggers and Automatic Remediation.

Inference-engine integration¶

Heimdall's reasoning capability is also wired into Auto-TLP through pkg/inference.HeimdallQC. When NORNICDB_AUTO_TLP_LLM_QC_ENABLED=true, candidate edges from the topological link-prediction algorithms are reviewed by the SLM before materialisation. The relevant feature flags:

• NORNICDB_AUTO_TLP_ENABLED — turn on automatic edge inference.
• NORNICDB_AUTO_TLP_LLM_QC_ENABLED — gate inference on SLM approval.
• NORNICDB_AUTO_TLP_LLM_AUGMENT_ENABLED — let the SLM suggest additional edges beyond what TLP found.

See Auto-TLP Heimdall and Feature Flags for the full surface.

Safety constraints¶

The Heimdall surface is intentionally narrow:

• Plugin actions execute in the host process with the same OS permissions as nornicdb serve. Plugins are not sandboxed; only load plugins from trusted sources.
• The SLM itself does not directly mutate the storage engine. All mutations go through the Cypher executor, the standard transaction layer, and the existing per-database RBAC enforcement.
• ActionOpcode (in pkg/heimdall/types.go) defines a finite enumerated set of operations the SLM is allowed to emit; any output outside that schema is rejected before execution.
• Chat content from users is treated as untrusted input: prompt-injection mitigations rely on system-prompt design and the ActionOpcode whitelist, not on input sanitization alone.

Token budget¶

Heimdall enforces a structured token budget at prompt-construction time so plugin authors can't blow up the system prompt. The defaults (pkg/heimdall/types.go):

const (
    DefaultMaxContextTokens      = 8192  // Total prompt budget
    DefaultMaxSystemPromptTokens = 6000  // System prompt + plugin contributions
    DefaultMaxUserMessageTokens  = 2000  // User message
)

These apply to all providers. Override via NORNICDB_HEIMDALL_MAX_CONTEXT_TOKENS, NORNICDB_HEIMDALL_MAX_SYSTEM_TOKENS, NORNICDB_HEIMDALL_MAX_USER_TOKENS, or YAML (heimdall.max_context_tokens etc.). For remote providers (Ollama, OpenAI) you can safely raise these to 32K or 128K to use the provider's full context window — see Heimdall Context & Tokens.

Configuration surface¶

The minimal env var set:

For the complete reference (including the token-budget overrides above) see Environment Variables Reference.

Resource expectations¶

For the default configuration (BGE-M3 embedding + qwen3-0.6b reasoning, both Q4_K_M):

Both fit comfortably on Apple Silicon's unified memory and on NVIDIA GPUs with 4 GB+ VRAM. CPU fallback is supported for both models when GPU memory is insufficient or unavailable.

Related Documentation¶

• Heimdall AI Assistant — operator setup and configuration.
• Heimdall Context & Tokens — token budgeting and provider-specific context windows.
• Heimdall Agentic Loop — how plugin actions and MCP tools chain together.
• Enabling MCP Tools — how to add store/recall/discover/link/task/tasks to the loop.
• Heimdall Plugins — writing custom plugins, lifecycle hooks, autonomous invocation.
• Event Triggers and Automatic Remediation — wiring database events through the SLM.
• Auto-TLP Heimdall — SLM-gated edge inference.
• Feature Flags — full feature-flag surface including Heimdall.