Skip to main content

Architecture Decision Record: Decoupled AI Compute Node for Local LLM Routing ⛵🤖

Status​

Decided (June 2026)

Context​

Athena Helm provides an offline voice companion mode that runs on-board via local LLM engines (Ollama running Gemma).

Initially, the Go telemetry server (backend) had its Ollama client target hardcoded to http://127.0.0.1:11434 (localhost). While this is convenient for development and local testing, it presents severe constraints in real-world yacht deployments:

  1. Compute Resource Limits: The yacht control computer (e.g., Raspberry Pi) that executes telemetry polling, NMEA 2000 bus decoding, and actuator control loops is a low-power, resource-constrained device. Running a multi-billion parameter LLM on the same machine risks CPU exhaustion, thermal throttling, and memory crashes, which threatens critical autopilot control safety.
  2. Dedicated Hardware Separation: A production-grade marine AI assistant requires specialized local neural network accelerators (e.g., an Orange Pi 5 with its NPU, a Jetson Orin Nano, or a dedicated mini-PC with an integrated GPU) placed on the boat's local subnet.

Decision​

We elected to decouple the local LLM server address from the Yacht Control computer by making the Ollama API endpoint configurable via the OLLAMA_API_URL environment variable.

  1. Environment Config: Modify callOllamaAPI in the Go server to retrieve OLLAMA_API_URL from the environment.
  2. Default Fallback: If OLLAMA_API_URL is empty, default to http://127.0.0.1:11434 to preserve standard development setups out-of-the-box.
  3. Resilience: If the configured remote Ollama server is unreachable or offline, fall back gracefully to the Go server's local deterministic rule-based parser.

Consequences​

  • Resource Protection & Autopilot Safety: Autopilot steering and NMEA 2000 bus reading remain responsive, unaffected by heavy local AI computation tasks.
  • Hardware Agility: Shipwrights can install a low-power Raspberry Pi at the helm to handle telemetry while placing a dedicated AI accelerator inside the cabin, communicating over local Wi-Fi/Ethernet.
  • Robust Fail-safes: Network hiccups between the control computer and the AI compute node do not disable companion commands; the system falls back transparently to local regex parsing.