Skip to main content

Developer Guide: Off-Grid Local LLMs & Voice APIs 🤖🎙️

To support yachts traveling across transoceanic passages completely off-grid, Athena Helm supports fully localized language models and speech generators.

🍓 Raspberry Pi 5 Stack (Lowest Cost)​

Under a low-computational load, we deploy Ollama onto raw hardware:

  1. Service Binding Configuration (server.ts) The server targets the local Ollama instance on: http://localhost:11434/api/chat

  2. Prompt Templates & System Prompts We ingest the boat's instant telemetry dashboard values inside Ollama's context window:

    {
      "model": "qwen2.5:3b-instruct",
      "messages": [
        { "role": "system", "content": "You are Athena, a yacht autopilot AI. Local boat telemetry: Pitch=4deg, Batt=88%" },
        { "role": "user", "content": "Suggest sail trim advice" }
      ],
      "stream": false
    }

🎛️ High-Power Dedicated GPU Server Core​

For megayachts or custom performance racing multi-hulls equipped with dedicated GPU accelerators (such as a compact Nvidia Jetson AGX board or an RTX 4060):

1. Whisper Large-v3 Speech-to-Text Setup​

Using Python-based faster-whisper via localized virtual envs:

  • Core latency is lower than <80ms for multi-line commands.
  • We bind a localized WebSocket inside the dashboard to pipeline microphone audio captures directly.

2. Kokoro Voice Synthesis Model (Text-to-Speech)​

  • Outputs highly photorealistic oceanic captain audio outputs.
  • Kokoro has an incredibly small memory footprint (~82M parameters), executing natively on standard CPU/GPU cores on the boat without lag.