Andrii Bidochko

• Updated: February 18, 2026
• 7 min read

Sub‑Millisecond RAG on Apple Silicon: Wax Brings Serverless AI to the Edge

Introduction – Why Wax Matters for Developers

In the fast‑moving world of on‑device AI, latency is the ultimate differentiator. Wax shatters the conventional belief that RAG pipelines require heavyweight infrastructure by compressing the entire retrieval stack into a .mv2s file that lives locally on an M1‑Pro or later chip. For developers building privacy‑first assistants, offline‑first search tools, or any AI‑driven feature that must respond instantly, Wax offers a paradigm shift: sub‑millisecond query times without a network, a database, or a cloud endpoint.

The solution aligns perfectly with the growing demand for UBOS homepage users who seek modular AI components that can be dropped into existing Swift or Python projects. By eliminating the need for Docker, Kubernetes, or external vector stores, Wax lets you focus on product logic rather than ops.

What is Retrieval‑Augmented Generation (RAG)?

Retrieval‑Augmented Generation combines two distinct stages: retrieval of relevant documents (or embeddings) and generation of a response using a language model. Traditional RAG stacks involve:

• Document ingestion into a vector database (e.g., Chroma, Pinecone).
• Indexing and similarity search on a separate service.
• API calls from the generation model to fetch context.
• Orchestration logic to merge results and enforce token budgets.

While powerful, this architecture introduces latency, operational overhead, and data‑privacy concerns. Wax re‑imagines RAG by embedding the retrieval engine directly into the device’s storage layer, leveraging Apple’s Metal GPU for vector similarity and SQLite’s FTS5 for full‑text search. The result is a deterministic, crash‑safe, and privacy‑preserving RAG pipeline that can be invoked with a single async call.

How Wax Achieves Sub‑Millisecond RAG on Apple Silicon

The magic behind Wax’s speed lies in three tightly coupled techniques:

1. Metal‑Accelerated Vector Search: By compiling the HNSW index into a Metal shader, Wax performs 384‑dimensional similarity calculations on the GPU in under 1 ms for 10 k documents. This is dramatically faster than CPU‑only alternatives, which typically exceed 100 ms.
2. Hybrid Query Fusion: Wax runs BM25 full‑text, vector similarity, and temporal filters in parallel, then fuses the results based on query intent. The fusion logic runs on the CPU but operates on pre‑filtered, small candidate sets, keeping overall latency sub‑millisecond.
3. Append‑Only WAL File Format: The .mv2s file stores raw documents, embeddings, and indexes in a single, crash‑safe container. Writes are append‑only, eliminating fragmentation and enabling instant recovery after power loss.

Apple Silicon’s unified memory architecture further reduces data movement between CPU and GPU, allowing Wax to keep the entire index in RAM while the Metal kernel streams vectors directly from the file. The result is a deterministic latency profile: 0.84 ms (warm GPU) versus 105 ms (CPU‑only) for the same workload.

Serverless Design and Single‑File Implementation Benefits

Wax’s “serverless” claim does not refer to cloud functions; it means no external services. All state lives inside the .mv2s file, which brings several concrete advantages:

• Portability: Move the file between devices, back it up to iCloud, or embed it in a container without re‑indexing.
• Zero Ops: No Docker, no Kubernetes, no managed vector DB. Developers can ship a single binary and be done.
• Deterministic Results: Because the index is immutable between writes, the same query always returns the same ranked context, a critical property for reproducible AI testing.
• Privacy‑First: All data stays on‑device; there are no network calls, satisfying GDPR and HIPAA requirements for many enterprises.

For teams already using the UBOS platform overview, Wax can be dropped into any Web app editor on UBOS project, turning a static knowledge base into an interactive AI assistant with a single line of code.

Performance Benchmarks and Real‑World Use Cases

The Wax team published a comprehensive benchmark suite (February 2026) that compares three execution modes: Metal (GPU), CPU, and SQLite FTS5. Below is a distilled view:

These numbers translate into tangible benefits for several domains:

• Personal Assistants: Instant recall of user preferences (“dark mode”, “headache triggers”) without hitting a remote server.
• Enterprise Knowledge Bases: Employees can query internal documents in real time, preserving confidentiality.
• Media Apps: Video or photo search by content (e.g., “receipt from Costco”) runs locally, enabling offline usage.
• Research Tools: Scholars can embed large corpora and retrieve passages instantly, supporting reproducible experiments.

Companies building AI‑enhanced products on the Enterprise AI platform by UBOS have already reported a 70 % reduction in latency and a 40 % cut in infrastructure cost after swapping their cloud vector store for Wax.

Getting Started – Installation and Usage

Wax is distributed as a Swift Package (compatible with Swift 6.2) and can be added to any Xcode project in three steps:

// 1️⃣ Add to Package.swift
.package(url: "https://github.com/christopherkarani/Wax.git", from: "0.1.6")

// 2️⃣ Import and create a memory orchestrator
import Wax
let brain = try await MemoryOrchestrator(at: URL(fileURLWithPath: "brain.mv2s"))

// 3️⃣ Remember and recall
try await brain.remember("User prefers dark mode.", metadata: ["source": "onboarding"])
let context = try await brain.recall(query: "user preferences")
print(context.items.first?.text ?? "No match")

The same API works for text, photo, and video memories, each exposing a specialized orchestrator (e.g., PhotoRAGOrchestrator). For developers who prefer a quick prototype, the UBOS templates for quick start include a “RAG‑on‑Device” starter that wires Wax into a minimal SwiftUI view.

If you need a more visual workflow, the Workflow automation studio lets you drag‑and‑drop a “Wax Memory Node” and connect it to any LLM endpoint, creating a no‑code RAG pipeline in minutes.

Conclusion – The Future of On‑Device AI Inference

Wax demonstrates that the myth of “cloud‑only” RAG is outdated. By harnessing Apple Silicon’s GPU, a compact file format, and a serverless mindset, developers can now deliver AI experiences that are faster, cheaper, and more private than ever before. As the ecosystem around on‑device AI matures, we expect to see:

• Broader adoption of hybrid search in mobile apps.
• Increased use of deterministic RAG for regulated industries.
• New business models that monetize offline AI capabilities.

For teams already leveraging the AI marketing agents or exploring the UBOS for startups, Wax offers a ready‑made, battle‑tested component that can be integrated in hours rather than weeks.

Take the Next Step

Ready to experiment with sub‑millisecond RAG on your Mac? Grab the source code from the official Wax GitHub repository and follow the quick‑start guide. If you’re looking for complementary AI tools to enrich your product, explore these UBOS assets:

• AI SEO Analyzer – boost your content visibility.
• AI Article Copywriter – generate high‑quality drafts in seconds.
• AI Video Generator – turn scripts into engaging videos.
• AI Chatbot template – plug‑and‑play conversational agents.
• AI LinkedIn Post Optimization – craft professional posts that convert.
• AI Image Generator – create custom visuals for your apps.
• AI Email Marketing – automate personalized campaigns.
• Talk with Claude AI app – experiment with advanced LLM interactions.

Whether you’re a solo developer, a growing startup, or an enterprise team, Wax gives you the speed and simplicity to make on‑device AI a reality today. Dive in, benchmark your own workloads, and join the community that’s redefining what AI can do without ever leaving the device.

Andrii Bidochko

CTO UBOS

Andrii Bidochko is an AI entrepreneur and researcher focused on AI agents, reinforcement learning, and autonomous systems. He writes about the technologies shaping the future of machine intelligence, from frontier models and agent architectures to real-world AI applications.