Andrii Bidochko

• Updated: July 1, 2026
• 6 min read

Protocol-Aware Tokenization and Architecture Co-Design for Wireless Packet Foundation Models

Direct Answer

The paper introduces a systematic study that isolates the impact of protocol‑aware tokenization versus model architecture when building foundation models for wireless packet traces. It shows that a carefully designed tokenizer contributes the bulk of accuracy gains, while the choice of backbone (deep GPT vs. Mamba‑2 state‑space) mainly trades speed and context length for a modest performance shift.

Background: Why This Problem Is Hard

Wireless networks generate massive streams of packet‑level data that encode protocol state, timing, and error‑correction information. Turning these raw traces into a form that large language‑style models can consume poses two intertwined challenges:

• Semantic granularity: Traditional byte‑or‑bit tokenizers treat packets as generic sequences, discarding the hierarchical meaning of fields such as MAC addresses, sequence numbers, and control flags.
• Model scalability: Even if a tokenizer preserves protocol semantics, the downstream architecture must handle long‑range dependencies (e.g., retransmission patterns) without exploding computational cost.

Prior work on network traffic modeling either relied on handcrafted feature extraction pipelines or applied generic tokenizers borrowed from natural‑language processing. Those approaches struggle to capture the strict syntactic rules of 802.11 and newer Wi‑Fi standards, leading to poor generalization across devices, firmware versions, and channel conditions.

What the Researchers Propose

The authors extend the earlier PLUME framework, which introduced a protocol‑aware tokenizer that maps each 802.11 field to a distinct token vocabulary. Their contribution is two‑fold:

1. Tokenizer scaling: They retain the same tokenization logic while expanding the vocabulary to cover newer amendments and optional extensions, ensuring that the tokenizer remains a faithful representation of the protocol.
2. Architecture co‑design: They evaluate the tokenizer on two divergent model families:
   • PLUME‑DEEP: a 24‑layer GPT‑style transformer that deepens the original 12‑layer design.
   • PLUME‑MAMBA: a Mamba‑2 state‑space model that replaces attention with a recurrence‑based kernel, offering higher throughput and longer context windows.

The key research question is whether performance improvements stem primarily from richer token semantics or from architectural sophistication.

How It Works in Practice

The end‑to‑end pipeline can be broken into three logical stages:

1. Packet Capture & Pre‑Processing

Raw 802.11 frames are collected using standard monitoring interfaces. Each frame is parsed to extract fields defined in the IEEE 802.11 specification (e.g., Frame Control, Duration, Address fields, Sequence Control, QoS Control). Missing or malformed fields are flagged but retained to preserve real‑world noise.

2. Protocol‑Aware Tokenization

The tokenizer maps each extracted field to a unique token ID. For example, a MAC address token is generated by concatenating a prefix (“MAC_”) with the hexadecimal representation, while control‑flag bits become separate binary tokens. This results in a deterministic, reversible sequence that mirrors the packet’s logical structure.

3. Model Inference

Two model families consume the token stream:

• PLUME‑DEEP (GPT‑style): Uses multi‑head self‑attention across 24 layers, enabling the model to learn complex cross‑packet relationships such as hidden retransmission loops.
• PLUME‑MAMBA (State‑Space): Replaces attention with a linear‑time state‑space kernel, allowing the model to process longer sequences (up to 2× the context length) with 1.7× higher throughput.

Both models are trained on a massive corpus of anonymized Wi‑Fi traffic, optimizing for next‑token prediction—a proxy for understanding protocol dynamics.

Evaluation & Results

The authors conduct a controlled 2×2 experiment, swapping tokenizers and architectures to isolate each factor’s contribution. Evaluation focuses on top‑1 accuracy for predicting the next packet token, a standard metric in sequence modeling.

Test Scenarios

• Standard 802.11 traffic: Captured from office Wi‑Fi networks under typical load.
• Edge‑case extensions: Includes frames with optional vendor‑specific information elements.
• Cross‑device validation: Trains on one set of access points and tests on a different hardware generation.

Key Findings

• Switching from a generic byte‑level tokenizer to the protocol‑aware tokenizer improves top‑1 accuracy by **32 points** (from ~66% to 98%).
• Deepening the GPT architecture (PLUME‑DEEP) adds another **2 points** of accuracy, reaching **98.2%**.
• The Mamba‑2 variant (PLUME‑MAMBA) attains **96.1%** accuracy while delivering **1.7× higher throughput** and **twice the context window** compared to PLUME‑DEEP.
• When the tokenizer is held constant, architectural changes shift performance by only a few percentage points, confirming that tokenization is the dominant lever.

These results demonstrate that a well‑engineered tokenizer can unlock near‑perfect predictive performance on wireless packet streams, while the choice of backbone primarily influences deployment characteristics such as latency and memory footprint.

Why This Matters for AI Systems and Agents

For practitioners building AI‑driven network management or security agents, the study offers a clear design hierarchy:

• First, invest in protocol‑aware tokenization. By aligning the token vocabulary with the underlying standards, models inherit domain knowledge without additional supervision.
• Second, select an architecture based on operational constraints. If real‑time inference on edge routers is required, PLUME‑MAMBA’s higher throughput and longer context are advantageous. For scenarios where absolute accuracy is paramount—such as forensic analysis of anomalous traffic—PLUME‑DEEP’s deeper transformer may be preferable.

This hierarchy simplifies the engineering trade‑off space, allowing teams to decouple “accuracy engineering” from “systems engineering.” The insight also extends to other protocol domains (e.g., 5G NR, Bluetooth), suggesting that tokenization can serve as a universal performance multiplier across networking AI.

Organizations looking to embed intelligent agents into their communication stacks can leverage these findings to accelerate time‑to‑value. For example, integrating a PLUME‑MAMBA model into a Enterprise AI platform by UBOS enables high‑throughput packet‑level inference without sacrificing the ability to capture long‑range temporal patterns.

What Comes Next

While the paper establishes tokenization as the primary performance driver, several open challenges remain:

• Generalization to emerging standards: Future Wi‑Fi generations (e.g., 802.11be) introduce new frame formats. Extending the tokenizer to accommodate these without retraining from scratch will be critical.
• Multi‑protocol fusion: Real‑world networks often carry mixed traffic (Wi‑Fi, Ethernet, cellular). A unified tokenizer that respects each protocol’s hierarchy could enable cross‑layer AI agents.
• Resource‑constrained deployment: Edge devices with limited memory may still struggle with 24‑layer transformers. Research into quantization, pruning, or hybrid attention‑state‑space models could bridge this gap.
• Explainability and debugging: Mapping model predictions back to specific protocol fields can aid network engineers in diagnosing anomalies. Developing visualization tools that link token‑level attention to packet semantics is an active area.

Addressing these directions will transform protocol‑aware foundation models from research prototypes into production‑grade components of autonomous network orchestration platforms.

For teams ready to experiment, the AI marketing agents page showcases how UBOS integrates custom tokenizers into its workflow automation studio, providing a low‑code entry point for building domain‑specific AI pipelines.

Read the full study on arXiv paper for detailed methodology, dataset specifications, and reproducibility instructions.

Andrii Bidochko

CTO UBOS

Andrii Bidochko is an AI entrepreneur and researcher focused on AI agents, reinforcement learning, and autonomous systems. He writes about the technologies shaping the future of machine intelligence, from frontier models and agent architectures to real-world AI applications.