Andrii Bidochko

• Updated: August 14, 2026
• 6 min read

Sensitivity Analysis of GRU, LSTM and Transformer Encoder in Classification of Automated Driving Systems

Direct Answer

The paper introduces a systematic robustness framework for classifying Level‑2 automated driving systems (ADS) using only vehicle telematics, and it benchmarks three popular sequence models—GRU, LSTM, and a Transformer encoder—against realistic signal degradations. This matters because reliable ADS identification under noisy real‑world conditions is a prerequisite for safety monitoring, regulatory compliance, and insurance analytics in the emerging Software‑Defined Vehicle ecosystem.

Background: Why This Problem Is Hard

Modern vehicles increasingly host multiple driver‑assist packages, ranging from factory‑installed solutions (e.g., Cadillac Super Cruise) to aftermarket add‑ons (e.g., Comma.ai Openpilot). For regulators, fleet operators, and insurers, knowing which system is active at any moment is essential for:

• Detecting unsafe hand‑over events.
• Auditing compliance with jurisdiction‑specific Level‑2 limits.
• Triggering post‑incident forensic analysis.

However, telematics streams—speed, steering angle, brake pressure, and binary event flags—are prone to:

• Sensor drift and additive noise.
• Communication glitches that corrupt event timestamps.
• Temporal jitter caused by asynchronous logging across ECUs.

Existing classification pipelines typically assume clean, perfectly synchronized data. They either rely on handcrafted rule sets that break when a single signal is lost, or they train deep models on pristine datasets, which leads to catastrophic performance drops when real‑world noise appears. The gap between laboratory accuracy and field reliability has thus remained largely unquantified.

What the Researchers Propose

The authors present two complementary contributions:

1. A modular robustness evaluation framework. It injects five families of realistic telematics corruptions—continuous‑channel noise, cross‑channel drift, temporal jitter, binary‑event burst loss, and spurious toggles—at five severity levels (L1–L5). The framework is model‑agnostic, allowing any sequence classifier to be stress‑tested under controlled degradation.
2. A comparative study of three sequence‑based classifiers. They train a Gated Recurrent Unit (GRU), a Long Short‑Term Memory network (LSTM), and a Transformer encoder on clean telematics data to distinguish four classes: Openpilot, Tesla Autopilot, Cadillac Super Cruise, and manual driving. The same models are then evaluated through the corruption pipeline.

By keeping the data source (vehicle telematics) constant and varying only the corruption intensity, the study isolates how each model’s architecture tolerates specific signal failures.

How It Works in Practice

The practical workflow can be broken down into three stages:

1. Data Ingestion & Pre‑processing

• Raw CAN‑bus logs are parsed into a uniform time‑series matrix (continuous channels) and a set of binary event streams (e.g., “lane‑keep‑assist active”).
• Missing timestamps are linearly interpolated, and each feature is normalized to zero‑mean, unit‑variance.

2. Model Training

• Each architecture receives the same sliding‑window input (e.g., 5‑second sequences sampled at 10 Hz).
• Training uses a cross‑entropy loss with class‑balanced weighting to mitigate the natural imbalance between manual and assisted driving samples.
• Two regimes are explored: “clean‑only” training and “threat‑matched” training, where a subset of corrupted samples is mixed into the training set.

3. Robustness Evaluation

• The corruption engine applies one of the five families to the test set, progressively increasing severity from L1 (minor perturbation) to L5 (extreme degradation).
• Macro‑F1 is computed for each severity level, ensuring equal emphasis on all four driving‑mode classes.
• Results are visualized as degradation curves, highlighting where each model’s performance collapses.

This pipeline is deliberately modular: swapping in a new model (e.g., a convolutional time‑series network) or adding a novel corruption type requires only a plug‑in, not a redesign of the entire evaluation suite.

Evaluation & Results

The authors conduct experiments on a curated telematics dataset comprising thousands of miles driven under each ADS and manual control. Key findings include:

Clean‑Data Performance

• GRU: macro‑F1 ≈ 0.92
• LSTM: macro‑F1 ≈ 0.90
• Transformer encoder: macro‑F1 ≈ 0.93

All three models exceed the 0.90 threshold, confirming that telematics alone can reliably differentiate the four classes when signals are pristine.

Threat‑Matched Training

Introducing a modest proportion (≈ 15 %) of corrupted samples during training yields macro‑F1 scores between 0.904 and 0.916 on clean data—a negligible penalty—while improving resilience to certain noise patterns.

Robustness Under Corruption

The degradation curves reveal a stark dichotomy:

• Event‑level corruptions (burst loss, delayed transitions, spurious toggles, cross‑feature inconsistencies) cause only a slight dip; macro‑F1 remains ≥ 0.87 even at the most severe L5 level.
• Temporal jitter—randomly shifting timestamps within a window—drastically collapses performance, driving macro‑F1 down to 0.44–0.50 across all three architectures at L5.

This suggests that the models are robust to binary‑signal glitches but highly sensitive to misaligned temporal information, a finding that aligns with the sequential nature of GRU, LSTM, and Transformer encoders.

Why This Matters for AI Systems and Agents

From a systems‑engineering perspective, the study provides actionable insights for anyone building AI‑driven vehicle monitoring platforms:

• Safety‑critical alerting. Knowing that temporal jitter is the Achilles’ heel enables designers to prioritize high‑precision timestamp synchronization (e.g., IEEE 802.1AS) in data pipelines.
• Regulatory reporting. Robust classification under event‑level noise means that compliance dashboards can rely on binary flag integrity checks without expensive redundancy.
• Insurance risk modeling. Accurate ADS identification feeds directly into usage‑based insurance (UBI) calculations; the robustness framework can be embedded as a validation layer before premium adjustments.
• Agent orchestration. In multi‑agent fleets where a supervisory AI must switch between different ADS controllers, the findings guide the design of fallback strategies that monitor timestamp drift as a trigger for safe‑mode activation.

Practitioners can leverage the UBOS platform overview to integrate the robustness engine into existing telematics ingestion pipelines, while the Workflow automation studio can automate the generation of corruption‑augmented test sets for continuous integration.

What Comes Next

While the paper makes a solid first step, several avenues remain open:

• Multi‑modal fusion. Combining telematics with camera or radar embeddings could mitigate temporal jitter by providing cross‑modal temporal anchors.
• Adaptive corruption modeling. Real‑world fleets exhibit non‑stationary noise patterns; a meta‑learning approach that updates corruption parameters online would keep the robustness evaluation current.
• Explainability. Post‑hoc analysis (e.g., attention maps in the Transformer) could reveal which signals the model relies on most, informing sensor redundancy strategies.
• Edge deployment. Benchmarking inference latency on automotive‑grade hardware (e.g., NVIDIA Drive AGX) will determine whether these models can run in real time on the vehicle itself.

Addressing these challenges will bring us closer to a future where Software‑Defined Vehicles can safely host multiple, interchangeable ADS packages without sacrificing oversight.

For developers interested in building AI‑powered monitoring agents, the AI marketing agents showcase how modular AI components can be orchestrated on the same platform, illustrating the broader applicability of the robustness framework beyond automotive use cases.

Read the full arXiv paper for detailed methodology, and explore more research‑focused articles on the UBOS homepage.

Andrii Bidochko

CTO UBOS

Andrii Bidochko is an AI entrepreneur and researcher focused on AI agents, reinforcement learning, and autonomous systems. He writes about the technologies shaping the future of machine intelligence, from frontier models and agent architectures to real-world AI applications.