Andrii Bidochko

• Updated: June 13, 2026
• 7 min read

Picid: A Modular Evaluation Infrastructure for Reproducible PHM Across Tasks and Domains

Direct Answer

Picid is a modular, open‑source evaluation infrastructure that codifies the entire Prognostics and Health Management (PHM) pipeline into a deterministic, leakage‑safe protocol. By standardizing data contracts, preprocessing steps, and metric calculations, Picid makes it possible to reproduce results across fault detection, diagnostics, and prognostics tasks—and to compare models fairly across wildly different domains such as batteries, turbofan engines, and building systems.

Background: Why This Problem Is Hard

PHM research sits at the intersection of signal processing, machine‑learning, and domain‑specific engineering. Practitioners routinely work with time‑series data that differ in sampling rates, sensor layouts, and failure definitions. Because each study typically defines its own data split, windowing strategy, and evaluation metric, two papers that claim “state‑of‑the‑art” performance are rarely comparable. The lack of a shared protocol leads to three concrete bottlenecks:

• Reproducibility gaps: Critical choices—such as how to align labels with sensor streams or whether to include future information in training—are often hidden in code repositories, making it hard for a third party to replicate results.
• Cross‑task inconsistency: Fault detection (binary classification), diagnostics (multi‑class classification), and prognostics (remaining‑useful‑life regression) each require different preprocessing and loss functions, yet most benchmark suites treat them as unrelated silos.
• Domain fragmentation: Datasets for bearings, batteries, hydraulics, and HVAC systems live in separate repositories with incompatible file formats and metadata conventions, forcing researchers to reinvent data loaders for every new experiment.

These challenges are amplified by the rapid adoption of deep learning models that are highly sensitive to data leakage and hyper‑parameter tuning. Without a unified evaluation backbone, progress stalls, and industry adoption of PHM solutions remains tentative.

What the Researchers Propose

The authors introduce Picid (Prognostics and Health Management Integrated Development), a framework that treats the PHM evaluation pipeline as an explicit, executable protocol. Picid’s design rests on three pillars:

1. Deterministic dataset construction: A declarative data contract specifies raw source files, preprocessing transforms, temporal windowing, and label alignment. The contract is compiled into a reproducible dataset object that guarantees no information leakage between training and test splits.
2. Task‑agnostic interface: Whether the downstream objective is detection, diagnosis, or remaining‑useful‑life prediction, Picid presents a unified fit/predict API. Model classes plug into the same interface, enabling side‑by‑side comparison of classification and regression families.
3. Extensible modularity: New datasets, preprocessing steps, or evaluation metrics can be added as plug‑ins without breaking the core protocol invariants. The framework enforces versioned contracts, so any change is traceable and reproducible.

In essence, Picid acts as a contract‑driven orchestrator that separates “what the data look like” from “how a model consumes the data,” thereby eliminating hidden assumptions that have historically plagued PHM research.

How It Works in Practice

Picid’s workflow can be visualized as a four‑stage pipeline:

1. Contract Definition: Researchers author a YAML/JSON contract that lists raw files, required preprocessing (e.g., detrending, normalization), temporal window size, and the target label schema (binary, multi‑class, or continuous). The contract also encodes the random seed for split generation.
2. Dataset Builder: The Picid engine parses the contract, loads the raw signals, applies the declared transforms, and slices the time series into overlapping windows. It then produces three immutable dataset objects—train, validation, and test—each with a guaranteed leakage‑free boundary.
3. Model Adapter: Any model that implements the Picid ModelAdapter interface can be registered. The adapter handles data reshaping, loss selection, and metric logging, allowing the same model code to run on classification or regression tasks without modification.
4. Evaluation Engine: After training, the engine computes a standardized metric suite (e.g., ROC‑AUC for detection, macro‑F1 for diagnostics, and RMSE for prognostics) and logs results in a reproducible JSON report. The report includes the exact contract version, code hash, and hardware configuration.

What sets Picid apart is its strict enforcement of “protocol invariants.” For example, the engine automatically checks that no future sensor readings appear in the training windows of a test instance—a common source of data leakage in time‑series forecasting. Because every step is declarative, swapping a preprocessing plug‑in or adding a new dataset requires only a contract edit, not a rewrite of the training loop.

Evaluation & Results

The authors validated Picid on a diverse benchmark comprising twelve publicly available PHM datasets: lithium‑ion batteries, rolling‑element bearings, turbofan engine degradation, hydraulic pump wear, air‑filter clogging, and commercial building HVAC performance. Across these domains, they trained thirteen models ranging from classical random forests to state‑of‑the‑art transformer‑based sequence learners.

Key findings include:

• Consistent ranking: When evaluated under Picid’s protocol, model performance rankings remained stable across multiple random seeds, confirming that the framework eliminates stochastic variance caused by inconsistent splits.
• Cross‑task fairness: The same model family (e.g., a Temporal Convolutional Network) achieved comparable relative gains on both fault detection (ROC‑AUC improvement of 4.2%) and remaining‑useful‑life prediction (RMSE reduction of 5.1%) when compared to baseline pipelines that used ad‑hoc preprocessing.
• Reproducibility proof: Independent researchers re‑ran the entire benchmark on a separate compute cluster and reproduced the published JSON reports within a 0.3% margin of error, demonstrating that Picid’s contract‑driven approach is truly portable.

Beyond raw numbers, the experiments highlighted a practical insight: many “state‑of‑the‑art” gains reported in prior literature vanished once the data were processed through Picid’s leakage‑safe windows. This suggests that a substantial portion of claimed progress may have stemmed from inadvertent information leakage rather than genuine algorithmic improvement.

Why This Matters for AI Systems and Agents

For AI practitioners building predictive maintenance agents, Picid offers a turnkey solution to the most painful part of the development cycle—data preparation and evaluation. By providing a deterministic, version‑controlled dataset, engineers can focus on model innovation rather than reinventing preprocessing pipelines for each new sensor suite.

Moreover, the unified interface enables seamless integration of PHM models into larger autonomous agent ecosystems. An agent responsible for fleet management can query a Picid‑trained prognostic model for remaining‑useful‑life estimates, while the same infrastructure supplies fault‑detection scores for real‑time alerting. This consistency reduces the engineering overhead of maintaining separate codebases for classification and regression tasks.

From an operational standpoint, Picid’s standardized reports feed directly into monitoring dashboards, supporting continuous model validation and drift detection. Companies can thus adopt a UBOS platform overview that leverages Picid’s contracts to orchestrate model retraining pipelines, ensuring that updates are reproducible and auditable.

Finally, the modular plug‑in architecture aligns with the emerging trend of “AI‑first” workflow automation. Teams can attach Picid’s evaluation engine to a Workflow automation studio, triggering automated benchmark runs whenever new sensor data arrive, and instantly surfacing performance regressions to data scientists.

What Comes Next

While Picid marks a significant step toward reproducible PHM research, several avenues remain open for expansion:

• Domain‑specific extensions: Adding plug‑ins for physics‑informed preprocessing (e.g., electrochemical models for batteries) could further narrow the gap between data‑driven and model‑based prognostics.
• Scalable cloud execution: Integrating Picid with container orchestration platforms would enable large‑scale hyper‑parameter sweeps across hundreds of datasets, a capability that aligns with the Enterprise AI platform by UBOS.
• Real‑time inference hooks: Extending the evaluation engine to support streaming inference would allow on‑device deployment of Picid‑validated models in edge scenarios such as industrial IoT gateways.
• Community‑driven benchmark hub: A public repository of contracts, model adapters, and result logs could foster collaborative improvement, similar to the model zoos used in computer vision.

Researchers interested in contributing can start by exploring the Picid paper on arXiv, cloning the open‑source repository, and submitting a new dataset contract. Early adopters in the startup ecosystem may find the UBOS for startups program a convenient entry point for building AI‑powered maintenance solutions that leverage Picid’s reproducibility guarantees.

In summary, Picid transforms the fragmented landscape of PHM evaluation into a coherent, extensible ecosystem. By codifying every step of the data‑to‑metric journey, it empowers engineers, researchers, and product teams to benchmark fairly, iterate faster, and ultimately deliver more reliable predictive maintenance agents.

Andrii Bidochko

CTO UBOS

Andrii Bidochko is an AI entrepreneur and researcher focused on AI agents, reinforcement learning, and autonomous systems. He writes about the technologies shaping the future of machine intelligence, from frontier models and agent architectures to real-world AI applications.