Andrii Bidochko

• Updated: August 27, 2026
• 7 min read

ATHENA: Knowledge-guided agentic neural architecture search for AutoFormer-based electronic health record modeling

Direct Answer

ATHENA is a knowledge‑guided, agentic neural architecture search (NAS) framework that automatically discovers high‑performing Transformer configurations for electronic health record (EHR) prediction tasks across multiple hospitals. By reusing a pretrained weight‑sharing supernet and leveraging cross‑hospital architecture priors, ATHENA cuts the computational cost of NAS while delivering models that are consistently better than manually tuned baselines.

Background: Why This Problem Is Hard

Transformer‑based models have become the de‑facto standard for clinical prediction because they can ingest irregular, multimodal time‑series data from EHRs. However, their performance hinges on a delicate balance of hyper‑parameters—depth, attention heads, feed‑forward dimensions, and tokenization strategies—that varies dramatically between prediction tasks (e.g., mortality vs. readmission) and between health systems (different coding practices, patient demographics, and data quality).

Current practice relies on expert intuition and exhaustive grid or random search, which is both time‑consuming and computationally expensive. Traditional NAS methods improve automation but typically require training each candidate architecture from scratch, a cost that scales poorly with the billions of parameters typical of modern Transformers. Moreover, most NAS pipelines treat each hospital as an isolated search problem, ignoring the wealth of architectural knowledge that could be transferred from one institution to another.

What the Researchers Propose

ATHENA introduces a three‑layered, agent‑driven NAS system that unifies three core ideas:

• Weight‑sharing supernet: A single over‑parameterized Transformer is pretrained once per hospital. Any candidate architecture can be instantiated as a subnet, inheriting the supernet’s weights and thus avoiding costly retraining.
• Cross‑hospital architecture prior: A two‑step prior extracts knowledge from source hospitals. First, a retrieval module selects high‑performing architectures that match the target task’s descriptor (e.g., “30‑day readmission”). Second, a SHAP‑based meta‑regression estimates the marginal contribution of each architectural component (layers, heads, etc.) across sites.
• Multi‑agent LLM orchestrator: Large language model agents propose new architectures, guided by the priors and real‑time validation feedback from the target hospital. The agents iteratively refine proposals until the search budget (30 candidates) is exhausted.

How It Works in Practice

The ATHENA workflow can be broken down into four sequential stages, each of which can be visualized in the diagram below.

1. Supernet Pretraining

For each participating hospital, a massive Transformer (the supernet) is trained on the full EHR dataset using a standard self‑supervised objective (e.g., masked event modeling). Because the supernet contains every possible sub‑configuration (different depths, head counts, etc.), it serves as a universal weight reservoir.

2. Prior Construction

When a new prediction task arrives at a target hospital, ATHENA first queries a repository of previously discovered architectures from source hospitals. The query uses a task descriptor vector (clinical outcome, observation window, patient cohort size). The retrieval engine returns the top‑k similar architectures. Next, a SHAP‑based meta‑regression model, trained on historical performance data, predicts how each architectural knob will affect the target task’s metric.

3. Agentic Search Loop

An LLM‑powered agent receives the prior information and proposes a candidate architecture. The candidate is instantiated as a subnet of the supernet, fine‑tuned on a small validation split of the target hospital’s data, and evaluated. The validation score is fed back to the agent, which updates its internal policy (via reinforcement‑style prompting) before suggesting the next candidate. This loop repeats until the allotted budget of 30 candidates is spent.

4. Final Model Selection

The architecture with the highest validation score is selected, fully fine‑tuned on the target hospital’s training set, and deployed for inference. Because the search reuses the supernet’s weights, the total GPU hours are an order of magnitude lower than conventional NAS.

Evaluation & Results

ATHENA was benchmarked on six clinically relevant prediction tasks (e.g., in‑hospital mortality, 30‑day readmission, sepsis onset) across two independent health systems—Hospital A (a large academic center) and Hospital B (a community network). The study compared ATHENA against four strong NAS baselines: random search, evolutionary NAS, gradient‑based DARTS, and a recent LLM‑guided NAS that does not share weights.

Key findings include:

• Performance parity or gain: ATHENA matched or outperformed all baselines in 9 out of 12 hospital‑task combinations, often achieving a 1‑3% absolute lift in AUROC over the best baseline.
• Search efficiency: All methods were given the same budget of 30 candidate evaluations. ATHENA’s weight‑sharing reduced the average fine‑tuning time per candidate from ~2 hours (baseline) to ~15 minutes, cutting total search time by ~80%.
• Stability across runs: Repeating the search three times per task showed that ATHENA’s selected architecture varied less than 10% of the time, whereas baseline methods produced divergent architectures in >40% of repeats.
• Cross‑hospital transfer: The architecture priors contributed an average 0.7 AUROC boost, confirming that knowledge from source hospitals is valuable even when data distributions differ.

These results demonstrate that ATHENA not only reduces computational overhead but also yields more reliable, high‑performing models for heterogeneous clinical environments.

Why This Matters for AI Systems and Agents

For AI practitioners building health‑focused agents, ATHENA offers a practical pathway to automate one of the most labor‑intensive steps—model architecture design. By encapsulating the search inside an LLM‑driven agent, developers can plug ATHENA into existing UBOS platform overview pipelines, letting the system propose, evaluate, and select models without manual intervention.

From an operational standpoint, the weight‑sharing supernet aligns with the emerging trend of “foundation models for tabular and time‑series data.” Organizations can maintain a single supernet per hospital, dramatically lowering storage and compute costs while still supporting dozens of downstream tasks.

Moreover, the SHAP‑based meta‑regression provides interpretable insights into which architectural components drive performance. This transparency is crucial for compliance teams that must justify model choices to regulators and clinicians.

Finally, the agentic search loop can be orchestrated alongside other AI agents—such as data‑curation bots or monitoring services—through the Workflow automation studio, enabling end‑to‑end pipelines that span data ingestion, model discovery, and continuous evaluation.

What Comes Next

While ATHENA marks a significant step forward, several open challenges remain:

• Scalability to larger health networks: Extending the supernet to dozens of hospitals will require federated training techniques to respect patient privacy.
• Dynamic task adaptation: Real‑world clinical workflows evolve; future work should explore continual‑learning extensions that update the architecture prior without retraining from scratch.
• Broader modality support: Incorporating imaging, genomics, or free‑text notes into the same supernet could unlock multimodal predictors.

Researchers and product teams interested in experimenting with ATHENA can start by integrating the framework with existing OpenAI ChatGPT integration for LLM orchestration, or by leveraging the Telegram integration on UBOS to receive real‑time search status updates.

As the healthcare AI ecosystem matures, the ability to transfer architectural knowledge across institutions will become a competitive differentiator. ATHENA’s blend of weight sharing, SHAP‑driven meta‑learning, and agentic search positions it as a template for the next generation of auto‑ML solutions in medicine.

Further Reading and Resources

For a deep dive into the technical details, consult the original ATHENA paper on arXiv. Additional UBOS resources that complement ATHENA’s workflow include:

• Enterprise AI platform by UBOS – scalable infrastructure for hosting supernets.
• UBOS solutions for SMBs – cost‑effective deployment options for smaller clinics.
• UBOS templates for quick start – pre‑built pipelines that can be adapted for ATHENA searches.
• About UBOS – learn more about the team behind the platform.

Andrii Bidochko

CTO UBOS

Andrii Bidochko is an AI entrepreneur and researcher focused on AI agents, reinforcement learning, and autonomous systems. He writes about the technologies shaping the future of machine intelligence, from frontier models and agent architectures to real-world AI applications.