Andrii Bidochko

• Updated: July 24, 2026
• 6 min read

A Coreset Selection Framework with Ensemble Aggregation for Image Classification

Direct Answer

The paper introduces SCOSS (Score‑Stratified Coreset Selection), a framework that builds compact, representative training subsets for image classification and then aggregates predictions from multiple independently sampled models. By doing so, it cuts training time and memory while preserving—or even improving—accuracy, a trade‑off that matters for any organization scaling visual AI workloads.

Background: Why This Problem Is Hard

Modern computer‑vision pipelines often rely on millions of labeled images. While large datasets boost model performance, they also inflate compute budgets, extend training cycles, and increase carbon footprints. Practitioners face two intertwined challenges:

• Sample relevance ambiguity: Not every image contributes equally to the decision boundary; identifying the most informative examples is non‑trivial.
• Variability across runs: Random seeds, data shuffling, and hyper‑parameter tweaks can cause the same model to behave differently, making a single “best” subset elusive.

Existing data‑reduction techniques—random down‑sampling, class‑balanced pruning, or heuristic‑based coresets—either ignore the nuanced distribution of difficulty across samples or assume a static model behavior. Consequently, they either sacrifice accuracy or fail to deliver consistent gains when the underlying classifier changes (e.g., from a linear SVM to a graph‑based neural net).

What the Researchers Propose

SCOSS tackles the above bottlenecks with a two‑pronged strategy:

Score‑Stratified Selection (SCOSS)

Instead of treating the training pool as a monolith, SCOSS first assigns each image a score that reflects its difficulty or informativeness (e.g., loss magnitude, margin, or a proxy confidence). The score range is then divided into equal‑width intervals, and a fixed number of samples is drawn from each interval. This stratification guarantees that easy, medium, and hard examples are all represented proportionally.

Ensemble Aggregation over Independent Subsets

To mitigate run‑to‑run variance, the framework repeats the SCOSS sampling process multiple times, training a separate model on each subset. At inference, predictions are combined—typically via majority voting or averaging—producing an ensemble that benefits from diverse views of the data while still operating on reduced training loads.

How It Works in Practice

The SCOSS pipeline can be visualized as a four‑stage workflow:

1. Score Computation: A lightweight “scorer” model (often a shallow network or a pre‑trained feature extractor) processes the full training set and outputs a scalar per image.
2. Stratified Partitioning: The score spectrum is split into k intervals (e.g., deciles). From each interval, a predetermined quota of images is randomly selected, forming a balanced coreset.
3. Independent Model Training: The process repeats n times, each time generating a new coreset and training a full‑scale classifier (such as Simple Graph Convolution (SGC) or a Support Vector Machine (SVM)).
4. Ensemble Prediction: At test time, each trained model emits a class probability vector; the final decision is the aggregated result across the n models.

What distinguishes SCOSS from a naïve random subset approach is the explicit preservation of difficulty diversity and the systematic reduction of stochastic training effects through ensemble voting.

Evaluation & Results

The authors benchmarked SCOSS on two image classification benchmarks: a standard coarse‑grained dataset and a fine‑grained counterpart that demands subtle visual discrimination. They compared four sampling strategies—random, class‑balanced random, moderate (fixed‑size heuristic), and SCOSS—across two classifiers (SGC and SVM) and varied the sampling ratio from 10 % to 50 % of the original training set.

Key Findings

• Competitive Accuracy: Across most ratios, SCOSS matched or exceeded the random baselines, especially for the graph‑based SGC where preserving hard examples proved critical.
• Best‑in‑Class for SGC: When using SGC, SCOSS consistently delivered the highest accuracy among all sampling methods, confirming that stratified difficulty sampling aligns well with graph convolutional learning dynamics.
• Efficiency Gains: Training on a 30 % coreset reduced wall‑clock time by roughly 2.5× while keeping the performance gap within 1 % of the full‑data model.
• Fine‑Grained Advantage: On the more challenging dataset, SGC trained with SCOSS outperformed SVMs even when the latter used the entire labeled pool, highlighting the synergy between stratified coresets and graph‑based feature propagation.

These results demonstrate that SCOSS is not merely a data‑reduction trick; it offers a principled pathway to maintain model fidelity while slashing resource consumption.

Why This Matters for AI Systems and Agents

For teams building AI‑driven products—whether autonomous visual inspection bots, recommendation engines that rely on image embeddings, or large‑scale content moderation pipelines—the ability to train on smaller, yet representative, subsets translates directly into cost savings and faster iteration cycles. SCOSS enables:

• Rapid prototyping: Data scientists can experiment with new architectures on a fraction of the data, shortening the feedback loop from weeks to days.
• Scalable deployment: Edge devices with limited memory can host models trained on SCOSS coresets without sacrificing critical accuracy.
• Robust ensemble design: By aggregating predictions from multiple lightweight models, system architects can achieve higher reliability—an essential trait for autonomous agents operating under uncertainty.

Moreover, the framework dovetails with modern AI orchestration platforms that support automated workflow pipelines. For example, integrating SCOSS into a Workflow automation studio can automate the scoring, sampling, and ensemble training steps, turning a research prototype into a production‑ready service with minimal manual overhead.

What Comes Next

While SCOSS marks a solid step forward, several avenues remain open for exploration:

• Dynamic scoring functions: Current experiments rely on a static scorer; future work could adapt scores online as the model learns, creating a feedback loop that refines the coreset in real time.
• Cross‑modal coresets: Extending stratified selection to multimodal datasets (e.g., image‑text pairs) could amplify efficiency gains for vision‑language models.
• Hardware‑aware sampling: Tailoring the coreset size to specific GPU or edge‑device memory constraints would make SCOSS a plug‑and‑play component for heterogeneous deployment environments.
• Integration with enterprise AI stacks: Embedding SCOSS into an Enterprise AI platform by UBOS would let large organizations manage data pipelines, model registries, and monitoring dashboards from a single console, turning research insights into scalable business value.

Addressing these challenges will not only sharpen the efficiency edge but also broaden the applicability of coreset‑based training to emerging AI agents that must learn continuously from streaming visual data.

For a deeper dive into the methodology and to access the open‑source implementation, see the original arXiv paper.

Andrii Bidochko

CTO UBOS

Andrii Bidochko is an AI entrepreneur and researcher focused on AI agents, reinforcement learning, and autonomous systems. He writes about the technologies shaping the future of machine intelligence, from frontier models and agent architectures to real-world AI applications.