Integrating OpenClaw Evaluation Data into CI/CD Pipelines for Automated Model Improvements

Prerequisites

1. OpenClaw CLI installed on the runner (available via pip install openclaw).
2. Repository secret OPENCLAW_API_KEY containing your OpenClaw token.
3. Docker image with your training script (e.g., myorg/model-trainer:latest).
4. Optional: UBOS partner program for managed compute resources.

Workflow file example

Create .github/workflows/openclaw-eval.yml in the root of your repo:

name: OpenClaw Evaluation & Auto‑Retrain

on:
  push:
    branches: [ main ]
  workflow_dispatch:

jobs:
  evaluate:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v3

      - name: Set up Python
        uses: actions/setup-python@v4
        with:
          python-version: '3.10'

      - name: Install OpenClaw CLI
        run: pip install openclaw

      - name: Run evaluation
        env:
          OPENCLAW_API_KEY: ${{ secrets.OPENCLAW_API_KEY }}
        run: |
          openclaw evaluate \
            --model-path ./model \
            --dataset ./data/test \
            --output metrics.json

      - name: Upload metrics as artifact
        uses: actions/upload-artifact@v3
        with:
          name: openclaw-metrics
          path: metrics.json

      - name: Check thresholds & trigger retrain
        id: check
        run: |
          THRESHOLD=0.85
          ACC=$(jq .accuracy metrics.json)
          if (( $(echo "$ACC < $THRESHOLD" | bc -l) )); then
            echo "retrain_needed=true" >> $GITHUB_OUTPUT
          else
            echo "retrain_needed=false" >> $GITHUB_OUTPUT
          fi

  retrain:
    needs: evaluate
    if: needs.evaluate.outputs.retrain_needed == 'true'
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v3

      - name: Pull Docker image
        run: docker pull myorg/model-trainer:latest

      - name: Run training container
        run: |
          docker run --rm \
            -v ${{ github.workspace }}:/workspace \
            myorg/model-trainer:latest \
            python train.py --data /workspace/data/train

      - name: Commit new model
        run: |
          git config user.name "github-actions"
          git config user.email "actions@github.com"
          git add model/
          git commit -m "🤖 Auto‑retrained model after OpenClaw trigger"
          git push

The workflow evaluates the model, compares the accuracy metric against a threshold, and automatically launches a Docker‑based retraining job when needed.

.gitlab-ci.yml example

Add the following to your repository root:

stages:
  - evaluate
  - retrain

variables:
  OPENCLAW_API_KEY: $OPENCLAW_API_KEY
  THRESHOLD: "0.90"

evaluate:
  stage: evaluate
  image: python:3.10-slim
  script:
    - pip install openclaw jq
    - openclaw evaluate --model-path model/ --dataset data/test/ --output metrics.json
    - cat metrics.json
    - |
      ACC=$(jq .accuracy metrics.json)
      if (( $(echo "$ACC < $THRESHOLD" | bc -l) )); then
        echo "RETRAIN=true" >> variables.env
      else
        echo "RETRAIN=false" >> variables.env
      fi
  artifacts:
    paths:
      - metrics.json
    reports:
      dotenv: variables.env

retrain:
  stage: retrain
  image: docker:latest
  services:
    - docker:dind
  script:
    - if [ "$RETRAIN" = "true" ]; then
        echo "Starting auto‑retrain...";
        docker pull myorg/model-trainer:latest;
        docker run --rm -v $CI_PROJECT_DIR:/workspace myorg/model-trainer:latest python train.py --data /workspace/data/train;
        git config --global user.email "gitlab-ci@example.com";
        git config --global user.name "GitLab CI";
        git add model/;
        git commit -m "🤖 Auto‑retrained model via GitLab CI";
        git push origin $CI_COMMIT_REF_NAME;
      else
        echo "Metrics meet threshold – no retrain needed.";
      fi
  only:
    - main

This pipeline mirrors the GitHub Actions flow but uses GitLab’s dotenv artifact to pass the RETRAIN flag between stages.

Jenkinsfile example

pipeline {
    agent any

    environment {
        OPENCLAW_API_KEY = credentials('openclaw-api-key')
        THRESHOLD = '0.88'
    }

    stages {
        stage('Checkout') {
            steps {
                checkout scm
            }
        }

        stage('Install OpenClaw') {
            steps {
                sh 'python -m pip install --upgrade pip'
                sh 'pip install openclaw jq'
            }
        }

        stage('Evaluate') {
            steps {
                sh '''
                    openclaw evaluate \\
                        --model-path ./model \\
                        --dataset ./data/test \\
                        --output metrics.json
                '''
                sh 'cat metrics.json'
                script {
                    def acc = sh(script: "jq .accuracy metrics.json", returnStdout: true).trim()
                    if (acc.toDouble() < env.THRESHOLD.toDouble()) {
                        env.RETRAIN = 'true'
                    } else {
                        env.RETRAIN = 'false'
                    }
                }
            }
            post {
                always {
                    archiveArtifacts artifacts: 'metrics.json', fingerprint: true
                }
            }
        }

        stage('Retrain') {
            when {
                expression { return env.RETRAIN == 'true' }
            }
            steps {
                sh '''
                    docker pull myorg/model-trainer:latest
                    docker run --rm -v $WORKSPACE:/workspace myorg/model-trainer:latest \\
                        python train.py --data /workspace/data/train
                '''
                sh '''
                    git config user.email "jenkins@ci.com"
                    git config user.name "Jenkins CI"
                    git add model/
                    git commit -m "🤖 Auto‑retrained model via Jenkins"
                    git push origin HEAD:${env.BRANCH_NAME}
                '''
            }
        }
    }

    post {
        always {
            echo "Pipeline finished. Retrain flag: ${env.RETRAIN}"
        }
        failure {
            mail to: 'devops-team@example.com',
                 subject: "OpenClaw pipeline failed",
                 body: "Check Jenkins job ${env.JOB_NAME} #${env.BUILD_NUMBER}"
        }
    }
}

Jenkins uses the when directive to conditionally execute the retraining stage. The credentials helper securely injects the OpenClaw API key.