Skip to main content
Geval integrates like any CLI: actions/checkout gives the job your repo files, you point geval check at a contract and a signals JSON file, then you interpret the exit code. There is no npm package for this engine—use the release binary.

​
What you must provide

InputWhere it comes from
Contract + policiesUsually committed (e.g. .geval/contract.yaml, .geval/policies/*.yaml). Same files as local geval check.
signals.jsonEither already in the repo (commit it) or created in the workflow by a script that talks to LangSmith, Braintrust, your API, etc.
Important: GitHub Actions can read any file you commit. You do not have to regenerate signals.json in CI unless you want fresh metrics every run. Choose one:
  • Committed signals — Commit .geval/signals.json (or signals/signals.json). In the workflow, skip any “generate” step and run: ./geval check --contract .geval/contract.yaml --signals .geval/signals.json
  • Generated signals — Add a step that writes signals.json (e.g. python scripts/export_from_langsmith.py > signals.json). Your script is your integration; Geval only needs valid JSON matching Signals and rules.
If you use LangSmith (or anything else), only the script that builds JSON changes—the geval check line stays the same.
Let the job fail when Geval returns BLOCK (exit 2). For REQUIRE_APPROVAL (exit 1), the job also fails unless you add custom logic (below). 
name: Geval quality gate

on:
  pull_request:
    branches: [main]

jobs:
  geval:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Install Geval
        run: |
          curl -fsSL https://github.com/geval-labs/geval/releases/latest/download/geval-linux-x86_64 -o geval
          chmod +x geval

      # Pick ONE of the next two steps:

      # A) Signals already in the repo — no generation
      # (remove this block if you use B)

      # B) Generate signals in CI (LangSmith, custom API, etc.)
      - name: Generate signals
        run: python .github/scripts/generate_signals.py > signals.json
        env:
          LANGSMITH_API_KEY: ${{ secrets.LANGSMITH_API_KEY }}

      - name: Validate contract
        run: ./geval validate-contract .geval/contract.yaml

      - name: Run Geval
        run: |
          ./geval check \
            --contract .geval/contract.yaml \
            --signals signals.json
If you use committed signals, replace the generate step with nothing and set --signals to the committed path, e.g. --signals .geval/signals.json. Exit codes: 0 = PASS, 1 = REQUIRE_APPROVAL, 2 = BLOCK. See Exit codes.

​
Optional: REQUIRE_APPROVAL does not fail the job (custom policy)

If you want BLOCK to fail CI but REQUIRE_APPROVAL to pass the job (so humans decide outside CI), you must capture the exit code in bash. GitHub’s default bash uses errexit (-e): if ./geval check exits non‑zero, the script stops and a following echo "exitcode=$?" never runs—so do not rely on that pattern. Correct pattern: 
- name: Run Geval
  id: geval
  run: |
    set +e
    ./geval check --contract .geval/contract.yaml --signals signals.json
    code=$?
    set -e
    echo "exitcode=$code" >> "$GITHUB_OUTPUT"
    exit 0

- name: Enforce decision
  run: |
    case "${{ steps.geval.outputs.exitcode }}" in
      0) echo "PASS" ;;
      1) echo "REQUIRE_APPROVAL — resolve in review process";;
      2) echo "BLOCK — failing job"; exit 1 ;;
      *) echo "Unexpected exit code"; exit 1 ;;
    esac
Do not combine continue-on-error: true on the same step with echo "exitcode=$?" without set +e—the step will exit before echo runs.

​
Handling REQUIRE_APPROVAL: extra logic

Exit code 1 means Geval’s merged outcome is REQUIRE_APPROVAL (a human or process should review before you treat the change as fully cleared). GitHub does not have a built-in “REQUIRE_APPROVAL” status—you choose what happens: fail the check, pass the check but label the PR, notify Slack, gate an Environment, or record an approval file with geval approve.
ExitMeaningTypical automation
0PASSNo extra action; merge allowed per your rules
1REQUIRE_APPROVALComment, label, notify, or require GitHub Environment reviewers
2BLOCKFail the job; do not merge until policies pass

​
1) Single job: fail only on BLOCK, label PR on REQUIRE_APPROVAL

Run check with set +e, map the code to a decision output, fail the step only when 2. A second step adds a label only when decision == require_approval. 
permissions:
  contents: read
  pull-requests: write

jobs:
  geval:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Install Geval
        run: |
          curl -fsSL https://github.com/geval-labs/geval/releases/latest/download/geval-linux-x86_64 -o geval
          chmod +x geval

      - name: Generate signals
        run: python .github/scripts/generate_signals.py > signals.json

      - name: Run Geval and classify
        id: geval
        run: |
          set +e
          ./geval check --contract .geval/contract.yaml --signals signals.json
          code=$?
          set -e
          echo "exitcode=$code" >> "$GITHUB_OUTPUT"
          case "$code" in
            0) echo "decision=pass" >> "$GITHUB_OUTPUT" ;;
            1) echo "decision=require_approval" >> "$GITHUB_OUTPUT" ;;
            2) echo "decision=block" >> "$GITHUB_OUTPUT" ;;
            *) echo "decision=error" >> "$GITHUB_OUTPUT" ;;
          esac
          if [ "$code" -eq 2 ]; then
            echo "BLOCK — failing job"
            exit 1
          fi
          if [ "$code" -ne 0 ] && [ "$code" -ne 1 ]; then
            echo "Unexpected exit $code"
            exit 1
          fi
          exit 0

      - name: Explain for comment
        if: github.event_name == 'pull_request' && steps.geval.outputs.decision != 'pass'
        run: |
          ./geval explain --contract .geval/contract.yaml --signals signals.json > geval-report.txt

      - name: Label PR — needs Geval review
        if: github.event_name == 'pull_request' && steps.geval.outputs.decision == 'require_approval'
        env:
          GH_TOKEN: ${{ github.token }}
        run: |
          gh pr edit "${{ github.event.pull_request.number }}" --add-label "geval:needs-approval"
          gh pr comment "${{ github.event.pull_request.number }}" --body-file geval-report.txt
Create the label geval:needs-approval once in the repo (or use gh label create). You can require that label to be removed before merge via a separate process, or use branch rules that require another check.

​
2) Follow-up job using needs (Slack, email, etc.)

Expose a job output so a second job runs only when the decision is require_approval. 
jobs:
  geval:
    runs-on: ubuntu-latest
    outputs:
      decision: ${{ steps.geval.outputs.decision }}
    steps:
      - uses: actions/checkout@v4
      - name: Install Geval
        run: |
          curl -fsSL https://github.com/geval-labs/geval/releases/latest/download/geval-linux-x86_64 -o geval
          chmod +x geval
      - name: Signals + check
        id: geval
        run: |
          python .github/scripts/generate_signals.py > signals.json
          set +e
          ./geval check --contract .geval/contract.yaml --signals signals.json
          code=$?
          set -e
          case "$code" in
            0) echo "decision=pass" >> "$GITHUB_OUTPUT" ;;
            1) echo "decision=require_approval" >> "$GITHUB_OUTPUT" ;;
            2) echo "decision=block" >> "$GITHUB_OUTPUT" ;;
            *) echo "decision=error" >> "$GITHUB_OUTPUT"; exit 1 ;;
          esac
          if [ "$code" -eq 2 ]; then exit 1; fi
          exit 0

  notify-approval:
    needs: geval
    if: needs.geval.outputs.decision == 'require_approval'
    runs-on: ubuntu-latest
    steps:
      - name: Notify reviewers
        run: echo "Add Slack/Teams/PagerDuty step here — PR needs human approval per Geval policy"
Replace the echo step with slackapi/slack-github-action, email, or your internal webhook.

​
3) GitHub Environments (manual approvers)

For deployment pipelines, attach a job to an Environment that has Required reviewers in repository settings. That job can run after Geval reports require_approval, or you can treat Geval BLOCK as “do not deploy” and use Environment approval only for production deploys—not a substitute for encoding Geval’s REQUIRE_APPROVAL in YAML, but a common combo.

​
4) Recording approval with geval approve

After a human agrees, someone can run locally or in a trusted workflow: 
geval approve --reason "Reviewed metrics and policy exceptions" --output .geval/approval.json
Commit approval.json or upload it as an artifact if your org’s policy expects an audit trail. geval reject writes a parallel record. See approve & reject. Wiring “merge only if approval file exists” is your policy layer (branch rules, required check, or a second workflow).

​
5) Failing CI on REQUIRE_APPROVAL (strictest)

If 1 and 2 should both fail the PR check, run geval check with no special handling—the default shell will exit non‑zero for both. No extra logic needed.

​
Optional: post geval explain on the PR

Use the GitHub CLI (needs pull-requests: write and GH_TOKEN): 
permissions:
  contents: read
  pull-requests: write

jobs:
  geval:
    steps:
      # ... checkout, install geval, signals, check ...
      - name: Comment on PR
        if: github.event_name == 'pull_request'
        env:
          GH_TOKEN: ${{ github.token }}
        run: |
          ./geval explain --contract .geval/contract.yaml --signals signals.json > geval-report.txt
          gh pr comment "${{ github.event.pull_request.number }}" --body-file geval-report.txt
Run explain after you know signals.json exists (same paths as check). If you only want a comment when check fails, wrap in if: conditions.

​
Multiple contracts

./geval check \
  -c .geval/contract.yaml \
  -c other-team/contract.yaml \
  --signals signals.json
Optional --combine-contracts (default worst_case): check.

​
Artifacts

geval check writes decision JSON under .geval/decisions/. Upload that folder as a workflow artifact if you want audit history in CI. See Decision artifacts.

​
See also

Exit codes · approve & reject · GitLab CI · Installation · Troubleshooting