Skip to main content

​
What is Geval?

Geval turns evaluation results into automated pass/fail decisions in CI/CD pipelines. It’s a lightweight, framework-agnostic tool that enforces quality contracts on your AI applications. Geval is:
  • ✅ Format-agnostic - Works with CSV, JSON, JSONL from any eval tool
  • ✅ Contract-based - Define quality requirements as code
  • ✅ CI-native - Exit codes and JSON output for automation
  • ✅ Framework-agnostic - Works with Promptfoo, LangSmith, OpenEvals, or custom tools
Geval is not:
  • ❌ An eval runner (it consumes eval outputs)
  • ❌ A monitoring tool (it’s for release gates)
  • ❌ A testing framework (it validates existing results)

Quick Start

Get up and running with Geval in minutes. Install, create your first contract, and run your first check.

​
Key Features

Multi-Format Support

Works with CSV, JSON, and JSONL files from any eval tool. No vendor lock-in.

CI/CD Integration

Exit codes and JSON output for seamless integration with any CI/CD system.

Baseline Comparisons

Compare against fixed thresholds, previous runs, or main branch baselines.

Contract-Based

Define quality requirements as code. Version it. Review it. Commit it.

​
How It Works

Your Evals → Geval Contract → CI Pass/Block Decision
  1. Define - Write quality contracts in YAML
  2. Integrate - Add to your CI/CD pipeline
  3. Enforce - Block bad releases automatically

View Examples

See real-world examples for performance monitoring, safety checks, and more.