Zero-instrumentation waste auditor for hand-rolled AI agent loops.
Frameworks can hide the cost and inefficiency of an agent loop.
loopcheck works from a plain JSON trace and answers:
- Are tools being called unnecessarily?
- Is the agent thrashing or retrying unchanged work?
- Did a prompt/model/tool change make the loop worse?
- Where is wasted cost accumulating?
It is the behavior module inside aisec-suite.
| Command | Purpose |
|---|---|
analyze |
Duplicate calls, loop thrashing, stale retries, wasted-cost estimates |
compare |
Regression detection between two runs |
report |
Aggregate waste across a trace directory |
--policy |
Suppress reviewed findings |
TraceRecorder |
Capture traces from OpenAI, Anthropic, or Groq-style loops |
loopcheck analyze trace.json
loopcheck analyze trace.json --json report.json --fail-on medium
loopcheck compare before.json after.json --fail-on-regression
loopcheck report traces/ --json rollup.jsonaisec behavior trace.json --json behavior-findings.jsonThe umbrella adapter normalizes specialist findings into the shared suite format.
Semantic redundancy is opt-in:
GROQ_API_KEY=... loopcheck analyze trace.json --judge groqDeterministic checks do not require a model or API key.
Cost estimates depend on trace token and pricing data. The semantic judge is bounded and model-dependent.
pip install -e ".[dev]"
pytest -qMIT