IRunJudge
AgentPrism.Abstractions.dllThe extension point that scores a completed production run.
public interface IRunJudgeRemarks
Section titled “Remarks”This interface does not wrap MAF’s AIJudgeLoopEvaluator.
Measured (MAF 1.16.0): LoopEvaluation does not return a score (only
ShouldReinvoke and Feedback), and LoopContext requires
a live AIAgent + AgentSession. It is not suited to scoring a
finished run —
Registered with TryAddEnumerable; multiple judges may score the same run.
If more than one IRunJudge is registered in a setup, the
online evaluation job runs all of them; each writes its own
RunScore row with Source = judge:{Name}.
Properties
Section titled “Properties”The judge’s name. Written into the RunScore.Source field as judge:{Name}.
string Name { get; }Property Value
Section titled “Property Value”Methods
Section titled “Methods”JudgeAsync(RunJudgeContext, CancellationToken)
Section titled “ JudgeAsync(RunJudgeContext, CancellationToken)”Scores the run.
ValueTask<RunJudgment> JudgeAsync(RunJudgeContext context, CancellationToken cancellationToken = default)Parameters
Section titled “Parameters”context RunJudgeContext
The context the judge sees.
cancellationToken CancellationToken
The cancellation token.
Returns
Section titled “Returns”The judge’s verdict.