OnlineEvaluationOptions
AgentPrism.Core.dllOnline evaluation settings.
public sealed class OnlineEvaluationOptionsInheritance
Section titled “Inheritance”object ← OnlineEvaluationOptions
Inherited Members
Section titled “Inherited Members”object.GetType(), object.ToString(), object.Equals(object?), object.Equals(object?, object?), object.ReferenceEquals(object?, object?), object.GetHashCode()
Remarks
Section titled “Remarks”Read from the AgentPrism:OnlineEvaluation configuration section.
Two-gate default. OnlineEvaluationOptions.Enabled defaults to
false (the plain no-surprises reading) AND
OnlineEvaluationOptions.SampleRate defaults to 0.0. Even when OnlineEvaluationOptions.Enabled
is turned on, no run is sampled, the judge model is never called, and not
a single cent is spent unless the rate is also given. The third defense is
OnlineEvaluationOptions.MaxScoresPerHour: even if the sample rate is miscalculated,
there is still an upper bound.
Constructors
Section titled “Constructors”OnlineEvaluationOptions()
Section titled “ OnlineEvaluationOptions()”public OnlineEvaluationOptions()Fields
Section titled “Fields”SectionName
Section titled “ SectionName”Configuration section name.
public const string SectionName = "AgentPrism:OnlineEvaluation"Field Value
Section titled “Field Value”Properties
Section titled “Properties”AgentNames
Section titled “ AgentNames”Only these agents are scored. Empty means all.
public IList<string> AgentNames { get; }Property Value
Section titled “Property Value”Enabled
Section titled “ Enabled”Whether online evaluation is enabled. Default false.
public bool Enabled { get; set; }Property Value
Section titled “Property Value”EvaluationWindow
Section titled “ EvaluationWindow”Window for the average calculation.
public TimeSpan EvaluationWindow { get; set; }Property Value
Section titled “Property Value”LowScoreThreshold
Section titled “ LowScoreThreshold”Low-score threshold, on a 0-100 scale.
public int LowScoreThreshold { get; set; }Property Value
Section titled “Property Value”MaxScoresPerHour
Section titled “ MaxScoresPerHour”Maximum number of runs sampled per hour per tenant.
public int MaxScoresPerHour { get; set; }Property Value
Section titled “Property Value”Remarks
Section titled “Remarks”The second line of defense for sampling: even if OnlineEvaluationOptions.SampleRate is miscalculated or traffic spikes, the absolute cost is bounded by this cap.
MinSampleSize
Section titled “ MinSampleSize”Minimum sample size required for an alert. A single low score does not produce an alert.
public int MinSampleSize { get; set; }Property Value
Section titled “Property Value”Remarks
Section titled “Remarks”The model is noisy; raising an alert from a single sample trains the on-call engineer to start ignoring notifications.
SampleRate
Section titled “ SampleRate”Fraction of completed runs to sample, 0.0-1.0.
public double SampleRate { get; set; }Property Value
Section titled “Property Value”Remarks
Section titled “Remarks”Default 0.0: even when OnlineEvaluationOptions.Enabled is turned on,
nothing is scored unless the rate is also given. Sampling itself is
deterministic - it is derived from the hash of the run
id; the same run is never evaluated twice, and a retry does not roll a
new die.