ProviderConcurrencyLimiter
AgentPrism.Core.dllLimits concurrent outgoing calls per model provider.
public sealed class ProviderConcurrencyLimiterInheritance
Section titled “Inheritance”object ← ProviderConcurrencyLimiter
Inherited Members
Section titled “Inherited Members”object.GetType(), object.ToString(), object.Equals(object?), object.Equals(object?, object?), object.ReferenceEquals(object?, object?), object.GetHashCode()
Remarks
Section titled “Remarks”The goal is to avoid producing a burst of 429 responses; the
circuit breaker only reacts once a provider is already failing. A request
over the limit waits for a slot instead of being rejected
— rejecting outright would turn a short traffic spike into exactly the
failure this option exists to prevent. The wait is bounded by the
caller’s own cancellation token, so it can never hang a request forever.
AgentPrismModelConcurrencyOptions.MaxConcurrentCallsPerProvider
is read via IOptionsMonitor.CurrentValue on every
call. When it is null — the default — ProviderConcurrencyLimiter.AcquireAsync
returns immediately with no semaphore allocated and no wait: the hot path
costs a single null check. Once a provider’s first limited call creates its
Threading.SemaphoreSlim, that provider’s numeric limit is fixed for the
semaphore’s lifetime — Threading.SemaphoreSlim cannot resize; a runtime
change to the configured number takes effect only for a provider that has
not yet been limited.
Constructors
Section titled “Constructors”ProviderConcurrencyLimiter(IOptionsMonitor<AgentPrismOptions>)
Section titled “ ProviderConcurrencyLimiter(IOptionsMonitor<AgentPrismOptions>)”Creates a new limiter.
public ProviderConcurrencyLimiter(IOptionsMonitor<AgentPrismOptions> optionsMonitor)Parameters
Section titled “Parameters”optionsMonitor IOptionsMonitor<AgentPrismOptions>
The runtime settings.
Exceptions
Section titled “Exceptions”optionsMonitor is null.
Methods
Section titled “Methods”AcquireAsync(string, CancellationToken)
Section titled “ AcquireAsync(string, CancellationToken)”Reserves an outgoing call slot for the given provider, waiting if the limit is currently reached.
public ValueTask<IDisposable?> AcquireAsync(string providerName, CancellationToken cancellationToken)Parameters
Section titled “Parameters”providerName string
The provider name. The limit is tracked per this name.
cancellationToken CancellationToken
The cancellation token. Canceling while waiting releases no slot (none was reserved) and throws OperationCanceledException.
Returns
Section titled “Returns”The reserved slot, or null when no limit is configured. Disposing the returned object releases the slot.
Exceptions
Section titled “Exceptions”providerName is empty.