6.3 KiB
W02 error, idempotency, and fencing contract
This document belongs to the current Agent gRPC contract. The package name is
not a protocol version; RequestMeta.protocol_version remains meaningful. It
does not change the SaaS/MQ contract.
Transport and domain errors
Handlers return normal gRPC status codes and, when a response message exists,
put the stable FailureCode in Failure.code as well:
FailureCode |
gRPC status | Retry rule |
|---|---|---|
INVALID_ARGUMENT |
InvalidArgument |
Fix the request; never retry unchanged |
UNAUTHENTICATED |
Unauthenticated |
Re-establish mTLS/session; do not replay business work |
PERMISSION_DENIED |
PermissionDenied |
Stop; require a new authorization |
FAILED_PRECONDITION |
FailedPrecondition |
Refresh state/barrier, then use the original operation ID only if allowed |
ABORTED |
Aborted |
Re-read the CAS revision; do not assume the operation applied |
RESOURCE_EXHAUSTED |
ResourceExhausted |
Wait for durable quota/resource release |
UNAVAILABLE |
Unavailable |
Reconnect and reconcile the original operation before any retry |
DEADLINE_EXCEEDED |
DeadlineExceeded |
Result is unknown unless a durable receipt exists |
NOT_FOUND |
NotFound |
Do not create a substitute execution |
ALREADY_EXISTS |
AlreadyExists |
Read the existing operation/result; do not create a second one |
A transport success is not an application receipt. RESULT_CODE_ACCEPTED means
that the receiving side durably recorded the request; APPLIED requires the
specified state transition and barrier evidence.
Idempotency keys
For execution and other durably deduplicated mutations,
RequestMeta.idempotency_key is scoped by
(agent_id, operation_id, idempotency_key) and the durable operation record
also stores the request content digest. The current task-level
ApplyApprovedTaskControl is an exception: it may omit the key, has no control
command ID or revision CAS, and processes every delivery. Reusing a key with different
content returns ABORTED/RESULT_CODE_CONFLICT; it never overwrites the first
request.
ActivateAgent:activation_operation_idis the idempotency key. A repeated identical request returns the same session generation and credential metadata.SetAdmissionState:barrier_idplus the expected admission generation is persisted. A replay cannot move the generation twice.Execute: the execution binding and permit ID are the deduplication identity. An unknown result is reconciled withQueryExecution; it is never retried as a new originate.ExecuteAuthorized(agent-authorized-origination.v0.1, isolated Mock only): the Dispatcher first persists oneissuedorrefuseddecision per execution. The Agent durably recordsUNKNOWNbefore invoking the mock adapter; identical operation replays return the stored receipt, changed content or a second operation for the same execution is a conflict. A lost reply or mock adapter failure requiresQueryExecution, never another originate or a new execution ID. An expired Dispatcher-issued deadline cannot invoke the adapter.GetExecutionPermit: the reservation, binding, expected revision and idempotency key are persisted. A permit is not issued after the reservation is released or fenced.ApplyTaskControl:(execution_id, expected_task_revision, action, idempotency_key)is CAS-checked.STOPis terminal;PAUSEmay be resumed only by a new authorized request.ApplyApprovedTaskControl: the active Dispatcher session and numeric tenant/task identity are checked. It closes local admission before hangup or drain and succeeds only after registered active calls end. The main approved-call runner must register those calls before live use. A timeout keeps the barrier closed; explicit redelivery applies again, with no control dedup.ReportExecutionEvent:(fact_id, content_sha256)is the durable fact key. An identical duplicate returns the original receipt; a digest mismatch is a conflict.RequestUpload/CompleteUpload:upload_idand the asset checksum are retained. Completion is returned only after therecording.uploadedfact is durably queued; a local PUT success alone is notrecording.uploaded.
Read-only methods may be retried, but callers still preserve the original
request and trace identity: GetAgentStatus, GetBootstrap, and
QueryExecution.
Fencing and session rules
- The Dispatcher creates a new opaque
dispatcher_epochwhen its active instance changes. The Agent accepts mutations only for the active epoch. - The Agent identity is the mTLS certificate plus the Dispatcher-approved
(agent_id, cell_id, boot_id, session_generation)binding. Self-reported identity or endpoint values are not authorization. - A newer boot or session generation fences older requests. The old request
returns
UNAUTHENTICATEDorABORTEDand cannot release an unknown lease. - The existing
Executepermit contains the dispatcher epoch, session generation, reservation andfencing_token; the Agent checks them before that legacy path. The separate Mock-onlyExecuteAuthorizedpath requires an active session and the Dispatcher-issued exclusive deadline, without a second business-policy calculation or an implicit permit/fallback. Its mixed/real execution is disabled pending separate authorization. - Admission close/drain is a prerequisite barrier.
SetAdmissionStateand the execution-boundApplyTaskControlrequire their expected generation/revision. The task-levelApplyApprovedTaskControlinstead checks the active Dispatcher session and keeps its local stop irreversible; Dispatcher durable task state remains authoritative. - When the result of a mutation is unknown, the caller first queries the
original operation/execution and records
UNKNOWNif evidence is absent. No new execution ID or new permit is invented for recovery.
Upload boundary
The Agent receives a restricted UploadGrant and uploads directly to the
approved OSS target. The Dispatcher never receives audio bytes. The Agent
reports only asset metadata/checksum through CompleteUpload; the Dispatcher
reliably queues the original recording.uploaded fact in MQ. This project does
not request a SaaS upload session, wait for verified, or invent an OSS ID;
MQ publisher confirmation is not SaaS application receipt.