游戏 AI / 系统课时
一次异常对话如何完整回放
一次异常对话如何完整回放
当 NPC 出现重复对白、记错玩家行为或突然改变立场时,最有效的排查方式不是重新猜 prompt,而是把异常请求按时间和版本完整重建出来。本课建立一条从请求日志到最终状态的可回放路径。
学习目标
- 能按 requestId 串起请求、模型输入、检索结果与状态版本
- 能区分“生成异常”和“上下文污染”两类根因
- 能通过最小回放脚本复现一次异常剧情
先确定一次回放需要什么
证据关联使用服务端分配的请求标识、尝试序号和授权范围。记录输入状态版本、检索结果引用、人格与提示版本、模型与解码配置、解析/规则结论和提交回执。跨 HTTP 服务可用 traceparent 传播链路上下文,trace ID 不承担授权职责。W3C Trace Context

例行日志优先保留版本、受控原因与证据引用。确需原始输入、输出或完整快照时,应由授权诊断人员通过受控存储读取,限定范围、保留期和用途。回放可用同一份模型输出复算解析与规则,重新请求模型则是一场新实验,可能产生不同结果。OWASP 日志指南
按 requestId 重建时间线
先定位可信请求和对应尝试。下面用阶段序号表达演示链路的顺序,读文件时可处理乱序行。真实分布式环境中,用父子调用、队列关联、数据库提交序号等因果证据;不同主机的墙钟时间不能单独证明先后。
保存以下合成证据为 events.jsonl。第一轮缺少后续证据。第二轮的移动目标不在许可范围内,日志中的规则拒绝与没有提交应一致。
{"scope":"save-01/main/p42/guard","request":"req-1","attempt":2,"seq":4,"stage":"candidate","target":[40,99]}
{"scope":"save-01/main/p42/guard","request":"req-1","attempt":2,"seq":1,"stage":"snapshot","rule":"move-v1","version":7,"allowed":[[40,75]]}
{"scope":"save-01/main/p42/guard","request":"req-1","attempt":1,"seq":1,"stage":"snapshot","rule":"move-v1","version":7,"allowed":[[40,75]]}
{"scope":"save-01/main/p42/guard","request":"req-1","attempt":2,"seq":3,"stage":"model_input","version":7,"prompt":"gate-v3","memory_ids":["m-1"]}
{"scope":"save-01/main/p42/guard","request":"req-1","attempt":2,"seq":2,"stage":"retrieval","memory_ids":["m-1","m-2"]}
{"scope":"save-01/main/p42/guard","request":"req-1","attempt":2,"seq":5,"stage":"validation","outcome":"rule_denied"}
{"scope":"save-01/main/p42/guard","request":"req-1","attempt":2,"seq":6,"stage":"commit","outcome":"not_attempted"}保存以下代码为 replay_request.py。它只复算已知版本的移动规则并对照记录,无模型调用、重投递或状态写入。合成日志中的 scope 是不透明演示标签,生产使用结构化作用域或经授权的内部引用。
import json
import sys
from collections import defaultdict
from pathlib import Path
STAGES = ("snapshot", "retrieval", "model_input", "candidate", "validation", "commit")
def inspect_attempt(rows):
by_stage = {row.get("stage"): row for row in rows}
if len(by_stage) != len(rows) or any(stage not in STAGES for stage in by_stage):
return "inconsistent_stages"
if any(type(row.get("seq")) is not int or row["seq"] != STAGES.index(stage) + 1
for stage, row in by_stage.items()):
return "inconsistent_stages"
missing = [stage for stage in STAGES if stage not in by_stage]
if missing:
return "incomplete: " + ",".join(missing)
state = by_stage["snapshot"]
supplied = by_stage["model_input"]
if state.get("rule") != "move-v1":
return "unsupported_rule_version"
if type(state.get("version")) is not int or state["version"] < 0:
return "invalid_snapshot"
if type(supplied.get("version")) is not int or supplied["version"] != state["version"]:
return "stale_context"
sent = supplied.get("memory_ids")
recalled = by_stage["retrieval"].get("memory_ids")
if any(not isinstance(ids, list) or any(not isinstance(key, str) for key in ids)
for ids in (sent, recalled)):
return "invalid_memory_evidence"
if not set(sent) <= set(recalled):
return "unexplained_memory"
target = by_stage["candidate"].get("target")
tiles = state.get("allowed")
if (not isinstance(tiles, list) or not isinstance(target, list)
or len(target) != 2 or any(type(value) is not int for value in target)):
return "invalid_action_evidence"
allowed = target in tiles
outcome = "candidate_ready" if allowed else "rule_denied"
if outcome != by_stage["validation"].get("outcome"):
return "validation_mismatch"
receipt = by_stage["commit"]
if not allowed:
return "replayed: rule_denied" if receipt.get("outcome") == "not_attempted" else "unexpected_commit"
if receipt.get("outcome") == "stale":
return "replayed: candidate_ready, stale_at_commit"
if (receipt.get("outcome") != "committed" or type(receipt.get("version")) is not int
or receipt["version"] != state["version"] + 1):
return "missing_valid_commit"
return "replayed: candidate_ready, commit_evidence_matched"
def replay(records, scope, request):
attempts = defaultdict(list)
for row in records:
if row["scope"] == scope and row["request"] == request:
attempts[row["attempt"]].append(row)
return [(number, inspect_attempt(rows)) for number, rows in sorted(attempts.items())]
if __name__ == "__main__":
if len(sys.argv) != 2:
raise SystemExit("usage: python replay_request.py <events.jsonl>")
with Path(sys.argv[1]).open(encoding="utf-8") as source:
rows = (json.loads(line) for line in source if line.strip())
results = replay(rows, "save-01/main/p42/guard", "req-1")
if not results:
raise SystemExit("no evidence for this scope and request")
for attempt, result in results:
print(f"attempt={attempt} {result}")python replay_request.py events.jsonlattempt=1 incomplete: retrieval,model_input,candidate,validation,commit
attempt=2 replayed: rule_denied证据不足时保留“部分回放”结论。完整证据表明模型确实提出了超范围移动,但本次规则拒绝且没有写入。示例限定模型输入里的记忆 ID 来自该次检索。若系统还会直接附加权威状态或会话事实,应分别记录来源。
判断异常到底发生在哪一层
如果 model_input 已经包含错误事实,优先检查记忆检索、状态快照和上下文组装;如果 model_input 正确而 model_output 异常,再检查模型版本、采样配置与输出约束。若输出正确但游戏表现错误,则继续看状态写入和事件消费。
这样排查可以把问题分成“输入污染”“生成异常”“执行异常”三段,而不是笼统归因于幻觉。尤其是重复对白,要比较相邻请求的 memoryIds、contextVersion 与 stateVersion,确认是否持续复用了同一份旧上下文。
回放结果要能被验证
验收时区分缺失、快照不符、模型输出不符、规则不符和意外提交。回放脚本只读诊断证据并复算可确定的规则。需要演练实际执行时,在隔离的合成或临时环境使用新请求标识,避免把生产事件再次投递。