游戏 AI / 系统课时

一次异常对话如何完整回放

一次异常对话如何完整回放

当 NPC 出现重复对白、记错玩家行为或突然改变立场时,最有效的排查方式不是重新猜 prompt,而是把异常请求按时间和版本完整重建出来。本课建立一条从请求日志到最终状态的可回放路径。

学习目标

  • 能按 requestId 串起请求、模型输入、检索结果与状态版本
  • 能区分“生成异常”和“上下文污染”两类根因
  • 能通过最小回放脚本复现一次异常剧情

先确定一次回放需要什么

证据关联使用服务端分配的请求标识、尝试序号和授权范围。记录输入状态版本、检索结果引用、人格与提示版本、模型与解码配置、解析/规则结论和提交回执。跨 HTTP 服务可用 traceparent 传播链路上下文,trace ID 不承担授权职责。W3C Trace Context

异常对话的完整回放链路

例行日志优先保留版本、受控原因与证据引用。确需原始输入、输出或完整快照时,应由授权诊断人员通过受控存储读取,限定范围、保留期和用途。回放可用同一份模型输出复算解析与规则,重新请求模型则是一场新实验,可能产生不同结果。OWASP 日志指南

按 requestId 重建时间线

先定位可信请求和对应尝试。下面用阶段序号表达演示链路的顺序,读文件时可处理乱序行。真实分布式环境中,用父子调用、队列关联、数据库提交序号等因果证据;不同主机的墙钟时间不能单独证明先后。

保存以下合成证据为 events.jsonl。第一轮缺少后续证据。第二轮的移动目标不在许可范围内,日志中的规则拒绝与没有提交应一致。

jsonl
{"scope":"save-01/main/p42/guard","request":"req-1","attempt":2,"seq":4,"stage":"candidate","target":[40,99]}
{"scope":"save-01/main/p42/guard","request":"req-1","attempt":2,"seq":1,"stage":"snapshot","rule":"move-v1","version":7,"allowed":[[40,75]]}
{"scope":"save-01/main/p42/guard","request":"req-1","attempt":1,"seq":1,"stage":"snapshot","rule":"move-v1","version":7,"allowed":[[40,75]]}
{"scope":"save-01/main/p42/guard","request":"req-1","attempt":2,"seq":3,"stage":"model_input","version":7,"prompt":"gate-v3","memory_ids":["m-1"]}
{"scope":"save-01/main/p42/guard","request":"req-1","attempt":2,"seq":2,"stage":"retrieval","memory_ids":["m-1","m-2"]}
{"scope":"save-01/main/p42/guard","request":"req-1","attempt":2,"seq":5,"stage":"validation","outcome":"rule_denied"}
{"scope":"save-01/main/p42/guard","request":"req-1","attempt":2,"seq":6,"stage":"commit","outcome":"not_attempted"}

保存以下代码为 replay_request.py。它只复算已知版本的移动规则并对照记录,无模型调用、重投递或状态写入。合成日志中的 scope 是不透明演示标签,生产使用结构化作用域或经授权的内部引用。

python
import json
import sys
from collections import defaultdict
from pathlib import Path

STAGES = ("snapshot", "retrieval", "model_input", "candidate", "validation", "commit")


def inspect_attempt(rows):
    by_stage = {row.get("stage"): row for row in rows}
    if len(by_stage) != len(rows) or any(stage not in STAGES for stage in by_stage):
        return "inconsistent_stages"
    if any(type(row.get("seq")) is not int or row["seq"] != STAGES.index(stage) + 1
           for stage, row in by_stage.items()):
        return "inconsistent_stages"
    missing = [stage for stage in STAGES if stage not in by_stage]
    if missing:
        return "incomplete: " + ",".join(missing)

    state = by_stage["snapshot"]
    supplied = by_stage["model_input"]
    if state.get("rule") != "move-v1":
        return "unsupported_rule_version"
    if type(state.get("version")) is not int or state["version"] < 0:
        return "invalid_snapshot"
    if type(supplied.get("version")) is not int or supplied["version"] != state["version"]:
        return "stale_context"
    sent = supplied.get("memory_ids")
    recalled = by_stage["retrieval"].get("memory_ids")
    if any(not isinstance(ids, list) or any(not isinstance(key, str) for key in ids)
           for ids in (sent, recalled)):
        return "invalid_memory_evidence"
    if not set(sent) <= set(recalled):
        return "unexplained_memory"
    target = by_stage["candidate"].get("target")
    tiles = state.get("allowed")
    if (not isinstance(tiles, list) or not isinstance(target, list)
            or len(target) != 2 or any(type(value) is not int for value in target)):
        return "invalid_action_evidence"
    allowed = target in tiles
    outcome = "candidate_ready" if allowed else "rule_denied"
    if outcome != by_stage["validation"].get("outcome"):
        return "validation_mismatch"
    receipt = by_stage["commit"]
    if not allowed:
        return "replayed: rule_denied" if receipt.get("outcome") == "not_attempted" else "unexpected_commit"
    if receipt.get("outcome") == "stale":
        return "replayed: candidate_ready, stale_at_commit"
    if (receipt.get("outcome") != "committed" or type(receipt.get("version")) is not int
            or receipt["version"] != state["version"] + 1):
        return "missing_valid_commit"
    return "replayed: candidate_ready, commit_evidence_matched"


def replay(records, scope, request):
    attempts = defaultdict(list)
    for row in records:
        if row["scope"] == scope and row["request"] == request:
            attempts[row["attempt"]].append(row)
    return [(number, inspect_attempt(rows)) for number, rows in sorted(attempts.items())]


if __name__ == "__main__":
    if len(sys.argv) != 2:
        raise SystemExit("usage: python replay_request.py <events.jsonl>")
    with Path(sys.argv[1]).open(encoding="utf-8") as source:
        rows = (json.loads(line) for line in source if line.strip())
        results = replay(rows, "save-01/main/p42/guard", "req-1")
    if not results:
        raise SystemExit("no evidence for this scope and request")
    for attempt, result in results:
        print(f"attempt={attempt} {result}")
bash
python replay_request.py events.jsonl
text
attempt=1 incomplete: retrieval,model_input,candidate,validation,commit
attempt=2 replayed: rule_denied

证据不足时保留“部分回放”结论。完整证据表明模型确实提出了超范围移动,但本次规则拒绝且没有写入。示例限定模型输入里的记忆 ID 来自该次检索。若系统还会直接附加权威状态或会话事实,应分别记录来源。

判断异常到底发生在哪一层

如果 model_input 已经包含错误事实,优先检查记忆检索、状态快照和上下文组装;如果 model_input 正确而 model_output 异常,再检查模型版本、采样配置与输出约束。若输出正确但游戏表现错误,则继续看状态写入和事件消费。

这样排查可以把问题分成“输入污染”“生成异常”“执行异常”三段,而不是笼统归因于幻觉。尤其是重复对白,要比较相邻请求的 memoryIds、contextVersion 与 stateVersion,确认是否持续复用了同一份旧上下文。

回放结果要能被验证

验收时区分缺失、快照不符、模型输出不符、规则不符和意外提交。回放脚本只读诊断证据并复算可确定的规则。需要演练实际执行时,在隔离的合成或临时环境使用新请求标识,避免把生产事件再次投递。

按自己的节奏,确认真正理解。

目录与阅读设置

阅读设置

17

记录一个想法

一次异常对话如何完整回放

0 / 10000 · 笔记保存到账号。未提交的修改会在关闭、离开或刷新后丢失。

登录哪都通