feat: improve skill import and runtime delivery
This commit is contained in:
lofyer
@@ -0,0 +1,88 @@
|
||||
---
|
||||
name: product-evidence
|
||||
version: 1.0.0
|
||||
description: |
|
||||
建立和维护产品市场材料的事实与证据清单，统一产品版本、功能状态、技术参数、
|
||||
术语、适用边界和可公开主张。用于开始任何产品介绍、投标参数、PPT、技术方案、
|
||||
白皮书或案例材料之前，也用于跨产物一致性核验。
|
||||
allowed-tools:
|
||||
- Read
|
||||
- Grep
|
||||
- Glob
|
||||
- Execute
|
||||
compatibility: Python 3.9+，校验脚本不依赖第三方包
|
||||
---
|
||||
|
||||
# 产品事实与证据
|
||||
|
||||
所有产品市场产物都必须从同一份 `product-evidence.json` 取事实。缺少依据时标记
|
||||
待核验，不允许由 Agent 补造参数、客户结果、认证、兼容性或竞争结论。
|
||||
|
||||
`<skill-dir>` 指本 `SKILL.md` 所在目录。
|
||||
|
||||
## 建立清单
|
||||
|
||||
```bash
|
||||
cp "<skill-dir>/templates/product-evidence.example.json" ./product-evidence.json
|
||||
```
|
||||
|
||||
逐项填写：
|
||||
|
||||
- `product`：产品名称、版本、类别、定位、成熟度和目标读者。
|
||||
- `terminology`：统一术语、定义和禁用旧称。
|
||||
- `features`：功能 ID、用户动作、结果、状态、版本范围和证据。
|
||||
- `parameters`：参数值、单位、测试条件、适用版本、公开级别和证据。
|
||||
- `claims`：允许对外使用的事实、目标或比较主张及适用产物。
|
||||
- `use_cases`：角色、问题、工作流、人工复核点和已有结果。
|
||||
- `differentiators`：比较对象、比较范围和支持证据。
|
||||
- `limitations`：部署条件、依赖、适用边界和必要人工复核。
|
||||
- `evidence`：来源、定位、核验日期、责任人和公开级别。
|
||||
- `prohibited_claims`：不得出现在任何材料中的绝对化或未经批准表述。
|
||||
|
||||
## 事实分级
|
||||
|
||||
- `released`：当前版本已提供，必须有可定位证据。
|
||||
- `beta`：可试用但存在范围限制，正文必须同时说明限制。
|
||||
- `planned`：仅可用将来时或规划表述，不得写成现有能力。
|
||||
- `deprecated`：不得作为当前卖点。
|
||||
|
||||
证据公开级别：
|
||||
|
||||
- `public`：可进入公开网站、彩页和公开演示。
|
||||
- `restricted`：只在授权客户或受控投标材料中使用。
|
||||
- `internal`：只用于内部判断，不能原样写入外发产物。
|
||||
|
||||
## 校验
|
||||
|
||||
先探测可用的 Python 3 解释器：Windows 优先使用 `python`，macOS/Linux
|
||||
优先使用 `python3`。下文 `<python>` 表示探测成功的解释器命令。
|
||||
|
||||
```bash
|
||||
<python> "<skill-dir>/scripts/validate_evidence.py" ./product-evidence.json
|
||||
<python> "<skill-dir>/scripts/validate_evidence.py" ./product-evidence.json --json
|
||||
<python> "<skill-dir>/scripts/validate_evidence.py" ./product-evidence.json \
|
||||
--strict --channel public
|
||||
```
|
||||
|
||||
结构错误、重复 ID、失效引用、无证据的已批准事实主张、非法状态或疑似密钥参数
|
||||
必须阻断。缺证据、占位符和待核验项在普通模式下告警，在 `--strict` 下阻断。
|
||||
|
||||
`--channel` 按目标渠道核对公开级别：已批准且带 `allowed_outputs` 的主张，其
|
||||
引用证据的 `disclosure` 不得低于渠道要求。对外产物必须以 `--strict` 加目标
|
||||
渠道运行通过后才能进入下游技能。
|
||||
|
||||
## 给下游技能的输入
|
||||
|
||||
调用任何产品产物技能时，同时提供：
|
||||
|
||||
1. 已通过校验的 `product-evidence.json`。
|
||||
2. 目标受众、使用场景、发布渠道和保密级别。
|
||||
3. 本次产物允许引用的证据范围。
|
||||
4. 截止日期、页数或篇幅、格式和品牌要求。
|
||||
|
||||
下游产物中的每个数字、兼容性、认证、客户效果和比较结论都应能追溯到清单 ID。
|
||||
|
||||
## 完成标准
|
||||
|
||||
清单结构校验通过；公开级别与使用渠道匹配；所有现有功能、参数和批准主张有
|
||||
证据；规划能力、限制和人工复核要求没有被省略；不含密钥、客户隐私或私有地址。
|
||||
@@ -0,0 +1,431 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Validate the shared product evidence manifest used by marketing skills."""
|
||||
|
||||
import argparse
|
||||
import datetime as dt
|
||||
import json
|
||||
import re
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
COLLECTIONS = (
|
||||
"features",
|
||||
"parameters",
|
||||
"claims",
|
||||
"use_cases",
|
||||
"differentiators",
|
||||
"limitations",
|
||||
"evidence",
|
||||
)
|
||||
REQUIRED_FIELDS = {
|
||||
"features": ("id", "name", "summary", "status"),
|
||||
"parameters": ("id", "name", "value", "conditions", "disclosure"),
|
||||
"claims": ("id", "text", "type", "status"),
|
||||
"use_cases": ("id", "name", "audience", "problem", "workflow", "outcome"),
|
||||
"differentiators": ("id", "statement", "comparison_scope"),
|
||||
"limitations": ("id", "text", "applies_to"),
|
||||
"evidence": (
|
||||
"id",
|
||||
"type",
|
||||
"title",
|
||||
"source",
|
||||
"locator",
|
||||
"verified_on",
|
||||
"owner",
|
||||
"disclosure",
|
||||
),
|
||||
}
|
||||
ALLOWED = {
|
||||
"product.maturity": {"released", "beta", "planned", "deprecated"},
|
||||
"features.status": {"released", "beta", "planned", "deprecated"},
|
||||
"claims.type": {"fact", "goal", "comparison"},
|
||||
"claims.status": {"approved", "draft", "rejected"},
|
||||
"disclosure": {"public", "restricted", "internal"},
|
||||
}
|
||||
OUTPUT_TYPES = {
|
||||
"feature-catalog",
|
||||
"technical-spec",
|
||||
"presentation",
|
||||
"technical-proposal",
|
||||
"one-pager",
|
||||
"whitepaper",
|
||||
"tender-response",
|
||||
"sales-demo",
|
||||
"case-study",
|
||||
"competitive-positioning",
|
||||
}
|
||||
DISCLOSURE_RANK = {"internal": 0, "restricted": 1, "public": 2}
|
||||
PLACEHOLDERS = re.compile(
|
||||
r"(?i)(?:\bTBD\b|\bTODO\b|待补充|待确认|placeholder|changeme)"
|
||||
)
|
||||
SECRET_QUERY = re.compile(r"(?i)(?:token|api[_-]?key|secret|password)=")
|
||||
|
||||
|
||||
def issue(path, message):
|
||||
return {"path": path, "message": message}
|
||||
|
||||
|
||||
def is_nonempty_string(value):
|
||||
return isinstance(value, str) and bool(value.strip())
|
||||
|
||||
|
||||
def validate_record_shape(collection, index, record, errors):
|
||||
path = f"{collection}[{index}]"
|
||||
if not isinstance(record, dict):
|
||||
errors.append(issue(path, "必须是对象"))
|
||||
return False
|
||||
for field in REQUIRED_FIELDS[collection]:
|
||||
value = record.get(field)
|
||||
if field in ("applies_to",):
|
||||
if not isinstance(value, list) or not value:
|
||||
errors.append(issue(f"{path}.{field}", "必须是非空数组"))
|
||||
elif not is_nonempty_string(value):
|
||||
errors.append(issue(f"{path}.{field}", "必须是非空字符串"))
|
||||
return True
|
||||
|
||||
|
||||
def evidence_disclosure_map(records):
|
||||
return {
|
||||
record.get("id"): record.get("disclosure")
|
||||
for record in records["evidence"]
|
||||
if isinstance(record, dict)
|
||||
}
|
||||
|
||||
|
||||
def validate_disclosure_chain(records, errors):
|
||||
"""A record must not be more public than the evidence backing it."""
|
||||
evidence_disclosure = evidence_disclosure_map(records)
|
||||
for collection in ("parameters", "claims"):
|
||||
for index, record in enumerate(records[collection]):
|
||||
if not isinstance(record, dict):
|
||||
continue
|
||||
own = record.get("disclosure")
|
||||
if own not in DISCLOSURE_RANK:
|
||||
continue
|
||||
for ref in record.get("evidence_ids", []) or []:
|
||||
backing = evidence_disclosure.get(ref)
|
||||
if backing not in DISCLOSURE_RANK:
|
||||
continue
|
||||
if DISCLOSURE_RANK[backing] < DISCLOSURE_RANK[own]:
|
||||
errors.append(
|
||||
issue(
|
||||
f"{collection}[{index}].disclosure",
|
||||
f"标记为 {own}，但证据 {ref} 仅为 {backing}",
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def validate_channel(records, channel, errors):
|
||||
"""Reject claims cleared for delivery whose evidence is too restricted."""
|
||||
target = DISCLOSURE_RANK[channel]
|
||||
evidence_disclosure = evidence_disclosure_map(records)
|
||||
for index, claim in enumerate(records["claims"]):
|
||||
if not isinstance(claim, dict) or claim.get("status") != "approved":
|
||||
continue
|
||||
if not claim.get("allowed_outputs"):
|
||||
continue
|
||||
for ref in claim.get("evidence_ids", []) or []:
|
||||
disclosure = evidence_disclosure.get(ref)
|
||||
if disclosure not in DISCLOSURE_RANK:
|
||||
continue
|
||||
if DISCLOSURE_RANK[disclosure] < target:
|
||||
errors.append(
|
||||
issue(
|
||||
f"claims[{index}].evidence_ids",
|
||||
f"证据 {ref} 为 {disclosure}，不足以支撑 {channel} 渠道",
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def validate_manifest(data, channel=None):
|
||||
errors = []
|
||||
warnings = []
|
||||
if not isinstance(data, dict):
|
||||
return [issue("$", "根节点必须是对象")], warnings
|
||||
if data.get("schema_version") != 1:
|
||||
errors.append(issue("schema_version", "当前仅支持整数 1"))
|
||||
|
||||
product = data.get("product")
|
||||
if not isinstance(product, dict):
|
||||
errors.append(issue("product", "必须是对象"))
|
||||
product = {}
|
||||
for field in ("name", "version", "category", "summary", "maturity"):
|
||||
if not is_nonempty_string(product.get(field)):
|
||||
errors.append(issue(f"product.{field}", "必须是非空字符串"))
|
||||
if product.get("maturity") not in ALLOWED["product.maturity"]:
|
||||
errors.append(
|
||||
issue(
|
||||
"product.maturity",
|
||||
f"必须是 {sorted(ALLOWED['product.maturity'])} 之一",
|
||||
)
|
||||
)
|
||||
|
||||
records = {}
|
||||
all_ids = {}
|
||||
for collection in COLLECTIONS:
|
||||
values = data.get(collection, [])
|
||||
if not isinstance(values, list):
|
||||
errors.append(issue(collection, "必须是数组"))
|
||||
values = []
|
||||
records[collection] = values
|
||||
for index, record in enumerate(values):
|
||||
if not validate_record_shape(collection, index, record, errors):
|
||||
continue
|
||||
record_id = record.get("id")
|
||||
if is_nonempty_string(record_id):
|
||||
if record_id in all_ids:
|
||||
errors.append(
|
||||
issue(
|
||||
f"{collection}[{index}].id",
|
||||
f"ID 与 {all_ids[record_id]} 重复：{record_id}",
|
||||
)
|
||||
)
|
||||
else:
|
||||
all_ids[record_id] = f"{collection}[{index}]"
|
||||
|
||||
evidence_ids = {
|
||||
record.get("id")
|
||||
for record in records["evidence"]
|
||||
if isinstance(record, dict) and is_nonempty_string(record.get("id"))
|
||||
}
|
||||
feature_ids = {
|
||||
record.get("id")
|
||||
for record in records["features"]
|
||||
if isinstance(record, dict) and is_nonempty_string(record.get("id"))
|
||||
}
|
||||
use_case_ids = {
|
||||
record.get("id")
|
||||
for record in records["use_cases"]
|
||||
if isinstance(record, dict) and is_nonempty_string(record.get("id"))
|
||||
}
|
||||
|
||||
for collection in COLLECTIONS:
|
||||
for index, record in enumerate(records[collection]):
|
||||
if not isinstance(record, dict):
|
||||
continue
|
||||
path = f"{collection}[{index}]"
|
||||
refs = record.get("evidence_ids", [])
|
||||
if refs is not None and not isinstance(refs, list):
|
||||
errors.append(issue(f"{path}.evidence_ids", "必须是数组"))
|
||||
refs = []
|
||||
for ref in refs or []:
|
||||
if ref not in evidence_ids:
|
||||
errors.append(
|
||||
issue(f"{path}.evidence_ids", f"引用不存在：{ref}")
|
||||
)
|
||||
for ref in record.get("use_case_ids", []) or []:
|
||||
if ref not in use_case_ids:
|
||||
errors.append(
|
||||
issue(f"{path}.use_case_ids", f"引用不存在：{ref}")
|
||||
)
|
||||
|
||||
for index, limitation in enumerate(records["limitations"]):
|
||||
if not isinstance(limitation, dict):
|
||||
continue
|
||||
for ref in limitation.get("applies_to", []) or []:
|
||||
if ref not in feature_ids and ref not in all_ids:
|
||||
errors.append(
|
||||
issue(
|
||||
f"limitations[{index}].applies_to",
|
||||
f"引用不存在：{ref}",
|
||||
)
|
||||
)
|
||||
|
||||
for index, feature in enumerate(records["features"]):
|
||||
if not isinstance(feature, dict):
|
||||
continue
|
||||
status = feature.get("status")
|
||||
if status not in ALLOWED["features.status"]:
|
||||
errors.append(
|
||||
issue(
|
||||
f"features[{index}].status",
|
||||
f"必须是 {sorted(ALLOWED['features.status'])} 之一",
|
||||
)
|
||||
)
|
||||
if status in {"released", "beta"} and not feature.get("evidence_ids"):
|
||||
warnings.append(
|
||||
issue(
|
||||
f"features[{index}].evidence_ids",
|
||||
"已发布或 beta 功能缺少证据",
|
||||
)
|
||||
)
|
||||
|
||||
for index, parameter in enumerate(records["parameters"]):
|
||||
if not isinstance(parameter, dict):
|
||||
continue
|
||||
disclosure = parameter.get("disclosure")
|
||||
if disclosure not in ALLOWED["disclosure"]:
|
||||
errors.append(
|
||||
issue(
|
||||
f"parameters[{index}].disclosure",
|
||||
f"必须是 {sorted(ALLOWED['disclosure'])} 之一",
|
||||
)
|
||||
)
|
||||
if not parameter.get("evidence_ids"):
|
||||
warnings.append(
|
||||
issue(
|
||||
f"parameters[{index}].evidence_ids",
|
||||
"技术参数缺少证据",
|
||||
)
|
||||
)
|
||||
|
||||
for index, claim in enumerate(records["claims"]):
|
||||
if not isinstance(claim, dict):
|
||||
continue
|
||||
if claim.get("type") not in ALLOWED["claims.type"]:
|
||||
errors.append(
|
||||
issue(
|
||||
f"claims[{index}].type",
|
||||
f"必须是 {sorted(ALLOWED['claims.type'])} 之一",
|
||||
)
|
||||
)
|
||||
if claim.get("status") not in ALLOWED["claims.status"]:
|
||||
errors.append(
|
||||
issue(
|
||||
f"claims[{index}].status",
|
||||
f"必须是 {sorted(ALLOWED['claims.status'])} 之一",
|
||||
)
|
||||
)
|
||||
if (
|
||||
claim.get("type") in {"fact", "comparison"}
|
||||
and claim.get("status") == "approved"
|
||||
and not claim.get("evidence_ids")
|
||||
):
|
||||
errors.append(
|
||||
issue(
|
||||
f"claims[{index}].evidence_ids",
|
||||
"已批准的事实或比较主张必须有证据",
|
||||
)
|
||||
)
|
||||
outputs = claim.get("allowed_outputs", [])
|
||||
if outputs is not None and not isinstance(outputs, list):
|
||||
errors.append(
|
||||
issue(f"claims[{index}].allowed_outputs", "必须是数组")
|
||||
)
|
||||
for output in outputs or []:
|
||||
if output not in OUTPUT_TYPES:
|
||||
errors.append(
|
||||
issue(
|
||||
f"claims[{index}].allowed_outputs",
|
||||
f"未知产物类型：{output}",
|
||||
)
|
||||
)
|
||||
|
||||
for index, evidence in enumerate(records["evidence"]):
|
||||
if not isinstance(evidence, dict):
|
||||
continue
|
||||
disclosure = evidence.get("disclosure")
|
||||
if disclosure not in ALLOWED["disclosure"]:
|
||||
errors.append(
|
||||
issue(
|
||||
f"evidence[{index}].disclosure",
|
||||
f"必须是 {sorted(ALLOWED['disclosure'])} 之一",
|
||||
)
|
||||
)
|
||||
verified_on = evidence.get("verified_on")
|
||||
if is_nonempty_string(verified_on):
|
||||
try:
|
||||
dt.date.fromisoformat(verified_on)
|
||||
except ValueError:
|
||||
errors.append(
|
||||
issue(
|
||||
f"evidence[{index}].verified_on",
|
||||
"必须使用 YYYY-MM-DD",
|
||||
)
|
||||
)
|
||||
source = evidence.get("source", "")
|
||||
if is_nonempty_string(source) and SECRET_QUERY.search(source):
|
||||
errors.append(
|
||||
issue(
|
||||
f"evidence[{index}].source",
|
||||
"来源中疑似包含密钥或令牌参数",
|
||||
)
|
||||
)
|
||||
|
||||
for index, differentiator in enumerate(records["differentiators"]):
|
||||
if isinstance(differentiator, dict) and not differentiator.get(
|
||||
"evidence_ids"
|
||||
):
|
||||
warnings.append(
|
||||
issue(
|
||||
f"differentiators[{index}].evidence_ids",
|
||||
"差异点缺少证据",
|
||||
)
|
||||
)
|
||||
|
||||
validate_disclosure_chain(records, errors)
|
||||
if channel in DISCLOSURE_RANK:
|
||||
validate_channel(records, channel, errors)
|
||||
|
||||
for path, value in walk_strings(data):
|
||||
if PLACEHOLDERS.search(value):
|
||||
warnings.append(issue(path, f"包含占位内容：{value[:60]}"))
|
||||
|
||||
if not evidence_ids:
|
||||
warnings.append(issue("evidence", "没有任何证据记录"))
|
||||
return errors, warnings
|
||||
|
||||
|
||||
def walk_strings(value, path="$"):
|
||||
if isinstance(value, dict):
|
||||
for key, item in value.items():
|
||||
yield from walk_strings(item, f"{path}.{key}")
|
||||
elif isinstance(value, list):
|
||||
for index, item in enumerate(value):
|
||||
yield from walk_strings(item, f"{path}[{index}]")
|
||||
elif isinstance(value, str):
|
||||
yield path, value
|
||||
|
||||
|
||||
def parse_args():
|
||||
parser = argparse.ArgumentParser(description="验证产品事实与证据清单")
|
||||
parser.add_argument("manifest", help="product-evidence.json")
|
||||
parser.add_argument("--json", action="store_true", help="输出 JSON")
|
||||
parser.add_argument(
|
||||
"--strict",
|
||||
action="store_true",
|
||||
help="存在占位符、缺证据等警告时也失败",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--channel",
|
||||
choices=sorted(DISCLOSURE_RANK),
|
||||
help="目标发布渠道，校验主张引用证据的公开级别是否足够",
|
||||
)
|
||||
return parser.parse_args()
|
||||
|
||||
|
||||
def main():
|
||||
args = parse_args()
|
||||
path = Path(args.manifest).expanduser().resolve()
|
||||
with path.open(encoding="utf-8") as handle:
|
||||
data = json.load(handle)
|
||||
errors, warnings = validate_manifest(data, channel=args.channel)
|
||||
report = {
|
||||
"file": str(path),
|
||||
"channel": args.channel,
|
||||
"valid": not errors and (not args.strict or not warnings),
|
||||
"errors": errors,
|
||||
"warnings": warnings,
|
||||
"stats": {
|
||||
collection: len(data.get(collection, []))
|
||||
if isinstance(data, dict) and isinstance(data.get(collection, []), list)
|
||||
else 0
|
||||
for collection in COLLECTIONS
|
||||
},
|
||||
}
|
||||
if args.json:
|
||||
print(json.dumps(report, ensure_ascii=False, indent=2))
|
||||
else:
|
||||
for level, items in (("错误", errors), ("警告", warnings)):
|
||||
for item in items:
|
||||
print(f"[{level}] {item['path']}：{item['message']}")
|
||||
print(
|
||||
f"核验完成：错误 {len(errors)}，警告 {len(warnings)}，"
|
||||
f"状态 {'通过' if report['valid'] else '失败'}"
|
||||
)
|
||||
return 0 if report["valid"] else 1
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -0,0 +1,120 @@
|
||||
{
|
||||
"schema_version": 1,
|
||||
"product": {
|
||||
"name": "示例产品",
|
||||
"version": "1.0",
|
||||
"category": "产品类别",
|
||||
"summary": "用一句可验证的话说明产品面向谁、解决什么问题。",
|
||||
"maturity": "released",
|
||||
"audiences": [
|
||||
"技术负责人",
|
||||
"业务负责人"
|
||||
],
|
||||
"deployment_modes": [
|
||||
"私有化部署"
|
||||
]
|
||||
},
|
||||
"terminology": [
|
||||
{
|
||||
"term": "标准术语",
|
||||
"definition": "术语在所有材料中的统一定义。",
|
||||
"avoid": [
|
||||
"不再使用的旧称谓"
|
||||
]
|
||||
}
|
||||
],
|
||||
"features": [
|
||||
{
|
||||
"id": "FEAT-001",
|
||||
"name": "示例功能",
|
||||
"summary": "说明用户可执行的动作和可观察结果。",
|
||||
"status": "released",
|
||||
"availability": "标准版本",
|
||||
"evidence_ids": [
|
||||
"EVD-001"
|
||||
],
|
||||
"use_case_ids": [
|
||||
"CASE-001"
|
||||
]
|
||||
}
|
||||
],
|
||||
"parameters": [
|
||||
{
|
||||
"id": "PAR-001",
|
||||
"name": "示例技术参数",
|
||||
"value": "已核验值",
|
||||
"unit": "",
|
||||
"conditions": "说明测试条件、版本和统计口径。",
|
||||
"scope": "标准版本",
|
||||
"disclosure": "public",
|
||||
"evidence_ids": [
|
||||
"EVD-001"
|
||||
]
|
||||
}
|
||||
],
|
||||
"claims": [
|
||||
{
|
||||
"id": "CLM-001",
|
||||
"text": "产品在指定版本和条件下提供示例功能。",
|
||||
"type": "fact",
|
||||
"status": "approved",
|
||||
"allowed_outputs": [
|
||||
"feature-catalog",
|
||||
"technical-proposal",
|
||||
"presentation"
|
||||
],
|
||||
"evidence_ids": [
|
||||
"EVD-001"
|
||||
]
|
||||
}
|
||||
],
|
||||
"use_cases": [
|
||||
{
|
||||
"id": "CASE-001",
|
||||
"name": "示例场景",
|
||||
"audience": "技术负责人",
|
||||
"problem": "说明现有工作中的具体问题。",
|
||||
"workflow": "说明产品参与的步骤和人工复核点。",
|
||||
"outcome": "只写已有证据支持的结果。",
|
||||
"evidence_ids": [
|
||||
"EVD-001"
|
||||
]
|
||||
}
|
||||
],
|
||||
"differentiators": [
|
||||
{
|
||||
"id": "DIF-001",
|
||||
"statement": "说明可验证的产品差异。",
|
||||
"comparison_scope": "明确比较对象和版本范围。",
|
||||
"evidence_ids": [
|
||||
"EVD-001"
|
||||
]
|
||||
}
|
||||
],
|
||||
"limitations": [
|
||||
{
|
||||
"id": "LIM-001",
|
||||
"text": "说明适用边界、依赖条件或人工复核要求。",
|
||||
"applies_to": [
|
||||
"FEAT-001"
|
||||
]
|
||||
}
|
||||
],
|
||||
"evidence": [
|
||||
{
|
||||
"id": "EVD-001",
|
||||
"type": "product-documentation",
|
||||
"title": "产品说明书",
|
||||
"source": "docs/product.md",
|
||||
"locator": "功能章节",
|
||||
"verified_on": "2026-01-01",
|
||||
"owner": "产品负责人",
|
||||
"disclosure": "public",
|
||||
"notes": "发布前重新核验版本。"
|
||||
}
|
||||
],
|
||||
"prohibited_claims": [
|
||||
"绝对领先",
|
||||
"百分之百准确"
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,113 @@
|
||||
import copy
|
||||
import importlib.util
|
||||
import json
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
SKILL_DIR = Path(__file__).resolve().parents[1]
|
||||
SCRIPT = SKILL_DIR / "scripts" / "validate_evidence.py"
|
||||
SPEC = importlib.util.spec_from_file_location("validate_evidence", SCRIPT)
|
||||
validate_evidence = importlib.util.module_from_spec(SPEC)
|
||||
SPEC.loader.exec_module(validate_evidence)
|
||||
|
||||
|
||||
class ProductEvidenceTests(unittest.TestCase):
|
||||
@classmethod
|
||||
def setUpClass(cls):
|
||||
template = SKILL_DIR / "templates" / "product-evidence.example.json"
|
||||
cls.valid_data = json.loads(template.read_text(encoding="utf-8"))
|
||||
|
||||
def test_example_manifest_is_valid(self):
|
||||
errors, warnings = validate_evidence.validate_manifest(self.valid_data)
|
||||
self.assertEqual(errors, [])
|
||||
self.assertEqual(warnings, [])
|
||||
|
||||
def test_rejects_duplicate_ids_and_unknown_references(self):
|
||||
data = copy.deepcopy(self.valid_data)
|
||||
data["parameters"][0]["id"] = "FEAT-001"
|
||||
data["claims"][0]["evidence_ids"] = ["EVD-MISSING"]
|
||||
errors, _ = validate_evidence.validate_manifest(data)
|
||||
messages = "\n".join(item["message"] for item in errors)
|
||||
self.assertIn("重复", messages)
|
||||
self.assertIn("引用不存在", messages)
|
||||
|
||||
def test_rejects_secret_in_evidence_source(self):
|
||||
data = copy.deepcopy(self.valid_data)
|
||||
data["evidence"][0]["source"] = "https://example.test/doc?token=secret"
|
||||
errors, _ = validate_evidence.validate_manifest(data)
|
||||
self.assertTrue(
|
||||
any("密钥" in item["message"] for item in errors)
|
||||
)
|
||||
|
||||
def test_placeholder_is_warning(self):
|
||||
data = copy.deepcopy(self.valid_data)
|
||||
data["product"]["summary"] = "TODO"
|
||||
errors, warnings = validate_evidence.validate_manifest(data)
|
||||
self.assertEqual(errors, [])
|
||||
self.assertTrue(warnings)
|
||||
|
||||
def test_rejects_record_more_public_than_evidence(self):
|
||||
data = copy.deepcopy(self.valid_data)
|
||||
for record in data["evidence"]:
|
||||
if record["id"] == "EVD-001":
|
||||
record["disclosure"] = "restricted"
|
||||
errors, _ = validate_evidence.validate_manifest(data)
|
||||
self.assertTrue(
|
||||
any("仅为 restricted" in item["message"] for item in errors)
|
||||
)
|
||||
|
||||
def test_restricted_parameter_is_not_a_public_channel_warning(self):
|
||||
data = copy.deepcopy(self.valid_data)
|
||||
data["parameters"][0]["disclosure"] = "restricted"
|
||||
for record in data["evidence"]:
|
||||
if record["id"] in data["parameters"][0]["evidence_ids"]:
|
||||
record["disclosure"] = "restricted"
|
||||
data["claims"][0]["evidence_ids"] = []
|
||||
data["claims"][0]["type"] = "goal"
|
||||
errors, warnings = validate_evidence.validate_manifest(
|
||||
data, channel="public"
|
||||
)
|
||||
self.assertEqual(errors, [])
|
||||
self.assertEqual(warnings, [])
|
||||
|
||||
def test_public_channel_rejects_internal_evidence(self):
|
||||
data = copy.deepcopy(self.valid_data)
|
||||
claim = data["claims"][0]
|
||||
claim["allowed_outputs"] = ["one-pager"]
|
||||
evidence_id = claim["evidence_ids"][0]
|
||||
for record in data["evidence"]:
|
||||
if record["id"] == evidence_id:
|
||||
record["disclosure"] = "internal"
|
||||
for parameter in data["parameters"]:
|
||||
if evidence_id in parameter.get("evidence_ids", []):
|
||||
parameter["disclosure"] = "internal"
|
||||
errors, _ = validate_evidence.validate_manifest(data, channel="public")
|
||||
self.assertTrue(
|
||||
any("不足以支撑 public" in item["message"] for item in errors)
|
||||
)
|
||||
errors, _ = validate_evidence.validate_manifest(data, channel="internal")
|
||||
self.assertEqual(errors, [])
|
||||
|
||||
def test_cli_json_report(self):
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
manifest = Path(tmp) / "product-evidence.json"
|
||||
manifest.write_text(
|
||||
json.dumps(self.valid_data, ensure_ascii=False),
|
||||
encoding="utf-8",
|
||||
)
|
||||
result = subprocess.run(
|
||||
[sys.executable, str(SCRIPT), str(manifest), "--json"],
|
||||
check=True,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
)
|
||||
report = json.loads(result.stdout)
|
||||
self.assertTrue(report["valid"])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
Reference in New Issue
Block a user