Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
c6ef6820e4 | ||
|
|
175c612d35 | ||
|
|
1566e2d3eb | ||
|
|
f7810ec3d9 |
@@ -0,0 +1,2 @@
|
||||
blank_issues_enabled: true
|
||||
contact_links: []
|
||||
@@ -0,0 +1,36 @@
|
||||
---
|
||||
name: 新技能提案
|
||||
about: 提新技能或技能改动前先过 SCOPE.md 门槛(默认拒绝,能组合就不新增)
|
||||
title: "[技能] "
|
||||
labels: skill
|
||||
---
|
||||
|
||||
## 1. 真实失败(必填)
|
||||
|
||||
在哪个真实任务/会话里出了问题?跑了什么、期望什么、实际什么。
|
||||
|
||||
> 假设性的「这样会更好」不算。说不清就说明还不到提的时候。
|
||||
|
||||
## 2. 为什么组合不出来(必填)
|
||||
|
||||
用现有哪个技能组合试过?为什么做不到?
|
||||
|
||||
> 能组合出来的,改现有技能,不新增。
|
||||
|
||||
## 3. 是否重复提案
|
||||
|
||||
- [ ] 已读 `.out-of-scope/` 三份拒绝档案,不是重复提案
|
||||
|
||||
## 4. 分桶与调用模式
|
||||
|
||||
- 初始分桶:`beta`(新技能一律先 beta)
|
||||
- 调用模式:`model-invoked` / `user-invoked`
|
||||
- 判定:这件事是不是「人下命令才开始」?是 → `user-invoked`(同时写 `disable-model-invocation: true`)
|
||||
|
||||
## 5. 最低交付
|
||||
|
||||
- [ ] `evals/evals.json` 3 条:正常路径 / 边界 / 失败恢复
|
||||
- [ ] `docs/skills/<name>.md` 四段式:What it does / When to reach for it / Common questions / It's working if
|
||||
- [ ] `docs/skill-routing.md` 已回改
|
||||
- [ ] `catalog.json` 已 regenerate
|
||||
- [ ] `CHANGELOG.md` + `plugin.json` 版本号
|
||||
@@ -0,0 +1,24 @@
|
||||
## 这个 PR 做了什么
|
||||
|
||||
<!-- 一句话,说明改的是哪个技能 / 哪套流程 -->
|
||||
|
||||
## SCOPE 门槛
|
||||
|
||||
- [ ] 真实失败:说清了在哪个真实任务里出的问题
|
||||
- [ ] 组合不出来:说明用现有技能组合为什么做不到
|
||||
- [ ] 已查 `.out-of-scope/`,不是重复提案
|
||||
|
||||
## 改动检查
|
||||
|
||||
- [ ] 分桶正确(新技能一律 `beta`;`promoted` 需要真实使用记录 + evals 通过)
|
||||
- [ ] 调用模式正确(`user-invoked` 必须写 `disable-model-invocation: true`)
|
||||
- [ ] `evals/evals.json` 3 条(正常 / 边界 / 失败恢复)
|
||||
- [ ] `docs/skills/<name>.md` 四段式
|
||||
- [ ] `docs/skill-routing.md` 已回改
|
||||
- [ ] `catalog.json` 已 regenerate(`python3 scripts/generate-catalog.py`)
|
||||
- [ ] `CHANGELOG.md` + `plugin.json` 版本号已更新
|
||||
- [ ] 四道门禁通过:`validate-skills.py --strict`、`scan-skill.sh`、`generate-catalog.py --check`、`tests/install-skills.test.sh`
|
||||
|
||||
## 人工拍板项(有则写明)
|
||||
|
||||
<!-- deprecated 删除、路由变更、promoted 晋升,需要人确认 -->
|
||||
@@ -25,3 +25,9 @@ __pycache__/
|
||||
|
||||
# Node
|
||||
node_modules/
|
||||
|
||||
# 定时任务产出(本机运行产物,不回写仓库;evals 结果报告除外)
|
||||
reports/skill-health/
|
||||
reports/monthly/
|
||||
reports/_scheduler/
|
||||
reports/evals/queue-*.md
|
||||
|
||||
@@ -53,3 +53,18 @@
|
||||
|
||||
- 补齐剩余 24 个技能(dev 类为主)的 `evals/evals.json`(各 3 条)与 `docs/skills/<name>.md`(四段式)。
|
||||
- `--coverage`:40 个 promoted+model-invoked 技能的 evals 与文档缺口 **24 → 0**。
|
||||
|
||||
### 继续推进(2026-11-01)
|
||||
|
||||
- 新增 beta 技能 `diagnosing-bugs`(难查 bug 诊断纪律:先建红循环再推理),补齐 evals 与四段式文档;路由表同步。
|
||||
- 新增 `scripts/monthly-review.sh`:一条命令跑月度审阅(覆盖率 / 扫描 / 第三方 / catalog / description 异常)。
|
||||
|
||||
### 治理落地:定时任务与提案入口(2026-11-01)
|
||||
|
||||
- 新增 `scripts/scheduled-task.sh`:`daily` / `weekly` / `monthly` 三个确定性任务,只跑门禁、扫漂移、生成队列;需要模型判断的部分(真跑 evals、审阅 routing)生成 queue 文件交 agent。
|
||||
- 新增 `scripts/install-schedulers.sh`:把节拍装成 macOS 用户级 launchd(`com.pipexerp.ai-proj-skill-{daily,weekly,monthly}`),不再依赖会话提醒;支持 `--status` / `--run` / `--uninstall`。
|
||||
- 新增 `.gitea/issue_template/new-skill.md`、`.gitea/issue_template/config.yaml`、`.gitea/pull_request_template.md`:提案与 PR 走同一套 SCOPE 门槛问题。
|
||||
- `README.md` 一键部署从 2 步改为 4 步(补 git hooks 与定时任务);`MAINTENANCE.md` 增加「定时任务」一节,写明脚本只做确定性动作的边界。
|
||||
- `.gitignore` 排除定时任务产出(`reports/skill-health/`、`reports/monthly/`、`reports/_scheduler/`、`reports/evals/queue-*.md`)。
|
||||
- 新增 `scripts/evals-status.py` 与 `reports/evals/TEMPLATE.md`:evals 覆盖率与回归报告模板。
|
||||
- `skill-creator` 实战记录 #1:用标准流程新建 `diagnosing-bugs`(beta,待观察后晋升)。
|
||||
|
||||
+24
-2
@@ -25,7 +25,8 @@
|
||||
|
||||
- 范围:上周改过的技能 + 3 个高风险技能(`dev-coding`、`dev-review`、`req-design`)。
|
||||
- 每技能取 1 条关键用例,做「有技能 / 无技能」对比,按 rubric 打分。
|
||||
- 结果记录到 `reports/evals/YYYY-Www.md`,进周报。
|
||||
- 结果记录到 `reports/evals/YYYY-Www.md`(模板 `reports/evals/TEMPLATE.md`),进周报。
|
||||
- 用 `python3 scripts/evals-status.py` 看哪些技能已有 evals、哪些已有回归报告。
|
||||
- 连续两次无增量的技能 → 按 `SCOPE.md` 评估降级。
|
||||
- 不做全量(61 技能 × 3 用例成本过高);全量只在治理大改时做。
|
||||
|
||||
@@ -34,7 +35,7 @@
|
||||
- 两台机器 `--audit` 漂移必须为 0(beta/deprecated 除外)。
|
||||
- 审 `catalog.json`:`description` 长度 <80 或 >600 的标黄;含「各种 / 等等 / 相关」这类泛化词的标出。
|
||||
- 第三方技能(arkcli 等)有变动时,评估是否影响团队技能;跑一次 `./scripts/scan-installed.sh`。
|
||||
- 跑 `python3 scripts/validate-skills.py --coverage`:报告缺 evals / 缺四段式文档的技能,按 `SCOPE.md` 的覆盖率路线图推进。
|
||||
- 跑 `./scripts/monthly-review.sh` 一次出齐:覆盖率、本仓库扫描、第三方扫描、catalog 一致性、description 异常项。
|
||||
|
||||
## 每季度首月 1 日 10:30 — 治理复核
|
||||
|
||||
@@ -44,6 +45,27 @@
|
||||
- 复核覆盖率:`--coverage` 的两项缺口是否按路线图收敛。
|
||||
- 更新本文件与 `CHANGELOG.md`。
|
||||
|
||||
## 定时任务(launchd,2026-11-01 起)
|
||||
|
||||
节拍不再挂在「会话提醒」上(会话一断就失效),改成宿主级 launchd:
|
||||
|
||||
```bash
|
||||
./scripts/install-schedulers.sh # 安装(幂等;换机器重跑一次即可复现)
|
||||
./scripts/install-schedulers.sh --status # 看状态与最近产出
|
||||
./scripts/install-schedulers.sh --run daily # 立刻跑一次(调试)
|
||||
./scripts/install-schedulers.sh --uninstall
|
||||
```
|
||||
|
||||
| 任务 | 时间 | 脚本做什么 | 产出 |
|
||||
| --- | --- | --- | --- |
|
||||
| `daily` | 每天 07:20 | 漂移 audit + 入库安全扫描 + 覆盖率 | `reports/skill-health/daily-YYYY-MM-DD.md`(`latest.md` 供 7:30 日报直接读) |
|
||||
| `weekly` | 每周一 10:00 | 算上周改动技能 + 3 个高风险项,生成 evals 抽样队列 | `reports/evals/queue-YYYY-MM-DD.md` |
|
||||
| `monthly` | 每月 1 日 10:00 | 一条命令跑完月度审阅;1/4/7/10 月附季度治理清单 | `reports/monthly/review-YYYY-MM.md`、`quarterly-YYYY-Qn.md` |
|
||||
|
||||
**边界**:脚本只做确定性动作。真跑 evals、审阅 routing 这类需要模型判断的部分,脚本只生成队列文件,由项目经理 Bot 在 7:30 日报里接手执行——分工不写在脚本里就会漏。
|
||||
|
||||
日志:`reports/_scheduler/<job>.log`。
|
||||
|
||||
## 责任分工
|
||||
|
||||
- 项目经理 Bot:执行节奏、出周/月报、盯漂移与门禁。
|
||||
|
||||
@@ -69,7 +69,7 @@ PRD 文档存储在思源笔记中,可以导出发送到飞书群,通过飞
|
||||
- **Codex**([Skills 文档](https://developers.openai.com/codex/skills))或 **Claude Code** 已安装
|
||||
- **ai-proj 账号 + MCP API Key**:联系管理员获取(Key 格式: `aiproj_pk_xxx`)
|
||||
|
||||
### 一键部署(2 步搞定)
|
||||
### 一键部署(4 步搞定)
|
||||
|
||||
```bash
|
||||
# 1. 克隆仓库
|
||||
@@ -78,8 +78,16 @@ cd ai-proj-helper
|
||||
|
||||
# 2. 默认安装到 Codex 的用户级标准目录 ~/.agents/skills
|
||||
./install-skills.sh
|
||||
|
||||
# 3. 装本地门禁(Gitea Actions 无 runner,commit/push 靠 git hook 兜底)
|
||||
./scripts/install-git-hooks.sh
|
||||
|
||||
# 4. 装技能治理定时任务(macOS launchd:每日漂移检查 / 每周 evals 队列 / 每月审阅)
|
||||
./scripts/install-schedulers.sh
|
||||
```
|
||||
|
||||
第 3、4 步只对「本仓库的维护者」需要——只装技能给别人用的,跑完第 2 步就可以停。
|
||||
|
||||
安装器会复制完整技能目录,包括 `SKILL.md`、references、scripts 和 assets。Codex 会自动发现 `~/.agents/skills` 中的技能;若没有出现,重启 Codex。
|
||||
|
||||
Claude Code 用户显式选择 Claude 目标;需要同时配置 MCP 和 marketplace 时运行 `init.sh`:
|
||||
|
||||
+16
-2
@@ -7,13 +7,14 @@
|
||||
"missing_evals": [],
|
||||
"missing_docs": []
|
||||
},
|
||||
"generated_at": "2026-10-31T21:19:23Z",
|
||||
"total": 61,
|
||||
"generated_at": "2026-10-31T21:33:50Z",
|
||||
"total": 62,
|
||||
"counts": {
|
||||
"promoted/user-invoked": 18,
|
||||
"deprecated/user-invoked": 1,
|
||||
"promoted/model-invoked": 40,
|
||||
"beta/user-invoked": 1,
|
||||
"beta/model-invoked": 1,
|
||||
"deprecated/model-invoked": 1
|
||||
},
|
||||
"skills": [
|
||||
@@ -420,6 +421,19 @@
|
||||
"has_evals": true,
|
||||
"has_docs": true
|
||||
},
|
||||
{
|
||||
"name": "diagnosing-bugs",
|
||||
"category": "dev",
|
||||
"version": "0.1.0",
|
||||
"status": "beta",
|
||||
"invocation": "model-invoked",
|
||||
"description": "难查 bug 的诊断纪律:先造出一条「现在就为这个 bug 变红」的紧反馈循环,再二分定位、最小验证、补回归测试。当 bug 难复现、间歇失败、或第一次看没头绪时使用。",
|
||||
"description_len": 84,
|
||||
"path": "skills-dev/diagnosing-bugs-plugin/skills/SKILL.md",
|
||||
"digest": "d93fb95c073858cc",
|
||||
"has_evals": true,
|
||||
"has_docs": true
|
||||
},
|
||||
{
|
||||
"name": "executing-plans",
|
||||
"category": "dev",
|
||||
|
||||
@@ -21,6 +21,7 @@
|
||||
| 设计 / 拆任务 | `req-design`、`dev-arch`、`db-migration` |
|
||||
| 编码 | `dev-coding`、`karpathy-guidelines` |
|
||||
| 测试 | `dev-test`、`req-test-gate` |
|
||||
| 难查的 bug / 间歇失败 | `diagnosing-bugs`(先建红循环再推理) |
|
||||
| 代码评审 | `dev-review`、`review-checklist` |
|
||||
| 发布(Slark 仓库/Desktop) | `slark-cicd` |
|
||||
| 发布(DSH 插件/宿主补丁) | `slark-dsh-cicd` |
|
||||
|
||||
@@ -0,0 +1,26 @@
|
||||
# diagnosing-bugs
|
||||
|
||||
## What it does
|
||||
|
||||
给难查的 bug 一套诊断纪律:先造出「现在就为这个 bug 变红」的紧反馈循环,再二分定位、一次验一个假设、最小改动修复,最后补一条曾经红过的回归测试。
|
||||
|
||||
## When to reach for it
|
||||
|
||||
- 间歇失败、偶发问题,第一次看没头绪
|
||||
- 已知两个好版本之间引入的回归(适合 git bisect)
|
||||
- 修过几次没修好的 bug
|
||||
|
||||
一眼可见的拼写/配置错误不需要它,直接修。
|
||||
|
||||
## Common questions
|
||||
|
||||
- **为什么不能先猜着改?** 没有红循环就没有验证手段,改了也不知道是不是这次改动生效。
|
||||
- **复现不出来怎么办?** 提高复现率(缩小输入、调整时序、加观测),或用 git bisect 从版本维度定位,而不是盲上。
|
||||
- **修完就完了吗?** 不,必须补一条回归测试,并证明修复前它是红的。
|
||||
- **和 dev-test 的区别?** dev-test 管测试怎么写;本技能管「怎么找到 bug」。
|
||||
|
||||
## It's working if
|
||||
|
||||
- 动手改代码前,先给出了一条现在为红的命令
|
||||
- 每次改动只变一个变量,并能说出上次结果的差异
|
||||
- 修复结论包含根因 + 回归测试 + 「曾经红过」的证据
|
||||
@@ -0,0 +1,15 @@
|
||||
# <YYYY-Www> 技能 evals 抽样回归
|
||||
|
||||
- 日期:YYYY-MM-DD
|
||||
- 范围:上周改动过的技能 + 高风险技能(dev-coding / dev-review / req-design)
|
||||
- 方法:每技能取 1 条关键用例,做「有技能 / 无技能」两组独立输出对比,按 rubric 打分(0–2)
|
||||
|
||||
| 技能 | 用例 | 有技能 | 无技能 | 差值 | 结论 |
|
||||
| --- | --- | --- | --- | --- | --- |
|
||||
| | | | | | |
|
||||
|
||||
## 结论与新发现
|
||||
|
||||
- 有效技能:
|
||||
- 无增量技能(按 SCOPE 评估是否降级):
|
||||
- 需要补的用例:
|
||||
Executable
+31
@@ -0,0 +1,31 @@
|
||||
#!/usr/bin/env python3
|
||||
"""列出各技能的 evals 与是否已有回归报告(reports/evals/<name>.md)。"""
|
||||
from __future__ import annotations
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
ROOT = Path(__file__).resolve().parent.parent
|
||||
rows = []
|
||||
for pj in sorted(ROOT.glob("skills-*/*-plugin/.claude-plugin/plugin.json")):
|
||||
meta = json.loads(pj.read_text())
|
||||
name = meta.get("install_name")
|
||||
if not name:
|
||||
continue
|
||||
ev = next((pj.parent.parent / "skills").rglob("evals/evals.json"), None)
|
||||
cases = 0
|
||||
if ev:
|
||||
try:
|
||||
cases = len(json.loads(ev.read_text()).get("evals", []))
|
||||
except Exception:
|
||||
cases = 0
|
||||
report = ROOT / "reports" / "evals" / f"{name}.md"
|
||||
rows.append((name, meta.get("status", "promoted"), cases, report.is_file()))
|
||||
|
||||
with_evals = sum(1 for r in rows if r[2] > 0)
|
||||
with_report = sum(1 for r in rows if r[3])
|
||||
print(f"{'技能':32} {'status':10} {'用例':4} 回归报告")
|
||||
for name, status, cases, has_report in rows:
|
||||
if cases == 0:
|
||||
continue
|
||||
print(f"{name:32} {status:10} {cases:<4} {'有' if has_report else '-'}")
|
||||
print(f"\n有 evals 的技能: {with_evals}/{len(rows)};已有回归报告: {with_report}/{with_evals}")
|
||||
Executable
+143
@@ -0,0 +1,143 @@
|
||||
#!/usr/bin/env bash
|
||||
# 安装 / 卸载 / 查看技能治理的定时任务(macOS 用户级 launchd)
|
||||
#
|
||||
# 为什么用 launchd 而不是会话提醒:会话提醒随会话消失,节拍就断了;
|
||||
# launchd 是宿主级,跟仓库绑定,换机器重跑一次即可复现。
|
||||
#
|
||||
# 用法:
|
||||
# ./scripts/install-schedulers.sh 安装(幂等,会先卸载再装)
|
||||
# ./scripts/install-schedulers.sh --status 看三个任务的加载状态
|
||||
# ./scripts/install-schedulers.sh --run daily 立刻跑一次(调试用)
|
||||
# ./scripts/install-schedulers.sh --uninstall 卸载
|
||||
set -uo pipefail
|
||||
|
||||
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
||||
TASK="$ROOT/scripts/scheduled-task.sh"
|
||||
AGENTS="$HOME/Library/LaunchAgents"
|
||||
LOGDIR="$ROOT/reports/_scheduler"
|
||||
PREFIX="com.pipexerp.ai-proj-skill"
|
||||
UID_N="$(id -u)"
|
||||
JOBS="daily weekly monthly"
|
||||
|
||||
SCHED_PATH="/opt/homebrew/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin"
|
||||
if command -v node >/dev/null 2>&1; then
|
||||
SCHED_PATH="$(dirname "$(command -v node)"):$SCHED_PATH"
|
||||
fi
|
||||
|
||||
log() { printf '%s\n' "$*"; }
|
||||
|
||||
calendar_xml() {
|
||||
case "$1" in
|
||||
daily) printf ' <key>Hour</key><integer>7</integer>\n <key>Minute</key><integer>20</integer>' ;;
|
||||
weekly) printf ' <key>Weekday</key><integer>1</integer>\n <key>Hour</key><integer>10</integer>\n <key>Minute</key><integer>0</integer>' ;;
|
||||
monthly) printf ' <key>Day</key><integer>1</integer>\n <key>Hour</key><integer>10</integer>\n <key>Minute</key><integer>0</integer>' ;;
|
||||
esac
|
||||
}
|
||||
|
||||
write_plist() {
|
||||
local job="$1" label="$PREFIX-$job" file="$AGENTS/$PREFIX-$job.plist" cal
|
||||
cal="$(calendar_xml "$job")"
|
||||
cat > "$file" <<PLIST
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
|
||||
<plist version="1.0">
|
||||
<dict>
|
||||
<key>Label</key><string>$label</string>
|
||||
<key>ProgramArguments</key>
|
||||
<array>
|
||||
<string>/bin/bash</string>
|
||||
<string>$TASK</string>
|
||||
<string>$job</string>
|
||||
</array>
|
||||
<key>WorkingDirectory</key><string>$ROOT</string>
|
||||
<key>EnvironmentVariables</key>
|
||||
<dict>
|
||||
<key>HOME</key><string>$HOME</string>
|
||||
<key>PATH</key><string>$SCHED_PATH</string>
|
||||
</dict>
|
||||
<key>StandardOutPath</key><string>$LOGDIR/$job.log</string>
|
||||
<key>StandardErrorPath</key><string>$LOGDIR/$job.err.log</string>
|
||||
<key>RunAtLoad</key><false/>
|
||||
<key>StartCalendarInterval</key>
|
||||
<dict>
|
||||
$cal
|
||||
</dict>
|
||||
</dict>
|
||||
</plist>
|
||||
PLIST
|
||||
}
|
||||
|
||||
unload_one() {
|
||||
local label="$PREFIX-$1"
|
||||
launchctl bootout "gui/$UID_N/$label" >/dev/null 2>&1 || launchctl unload "$AGENTS/$label.plist" >/dev/null 2>&1
|
||||
}
|
||||
|
||||
install() {
|
||||
mkdir -p "$AGENTS" "$LOGDIR"
|
||||
chmod +x "$TASK" 2>/dev/null
|
||||
local job label
|
||||
for job in $JOBS; do
|
||||
label="$PREFIX-$job"
|
||||
write_plist "$job"
|
||||
unload_one "$job"
|
||||
if launchctl bootstrap "gui/$UID_N" "$AGENTS/$label.plist" 2>/dev/null; then
|
||||
log "已加载 $label"
|
||||
elif launchctl load -w "$AGENTS/$label.plist" 2>/dev/null; then
|
||||
log "已加载(load) $label"
|
||||
else
|
||||
log "加载失败 $label(检查 plist: $AGENTS/$label.plist)"
|
||||
fi
|
||||
done
|
||||
log ""
|
||||
log "触发时间:daily 每天 07:20 / weekly 每周一 10:00 / monthly 每月 1 日 10:00(1/4/7/10 月附季度清单)"
|
||||
log "日志:${LOGDIR#$ROOT/}/<job>.log"
|
||||
log "产出:reports/skill-health/、reports/evals/queue-*.md、reports/monthly/"
|
||||
}
|
||||
|
||||
status() {
|
||||
local job label
|
||||
for job in $JOBS; do
|
||||
label="$PREFIX-$job"
|
||||
if launchctl print "gui/$UID_N/$label" >/dev/null 2>&1; then
|
||||
printf '%-8s 已加载 %s\n' "$job" \
|
||||
"$(launchctl print "gui/$UID_N/$label" 2>/dev/null | grep -E 'state =|last exit code' | tr -d '\t' | tr '\n' ' ')"
|
||||
else
|
||||
printf '%-8s 未加载\n' "$job"
|
||||
fi
|
||||
done
|
||||
log ""
|
||||
log "日志目录:${LOGDIR#$ROOT/}"
|
||||
log "最近产出:"
|
||||
ls -1t "$ROOT/reports/skill-health"/daily-*.md "$ROOT/reports/evals"/queue-*.md "$ROOT/reports/monthly"/review-*.md 2>/dev/null | head -3 | sed "s#^$ROOT/# - #"
|
||||
}
|
||||
|
||||
run_now() {
|
||||
local job="${1:-}"
|
||||
case "$job" in daily|weekly|monthly) ;; *) log "用法: $0 --run daily|weekly|monthly"; exit 2 ;; esac
|
||||
mkdir -p "$LOGDIR"
|
||||
local label="$PREFIX-$job"
|
||||
if launchctl print "gui/$UID_N/$label" >/dev/null 2>&1; then
|
||||
launchctl kickstart -k "gui/$UID_N/$label" && log "已触发 $job(看 ${LOGDIR#$ROOT/}/$job.log)"
|
||||
else
|
||||
log "任务未加载,直接前台执行"
|
||||
/bin/bash "$TASK" "$job"
|
||||
fi
|
||||
}
|
||||
|
||||
uninstall() {
|
||||
local job label
|
||||
for job in $JOBS; do
|
||||
label="$PREFIX-$job"
|
||||
unload_one "$job"
|
||||
rm -f "$AGENTS/$label.plist"
|
||||
log "已卸载 $label"
|
||||
done
|
||||
}
|
||||
|
||||
case "${1:-install}" in
|
||||
install|--install) install ;;
|
||||
--status|status) status ;;
|
||||
--run) run_now "${2:-}" ;;
|
||||
--uninstall|uninstall) uninstall ;;
|
||||
*) log "用法: $0 [install|--status|--run daily|weekly|monthly|--uninstall]"; exit 2 ;;
|
||||
esac
|
||||
Executable
+25
@@ -0,0 +1,25 @@
|
||||
#!/usr/bin/env bash
|
||||
# 月度技能审阅:一条命令跑完 MAINTENANCE 里的月度项。
|
||||
# 用法: ./scripts/monthly-review.sh
|
||||
set -uo pipefail
|
||||
cd "$(git rev-parse --show-toplevel)" || exit 1
|
||||
echo "== 技能月度审阅 $(date '+%F %H:%M') =="
|
||||
echo; echo "-- 1. 覆盖率(promoted + model-invoked)--"
|
||||
python3 scripts/validate-skills.py --coverage
|
||||
echo; echo "-- 2. 本仓库安全扫描 --"
|
||||
./scripts/scan-skill.sh | tail -2
|
||||
echo; echo "-- 3. 第三方已装技能扫描 --"
|
||||
./scripts/scan-installed.sh | tail -2
|
||||
echo; echo "-- 4. catalog 是否最新 --"
|
||||
python3 scripts/generate-catalog.py --check
|
||||
echo; echo "-- 5. catalog 摘要与 description 异常项 --"
|
||||
python3 - <<'PY'
|
||||
import json
|
||||
d = json.load(open('catalog.json'))
|
||||
print("总数:", d['total'], "| 分布:", d['counts'])
|
||||
bad = [(x['name'], x['description_len']) for x in d['skills']
|
||||
if x['description_len'] < 60 or x['description_len'] > 600]
|
||||
print("description 长度异常:", bad if bad else "无")
|
||||
cov = d.get('coverage', {})
|
||||
print(f"覆盖率: evals {cov.get('with_evals')}/{cov.get('required')},文档 {cov.get('with_docs')}/{cov.get('required')}")
|
||||
PY
|
||||
Executable
+162
@@ -0,0 +1,162 @@
|
||||
#!/usr/bin/env bash
|
||||
# 技能治理定时任务入口(由 launchd 调用,见 scripts/install-schedulers.sh)
|
||||
#
|
||||
# 设计原则:脚本只做「确定性」的事——跑门禁、扫漂移、生成清单。
|
||||
# 需要模型判断的部分(真跑 evals、人工审阅 routing)不在脚本里做,只生成 queue 文件,
|
||||
# 由项目经理 Bot 在 7:30 项目日报里接手执行。这样节拍不依赖任何会话的记忆。
|
||||
#
|
||||
# 用法:
|
||||
# ./scripts/scheduled-task.sh daily # 每天 07:20,先于 7:30 项目日报
|
||||
# ./scripts/scheduled-task.sh weekly # 每周一 10:00
|
||||
# ./scripts/scheduled-task.sh monthly # 每月 1 日 10:00(1/4/7/10 月附带季度清单)
|
||||
set -uo pipefail
|
||||
|
||||
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
||||
cd "$ROOT" || exit 1
|
||||
|
||||
JOB="${1:-daily}"
|
||||
DAY="$(date '+%F')"
|
||||
HEALTH_DIR="$ROOT/reports/skill-health"
|
||||
EVALS_DIR="$ROOT/reports/evals"
|
||||
MONTHLY_DIR="$ROOT/reports/monthly"
|
||||
mkdir -p "$HEALTH_DIR" "$EVALS_DIR" "$MONTHLY_DIR"
|
||||
|
||||
log() { printf '[%s] %s\n' "$(date '+%H:%M:%S')" "$*"; }
|
||||
|
||||
# ---------- 公共采集:漂移 / 入库扫描 / 覆盖率 / 第三方 ----------
|
||||
DRIFT_LINE=""; SCAN_LINE=""; COV_LINE=""; THIRD_LINE=""
|
||||
DRIFT_N="0"; HIGH_N="0"
|
||||
collect_health() {
|
||||
local audit
|
||||
audit="$(./install-skills.sh --audit 2>&1)"
|
||||
DRIFT_LINE="$(printf '%s\n' "$audit" | grep -E '[0-9]+ 个漂移' | tail -1)"
|
||||
SCAN_LINE="$(./scripts/scan-skill.sh 2>&1 | grep '扫描完成' | tail -1)"
|
||||
COV_LINE="$(/usr/bin/python3 scripts/validate-skills.py --coverage 2>&1 | grep '覆盖率' | tail -1)"
|
||||
THIRD_LINE="$(./scripts/scan-installed.sh 2>&1 | grep '扫描完成' | tail -1)"
|
||||
DRIFT_N="$(printf '%s' "$DRIFT_LINE" | grep -oE '[0-9]+ 个漂移' | grep -oE '^[0-9]+' || true)"
|
||||
HIGH_N="$(printf '%s' "$SCAN_LINE" | grep -oE '[0-9]+ 个高危' | grep -oE '^[0-9]+' || true)"
|
||||
DRIFT_N="${DRIFT_N:-0}"
|
||||
HIGH_N="${HIGH_N:-0}"
|
||||
AUDIT_OUT="$audit"
|
||||
}
|
||||
|
||||
# ---------- 每日:漂移 + 入库扫描,写一份可被日报直接读的健康快照 ----------
|
||||
run_daily() {
|
||||
local md="$HEALTH_DIR/daily-$DAY.md" todo=0
|
||||
collect_health
|
||||
{
|
||||
echo "# 技能健康 $DAY"
|
||||
echo
|
||||
echo "- 漂移:${DRIFT_LINE:-未采集}"
|
||||
echo "- 入库扫描:${SCAN_LINE:-未采集}"
|
||||
echo "- 覆盖率:${COV_LINE:-未采集}"
|
||||
echo "- 第三方技能:${THIRD_LINE:-未采集}"
|
||||
echo
|
||||
echo "## 需要处理"
|
||||
if [ "$DRIFT_N" != "0" ]; then
|
||||
echo "- 漂移 ${DRIFT_N} 个(未安装 / 版本落后),按 MAINTENANCE「每日」处理"
|
||||
todo=1
|
||||
fi
|
||||
if [ "$HIGH_N" != "0" ]; then
|
||||
echo "- 入库扫描高危 ${HIGH_N} 个,立即处理,不等日报"
|
||||
todo=1
|
||||
fi
|
||||
[ "$todo" = "0" ] && echo "无"
|
||||
echo
|
||||
echo "## 原始输出(audit 尾部)"
|
||||
echo '```'
|
||||
printf '%s\n' "$AUDIT_OUT" | tail -15
|
||||
echo '```'
|
||||
} > "$md"
|
||||
cp "$md" "$HEALTH_DIR/latest.md"
|
||||
log "daily -> ${md#$ROOT/}"
|
||||
log "漂移 ${DRIFT_N} / 高危 ${HIGH_N} / ${SCAN_LINE:-扫描未采集}"
|
||||
}
|
||||
|
||||
# ---------- 每周:生成 evals 抽样队列(执行仍需模型,见 MAINTENANCE「每周一」)----------
|
||||
run_weekly() {
|
||||
local md="$EVALS_DIR/queue-$DAY.md"
|
||||
local changed changed_total highrisk reports
|
||||
changed_total="$(git log --since='7 days ago' --name-only --pretty=format: 2>/dev/null \
|
||||
| grep -oE '^skills-[a-z]+/[^/]+-plugin/' | sort -u | wc -l | tr -d ' ')"
|
||||
changed="$(git log --since='7 days ago' --name-only --pretty=format: 2>/dev/null \
|
||||
| grep -oE '^skills-[a-z]+/[^/]+-plugin/' \
|
||||
| sed -E 's#^skills-[a-z]+/##; s#-plugin/$##' \
|
||||
| sort | uniq -c | sort -rn | head -5 | awk '{print $2}' | tr '\n' ' ')"
|
||||
highrisk="dev-coding dev-review req-design"
|
||||
reports="$(ls "$EVALS_DIR"/*.md 2>/dev/null | grep -vE 'TEMPLATE|queue-' | wc -l | tr -d ' ')"
|
||||
{
|
||||
echo "# 本周 evals 抽样队列 $DAY"
|
||||
echo
|
||||
echo "现有回归报告:${reports:-0} 份(目标:每周至少 +1 份)"
|
||||
echo
|
||||
echo "## 范围"
|
||||
echo
|
||||
echo "- 上周改动技能共 ${changed_total:-0} 个,按改动次数取前 5:${changed:-(无)}"
|
||||
echo "- 高风险固定项:$highrisk"
|
||||
echo
|
||||
echo "## 做法(每个技能)"
|
||||
echo
|
||||
echo "1. 取 1 条关键用例,跑「有技能 / 无技能」两遍;"
|
||||
echo "2. 按 rubric 打分,写增量;"
|
||||
echo "3. 结果写入 \`reports/evals/$DAY-<技能名>.md\`(模板 \`reports/evals/TEMPLATE.md\`)。"
|
||||
echo
|
||||
echo "## 判读"
|
||||
echo
|
||||
echo "- 连续两次无增量 → 按 \`SCOPE.md\` 评估降级。"
|
||||
echo "- 全量只在治理大改时做,不做每周全量。"
|
||||
echo
|
||||
echo "> 本文件由 launchd 每周一 10:00 自动生成;脚本不调模型,执行由 agent 接手。"
|
||||
} > "$md"
|
||||
log "weekly -> ${md#$ROOT/}(报告现有 ${reports:-0} 份)"
|
||||
}
|
||||
|
||||
# ---------- 每月:一条命令出齐审阅项;季度月附治理清单 ----------
|
||||
run_monthly() {
|
||||
local mo md mon q
|
||||
mo="$(date '+%Y-%m')"
|
||||
mon="$(date '+%m')"
|
||||
md="$MONTHLY_DIR/review-$mo.md"
|
||||
{
|
||||
echo "# 技能月度审阅 $mo"
|
||||
echo
|
||||
./scripts/monthly-review.sh 2>&1
|
||||
} > "$md"
|
||||
collect_health
|
||||
{
|
||||
echo
|
||||
echo "## 补充:漂移与第三方"
|
||||
echo
|
||||
echo "- ${DRIFT_LINE:-漂移未采集}"
|
||||
echo "- ${THIRD_LINE:-第三方未采集}"
|
||||
} >> "$md"
|
||||
log "monthly -> ${md#$ROOT/}"
|
||||
case "$mon" in
|
||||
01|04|07|10)
|
||||
q="$(date '+%Y')-Q$(( (10#$mon - 1) / 3 + 1 ))"
|
||||
local mq="$MONTHLY_DIR/quarterly-$q.md"
|
||||
{
|
||||
echo "# 季度治理复核 $q"
|
||||
echo
|
||||
echo "按 MAINTENANCE「每季度首月 1 日 10:30」逐项打勾:"
|
||||
echo
|
||||
echo "- [ ] 过一遍 \`docs/skill-routing.md\`:删掉不再用的引用、补上新技能"
|
||||
echo "- [ ] 清理 \`deprecated\`:满 3 个月且无回滚需求 → 删除插件目录"
|
||||
echo "- [ ] 复核分桶与调用模式:还有没有命令型技能占着 \`model-invoked\`"
|
||||
echo "- [ ] 复核覆盖率:\`--coverage\` 两项缺口是否按路线图收敛"
|
||||
echo "- [ ] 更新 \`MAINTENANCE.md\` 与 \`CHANGELOG.md\`"
|
||||
} > "$mq"
|
||||
log "quarterly -> ${mq#$ROOT/}"
|
||||
;;
|
||||
esac
|
||||
}
|
||||
|
||||
case "$JOB" in
|
||||
daily) run_daily ;;
|
||||
weekly) run_weekly ;;
|
||||
monthly) run_monthly ;;
|
||||
*)
|
||||
echo "用法: $0 daily|weekly|monthly" >&2
|
||||
exit 2
|
||||
;;
|
||||
esac
|
||||
@@ -0,0 +1,11 @@
|
||||
{
|
||||
"name": "diagnosing-bugs-plugin",
|
||||
"description": "难查 bug 的诊断纪律:先造出会变红的紧反馈循环,再二分定位、最小验证、补回归测试。",
|
||||
"version": "0.1.0",
|
||||
"author": { "name": "qiudl" },
|
||||
"install_name": "diagnosing-bugs",
|
||||
"install_type": "skill",
|
||||
"dir_category": "dev",
|
||||
"status": "beta",
|
||||
"invocation": "model-invoked"
|
||||
}
|
||||
@@ -0,0 +1,62 @@
|
||||
---
|
||||
name: diagnosing-bugs
|
||||
description: 难查 bug 的诊断纪律:先造出一条「现在就为这个 bug 变红」的紧反馈循环,再二分定位、最小验证、补回归测试。当 bug 难复现、间歇失败、或第一次看没头绪时使用。
|
||||
metadata:
|
||||
invocation: model-invoked
|
||||
owner: qiudl
|
||||
---
|
||||
|
||||
# 难查 bug 诊断
|
||||
|
||||
## 0. 一句话
|
||||
|
||||
**没有红循环就不推理**:先有一条几秒内能跑、且现在就为这个 bug 变红的命令,再开始猜。
|
||||
|
||||
## 1. 何时用
|
||||
|
||||
- 间歇性失败、偶发问题
|
||||
- 第一次看没头绪、日志线索不足
|
||||
- 已知两个好版本之间引入的回归
|
||||
- 别人修过几次没修好的 bug
|
||||
|
||||
何时不用:一眼可见的拼写/配置错误、明确报错指向单行 → 直接修。
|
||||
|
||||
## 2. 铁律
|
||||
|
||||
1. **先复现**:能手动复现最好;不能就缩小到最小输入/最小步骤。
|
||||
2. **建红循环**:一条命令(测试/脚本/curl),现在跑必须是红的,且失败原因就是这个 bug。
|
||||
3. **一次只验一个假设**:改一个变量,跑一次,记录结果。
|
||||
4. **修完补回归测试**:确认测试在修复前是红的(临时回退验证一次)。
|
||||
|
||||
## 3. 步骤
|
||||
|
||||
1. 收集信息:报错、日志、复现步骤、最近改动(`git log`/`git bisect`)。
|
||||
2. 写最小复现:能稳定触发的最小输入或最小步骤。
|
||||
3. 建立反馈循环:命令固定、几秒内出结果、失败信息可读。
|
||||
4. 二分定位:git bisect / 输入二分 / 代码路径二分,把范围砍半。
|
||||
5. 假设 → 最小改动 → 跑循环验证;不做"顺手重构"。
|
||||
6. 回归测试:补一条覆盖该 bug 的测试,并证明它曾经红过。
|
||||
7. 记录根因与预防(进复盘或 memory)。
|
||||
|
||||
## 4. 反模式
|
||||
|
||||
- 没复现就开始改代码
|
||||
- 同时改多个地方,不知道哪个生效
|
||||
- 靠"看了日志觉得是缓存问题"直接改
|
||||
- 修完不补回归测试
|
||||
- 间歇失败归因于"环境问题"就收工
|
||||
|
||||
## 5. 验证清单
|
||||
|
||||
- [ ] 有一个现在为红的命令(不是偶发)
|
||||
- [ ] 每次改动只变一个变量,并有结果记录
|
||||
- [ ] 修复后同一命令变绿
|
||||
- [ ] 回归测试存在,且回退修复后它会红
|
||||
- [ ] 根因已写清(不是"不知道,反正好了")
|
||||
|
||||
## 6. 与其他技能
|
||||
|
||||
- 需要写测试规范 → `dev-test`
|
||||
- 需要判断是不是设计缺陷 → `defect-analysis`
|
||||
- 修复后要评审 → `dev-review`
|
||||
- 任务状态与证据 → `ai-proj`(REQ 未 released 不算完成)
|
||||
@@ -0,0 +1,27 @@
|
||||
{
|
||||
"skill_name": "diagnosing-bugs",
|
||||
"skill_version": "0.1.0",
|
||||
"evals": [
|
||||
{
|
||||
"id": 1,
|
||||
"kind": "happy_path",
|
||||
"prompt": "这个接口偶尔返回 500,日志看不出原因,帮查一下",
|
||||
"expected_output": "先要求/建立最小复现与一条现在为红的命令(测试或 curl 循环),说明偶发失败要先提高复现率,再二分定位;不在没有红循环时直接给修复方案",
|
||||
"files": []
|
||||
},
|
||||
{
|
||||
"id": 2,
|
||||
"kind": "edge_case",
|
||||
"prompt": "我看日志就感觉是缓存没失效,直接把缓存清一下就好",
|
||||
"expected_output": "不直接改:把假设变成可验证实验(一条能证伪的命令),先证明是缓存问题再改,并说明如何确认修复",
|
||||
"files": []
|
||||
},
|
||||
{
|
||||
"id": 3,
|
||||
"kind": "failure_recovery",
|
||||
"prompt": "试了几次复现不出来,算了先上线吧",
|
||||
"expected_output": "拒绝盲上:改为加观测(日志/埋点)、提高复现率(输入二分/时序调整)、或用 git bisect 定位引入点;明确记录未复现风险",
|
||||
"files": []
|
||||
}
|
||||
]
|
||||
}
|
||||
Reference in New Issue
Block a user