hehaiguang1123
a6f05ab2d5
- Phase 0: AGENTS.md cleanup (dedup quotes, renumber sections, merge qmd) - Phase 1: typed relations (manage-relations.py, graph-search.py, check-staleness.py, detect-conflicts.py) - Phase 2: frontmatter validator, weekly lint, knowledge promotion, git hooks - Fix .gitignore to track tools/ and .githooks/ - Fix git remote URL (remove plaintext token) - New wiki pages: 504 pages, 34 raw sources
893 lines
28 KiB
Markdown
893 lines
28 KiB
Markdown
---
|
||
categories:
|
||
- "[[Tools]]"
|
||
- "[[Documentation]]"
|
||
tags:
|
||
- wiki
|
||
- planning
|
||
- architecture
|
||
created: 2026-06-30
|
||
type: reference
|
||
---
|
||
|
||
# LLM Wiki v2 升级 — 技术实施方案
|
||
|
||
> 基于 Karpathy v1 + Rohit v2 理念的本仓库升级方案。
|
||
>
|
||
> 评估基线：v1 实现度 75%，v2 实现度 15%，总体成熟度 48% → 目标 85%+
|
||
|
||
---
|
||
|
||
## 目录
|
||
|
||
- [Phase 0 — 即时修复（1-2h）](#phase-0--即时修复1-2h)
|
||
- [Phase 1 — 短期升级（2-4 周）](#phase-1--短期升级2-4-周)
|
||
- [Phase 2 — 中期自动化（1-2 月）](#phase-2--中期自动化1-2-月)
|
||
- [Phase 3 — 长期进阶（3-6 月）](#phase-3--长期进阶3-6-月)
|
||
- [附录：文件清单](#附录文件清单)
|
||
|
||
---
|
||
|
||
## Phase 0 — 即时修复（1-2h）
|
||
|
||
**目标**: 清理 Schema 债务，提升基础数据质量
|
||
|
||
### 任务 A-0: 修复 AGENTS.md 冗余
|
||
|
||
| 问题 | 文件 | 行号 | 操作 |
|
||
|------|------|------|------|
|
||
| Karpathy quote 重复 x3 | AGENTS.md | 1294, 1297 | 删除第 1297 行的重复 quote，保留第 1294 行 |
|
||
| 编号冲突（两个第 5 节） | AGENTS.md | 777, 887 | 重编号：WorkBuddy → `## 5.`，工具目录 → `## 5.5` 保留，代码风格 → `## 6.` 并调整后续编号 |
|
||
| qmd 说明附录末尾 | AGENTS.md | 末尾 | 整合到 `## 6. 工具链` 节 |
|
||
|
||
具体操作：
|
||
|
||
1. 删除重复 quote：
|
||
```
|
||
Location: ~L1294-L1297
|
||
old: > "这种个性化方式..."
|
||
new: (delete line L1297)
|
||
```
|
||
|
||
2. 重新编号冲突节：
|
||
```
|
||
~L887: "## 5. 代码风格与工具" → "## 6. 代码风格与工具"
|
||
~L891: "### 5.1 脚本规范" → "### 6.1"
|
||
~L898: "### 5.2 Lint" → "### 6.2"
|
||
```
|
||
|
||
同时触发后续引用更新：AGENTS.md 内部引用的锚点需更新。
|
||
|
||
3. 将 qmd 说明（当前末尾）移入 `## 6. 工具链`：
|
||
- 当前末尾的 qmd 说明（`## opencode-mem` 之前的块）拆出
|
||
- 在 `### 6.2 Lint 工具脚本` 之后，新增 `### 6.3 qmd 本地搜索引擎`
|
||
|
||
### 任务 B-0: 提高行号标注覆盖率
|
||
|
||
**当前**: 880 条 `[raw:…]` 引用中，492 条有行号（55.9%）
|
||
|
||
**工具**: `tools/scripts/fix-raw-citations.py`（新建）
|
||
|
||
```python
|
||
#!/usr/bin/env python
|
||
# -*- coding: utf-8 -*-
|
||
"""
|
||
fix-raw-citations.py — 扫描 wiki/ 页面，对缺失行号的 [raw:file] 引用
|
||
自动从原始 raw 文件中查找匹配文本并补上行号。
|
||
"""
|
||
import re, os, glob, sys
|
||
from pathlib import Path
|
||
|
||
VAULT = Path(r"D:\Applications\app\kepano-obsidian-main")
|
||
WIKI_DIR = VAULT / "wiki"
|
||
RAW_DIR = VAULT / "raw"
|
||
|
||
def extract_context(text: str, keyword: str, context_lines: int = 3) -> str:
|
||
"""在 raw 文件中找到 keyword 所在的行号范围"""
|
||
lines = text.splitlines()
|
||
for i, line in enumerate(lines, 1):
|
||
if keyword in line:
|
||
start = max(1, i - context_lines)
|
||
end = min(len(lines), i + context_lines)
|
||
return f"{start}-{end}"
|
||
return ""
|
||
|
||
def process_file(filepath: Path):
|
||
content = filepath.read_text(encoding="utf-8")
|
||
# Match [raw:filename] without line number
|
||
pattern = r'\[raw:([^:\]]+)\]' # [raw:filename]
|
||
matches = list(re.finditer(pattern, content))
|
||
if not matches:
|
||
return content, 0
|
||
|
||
raw_text_cache = {}
|
||
fixes = 0
|
||
for m in reversed(matches): # iterate backwards to preserve offsets
|
||
raw_filename = m.group(1)
|
||
if raw_filename not in raw_text_cache:
|
||
raw_path = RAW_DIR / f"{raw_filename}.md"
|
||
if raw_path.exists():
|
||
raw_text_cache[raw_filename] = raw_path.read_text(encoding="utf-8")
|
||
else:
|
||
continue
|
||
|
||
# Get text around the citation (find keyword in surrounding context)
|
||
start = max(0, m.start() - 200)
|
||
end = min(len(content), m.end() + 200)
|
||
context = content[start:end]
|
||
|
||
# Try to extract a key phrase from context to search in raw
|
||
# Simple approach: get 3 words before the citation
|
||
before = content[max(0, m.start()-50):m.start()]
|
||
words = re.findall(r'[\w\u4e00-\u9fff]+', before)
|
||
keyword = " ".join(words[-5:]) if len(words) >= 5 else " ".join(words)
|
||
|
||
line_range = extract_context(raw_text_cache[raw_filename], keyword)
|
||
if line_range:
|
||
old = m.group(0)
|
||
new = f"[raw:{raw_filename}:{line_range}]"
|
||
content = content[:m.start()] + new + content[m.end():]
|
||
fixes += 1
|
||
|
||
return content, fixes
|
||
|
||
if __name__ == "__main__":
|
||
total_fixes = 0
|
||
for md_file in sorted(WIKI_DIR.glob("*.md")):
|
||
if md_file.name in ("index.md", "log.md"):
|
||
continue
|
||
content, fixes = process_file(md_file)
|
||
if fixes:
|
||
md_file.write_text(content, encoding="utf-8")
|
||
print(f"{md_file.name}: +{fixes} line numbers")
|
||
total_fixes += fixes
|
||
print(f"\nTotal: {total_fixes} citations updated")
|
||
```
|
||
|
||
**运行**:
|
||
```bash
|
||
$env:PYTHONIOENCODING="utf-8"
|
||
python tools/scripts/fix-raw-citations.py
|
||
```
|
||
|
||
**预期效果**: 行号覆盖率从 56% → 75%+
|
||
|
||
### 任务 C-0: 更新 index.md 指标
|
||
|
||
**当前问题**: 声称 478+ wiki 页面，实际 506 页
|
||
|
||
**操作**: 运行统计后手动编辑 index.md
|
||
|
||
```bash
|
||
# 统计当前指标
|
||
python -c "
|
||
import os
|
||
wiki = r'D:\Applications\app\kepano-obsidian-main\wiki'
|
||
files = [f for f in os.listdir(wiki) if f.endswith('.md') and f not in ('index.md','log.md')]
|
||
print(f'Wiki pages: {len(files)}')
|
||
raw = r'D:\Applications\app\kepano-obsidian-main\raw'
|
||
raw_files = [f for f in os.listdir(raw) if f.endswith('.md') and not f.startswith('_')]
|
||
print(f'Raw sources: {len(raw_files)}')
|
||
# Type breakdown
|
||
types = {}
|
||
for f in files:
|
||
content = open(os.path.join(wiki, f), encoding='utf-8').read()
|
||
m = __import__('re').search(r'type:\s*(\S+)', content)
|
||
if m: t = m.group(1); types[t] = types.get(t, 0) + 1
|
||
for t, c in sorted(types.items(), key=lambda x: -x[1]):
|
||
print(f' {t}: {c}')
|
||
"
|
||
```
|
||
|
||
更新 `wiki/index.md` 中概览表格的数字。
|
||
|
||
---
|
||
|
||
## Phase 1 — 短期升级（2-4 周）
|
||
|
||
**目标**: 实现 v2 两个核心特性——Typed relationships 和内存生命周期
|
||
|
||
### 任务 A-1: Typed relationships
|
||
|
||
#### 1.1 前端：Frontmatter 扩展
|
||
|
||
在 `AGENTS.md` 中定义新的 frontmatter 字段：
|
||
|
||
```yaml
|
||
# 新增可选字段
|
||
relations:
|
||
- type: depends_on | conflicts_with | supersedes | caused_by | supports | extends | part_of | example_of
|
||
target: "[[页面名称]]"
|
||
description: "关系说明（可选）" # NEW
|
||
confidence: 1-5 # NEW
|
||
```
|
||
|
||
**边类型一览**：
|
||
|
||
| 类型 | 含义 | 例 |
|
||
|------|------|-----|
|
||
| `depends_on` | A 理解依赖 B | [[三湾改编]] depends_on [[兵为将有]] |
|
||
| `conflicts_with` | A 与 B 矛盾的声明 | [[RAG vs 持久化知识库]] conflicts_with [[纯 RAG 方案]] |
|
||
| `supersedes` | A 替代了 B（新知识覆盖旧知识） | [[LLM Wiki v2]] supersedes [[LLM Wiki v1]] |
|
||
| `caused_by` | A 由 B 导致 | [[八月失败]] caused_by [[杜修经]] |
|
||
| `supports` | A 提供证据支持 B | [[ALEKS 研究]] supports [[AI 个性化学习]] |
|
||
| `extends` | A 扩展了 B 的概念 | [[AIEOU]] extends [[ETEE 生命周期框架]] |
|
||
| `part_of` | A 是 B 的一部分 | [[支部建在连上]] part_of [[三湾改编]] |
|
||
| `example_of` | A 是 B 的一个实例 | [[Duolingo]] example_of [[AI 教育应用]] |
|
||
|
||
#### 1.2 工具：relations 编辑脚本
|
||
|
||
```python
|
||
# tools/scripts/manage-relations.py
|
||
"""
|
||
用法：
|
||
python manage-relations.py add <page> --type supersedes --target "旧页面"
|
||
python manage-relations.py list <page>
|
||
python manage-relations.py graph <page> # 输出 DOT 格式（Graphviz）
|
||
"""
|
||
```
|
||
|
||
**核心逻辑 — 添加关系**：
|
||
|
||
```python
|
||
import yaml, re, sys, json
|
||
from pathlib import Path
|
||
|
||
WIKI = Path(r"D:\Applications\app\kepano-obsidian-main\wiki")
|
||
|
||
def read_frontmatter(filepath):
|
||
content = filepath.read_text(encoding="utf-8")
|
||
m = re.match(r"^---\n(.*?)\n---", content, re.DOTALL)
|
||
if not m: return {}, content
|
||
try:
|
||
front = yaml.safe_load(m.group(1)) or {}
|
||
except:
|
||
front = {}
|
||
return front, content
|
||
|
||
def write_frontmatter(filepath, front, body):
|
||
new_yaml = yaml.dump(front, allow_unicode=True, default_flow_style=False, sort_keys=False)
|
||
filepath.write_text(f"---\n{new_yaml}---\n{body.lstrip()}", encoding="utf-8")
|
||
|
||
def add_relation(page, rel_type, target, desc="", confidence=3):
|
||
fp = WIKI / f"{page}.md"
|
||
if not fp.exists():
|
||
print(f"ERROR: {page}.md not found"); return
|
||
front, body = read_frontmatter(fp)
|
||
if "relations" not in front: front["relations"] = []
|
||
# dedup
|
||
for r in front["relations"]:
|
||
if r.get("type") == rel_type and r.get("target") == target:
|
||
print(f" Already exists: {rel_type} [[{target}]]")
|
||
return
|
||
entry = {"type": rel_type, "target": target}
|
||
if desc: entry["description"] = desc
|
||
if confidence != 3: entry["confidence"] = confidence
|
||
front["relations"].append(entry)
|
||
write_frontmatter(fp, front, body)
|
||
print(f" Added: {page} --{rel_type}--> [[{target}]]")
|
||
```
|
||
|
||
#### 1.3 增量迁移策略
|
||
|
||
不要一次性给所有 506 页添加 relations。**新页面优先**：
|
||
|
||
1. 新创建的 wiki 页面**必须**（使用 AGENTS.md 规范要求）包含 relations
|
||
2. 对现有的 high-traffic 页面（被最多其他页面引用的 top-50）按批次添加
|
||
3. Lint 报告 `relations: none` 的页面数量
|
||
|
||
**优先级排序**：
|
||
|
||
```bash
|
||
# 找出被引用最多的 Top 50 页面
|
||
grep -roh "\[\[[^]]*\]\]" wiki/*.md | sort | uniq -c | sort -rn | head -50
|
||
```
|
||
|
||
#### 1.4 搜索集成：graph-aware search
|
||
|
||
**方案**: 无需新搜索引擎。在 qmd 的基础上加一层关系扩展：
|
||
|
||
```python
|
||
# tools/scripts/graph-search.py
|
||
"""
|
||
用法：
|
||
python graph-search.py "查询词"
|
||
|
||
流程：
|
||
1. qmd vsearch "查询词" → top 10 结果
|
||
2. 对每个结果，读取 relations → 获取相邻节点
|
||
3. 去重后返回 (直接关联 + 关系扩展后的) top 15
|
||
"""
|
||
```
|
||
|
||
核心代码：
|
||
|
||
```python
|
||
import subprocess, json, yaml, re
|
||
from pathlib import Path
|
||
|
||
QMD = r'node "C:\Users\hhhh2024\AppData\Roaming\npm\node_modules\@tobilu\qmd\dist\cli\qmd.js"'
|
||
WIKI = Path(r"D:\Applications\app\kepano-obsidian-main\wiki")
|
||
|
||
def qmd_search(query: str, count=10):
|
||
result = subprocess.run(
|
||
f'{QMD} vsearch "{query}" -c wiki -n {count}',
|
||
capture_output=True, text=True, shell=True
|
||
)
|
||
return [line.strip() for line in result.stdout.splitlines() if line.strip()]
|
||
|
||
def get_relations(page_name: str):
|
||
fp = WIKI / f"{page_name}.md"
|
||
if not fp.exists(): return []
|
||
content = fp.read_text(encoding="utf-8")
|
||
m = re.match(r"^---\n(.*?)\n---", content, re.DOTALL)
|
||
if not m: return []
|
||
try:
|
||
front = yaml.safe_load(m.group(1)) or {}
|
||
except:
|
||
return []
|
||
return front.get("relations", [])
|
||
|
||
def expand_results(results):
|
||
expanded = list(results)
|
||
for r in results:
|
||
rels = get_relations(r)
|
||
for rel in rels:
|
||
target = rel.get("target", "").strip("[]")
|
||
if target and target not in expanded:
|
||
expanded.append(target)
|
||
return expanded[:15]
|
||
|
||
if __name__ == "__main__":
|
||
query = sys.argv[1]
|
||
results = qmd_search(query)
|
||
print("=== Direct matches ===")
|
||
for r in results: print(f" [[{r}]]")
|
||
expanded = expand_results(results)
|
||
new_items = [e for e in expanded if e not in results]
|
||
if new_items:
|
||
print("\n=== Graph-expanded (via relations) ===")
|
||
for e in new_items: print(f" [[{e}]]")
|
||
```
|
||
|
||
---
|
||
|
||
### 任务 B-1: 内存生命周期
|
||
|
||
#### 2.1 Frontmatter 扩展
|
||
|
||
```yaml
|
||
# 新增可选字段
|
||
confidence: 3 # 1-5: 1=推测, 2=单源未验证, 3=已验证, 4=多源一致, 5=无可争议
|
||
status: active # active | superseded | deprecated | tentative | needs-review
|
||
superseded_by: "[[新页面]]" # 仅 status=superseded 时
|
||
last_reviewed: 2026-06-30
|
||
review_interval_days: 180 # 默认 180 天（技术类建议 90 天）
|
||
```
|
||
|
||
#### 2.2 时效检测脚本
|
||
|
||
```python
|
||
# tools/scripts/check-staleness.py
|
||
"""
|
||
扫描 wiki/ 中所有页面的 last_reviewed 字段，
|
||
计算是否超过 review_interval_days，输出过时页面列表。
|
||
"""
|
||
from datetime import date, timedelta
|
||
import yaml, re, sys
|
||
from pathlib import Path
|
||
|
||
WIKI = Path(r"D:\Applications\app\kepano-obsidian-main\wiki")
|
||
TODAY = date.today()
|
||
DEFAULT_INTERVAL = 180
|
||
|
||
stale = []
|
||
no_review = []
|
||
|
||
for md_file in sorted(WIKI.glob("*.md")):
|
||
if md_file.name in ("index.md", "log.md"):
|
||
continue
|
||
content = md_file.read_text(encoding="utf-8")
|
||
m = re.match(r"^---\n(.*?)\n---", content, re.DOTALL)
|
||
if not m: continue
|
||
try:
|
||
front = yaml.safe_load(m.group(1)) or {}
|
||
except:
|
||
continue
|
||
|
||
last_str = front.get("last_reviewed", "")
|
||
interval = front.get("review_interval_days", DEFAULT_INTERVAL)
|
||
|
||
if not last_str:
|
||
no_review.append(md_file.stem)
|
||
continue
|
||
|
||
try:
|
||
last = date.fromisoformat(last_str)
|
||
if (TODAY - last).days > interval:
|
||
stale.append((md_file.stem, last_str, (TODAY - last).days - interval))
|
||
except:
|
||
no_review.append(md_file.stem)
|
||
|
||
print(f"=== Stale pages (overdue): {len(stale)} ===")
|
||
for name, last, overdue in sorted(stale, key=lambda x: -x[2]):
|
||
print(f" {name}: last reviewed {last}, overdue by {overdue} days")
|
||
|
||
if no_review:
|
||
print(f"\n=== No review date: {len(no_review)} ===")
|
||
for n in no_review[:20]:
|
||
print(f" {n}")
|
||
if len(no_review) > 20:
|
||
print(f" ... and {len(no_review)-20} more")
|
||
```
|
||
|
||
#### 2.3 Lint 集成
|
||
|
||
将时效检测和矛盾检测集成到现有的 Lint 工作流：
|
||
|
||
```powershell
|
||
# tools/scripts/check-staleness.ps1
|
||
$env:PYTHONIOENCODING="utf-8"
|
||
python tools/scripts/check-staleness.py
|
||
```
|
||
|
||
矛盾检测逻辑：
|
||
|
||
```python
|
||
def detect_conflicts():
|
||
"""扫描所有页面的 relations 字段中的 conflicts_with 对，
|
||
确认双方的声明是否都还引用对方。"""
|
||
conflicts = []
|
||
for md_file in WIKI.glob("*.md"):
|
||
if md_file.name in ("index.md", "log.md"): continue
|
||
content = md_file.read_text(encoding="utf-8")
|
||
m = re.match(r"^---\n(.*?)\n---", content, re.DOTALL)
|
||
if not m: continue
|
||
try: front = yaml.safe_load(m.group(1)) or {}
|
||
except: continue
|
||
for rel in front.get("relations", []):
|
||
if rel["type"] == "conflicts_with":
|
||
target = rel["target"].strip("[]")
|
||
# Check if target also references us
|
||
conflicts.append((md_file.stem, target))
|
||
return conflicts
|
||
```
|
||
|
||
#### 2.4 AGENTS.md 规范更新
|
||
|
||
在 wiki 页面模板（`AGENTS.md` 第 2.4 节）增加：
|
||
|
||
```yaml
|
||
# 更新所有模板，增加：
|
||
relations:
|
||
# - type: depends_on | supersedes | etc.
|
||
# target: "[[页面]]"
|
||
confidence: 3
|
||
status: active
|
||
last_reviewed: "{{today}}"
|
||
review_interval_days: 180
|
||
```
|
||
|
||
---
|
||
|
||
## Phase 2 — 中期自动化（1-2 月）
|
||
|
||
**目标**: 事件驱动自动化 + Consolidation tiers
|
||
|
||
### 任务 A-2: 事件驱动自动化
|
||
|
||
#### 1.1 Git hooks 配置
|
||
|
||
```bash
|
||
# .githooks/pre-commit
|
||
#!/bin/sh
|
||
# Pre-commit hook: 验证被修改的 wiki 页面 frontmatter
|
||
|
||
CHANGED=$(git diff --cached --name-only --diff-filter=ACM | grep '^wiki/.*\.md$')
|
||
if [ -z "$CHANGED" ]; then exit 0; fi
|
||
|
||
python tools/scripts/validate-frontmatter.py --files $CHANGED
|
||
if [ $? -ne 0 ]; then
|
||
echo "ERROR: Frontmatter validation failed. Commit rejected."
|
||
exit 1
|
||
fi
|
||
```
|
||
|
||
```bash
|
||
# Config:
|
||
git config core.hooksPath .githooks
|
||
```
|
||
|
||
#### 1.2 validate-frontmatter.py 脚本
|
||
|
||
```python
|
||
# tools/scripts/validate-frontmatter.py
|
||
"""
|
||
校验 wiki 页面的 frontmatter 完整性（pre-commit 用）。
|
||
检查项：
|
||
- 必须包含 categories (含 [[LLM Wiki]])
|
||
- 必须包含 tags (含 wiki)
|
||
- 必须包含 type
|
||
- 如果 status != active，必须说明 superseded_by/deprecated 原因
|
||
- relations 的 target 必须指向存在的页面
|
||
"""
|
||
|
||
REQUIRED_CATEGORIES = ["[[LLM Wiki]]"]
|
||
REQUIRED_TAGS = ["wiki"]
|
||
VALID_TYPES = ["concept", "entity", "tool", "reference", "place",
|
||
"institution", "method", "knowledge-card", "synthesis",
|
||
"index", "log", "research-report", "lesson"]
|
||
|
||
def validate(filepath):
|
||
errors = []
|
||
content = filepath.read_text(encoding="utf-8")
|
||
m = re.match(r"^---\n(.*?)\n---", content, re.DOTALL)
|
||
if not m: return ["No frontmatter"]
|
||
try: front = yaml.safe_load(m.group(1)) or {}
|
||
except: return ["YAML parse error"]
|
||
|
||
cats = front.get("categories", [])
|
||
if not any(c.strip("[]") == "LLM Wiki" for c in cats):
|
||
errors.append("Missing [[LLM Wiki]] in categories")
|
||
tags = front.get("tags", [])
|
||
if "wiki" not in tags:
|
||
errors.append("Missing 'wiki' in tags")
|
||
if not front.get("type"):
|
||
errors.append("Missing type")
|
||
elif front["type"] not in VALID_TYPES:
|
||
errors.append(f"Invalid type: {front['type']}")
|
||
if not front.get("source"):
|
||
errors.append("Missing source")
|
||
for rel in front.get("relations", []):
|
||
target = rel.get("target", "").strip("[]")
|
||
if target and not (WIKI / f"{target}.md").exists():
|
||
errors.append(f"Relation target [[{target}]] not found")
|
||
return errors
|
||
```
|
||
|
||
#### 1.3 Windows 定时任务
|
||
|
||
```powershell
|
||
# 安装每周 Lint 定时任务
|
||
$action = New-ScheduledTaskAction -Execute "powershell.exe" `
|
||
-Argument "-NoProfile -ExecutionPolicy Bypass -File D:\Applications\app\kepano-obsidian-main\tools\scripts\weekly-lint.ps1"
|
||
|
||
$trigger = New-ScheduledTaskTrigger -Weekly -DaysOfWeek Sunday -At 10:00PM
|
||
|
||
Register-ScheduledTask -TaskName "ObspVault-WeeklyLint" `
|
||
-Action $action -Trigger $trigger -Description "Weekly LLM Wiki health check"
|
||
```
|
||
|
||
```powershell
|
||
# tools/scripts/weekly-lint.ps1
|
||
$env:PYTHONIOENCODING="utf-8"
|
||
$logFile = "D:\Applications\app\kepano-obsidian-main\tools\data\lint-report-$(Get-Date -Format 'yyyy-MM-dd').log"
|
||
|
||
Write-Output "=== Weekly Lint $(Get-Date) ===" > $logFile
|
||
|
||
# 1. Orphan detection
|
||
Write-Output "`n=== Orphans ===" >> $logFile
|
||
python tools/scripts/check-orphans.py >> $logFile 2>&1
|
||
|
||
# 2. Broken links
|
||
Write-Output "`n=== Broken Links ===" >> $logFile
|
||
& ".\tools\scripts\wiki-lint-broken-v2.ps1" >> $logFile 2>&1
|
||
|
||
# 3. Staleness
|
||
Write-Output "`n=== Staleness ===" >> $logFile
|
||
python tools/scripts/check-staleness.py >> $logFile 2>&1
|
||
|
||
# 4. Discrepancies
|
||
Write-Output "`n=== Conflicts ===" >> $logFile
|
||
python tools/scripts/detect-conflicts.py >> $logFile 2>&1
|
||
|
||
Write-Output "`nDone: $(Get-Date)" >> $logFile
|
||
```
|
||
|
||
#### 1.4 session-end 结晶机制
|
||
|
||
在 `AGENTS.md` 增加自动化指引：
|
||
|
||
```markdown
|
||
## 结晶机制
|
||
|
||
每次 LLM 对话结束时，Agent 必须：
|
||
|
||
1. 检查对话中是否产生了**可复用的知识**（新概念、新的实体信息、经验教训）
|
||
2. 如果产生，创建或更新对应的 wiki 页面
|
||
3. 在 wiki/log.md 追加条目
|
||
4. 执行 `git add + git commit`（自动提交）
|
||
|
||
### 自动判断标准
|
||
|
||
| 信号 | 动作 |
|
||
|------|------|
|
||
| 用户问了从未问过的问题 | 若回答中包含新知识，创建新页面 |
|
||
| 引用了外部来源 | 存入 raw/，创建 wiki 页面 |
|
||
| 修复了错误或过时信息 | 更新对应页面，设置为 superseded |
|
||
```
|
||
|
||
---
|
||
|
||
### 任务 B-2: Consolidation tiers
|
||
|
||
#### 2.1 层级定义
|
||
|
||
```
|
||
wiki/
|
||
├── working/ ← NEW: 临时笔记、对话草稿（TTL 7 天）
|
||
├── semantic/ ← 当前 wiki/ 大部分页面
|
||
├── procedural/ ← 操作手册、最佳实践、工作流
|
||
└── archive/ ← 已有的过时页面归档
|
||
```
|
||
|
||
**迁移规则**：
|
||
|
||
| 来源 → 目标 | 条件 | 自动化程度 |
|
||
|------------|------|-----------|
|
||
| root level → working/ | 任何新建的零散 `.md` | 手动 |
|
||
| working/ → semantic/ | 有 source 字段 + type + 完成度 > 70% | 脚本辅助 |
|
||
| semantic/ → procedural/ | 面向操作的 knowledge（how-to、配方、操作指南） | 手动 |
|
||
| active → archive/ | status=deprecated OR superseded | 自动（Lint 时） |
|
||
|
||
#### 2.2 层级提升脚本
|
||
|
||
```python
|
||
# tools/scripts/promote-knowledge.py
|
||
"""
|
||
检查 working/ 中的页面，分析是否可以提升到 semantic/。
|
||
标准：
|
||
- 有完整的 frontmatter（categories, tags, type, source）
|
||
- 有 body 内容（>100 字）
|
||
- 创建时间 > 7 天
|
||
"""
|
||
from datetime import date, timedelta
|
||
import re, yaml, shutil
|
||
from pathlib import Path
|
||
|
||
WIKI = Path(r"D:\Applications\app\kepano-obsidian-main\wiki")
|
||
|
||
def check_promotable(filepath: Path) -> tuple[bool, list[str]]:
|
||
"""返回 (是否可提升, 原因列表)"""
|
||
content = filepath.read_text(encoding="utf-8")
|
||
m = re.match(r"^---\n(.*?)\n---", content, re.DOTALL)
|
||
if not m: return False, ["No frontmatter"]
|
||
try: front = yaml.safe_load(m.group(1)) or {}
|
||
except: return False, ["YAML parse error"]
|
||
|
||
body = content[m.end():].strip()
|
||
reasons = []
|
||
if not front.get("categories"): reasons.append("Missing categories")
|
||
if not front.get("tags"): reasons.append("Missing tags")
|
||
if not front.get("type"): reasons.append("Missing type")
|
||
if not front.get("source"): reasons.append("Missing source")
|
||
if len(body) < 100: reasons.append("Body too short (<100 chars)")
|
||
return len(reasons) == 0, reasons
|
||
```
|
||
|
||
---
|
||
|
||
## Phase 3 — 长期进阶（3-6 月）
|
||
|
||
**目标**: 图遍历搜索 + 高级自修正
|
||
|
||
### 任务 A-3: 图遍历搜索
|
||
|
||
#### 1.1 图存储层（轻量级起步）
|
||
|
||
从 Neo4j 方案改为 **Sqlite + JSON 混合**方案以降低复杂度：
|
||
|
||
```python
|
||
# tools/scripts/knowledge-graph.py
|
||
"""
|
||
维护 wiki/ 的 typed relationships 图。
|
||
存储：tools/data/knowledge-graph.json + Sqlite 索引
|
||
"""
|
||
|
||
GRAPH_DB = Path(r"D:\Applications\app\kepano-obsidian-main\tools\data\knowledge-graph.json")
|
||
|
||
def build_graph():
|
||
"""重建完整图（从所有 wiki 页面的 relations 字段）"""
|
||
graph = {"nodes": [], "edges": []}
|
||
seen_nodes = set()
|
||
for md_file in sorted(WIKI.glob("*.md")):
|
||
if md_file.name in ("index.md", "log.md"): continue
|
||
content = md_file.read_text(encoding="utf-8")
|
||
m = re.match(r"^---\n(.*?)\n---", content, re.DOTALL)
|
||
if not m: continue
|
||
try: front = yaml.safe_load(m.group(1)) or {}
|
||
except: continue
|
||
name = md_file.stem
|
||
if name not in seen_nodes:
|
||
graph["nodes"].append({
|
||
"id": name,
|
||
"type": front.get("type", "unknown"),
|
||
"confidence": front.get("confidence", 3)
|
||
})
|
||
seen_nodes.add(name)
|
||
for rel in front.get("relations", []):
|
||
target = rel.get("target", "").strip("[]")
|
||
if not target: continue
|
||
if target not in seen_nodes:
|
||
graph["nodes"].append({
|
||
"id": target,
|
||
"type": "unknown", # will be resolved
|
||
"confidence": 3
|
||
})
|
||
seen_nodes.add(target)
|
||
graph["edges"].append({
|
||
"source": name,
|
||
"target": target,
|
||
"type": rel.get("type", "related"),
|
||
"confidence": rel.get("confidence", 3)
|
||
})
|
||
GRAPH_DB.write_text(json.dumps(graph, ensure_ascii=False, indent=2), encoding="utf-8")
|
||
return graph
|
||
```
|
||
|
||
#### 1.2 图遍历 — 证据链查询
|
||
|
||
```python
|
||
def find_evidence_chain(target_page: str):
|
||
"""从 target 反向追溯所有 caused_by/supports 关系的源头"""
|
||
graph = json.loads(GRAPH_DB.read_text(encoding="utf-8"))
|
||
adj = {}
|
||
for e in graph["edges"]:
|
||
adj.setdefault(e["target"], []).append((e["source"], e["type"]))
|
||
|
||
chain = []
|
||
visited = set()
|
||
def dfs(node, depth=0):
|
||
if node in visited or depth > 5: return
|
||
visited.add(node)
|
||
if depth > 0:
|
||
chain.append((" " * depth) + f"← {node}")
|
||
for src, rel in adj.get(node, []):
|
||
dfs(src, depth + 1)
|
||
|
||
dfs(target_page)
|
||
return chain
|
||
```
|
||
|
||
#### 1.3 qmd 集成（graph-expansion 模块）
|
||
|
||
在 qmd 搜索结果后，用 graph 做扩展：
|
||
|
||
```python
|
||
def graph_expanded_search(query: str) -> list[dict]:
|
||
"""融合搜索：qmd BM25/向量结果 → graph 关系扩展 → 去重重排序"""
|
||
direct = qmd_search(query) # list of page names
|
||
graph = json.loads(GRAPH_DB.read_text(encoding="utf-8"))
|
||
|
||
# Build node -> adjacent nodes
|
||
adj = {}
|
||
for e in graph["edges"]:
|
||
adj.setdefault(e["source"], []).append((e["target"], e["type"], e["confidence"]))
|
||
adj.setdefault(e["target"], []).append((e["source"], "inverse_" + e["type"], e["confidence"]))
|
||
|
||
expanded = list(direct)
|
||
for page in direct:
|
||
for target, rel_type, conf in adj.get(page, []):
|
||
if target not in expanded and conf >= 3:
|
||
expanded.append(target)
|
||
|
||
return [{"page": p, "relevance": "direct" if p in direct else "graph"} for p in expanded]
|
||
```
|
||
|
||
---
|
||
|
||
### 任务 B-3: 高级自修正
|
||
|
||
#### 3.1 LLM 评审器
|
||
|
||
```python
|
||
# tools/scripts/review-pages.py
|
||
"""
|
||
使用本地 LLM（通过 Ollama/llama.cpp）审核 wiki 页面的内容准确性。
|
||
可检测：
|
||
- 声明与 raw 来源不匹配（数据偏差）
|
||
- 声明之间的矛盾（跨页面）
|
||
- 过时的信息（last_reviewed 太久）
|
||
|
||
用法：
|
||
python review-pages.py --stale # 审查过时页面
|
||
python review-pages.py --random 5 # 随机抽 5 页
|
||
"""
|
||
|
||
import subprocess, json, re, random
|
||
from pathlib import Path
|
||
|
||
OLLAMA_MODEL = "qwen2.5:7b" # 本地模型
|
||
|
||
def review_page(page_name: str) -> dict:
|
||
"""用 LLM 审核给定页面"""
|
||
fp = Path(r"D:\Applications\app\kepano-obsidian-main\wiki") / f"{page_name}.md"
|
||
content = fp.read_text(encoding="utf-8")
|
||
|
||
prompt = f"""请审核以下 wiki 页面，输出 JSON 格式的审核结果：
|
||
{{
|
||
"accuracy_score": 1-5,
|
||
"issues": ["问题1", "问题2"],
|
||
"confidence_match": true/false,
|
||
"suggested_updates": ["建议1"]
|
||
}}
|
||
|
||
页面内容：
|
||
{content[:4000]} # truncate to fit context
|
||
"""
|
||
result = subprocess.run(
|
||
["ollama", "run", OLLAMA_MODEL, prompt],
|
||
capture_output=True, text=True, timeout=120
|
||
)
|
||
try:
|
||
return json.loads(result.stdout)
|
||
except:
|
||
return {"accuracy_score": 3, "issues": ["Parse failed"], "confidence_match": False}
|
||
```
|
||
|
||
#### 3.2 矛盾自动标注
|
||
|
||
当检测到跨页面矛盾时，自动在双方页面添加 callout：
|
||
|
||
```markdown
|
||
> [!WARNING] 可能矛盾
|
||
> 本页声称「A = 3」，但 [[其他页面]] 声称「A = 4」。
|
||
> 来源对比：[[raw/source1]]:42 vs [[raw/source2]]:87
|
||
> 需要人工复核。
|
||
```
|
||
|
||
#### 3.3 低置信度自动建议
|
||
|
||
```python
|
||
def auto_suggest_low_confidence():
|
||
"""对 confidence <= 2 的页面，自动生成"需要更多证据"提示"""
|
||
for md_file in WIKI.glob("*.md"):
|
||
if md_file.name in ("index.md", "log.md"): continue
|
||
content = md_file.read_text(encoding="utf-8")
|
||
m = re.match(r"^---\n(.*?)\n---", content, re.DOTALL)
|
||
if not m: continue
|
||
try: front = yaml.safe_load(m.group(1)) or {}
|
||
except: continue
|
||
conf = front.get("confidence", 3)
|
||
if conf is not None and conf <= 2:
|
||
print(f"Suggestion: {md_file.stem} (confidence={conf}) — needs more evidence")
|
||
```
|
||
|
||
---
|
||
|
||
## 附录：文件清单
|
||
|
||
### 新建文件
|
||
|
||
| 文件 | 阶段 | 用途 |
|
||
|------|------|------|
|
||
| `tools/scripts/fix-raw-citations.py` | P0 | 行号自动标注 |
|
||
| `tools/scripts/manage-relations.py` | P1 | Typed relationships 管理 |
|
||
| `tools/scripts/graph-search.py` | P1 | 图感知搜索 |
|
||
| `tools/scripts/check-staleness.py` | P1 | 时效检测 |
|
||
| `tools/scripts/detect-conflicts.py` | P1 | 矛盾检测 |
|
||
| `tools/scripts/validate-frontmatter.py` | P2 | Pre-commit frontmatter 验证 |
|
||
| `tools/scripts/weekly-lint.ps1` | P2 | 定时 Lint 入口 |
|
||
| `tools/scripts/promote-knowledge.py` | P2 | 层级提升 |
|
||
| `tools/scripts/knowledge-graph.py` | P3 | 图存储和遍历 |
|
||
| `tools/scripts/review-pages.py` | P3 | LLM 评审器 |
|
||
| `.githooks/pre-commit` | P2 | Git hook |
|
||
| `.githooks/post-merge` | P2 | Git hook |
|
||
| `tools/data/knowledge-graph.json` | P3 | 图数据库文件 |
|
||
|
||
### 修改文件
|
||
|
||
| 文件 | 阶段 | 变更 |
|
||
|------|------|------|
|
||
| `AGENTS.md` | P0/P1 | 修复冗余 + 新增 v2 特性定义 + 更新模板 |
|
||
| `wiki/index.md` | P0 | 更新指标数字 |
|
||
| `wiki/log.md` | P0/P1/P2 | 追加每次操作日志 |
|
||
|
||
### 依赖安装
|
||
|
||
```bash
|
||
pip install pyyaml # 已安装，确认版本
|
||
# Phase 3 可能需要：
|
||
pip install networkx # 轻量图分析（替代 Neo4j）
|
||
```
|