Skill Judge

# Skill Judge A comprehensive evaluation framework for assessing Agent Skill quality against official specifications and best practices. This skill provides multi-dimensional scoring and actionable improvement suggestions for SKILL.md files and skill packages. ## Purpose Skill Judge exists to solve a critical problem: **most Skills waste tokens on knowledge Claude already has**. The skill helps you evaluate whether a Skill actually adds value by measuring its "knowledge delta" - the gap between what the Skill provides and what Claude already knows. A good Skill should be a compressed expert brain, not a tutorial. ### The Core Formula > **Good Skill = Expert-only Knowledge - What Claude Already Knows** This skill helps you identify: - Token-wasting redundant content (things Claude already knows) - Genuine expert knowledge that adds value - Structural issues that prevent Skills from being activated or used effectively ## When to Use Use Skill Judge when you need to: - **Review a Skill before publishing**: Evaluate quality and identify improvements - **Audit existing Skills**: Systematic assessment against best practices - **Improve a SKILL.md file**: Get specific, actionable suggestions - **Learn Skill design patterns**: Understand what makes a great Skill - **Compare Skills**: Assess relative quality using consistent criteria **Trigger phrases**: - "Evaluate this skill" - "Review my SKILL.md" - "Audit this skill" - "Score this skill" - "How can I improve this skill?" - "Is this skill well-designed?" ## How It Works ### Evaluation Protocol 1. **First Pass - Knowledge Delta Scan**: Read the SKILL.md and categorize each section as: - **[E] Expert**: Claude genuinely doesn't know this (value-add) - **[A] Activation**: Claude knows but brief reminder is useful (acceptable) - **[R] Redundant**: Claude definitely knows this (should delete) 2. **Structure Analysis**: Check frontmatter validity, line count, reference files, design pattern, and loading triggers 3. **Score Each Dimension**: Evaluate against 8 dimensions with specific evidence and justifications 4. **Calculate Total and Grade**: Sum scores (max 120 points) and assign grade 5. **Generate Report**: Produce structured report with scores, critical issues, and improvements ### The 8 Evaluation Dimensions (120 points total) | Dimension | Max Points | What It Measures | |-----------|------------|------------------| | **D1: Knowledge Delta** | 20 | Does the Skill add genuine expert knowledge? (THE CORE DIMENSION) | | **D2: Mindset + Procedures** | 15 | Does it transfer expert thinking patterns and domain-specific workflows? | | **D3: Anti-Pattern Quality** | 15 | Does it have effective NEVER lists with specific reasons? | | **D4: Specification Compliance** | 15 | Is the frontmatter valid? Is the description comprehensive? | | **D5: Progressive Disclosure** | 15 | Is content properly layered for on-demand loading? | | **D6: Freedom Calibration** | 15 | Is specificity appropriate for task fragility? | | **D7: Pattern Recognition** | 10 | Does it follow an established official pattern? | | **D8: Practical Usability** | 15 | Can an Agent actually use this Skill effectively? | ### Grading Scale | Grade | Percentage | Meaning | |-------|------------|---------| | A | 90%+ (108+) | Excellent - production-ready expert Skill | | B | 80-89% (96-107) | Good - minor improvements needed | | C | 70-79% (84-95) | Adequate - clear improvement path | | D | 60-69% (72-83) | Below Average - significant issues | | F | <60% ( "Would an expert in this domain, looking at this Skill, say: 'Yes, this captures knowledge that took me years to learn'?" If yes, the Skill has genuine value. If no, it's compressing what Claude already knows. ## Quick Reference Checklist ``` SKILL EVALUATION QUICK CHECK KNOWLEDGE DELTA (most important): [ ] No "What is X" explanations for basic concepts [ ] No step-by-step tutorials for standard operations [ ] Has decision trees for non-obvious choices [ ] Has trade-offs only experts would know [ ] Has edge cases from real-world experience MINDSET + PROCEDURES: [ ] Transfers thinking patterns (how to think about problems) [ ] Has "Before doing X, ask yourself..." frameworks [ ] Includes domain-specific procedures Claude wouldn't know ANTI-PATTERNS: [ ] Has explicit NEVER list [ ] Anti-patterns are specific, not vague [ ] Includes WHY (non-obvious reasons) SPECIFICATION: [ ] Valid YAML frontmatter [ ] Description answers: WHAT, WHEN, KEYWORDS [ ] Description specific enough for Agent activation STRUCTURE: [ ] SKILL.md < 500 lines (ideal High freedom (principles) [ ] Fragile operations -> Low freedom (exact scripts) USABILITY: [ ] Decision trees for multi-path scenarios [ ] Working code examples [ ] Error handling and fallbacks ``` ## Prerequisites None. Skill Judge is self-contained and requires no external tools or dependencies. ## Related Concepts - **Tool vs Skill**: Tools define capability boundaries (what Claude CAN do). Skills inject knowledge (what Claude KNOWS how to do). - **Progressive Disclosure**: Three-layer loading system (metadata -> SKILL.md body -> resources) - **Freedom Calibration**: Matching constraint level to task fragility