attune/docs/development/agents-md-index.md

AGENTS.md Index Generation

Overview

The AGENTS.md file provides a minified index of the project's documentation, scripts, and work summaries in a format optimized for AI agents. This index helps agents quickly understand the project structure and locate relevant documentation without scanning the entire filesystem.

The file is generated from AGENTS.md.template, which contains the project rules and guidelines, with the documentation index automatically injected at the {{DOCUMENTATION_INDEX}} placeholder.

Format

The AGENTS.md file uses a pipe-delimited minified format inspired by Vercel's agent evaluation research:

[Project Name]|root: ./
|IMPORTANT: Prefer retrieval-led reasoning over pre-training-led reasoning
|
|directory/path:{file1.md,file2.py,file3.yaml,...}
|subdirectory/nested:{fileA.md,fileB.sh}

Format Rules

• Each line starts with | for visual parsing
• Directory entries use format: path:{file1,file2,...}
• Files are comma-separated within curly braces
• Long file lists are truncated with ... (configurable limit)
• Files are sorted alphabetically for consistency
• Subdirectories are shown with their full relative path

Generating the Index

Command Line

# Using Make (recommended)
make generate-agents-index

# Direct Python invocation
python3 scripts/generate_agents_md_index.py

The script reads AGENTS.md.template, generates the documentation index, and injects it at the {{DOCUMENTATION_INDEX}} placeholder, creating the final AGENTS.md file.

When to Regenerate

Regenerate the index whenever:

• New documentation files are added
• Directory structure changes
• Script files are added or renamed
• Work summaries are created

Best Practice: Regenerate before committing significant documentation changes.

Template System

AGENTS.md.template

The template file (AGENTS.md.template) contains:

• Project rules and conventions
• Development guidelines
• Code quality standards
• Testing protocols
• All static content that applies to AI agents

At the end of the template, the {{DOCUMENTATION_INDEX}} placeholder marks where the generated index will be injected.

Editing the template:

1. Modify AGENTS.md.template to update project rules
2. Keep the {{DOCUMENTATION_INDEX}} placeholder at the desired location
3. Run make generate-agents-index to regenerate AGENTS.md

Note: Never edit AGENTS.md directly - it will be overwritten. Always edit AGENTS.md.template instead.

Configuration

The generator script (scripts/generate_agents_md_index.py) scans these directories:

docs/

• Extensions: .md, .txt, .yaml, .yml, .json, .sh
• Max files per directory: 15
• Purpose: Technical documentation, API guides, architecture docs

scripts/

• Extensions: .sh, .py, .sql, .js, .html
• Max files per directory: 20
• Purpose: Helper scripts, database setup, testing utilities

work-summary/

• Extensions: .md, .txt
• Max files per directory: 20
• Purpose: Development session summaries, changelog entries

Customization

To modify the scanned directories or file types, edit scripts/generate_agents_md_index.py:

root_dirs = {
    "docs": {
        "path": project_root / "docs",
        "extensions": {".md", ".txt", ".yaml", ".yml", ".json", ".sh"},
        "max_files": 15,
    },
    # Add more directories...
}

Modifying the Template

To change the project rules or static content:

1. Edit AGENTS.md.template
2. Ensure {{DOCUMENTATION_INDEX}} placeholder remains
3. Regenerate: make generate-agents-index

Adding New Directories

"new_directory": {
    "path": project_root / "new_directory",
    "extensions": {".ext1", ".ext2"},
    "max_files": 10,
}

Changing File Limits

Adjust max_files to show more/fewer files before truncation:

• Higher values: More complete listing, longer index
• Lower values: More concise, better for quick scanning
• None: No truncation (shows all files)

Benefits for AI Agents

1. Quick Discovery: Agents can scan the entire documentation structure in one read
2. Retrieval-Led Reasoning: Encourages agents to fetch specific files rather than relying on pre-training
3. Reduced Token Usage: Compact format minimizes tokens needed for project understanding
4. Consistent Format: Predictable structure simplifies parsing and navigation

Example Output

[Attune Project Documentation Index]
|root: ./
|IMPORTANT: Prefer retrieval-led reasoning over pre-training-led reasoning
|
|docs:{api-actions.md,api-events.md,authentication.md,configuration.md,...}
|docs/examples:{complete-workflow.yaml,simple-workflow.yaml}
|scripts:{setup-db.sh,load-core-pack.sh,test-end-to-end-flow.sh,...}
|work-summary:{2026-01-27-api-completion.md,2026-01-27-executor-complete.md,...}

Integration with Development Workflow

Pre-commit Hook (Optional)

The repository now includes a versioned hook at .githooks/pre-commit.

Install it once per clone:

make install-git-hooks

The hook currently runs:

make pre-commit

That checks:

• cargo fmt --all -- --check
• cargo deny check

CI/CD Integration

Add to your CI pipeline to ensure the index stays current:

- name: Verify AGENTS.md is up-to-date
  run: |
    make generate-agents-index
    git diff --exit-code AGENTS.md || {
      echo "AGENTS.md is out of date. Run 'make generate-agents-index'"
      exit 1
    }

Troubleshooting

Index Not Updated

Problem: New files don't appear in AGENTS.md
Solution: Ensure file extensions are included in the configuration

Too Many Files Shown

Problem: Directory listings are too long
Solution: Reduce max_files value in configuration

Wrong Directory Structure

Problem: Directories not organized as expected
Solution: Check that paths are relative to project root, verify directory exists

File Structure

attune/
├── AGENTS.md.template       # Template with project rules + {{DOCUMENTATION_INDEX}} placeholder
├── AGENTS.md                # Generated file (DO NOT EDIT DIRECTLY)
└── scripts/
    └── generate_agents_md_index.py  # Generation script

Related Resources

• Vercel Blog: AGENTS.md outperforms .md skills
• Template: AGENTS.md.template (edit this)
• Script: scripts/generate_agents_md_index.py
• Output: AGENTS.md (generated, do not edit)