attune-system/attune

Fork 0

Files

David Culbreth 3b14c65998 re-uploading work

2026-02-04 17:46:30 -06:00

8.5 KiB

Raw Blame History

Session: Documentation Reorganization and AGENTS.md Index - 2026-01-30

Objectives

Create a Python script to generate AGENTS.md index file in minified format
Organize existing documentation into logical subdirectories
Improve documentation discoverability for both humans and AI agents

Work Completed

1. AGENTS.md Index Generation Script

Created scripts/generate_agents_md_index.py with the following features:

Scans docs/, scripts/, and work-summary/ directories
Generates minified index format inspired by Vercel's agent evaluation research
Configurable file extensions per directory type
Automatic file truncation with ... indicator for long lists
Alphabetical sorting for consistency
Robust error handling for missing directories

Configuration:

docs/: .md, .txt, .yaml, .yml, .json, .sh (max 15 files per dir)
scripts/: .sh, .py, .sql, .js, .html (max 20 files per dir)
work-summary/: .md, .txt (max 20 files per dir)

Template System:

Uses AGENTS.md.template as base (contains project rules and guidelines)
Injects generated index at {{DOCUMENTATION_INDEX}} placeholder
Final output: AGENTS.md (446 lines: 408 template + 38 index)

Integration:

Added make generate-agents-index command to Makefile
Created documentation at docs/development/agents-md-index.md

2. Documentation Reorganization

docs/ Structure (16 subdirectories)

Organized 102 documentation files into logical categories:

docs/api/ (14 files) - API endpoint documentation, OpenAPI specs
docs/architecture/ (9 files) - Service architecture, system design
docs/authentication/ (6 files) - Auth, security, secrets management
docs/cli/ (2 files) - CLI documentation
docs/configuration/ (4 files) - Configuration guides
docs/dependencies/ (9 files) - Dependency management, migrations
docs/deployment/ (2 files) - Production deployment, operations
docs/development/ (5 files) - Developer guides, tooling
docs/examples/ (5 files) - Example configs and workflows
docs/guides/ (5 files) - Getting started, quickstarts
docs/migrations/ (1 file) - Database migration docs
docs/packs/ (7 files) - Pack system documentation
docs/performance/ (5 files) - Performance optimization
docs/plans/ (1 file) - Future planning
docs/sensors/ (6 files) - Sensor system documentation
docs/testing/ (7 files) - Testing strategies and guides
docs/web-ui/ (1 file) - Web UI documentation
docs/webhooks/ (2 files) - Webhook system
docs/workflows/ (10 files) - Workflow engine documentation

work-summary/ Structure (7 subdirectories)

Organized 213 work summary files into logical categories:

work-summary/status/ (9 files) - Current status, TODO lists
work-summary/phases/ (28 files) - Phase completions, planning, analysis
work-summary/sessions/ (155 files) - Daily development session notes
work-summary/features/ (6 files) - Feature implementation summaries
work-summary/migrations/ (7 files) - Migration and refactoring summaries
work-summary/changelogs/ (11 files) - Changelogs, completion summaries

Created comprehensive navigation guides:

docs/README.md - Main documentation entry point with:
- Quick navigation by role (new developer, pack builder, etc.)
- Directory structure overview
- Search tips and best practices
- Contributing guidelines
work-summary/README.md - Work summary navigation with:
- Directory structure explanation
- Search strategies by time period and topic
- Writing guidelines for session notes
- Best practices
docs/development/documentation-organization.md - Detailed guide covering:
- Purpose and contents of each subdirectory
- When to use each category
- Maintenance guidelines
- Reorganization history and rationale

Decisions Made

Committed AGENTS.md instead of gitignoring it: Since it's useful for AI agents, small in size, and changes infrequently
Template-based generation: Used AGENTS.md.template as base to combine project rules with generated documentation index in a single file
Subdirectory structure: Chose fine-grained categorization (16 subdirs for docs) over coarse-grained to improve discoverability
work-summary organization: Separated by type (status, phases, sessions, features) rather than by date to make it easier to find specific information
Phase summaries location: Moved from docs/ to work-summary/phases/ since they're historical work summaries rather than technical documentation
File truncation limits: Set conservative limits (15-20 files) for AGENTS.md index to keep it concise while showing enough context

Technical Details

Template System

AGENTS.md.template (408 lines)
    ├── Project rules and guidelines
    ├── Development conventions
    ├── Code quality standards
    └── {{DOCUMENTATION_INDEX}} placeholder

          ↓ (script generates and injects)

AGENTS.md (446 lines)
    ├── All template content
    └── Generated documentation index (38 lines)

Generated Index Format

[Attune Project Documentation Index]
|root: ./
|IMPORTANT: Prefer retrieval-led reasoning over pre-training-led reasoning
|
|docs/api:{api-actions.md,api-completion-plan.md,...}
|docs/architecture:{executor-service.md,notifier-service.md,...}
|work-summary/sessions:{2024-01-13-event-enforcement-endpoints.md,...}

Index lines: 38 (16 for docs subdirs, 7 for work-summary subdirs, metadata)

File Statistics

docs/: 102 files organized into 16 categories
work-summary/: 213 files organized into 7 categories
Total organized: 315 files

Benefits

Improved Discoverability: Related documentation now grouped together
Better Navigation: README files provide clear entry points
AI Agent Optimization: Minified AGENTS.md index enables faster context loading
Cleaner Root Directories: No more cluttered docs/ and work-summary/ roots
Easier Maintenance: Clear conventions for where new docs should go
Better Onboarding: New developers can quickly find relevant documentation

Files Created

scripts/generate_agents_md_index.py - Index generation script with template support
AGENTS.md - Generated file combining template + index (446 lines)
docs/README.md - Documentation navigation guide
docs/development/agents-md-index.md - AGENTS.md feature documentation (updated for template)
docs/development/documentation-organization.md - Organization guide
work-summary/README.md - Work summary navigation guide

Files Modified

Makefile - Added generate-agents-index target
AGENTS.md.template - Added {{DOCUMENTATION_INDEX}} placeholder
All documentation files moved to subdirectories (preserved via git mv for history)

Testing

✅ Script generates valid AGENTS.md output
✅ Handles edge cases (empty directories, missing paths, large file lists)
✅ File truncation works correctly
✅ Alphabetical sorting verified
✅ Make target integrates successfully
✅ All files successfully organized into subdirectories
✅ No files left in root (except README.md in each directory)

Next Steps

Always edit AGENTS.md.template for project rules changes, never AGENTS.md directly
Run make generate-agents-index after adding new documentation or changing template
Follow the guidelines in docs/development/documentation-organization.md for future docs
Consider adding pre-commit hook to auto-regenerate AGENTS.md when docs or template change
Update any hardcoded documentation paths in code or other docs if needed

Vercel Blog: AGENTS.md outperforms .md skills in agent evals
Documentation: docs/development/agents-md-index.md
Organization guide: docs/development/documentation-organization.md

Notes

Template-based approach combines project rules with dynamic index in single file
The minified index format significantly reduces token usage for AI agents scanning the project
Directory organization follows common documentation patterns (guides, api, architecture, etc.)
Session files remain dated for chronological tracking while other categories use thematic organization
README files serve as navigation hubs for both directories
AGENTS.md is generated - always edit AGENTS.md.template instead

Session completed: 2026-01-30 Total files organized: 315 New subdirectories: 23

8.5 KiB Raw Blame History