193 lines
8.5 KiB
Markdown
193 lines
8.5 KiB
Markdown
# Session: Documentation Reorganization and AGENTS.md Index - 2026-01-30
|
|
|
|
## Objectives
|
|
- Create a Python script to generate AGENTS.md index file in minified format
|
|
- Organize existing documentation into logical subdirectories
|
|
- Improve documentation discoverability for both humans and AI agents
|
|
|
|
## Work Completed
|
|
|
|
### 1. AGENTS.md Index Generation Script
|
|
Created `scripts/generate_agents_md_index.py` with the following features:
|
|
- Scans `docs/`, `scripts/`, and `work-summary/` directories
|
|
- Generates minified index format inspired by Vercel's agent evaluation research
|
|
- Configurable file extensions per directory type
|
|
- Automatic file truncation with `...` indicator for long lists
|
|
- Alphabetical sorting for consistency
|
|
- Robust error handling for missing directories
|
|
|
|
**Configuration:**
|
|
- `docs/`: .md, .txt, .yaml, .yml, .json, .sh (max 15 files per dir)
|
|
- `scripts/`: .sh, .py, .sql, .js, .html (max 20 files per dir)
|
|
- `work-summary/`: .md, .txt (max 20 files per dir)
|
|
|
|
**Template System:**
|
|
- Uses `AGENTS.md.template` as base (contains project rules and guidelines)
|
|
- Injects generated index at `{{DOCUMENTATION_INDEX}}` placeholder
|
|
- Final output: `AGENTS.md` (446 lines: 408 template + 38 index)
|
|
|
|
**Integration:**
|
|
- Added `make generate-agents-index` command to Makefile
|
|
- Created documentation at `docs/development/agents-md-index.md`
|
|
|
|
### 2. Documentation Reorganization
|
|
|
|
#### docs/ Structure (16 subdirectories)
|
|
Organized 102 documentation files into logical categories:
|
|
|
|
- **`docs/api/`** (14 files) - API endpoint documentation, OpenAPI specs
|
|
- **`docs/architecture/`** (9 files) - Service architecture, system design
|
|
- **`docs/authentication/`** (6 files) - Auth, security, secrets management
|
|
- **`docs/cli/`** (2 files) - CLI documentation
|
|
- **`docs/configuration/`** (4 files) - Configuration guides
|
|
- **`docs/dependencies/`** (9 files) - Dependency management, migrations
|
|
- **`docs/deployment/`** (2 files) - Production deployment, operations
|
|
- **`docs/development/`** (5 files) - Developer guides, tooling
|
|
- **`docs/examples/`** (5 files) - Example configs and workflows
|
|
- **`docs/guides/`** (5 files) - Getting started, quickstarts
|
|
- **`docs/migrations/`** (1 file) - Database migration docs
|
|
- **`docs/packs/`** (7 files) - Pack system documentation
|
|
- **`docs/performance/`** (5 files) - Performance optimization
|
|
- **`docs/plans/`** (1 file) - Future planning
|
|
- **`docs/sensors/`** (6 files) - Sensor system documentation
|
|
- **`docs/testing/`** (7 files) - Testing strategies and guides
|
|
- **`docs/web-ui/`** (1 file) - Web UI documentation
|
|
- **`docs/webhooks/`** (2 files) - Webhook system
|
|
- **`docs/workflows/`** (10 files) - Workflow engine documentation
|
|
|
|
#### work-summary/ Structure (7 subdirectories)
|
|
Organized 213 work summary files into logical categories:
|
|
|
|
- **`work-summary/status/`** (9 files) - Current status, TODO lists
|
|
- **`work-summary/phases/`** (28 files) - Phase completions, planning, analysis
|
|
- **`work-summary/sessions/`** (155 files) - Daily development session notes
|
|
- **`work-summary/features/`** (6 files) - Feature implementation summaries
|
|
- **`work-summary/migrations/`** (7 files) - Migration and refactoring summaries
|
|
- **`work-summary/changelogs/`** (11 files) - Changelogs, completion summaries
|
|
|
|
### 3. Navigation Documentation
|
|
Created comprehensive navigation guides:
|
|
|
|
- **`docs/README.md`** - Main documentation entry point with:
|
|
- Quick navigation by role (new developer, pack builder, etc.)
|
|
- Directory structure overview
|
|
- Search tips and best practices
|
|
- Contributing guidelines
|
|
|
|
- **`work-summary/README.md`** - Work summary navigation with:
|
|
- Directory structure explanation
|
|
- Search strategies by time period and topic
|
|
- Writing guidelines for session notes
|
|
- Best practices
|
|
|
|
- **`docs/development/documentation-organization.md`** - Detailed guide covering:
|
|
- Purpose and contents of each subdirectory
|
|
- When to use each category
|
|
- Maintenance guidelines
|
|
- Reorganization history and rationale
|
|
|
|
## Decisions Made
|
|
|
|
1. **Committed AGENTS.md instead of gitignoring it**: Since it's useful for AI agents, small in size, and changes infrequently
|
|
|
|
2. **Template-based generation**: Used `AGENTS.md.template` as base to combine project rules with generated documentation index in a single file
|
|
|
|
3. **Subdirectory structure**: Chose fine-grained categorization (16 subdirs for docs) over coarse-grained to improve discoverability
|
|
|
|
4. **work-summary organization**: Separated by type (status, phases, sessions, features) rather than by date to make it easier to find specific information
|
|
|
|
5. **Phase summaries location**: Moved from `docs/` to `work-summary/phases/` since they're historical work summaries rather than technical documentation
|
|
|
|
6. **File truncation limits**: Set conservative limits (15-20 files) for AGENTS.md index to keep it concise while showing enough context
|
|
|
|
## Technical Details
|
|
|
|
### Template System
|
|
```
|
|
AGENTS.md.template (408 lines)
|
|
├── Project rules and guidelines
|
|
├── Development conventions
|
|
├── Code quality standards
|
|
└── {{DOCUMENTATION_INDEX}} placeholder
|
|
|
|
↓ (script generates and injects)
|
|
|
|
AGENTS.md (446 lines)
|
|
├── All template content
|
|
└── Generated documentation index (38 lines)
|
|
```
|
|
|
|
### Generated Index Format
|
|
```
|
|
[Attune Project Documentation Index]
|
|
|root: ./
|
|
|IMPORTANT: Prefer retrieval-led reasoning over pre-training-led reasoning
|
|
|
|
|
|docs/api:{api-actions.md,api-completion-plan.md,...}
|
|
|docs/architecture:{executor-service.md,notifier-service.md,...}
|
|
|work-summary/sessions:{2024-01-13-event-enforcement-endpoints.md,...}
|
|
```
|
|
|
|
Index lines: 38 (16 for docs subdirs, 7 for work-summary subdirs, metadata)
|
|
|
|
### File Statistics
|
|
- **docs/**: 102 files organized into 16 categories
|
|
- **work-summary/**: 213 files organized into 7 categories
|
|
- **Total organized**: 315 files
|
|
|
|
## Benefits
|
|
|
|
1. **Improved Discoverability**: Related documentation now grouped together
|
|
2. **Better Navigation**: README files provide clear entry points
|
|
3. **AI Agent Optimization**: Minified AGENTS.md index enables faster context loading
|
|
4. **Cleaner Root Directories**: No more cluttered docs/ and work-summary/ roots
|
|
5. **Easier Maintenance**: Clear conventions for where new docs should go
|
|
6. **Better Onboarding**: New developers can quickly find relevant documentation
|
|
|
|
## Files Created
|
|
- `scripts/generate_agents_md_index.py` - Index generation script with template support
|
|
- `AGENTS.md` - Generated file combining template + index (446 lines)
|
|
- `docs/README.md` - Documentation navigation guide
|
|
- `docs/development/agents-md-index.md` - AGENTS.md feature documentation (updated for template)
|
|
- `docs/development/documentation-organization.md` - Organization guide
|
|
- `work-summary/README.md` - Work summary navigation guide
|
|
|
|
## Files Modified
|
|
- `Makefile` - Added `generate-agents-index` target
|
|
- `AGENTS.md.template` - Added `{{DOCUMENTATION_INDEX}}` placeholder
|
|
- All documentation files moved to subdirectories (preserved via `git mv` for history)
|
|
|
|
## Testing
|
|
- ✅ Script generates valid AGENTS.md output
|
|
- ✅ Handles edge cases (empty directories, missing paths, large file lists)
|
|
- ✅ File truncation works correctly
|
|
- ✅ Alphabetical sorting verified
|
|
- ✅ Make target integrates successfully
|
|
- ✅ All files successfully organized into subdirectories
|
|
- ✅ No files left in root (except README.md in each directory)
|
|
|
|
## Next Steps
|
|
1. **Always edit** `AGENTS.md.template` for project rules changes, never `AGENTS.md` directly
|
|
2. Run `make generate-agents-index` after adding new documentation or changing template
|
|
3. Follow the guidelines in `docs/development/documentation-organization.md` for future docs
|
|
4. Consider adding pre-commit hook to auto-regenerate AGENTS.md when docs or template change
|
|
5. Update any hardcoded documentation paths in code or other docs if needed
|
|
|
|
## Related
|
|
- Vercel Blog: [AGENTS.md outperforms .md skills in agent evals](https://vercel.com/blog/agents-md-outperforms-skills-in-our-agent-evals)
|
|
- Documentation: `docs/development/agents-md-index.md`
|
|
- Organization guide: `docs/development/documentation-organization.md`
|
|
|
|
## Notes
|
|
- Template-based approach combines project rules with dynamic index in single file
|
|
- The minified index format significantly reduces token usage for AI agents scanning the project
|
|
- Directory organization follows common documentation patterns (guides, api, architecture, etc.)
|
|
- Session files remain dated for chronological tracking while other categories use thematic organization
|
|
- README files serve as navigation hubs for both directories
|
|
- `AGENTS.md` is generated - always edit `AGENTS.md.template` instead
|
|
|
|
---
|
|
|
|
*Session completed: 2026-01-30*
|
|
*Total files organized: 315*
|
|
*New subdirectories: 23* |