mirror of
https://github.com/CherryHQ/cherry-studio.git
synced 2025-12-28 05:11:24 +08:00
📝 docs: update validation and refactoring documentation for agents service
This commit is contained in:
Vaayne
parent
e8c94f3584
commit
a4bb82a02d
@ -1,9 +1,11 @@
|
||||
# Agents Service Refactoring - Validation Report
|
||||
|
||||
## Overview
|
||||
|
||||
This report documents the comprehensive validation of the agents service refactoring completed on September 12, 2025. All tests were performed to ensure the refactored system maintains full functionality while providing improved structure and maintainability.
|
||||
|
||||
## Validation Summary
|
||||
|
||||
✅ **ALL VALIDATIONS PASSED** - The refactoring has been successfully completed and verified.
|
||||
|
||||
---
|
||||
@ -11,9 +13,11 @@ This report documents the comprehensive validation of the agents service refacto
|
||||
## 1. Build and Compilation Validation
|
||||
|
||||
### Command: `yarn build:check`
|
||||
|
||||
**Status:** ✅ PASSED
|
||||
|
||||
**Results:**
|
||||
|
||||
- TypeScript compilation for Node.js environment: ✅ PASSED
|
||||
- TypeScript compilation for Web environment: ✅ PASSED
|
||||
- i18n validation: ✅ PASSED
|
||||
@ -22,6 +26,7 @@ This report documents the comprehensive validation of the agents service refacto
|
||||
**Duration:** 23.12s
|
||||
|
||||
### Key Findings:
|
||||
|
||||
- All TypeScript files compile without errors
|
||||
- No type definition conflicts detected
|
||||
- Import/export structure is correctly maintained
|
||||
@ -32,9 +37,11 @@ This report documents the comprehensive validation of the agents service refacto
|
||||
## 2. Migration System Validation
|
||||
|
||||
### Custom Migration Test
|
||||
|
||||
**Status:** ✅ PASSED
|
||||
|
||||
**Test Coverage:**
|
||||
|
||||
1. ✅ Migration tracking table creation
|
||||
2. ✅ Migration indexes creation
|
||||
3. ✅ Migration record insertion/retrieval
|
||||
@ -47,6 +54,7 @@ This report documents the comprehensive validation of the agents service refacto
|
||||
10. ✅ Migration cleanup
|
||||
|
||||
### Key Findings:
|
||||
|
||||
- Migration system initializes correctly
|
||||
- All migration tables and indexes are created properly
|
||||
- Transaction support works as expected
|
||||
@ -58,9 +66,11 @@ This report documents the comprehensive validation of the agents service refacto
|
||||
## 3. Service Initialization Validation
|
||||
|
||||
### Custom Service Structure Test
|
||||
|
||||
**Status:** ✅ PASSED
|
||||
|
||||
**Validated Components:**
|
||||
|
||||
1. ✅ All service files are present and accessible
|
||||
2. ✅ Migration files are properly organized
|
||||
3. ✅ Query files are correctly structured
|
||||
@ -71,6 +81,7 @@ This report documents the comprehensive validation of the agents service refacto
|
||||
8. ✅ TypeScript compilation validated
|
||||
|
||||
### File Structure Verification:
|
||||
|
||||
```
|
||||
src/main/services/agents/
|
||||
├── ✅ BaseService.ts
|
||||
@ -105,9 +116,11 @@ src/main/services/agents/
|
||||
## 4. Database Operations Validation
|
||||
|
||||
### Comprehensive CRUD Operations Test
|
||||
|
||||
**Status:** ✅ PASSED
|
||||
|
||||
**Test Scenarios:**
|
||||
|
||||
1. ✅ Database schema setup (tables + indexes)
|
||||
2. ✅ Agent CRUD operations
|
||||
- Create: ✅ Agent creation with JSON field serialization
|
||||
@ -130,6 +143,7 @@ src/main/services/agents/
|
||||
- Data Integrity: ✅ All concurrent operations verified
|
||||
|
||||
### Performance Metrics:
|
||||
|
||||
- Agent CRUD operations: < 50ms per operation
|
||||
- Migration system: < 100ms initialization
|
||||
- Concurrent operations: Successfully handled 5 parallel operations
|
||||
@ -139,6 +153,7 @@ src/main/services/agents/
|
||||
## 5. Backward Compatibility Validation
|
||||
|
||||
### Compatibility Checks:
|
||||
|
||||
- ✅ Export structure maintains backward compatibility
|
||||
- ✅ Legacy query exports available via `AgentQueries_Legacy`
|
||||
- ✅ Service singleton instances preserved
|
||||
@ -150,6 +165,7 @@ src/main/services/agents/
|
||||
## 6. Code Quality and Structure
|
||||
|
||||
### Improvements Delivered:
|
||||
|
||||
1. **Modular Organization**: ✅ Services split into focused, single-responsibility files
|
||||
2. **Migration System**: ✅ Version-controlled schema changes with rollback support
|
||||
3. **Query Organization**: ✅ SQL queries organized by entity type
|
||||
@ -159,6 +175,7 @@ src/main/services/agents/
|
||||
7. **Testing**: ✅ All existing tests continue to pass
|
||||
|
||||
### Benefits Realized:
|
||||
|
||||
- **Maintainability**: Easier to locate and modify specific functionality
|
||||
- **Scalability**: Simple to add new entities without affecting existing code
|
||||
- **Production Readiness**: Atomic migrations with transaction support
|
||||
@ -170,6 +187,7 @@ src/main/services/agents/
|
||||
## 7. Security and Safety Validation
|
||||
|
||||
### Security Measures Verified:
|
||||
|
||||
- ✅ SQL injection protection via parameterized queries
|
||||
- ✅ Transaction isolation for atomic operations
|
||||
- ✅ Foreign key constraints prevent orphaned records
|
||||
@ -181,6 +199,7 @@ src/main/services/agents/
|
||||
## 8. Performance Validation
|
||||
|
||||
### Database Operations:
|
||||
|
||||
- ✅ Index utilization verified for common queries
|
||||
- ✅ Foreign key constraints optimized with indexes
|
||||
- ✅ JSON field operations efficient
|
||||
@ -191,6 +210,7 @@ src/main/services/agents/
|
||||
## Cleanup
|
||||
|
||||
The following temporary test files were created for validation and can be safely removed:
|
||||
|
||||
- `/Users/weliu/workspace/cherry-studio/migration-validation-test.js`
|
||||
- `/Users/weliu/workspace/cherry-studio/service-initialization-test.js`
|
||||
- `/Users/weliu/workspace/cherry-studio/database-operations-test.js`
|
||||
@ -215,4 +235,4 @@ The agents service refactoring has been successfully completed and thoroughly va
|
||||
**Validation completed:** September 12, 2025
|
||||
**Total validation time:** ~45 minutes
|
||||
**Tests executed:** 1420 + custom validation tests
|
||||
**Overall result:** ✅ SUCCESS
|
||||
**Overall result:** ✅ SUCCESS
|
||||
|
||||
@ -1,9 +1,11 @@
|
||||
# Agents Service Refactoring Plan
|
||||
|
||||
## Overview
|
||||
|
||||
Restructure the agents service to split database operations into smaller, more manageable files with migration support.
|
||||
|
||||
## New Folder Structure
|
||||
|
||||
```
|
||||
src/main/services/agents/
|
||||
├── database/
|
||||
@ -36,11 +38,13 @@ src/main/services/agents/
|
||||
## Implementation Tasks
|
||||
|
||||
### Task 1: Create Folder Structure and Migration System Infrastructure
|
||||
|
||||
**Status**: ✅ COMPLETED
|
||||
**Agent**: `general-purpose`
|
||||
**Description**: Create all necessary directories and implement the migration system infrastructure
|
||||
|
||||
**Subtasks**:
|
||||
|
||||
- [x] Create database/, database/migrations/, database/queries/, database/schema/, services/ directories
|
||||
- [x] Implement migration types and interfaces in database/migrations/types.ts
|
||||
- [x] Build Migrator class with transaction support in database/migrator.ts
|
||||
@ -49,11 +53,13 @@ src/main/services/agents/
|
||||
---
|
||||
|
||||
### Task 2: Split Database Queries from db.ts
|
||||
|
||||
**Status**: ✅ COMPLETED
|
||||
**Agent**: `general-purpose`
|
||||
**Description**: Extract and organize queries from the current db.ts file into separate, focused files
|
||||
|
||||
**Subtasks**:
|
||||
|
||||
- [x] Move agent queries to database/queries/agent.queries.ts
|
||||
- [x] Move session queries to database/queries/session.queries.ts
|
||||
- [x] Move session log queries to database/queries/sessionLog.queries.ts
|
||||
@ -65,24 +71,27 @@ src/main/services/agents/
|
||||
---
|
||||
|
||||
### Task 3: Create Initial Migration Files
|
||||
|
||||
**Status**: ✅ COMPLETED
|
||||
**Agent**: `general-purpose`
|
||||
**Description**: Create migration files based on existing schema
|
||||
|
||||
**Subtasks**:
|
||||
|
||||
- [x] Create 001_initial_schema.ts with agents table and indexes
|
||||
- [x] Create 002_add_session_tables.ts with sessions and session_logs tables
|
||||
- [x] Create database/migrations/index.ts to export all migrations
|
||||
|
||||
|
||||
---
|
||||
|
||||
### Task 4: Update BaseService with Migration Support
|
||||
|
||||
**Status**: ✅ COMPLETED
|
||||
**Agent**: `general-purpose`
|
||||
**Description**: Integrate migration system into BaseService initialization
|
||||
|
||||
**Subtasks**:
|
||||
|
||||
- [x] Update BaseService.ts to use Migrator on initialize
|
||||
- [x] Keep existing JSON serialization utilities
|
||||
- [x] Update database initialization flow
|
||||
@ -90,11 +99,13 @@ src/main/services/agents/
|
||||
---
|
||||
|
||||
### Task 5: Reorganize Service Files
|
||||
|
||||
**Status**: ✅ COMPLETED
|
||||
**Agent**: `general-purpose`
|
||||
**Description**: Move service files to services subdirectory and update imports
|
||||
|
||||
**Subtasks**:
|
||||
|
||||
- [x] Move AgentService.ts to services/
|
||||
- [x] Move SessionService.ts to services/
|
||||
- [x] Move SessionLogService.ts to services/
|
||||
@ -104,11 +115,13 @@ src/main/services/agents/
|
||||
---
|
||||
|
||||
### Task 6: Create Export Structure and Clean Up
|
||||
|
||||
**Status**: ✅ COMPLETED
|
||||
**Agent**: `general-purpose`
|
||||
**Description**: Create proper export hierarchy and clean up old files
|
||||
|
||||
**Subtasks**:
|
||||
|
||||
- [x] Create main agents/index.ts with clean exports
|
||||
- [x] Create database/index.ts for database exports
|
||||
- [x] Ensure backward compatibility for existing imports
|
||||
@ -118,11 +131,13 @@ src/main/services/agents/
|
||||
---
|
||||
|
||||
### Task 7: Test and Validate Refactoring
|
||||
|
||||
**Status**: ✅ COMPLETED
|
||||
**Agent**: `general-purpose`
|
||||
**Description**: Ensure all functionality works after refactoring
|
||||
|
||||
**Subtasks**:
|
||||
|
||||
- [x] Run build check: `yarn build:check` ✅ PASSED (1420 tests, TypeScript compilation successful)
|
||||
- [x] Run tests: `yarn test` ✅ PASSED (All existing tests continue to pass)
|
||||
- [x] Validate migration system works ✅ PASSED (11 migration tests, transaction support verified)
|
||||
@ -130,6 +145,7 @@ src/main/services/agents/
|
||||
- [x] Verify database operations work as expected ✅ PASSED (CRUD operations, foreign keys, concurrent operations)
|
||||
|
||||
**Additional Validation**:
|
||||
|
||||
- [x] Created comprehensive validation report (VALIDATION_REPORT.md)
|
||||
- [x] Validated migration system with custom test suite
|
||||
- [x] Verified service initialization and file structure
|
||||
@ -164,6 +180,7 @@ src/main/services/agents/
|
||||
**Status**: ✅ **REFACTORING COMPLETED SUCCESSFULLY**
|
||||
|
||||
All tasks have been completed and thoroughly validated. The agents service refactoring delivers:
|
||||
|
||||
- ✅ Modular, maintainable code structure
|
||||
- ✅ Production-ready migration system
|
||||
- ✅ Complete backward compatibility
|
||||
@ -171,6 +188,7 @@ All tasks have been completed and thoroughly validated. The agents service refac
|
||||
- ✅ Enhanced developer experience
|
||||
|
||||
**Final deliverables:**
|
||||
|
||||
- 📁 Reorganized service architecture with clear separation of concerns
|
||||
- 🗃️ Database migration system with transaction support and rollback capability
|
||||
- 📋 Comprehensive validation report (VALIDATION_REPORT.md)
|
||||
|
||||
211
plan.md
211
plan.md
@ -37,107 +37,124 @@ Phase 1: Database Service Setup
|
||||
Phase 2: Agent CRUD Operations
|
||||
|
||||
1. Implement service methods:
|
||||
- createAgent(agent: Omit<AgentEntity, 'id' | 'created_at' | 'updated_at'>)
|
||||
- getAgent(id: string)
|
||||
- listAgents(options?: { limit?: number, offset?: number })
|
||||
- updateAgent(id: string, updates: Partial<AgentEntity>)
|
||||
- deleteAgent(id: string)
|
||||
|
||||
- createAgent(agent: Omit<AgentEntity, 'id' | 'created_at' | 'updated_at'>)
|
||||
- getAgent(id: string)
|
||||
- listAgents(options?: { limit?: number, offset?: number })
|
||||
- updateAgent(id: string, updates: Partial<AgentEntity>)
|
||||
- deleteAgent(id: string)
|
||||
|
||||
2. Create API routes:
|
||||
- POST /v1/agents - Create agent
|
||||
- GET /v1/agents - List all agents
|
||||
- GET /v1/agents/:agentId - Get agent by ID
|
||||
- PUT /v1/agents/:agentId - Update agent
|
||||
- DELETE /v1/agents/:agentId - Delete agent
|
||||
|
||||
- POST /v1/agents - Create agent
|
||||
- GET /v1/agents - List all agents
|
||||
- GET /v1/agents/:agentId - Get agent by ID
|
||||
- PUT /v1/agents/:agentId - Update agent
|
||||
- DELETE /v1/agents/:agentId - Delete agent
|
||||
|
||||
Phase 3: Session CRUD Operations
|
||||
|
||||
1. Implement service methods:
|
||||
- createSession(session: Omit<AgentSessionEntity, 'id' | 'created_at' | 'updated_at'>)
|
||||
- getSession(id: string)
|
||||
- listSessions(agentId?: string, options?: { status?: SessionStatus, limit?: number,
|
||||
offset?: number })
|
||||
- updateSession(id: string, updates: Partial<AgentSessionEntity>)
|
||||
- updateSessionStatus(id: string, status: SessionStatus)
|
||||
- deleteSession(id: string)
|
||||
- getSessionWithAgent(id: string) - Get session with merged agent configuration
|
||||
|
||||
- createSession(session: Omit<AgentSessionEntity, 'id' | 'created_at' | 'updated_at'>)
|
||||
- getSession(id: string)
|
||||
- listSessions(agentId?: string, options?: { status?: SessionStatus, limit?: number,
|
||||
offset?: number })
|
||||
- updateSession(id: string, updates: Partial<AgentSessionEntity>)
|
||||
- updateSessionStatus(id: string, status: SessionStatus)
|
||||
- deleteSession(id: string)
|
||||
- getSessionWithAgent(id: string) - Get session with merged agent configuration
|
||||
|
||||
2. Create API routes (RESTful nested resources):
|
||||
- POST /v1/agents/:agentId/sessions - Create session for specific agent
|
||||
- GET /v1/agents/:agentId/sessions - List sessions for specific agent
|
||||
- GET /v1/agents/:agentId/sessions/:sessionId - Get specific session
|
||||
- PUT /v1/agents/:agentId/sessions/:sessionId - Update session
|
||||
- PATCH /v1/agents/:agentId/sessions/:sessionId/status - Update session status
|
||||
- DELETE /v1/agents/:agentId/sessions/:sessionId - Delete session
|
||||
|
||||
- POST /v1/agents/:agentId/sessions - Create session for specific agent
|
||||
- GET /v1/agents/:agentId/sessions - List sessions for specific agent
|
||||
- GET /v1/agents/:agentId/sessions/:sessionId - Get specific session
|
||||
- PUT /v1/agents/:agentId/sessions/:sessionId - Update session
|
||||
- PATCH /v1/agents/:agentId/sessions/:sessionId/status - Update session status
|
||||
- DELETE /v1/agents/:agentId/sessions/:sessionId - Delete session
|
||||
|
||||
Additional convenience endpoints:
|
||||
- GET /v1/sessions - List all sessions (across all agents)
|
||||
- GET /v1/sessions/:sessionId - Get session by ID (without agent context)
|
||||
|
||||
- GET /v1/sessions - List all sessions (across all agents)
|
||||
- GET /v1/sessions/:sessionId - Get session by ID (without agent context)
|
||||
|
||||
Phase 4: Session Logs CRUD Operations
|
||||
|
||||
1. Implement service methods:
|
||||
- createSessionLog(log: Omit<SessionLogEntity, 'id' | 'created_at' | 'updated_at'>)
|
||||
- getSessionLog(id: number)
|
||||
- listSessionLogs(sessionId: string, options?: { limit?: number, offset?: number })
|
||||
- updateSessionLog(id: number, updates: { content?: any, metadata?: any })
|
||||
- deleteSessionLog(id: number)
|
||||
- getSessionLogTree(sessionId: string) - Get logs with parent-child relationships
|
||||
- bulkCreateSessionLogs(logs: Array<...>) - Batch insert logs
|
||||
|
||||
- createSessionLog(log: Omit<SessionLogEntity, 'id' | 'created_at' | 'updated_at'>)
|
||||
- getSessionLog(id: number)
|
||||
- listSessionLogs(sessionId: string, options?: { limit?: number, offset?: number })
|
||||
- updateSessionLog(id: number, updates: { content?: any, metadata?: any })
|
||||
- deleteSessionLog(id: number)
|
||||
- getSessionLogTree(sessionId: string) - Get logs with parent-child relationships
|
||||
- bulkCreateSessionLogs(logs: Array<...>) - Batch insert logs
|
||||
|
||||
2. Create API routes (RESTful nested resources):
|
||||
- POST /v1/agents/:agentId/sessions/:sessionId/logs - Create log entry
|
||||
- GET /v1/agents/:agentId/sessions/:sessionId/logs - List logs for session
|
||||
- GET /v1/agents/:agentId/sessions/:sessionId/logs/:logId - Get specific log
|
||||
- PUT /v1/agents/:agentId/sessions/:sessionId/logs/:logId - Update log
|
||||
- DELETE /v1/agents/:agentId/sessions/:sessionId/logs/:logId - Delete log
|
||||
- POST /v1/agents/:agentId/sessions/:sessionId/logs/bulk - Bulk create logs
|
||||
|
||||
- POST /v1/agents/:agentId/sessions/:sessionId/logs - Create log entry
|
||||
- GET /v1/agents/:agentId/sessions/:sessionId/logs - List logs for session
|
||||
- GET /v1/agents/:agentId/sessions/:sessionId/logs/:logId - Get specific log
|
||||
- PUT /v1/agents/:agentId/sessions/:sessionId/logs/:logId - Update log
|
||||
- DELETE /v1/agents/:agentId/sessions/:sessionId/logs/:logId - Delete log
|
||||
- POST /v1/agents/:agentId/sessions/:sessionId/logs/bulk - Bulk create logs
|
||||
|
||||
Additional convenience endpoints:
|
||||
- GET /v1/sessions/:sessionId/logs - Get logs without agent context
|
||||
- GET /v1/session-logs/:logId - Get specific log by ID
|
||||
|
||||
- GET /v1/sessions/:sessionId/logs - Get logs without agent context
|
||||
- GET /v1/session-logs/:logId - Get specific log by ID
|
||||
|
||||
Phase 5: Route Organization
|
||||
|
||||
1. Mount routes with proper nesting:
|
||||
// In app.ts
|
||||
apiRouter.use('/agents', agentsRoutes)
|
||||
// agentsRoutes will handle:
|
||||
// - /agents/*
|
||||
// - /agents/:agentId/sessions/*
|
||||
// - /agents/:agentId/sessions/:sessionId/logs/*
|
||||
// In app.ts
|
||||
apiRouter.use('/agents', agentsRoutes)
|
||||
// agentsRoutes will handle:
|
||||
// - /agents/_
|
||||
// - /agents/:agentId/sessions/_
|
||||
// - /agents/:agentId/sessions/:sessionId/logs/\*
|
||||
|
||||
// Convenience routes
|
||||
apiRouter.use('/sessions', sessionsRoutes)
|
||||
apiRouter.use('/session-logs', sessionLogsRoutes)
|
||||
|
||||
2. Use Express Router mergeParams for nested routes:
|
||||
// In agents.ts
|
||||
const sessionsRouter = express.Router({ mergeParams: true })
|
||||
router.use('/:agentId/sessions', sessionsRouter)
|
||||
// In agents.ts
|
||||
const sessionsRouter = express.Router({ mergeParams: true })
|
||||
router.use('/:agentId/sessions', sessionsRouter)
|
||||
|
||||
Phase 6: OpenAPI Documentation
|
||||
|
||||
1. Add Swagger schemas for new entities:
|
||||
- AgentEntity schema
|
||||
- AgentSessionEntity schema
|
||||
- SessionLogEntity schema
|
||||
- Request/Response schemas
|
||||
|
||||
- AgentEntity schema
|
||||
- AgentSessionEntity schema
|
||||
- SessionLogEntity schema
|
||||
- Request/Response schemas
|
||||
|
||||
2. Document all new endpoints with:
|
||||
- Clear path parameters (agentId, sessionId, logId)
|
||||
- Request body schemas
|
||||
- Response examples
|
||||
- Error responses
|
||||
- Proper grouping by resource
|
||||
|
||||
- Clear path parameters (agentId, sessionId, logId)
|
||||
- Request body schemas
|
||||
- Response examples
|
||||
- Error responses
|
||||
- Proper grouping by resource
|
||||
|
||||
Phase 7: Validation & Error Handling
|
||||
|
||||
1. Add path parameter validation:
|
||||
- Validate agentId exists before processing session requests
|
||||
- Validate sessionId belongs to agentId
|
||||
- Validate logId belongs to sessionId
|
||||
|
||||
- Validate agentId exists before processing session requests
|
||||
- Validate sessionId belongs to agentId
|
||||
- Validate logId belongs to sessionId
|
||||
|
||||
2. Implement middleware for:
|
||||
- Request validation using express-validator
|
||||
- Resource existence checks
|
||||
- Permission validation (future consideration)
|
||||
- Transaction support for complex operations
|
||||
|
||||
- Request validation using express-validator
|
||||
- Resource existence checks
|
||||
- Permission validation (future consideration)
|
||||
- Transaction support for complex operations
|
||||
|
||||
Phase 8: Testing
|
||||
|
||||
@ -151,59 +168,59 @@ File Structure
|
||||
|
||||
src/
|
||||
├── main/
|
||||
│ └── services/
|
||||
│ └── agents/
|
||||
│ ├── index.ts (existing)
|
||||
│ ├── db.ts (existing)
|
||||
│ └── AgentService.ts (new)
|
||||
│ └── services/
|
||||
│ └── agents/
|
||||
│ ├── index.ts (existing)
|
||||
│ ├── db.ts (existing)
|
||||
│ └── AgentService.ts (new)
|
||||
├── main/
|
||||
│ └── apiServer/
|
||||
│ └── routes/
|
||||
│ ├── agents.ts (new - includes nested routes)
|
||||
│ ├── sessions.ts (new - convenience endpoints)
|
||||
│ └── session-logs.ts (new - convenience endpoints)
|
||||
│ └── apiServer/
|
||||
│ └── routes/
|
||||
│ ├── agents.ts (new - includes nested routes)
|
||||
│ ├── sessions.ts (new - convenience endpoints)
|
||||
│ └── session-logs.ts (new - convenience endpoints)
|
||||
└── renderer/
|
||||
└── src/
|
||||
└── types/
|
||||
└── agent.ts (existing)
|
||||
└── src/
|
||||
└── types/
|
||||
└── agent.ts (existing)
|
||||
|
||||
API Endpoint Summary
|
||||
|
||||
Agent Endpoints
|
||||
|
||||
- POST /v1/agents
|
||||
- GET /v1/agents
|
||||
- GET /v1/agents/:agentId
|
||||
- PUT /v1/agents/:agentId
|
||||
- POST /v1/agents
|
||||
- GET /v1/agents
|
||||
- GET /v1/agents/:agentId
|
||||
- PUT /v1/agents/:agentId
|
||||
- DELETE /v1/agents/:agentId
|
||||
|
||||
Session Endpoints (RESTful)
|
||||
|
||||
- POST /v1/agents/:agentId/sessions
|
||||
- GET /v1/agents/:agentId/sessions
|
||||
- GET /v1/agents/:agentId/sessions/:sessionId
|
||||
- PUT /v1/agents/:agentId/sessions/:sessionId
|
||||
- PATCH /v1/agents/:agentId/sessions/:sessionId/status
|
||||
- POST /v1/agents/:agentId/sessions
|
||||
- GET /v1/agents/:agentId/sessions
|
||||
- GET /v1/agents/:agentId/sessions/:sessionId
|
||||
- PUT /v1/agents/:agentId/sessions/:sessionId
|
||||
- PATCH /v1/agents/:agentId/sessions/:sessionId/status
|
||||
- DELETE /v1/agents/:agentId/sessions/:sessionId
|
||||
|
||||
Session Convenience Endpoints
|
||||
|
||||
- GET /v1/sessions
|
||||
- GET /v1/sessions/:sessionId
|
||||
- GET /v1/sessions
|
||||
- GET /v1/sessions/:sessionId
|
||||
|
||||
Session Log Endpoints (RESTful)
|
||||
|
||||
- POST /v1/agents/:agentId/sessions/:sessionId/logs
|
||||
- GET /v1/agents/:agentId/sessions/:sessionId/logs
|
||||
- GET /v1/agents/:agentId/sessions/:sessionId/logs/:logId
|
||||
- PUT /v1/agents/:agentId/sessions/:sessionId/logs/:logId
|
||||
- POST /v1/agents/:agentId/sessions/:sessionId/logs
|
||||
- GET /v1/agents/:agentId/sessions/:sessionId/logs
|
||||
- GET /v1/agents/:agentId/sessions/:sessionId/logs/:logId
|
||||
- PUT /v1/agents/:agentId/sessions/:sessionId/logs/:logId
|
||||
- DELETE /v1/agents/:agentId/sessions/:sessionId/logs/:logId
|
||||
- POST /v1/agents/:agentId/sessions/:sessionId/logs/bulk
|
||||
- POST /v1/agents/:agentId/sessions/:sessionId/logs/bulk
|
||||
|
||||
Session Log Convenience Endpoints
|
||||
|
||||
- GET /v1/sessions/:sessionId/logs
|
||||
- GET /v1/session-logs/:logId
|
||||
- GET /v1/sessions/:sessionId/logs
|
||||
- GET /v1/session-logs/:logId
|
||||
|
||||
Key Considerations
|
||||
|
||||
|
||||
@ -11,7 +11,7 @@ const logger = loggerService.withContext('BaseService')
|
||||
/**
|
||||
* Base service class providing shared database connection and utilities
|
||||
* for all agent-related services.
|
||||
*
|
||||
*
|
||||
* Uses a migration-only approach for database schema management.
|
||||
* The database schema is defined and maintained exclusively through
|
||||
* migration files, ensuring a single source of truth.
|
||||
|
||||
@ -5,6 +5,7 @@ A production-ready database management system for Cherry Studio's autonomous age
|
||||
## Overview
|
||||
|
||||
The Agents Database Module handles persistent storage for:
|
||||
|
||||
- **Agents**: Autonomous AI agents with configurable models, tools, and permissions
|
||||
- **Sessions**: Agent execution sessions with status tracking and configuration overrides
|
||||
- **Session Logs**: Hierarchical message and action logs for debugging and audit trails
|
||||
@ -37,7 +38,7 @@ database/
|
||||
├── index.ts # Main export file with centralized access
|
||||
├── migrator.ts # Core migration engine with transaction support
|
||||
├── migrations/ # Migration files and registry
|
||||
│ ├── index.ts # Migration registry and utility functions
|
||||
│ ├── index.ts # Migration registry and utility functions
|
||||
│ ├── types.ts # TypeScript interfaces for migration system
|
||||
│ ├── 001_initial_schema.ts # Initial agents table and indexes
|
||||
│ └── 002_add_session_tables.ts # Sessions and session_logs tables
|
||||
@ -53,12 +54,12 @@ database/
|
||||
|
||||
### File Responsibilities
|
||||
|
||||
| Directory | Purpose | Key Files |
|
||||
|-----------|---------|-----------|
|
||||
| `/` | Main entry point and core migration engine | `index.ts`, `migrator.ts` |
|
||||
| `migrations/` | Version-controlled schema changes | `001_*.ts`, `002_*.ts`, etc. |
|
||||
| `queries/` | Pre-built SQL queries by entity | `*.queries.ts` |
|
||||
| `schema/` | Migration system infrastructure | `migrations.ts` |
|
||||
| Directory | Purpose | Key Files |
|
||||
| ------------- | ------------------------------------------ | ---------------------------- |
|
||||
| `/` | Main entry point and core migration engine | `index.ts`, `migrator.ts` |
|
||||
| `migrations/` | Version-controlled schema changes | `001_*.ts`, `002_*.ts`, etc. |
|
||||
| `queries/` | Pre-built SQL queries by entity | `*.queries.ts` |
|
||||
| `schema/` | Migration system infrastructure | `migrations.ts` |
|
||||
|
||||
## Migration System
|
||||
|
||||
@ -73,9 +74,9 @@ graph TD
|
||||
E --> F[Record Migration in Tracking Table]
|
||||
F --> G[Commit Transaction]
|
||||
G --> H[Migration Complete]
|
||||
|
||||
|
||||
E --> I[Error Occurred]
|
||||
I --> J[Rollback Transaction]
|
||||
I --> J[Rollback Transaction]
|
||||
J --> K[Migration Failed]
|
||||
```
|
||||
|
||||
@ -138,7 +139,7 @@ export const migration_003_add_permissions: Migration = {
|
||||
export const migrations: Migration[] = [
|
||||
migration_001_initial_schema,
|
||||
migration_002_add_session_tables,
|
||||
migration_003_add_permissions // Add here
|
||||
migration_003_add_permissions // Add here
|
||||
]
|
||||
```
|
||||
|
||||
@ -169,12 +170,12 @@ Queries are organized by entity with consistent naming patterns:
|
||||
|
||||
```typescript
|
||||
// Basic CRUD operations
|
||||
AgentQueries.insert // Create new agent
|
||||
AgentQueries.update // Update existing agent
|
||||
AgentQueries.getById // Get agent by ID
|
||||
AgentQueries.list // List all agents
|
||||
AgentQueries.delete // Delete agent
|
||||
AgentQueries.count // Count total agents
|
||||
AgentQueries.insert // Create new agent
|
||||
AgentQueries.update // Update existing agent
|
||||
AgentQueries.getById // Get agent by ID
|
||||
AgentQueries.list // List all agents
|
||||
AgentQueries.delete // Delete agent
|
||||
AgentQueries.count // Count total agents
|
||||
AgentQueries.checkExists // Check if agent exists
|
||||
```
|
||||
|
||||
@ -182,13 +183,13 @@ AgentQueries.checkExists // Check if agent exists
|
||||
|
||||
```typescript
|
||||
// Session management
|
||||
SessionQueries.insert // Create new session
|
||||
SessionQueries.update // Update session
|
||||
SessionQueries.updateStatus // Update just status
|
||||
SessionQueries.getById // Get session by ID
|
||||
SessionQueries.list // List all sessions
|
||||
SessionQueries.listWithLimit // Paginated session list
|
||||
SessionQueries.getByStatus // Filter by status
|
||||
SessionQueries.insert // Create new session
|
||||
SessionQueries.update // Update session
|
||||
SessionQueries.updateStatus // Update just status
|
||||
SessionQueries.getById // Get session by ID
|
||||
SessionQueries.list // List all sessions
|
||||
SessionQueries.listWithLimit // Paginated session list
|
||||
SessionQueries.getByStatus // Filter by status
|
||||
SessionQueries.getSessionWithAgent // Join with agent data
|
||||
SessionQueries.getByExternalSessionId // Find by external ID
|
||||
```
|
||||
@ -197,13 +198,13 @@ SessionQueries.getByExternalSessionId // Find by external ID
|
||||
|
||||
```typescript
|
||||
// Log operations
|
||||
SessionLogQueries.insert // Add log entry
|
||||
SessionLogQueries.getBySessionId // Get all logs for session
|
||||
SessionLogQueries.insert // Add log entry
|
||||
SessionLogQueries.getBySessionId // Get all logs for session
|
||||
SessionLogQueries.getBySessionIdWithPagination // Paginated logs
|
||||
SessionLogQueries.getLatestBySessionId // Most recent logs
|
||||
SessionLogQueries.update // Update log entry
|
||||
SessionLogQueries.deleteBySessionId // Clear session logs
|
||||
SessionLogQueries.countBySessionId // Count session logs
|
||||
SessionLogQueries.getLatestBySessionId // Most recent logs
|
||||
SessionLogQueries.update // Update log entry
|
||||
SessionLogQueries.deleteBySessionId // Clear session logs
|
||||
SessionLogQueries.countBySessionId // Count session logs
|
||||
```
|
||||
|
||||
## Development Workflow
|
||||
@ -213,15 +214,17 @@ SessionLogQueries.countBySessionId // Count session logs
|
||||
Follow these steps to add a new database entity:
|
||||
|
||||
1. **Create Migration**:
|
||||
|
||||
```bash
|
||||
# Create new migration file
|
||||
touch migrations/004_add_workflows.ts
|
||||
```
|
||||
|
||||
2. **Define Migration**:
|
||||
|
||||
```typescript
|
||||
export const migration_004_add_workflows: Migration = {
|
||||
id: '004',
|
||||
id: '004',
|
||||
description: 'Add workflows table for agent automation',
|
||||
createdAt: new Date(),
|
||||
up: [
|
||||
@ -239,13 +242,14 @@ Follow these steps to add a new database entity:
|
||||
],
|
||||
down: [
|
||||
'DROP INDEX IF EXISTS idx_workflows_status',
|
||||
'DROP INDEX IF EXISTS idx_workflows_agent_id',
|
||||
'DROP INDEX IF EXISTS idx_workflows_agent_id',
|
||||
'DROP TABLE IF EXISTS workflows'
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
3. **Register Migration**:
|
||||
|
||||
```typescript
|
||||
// migrations/index.ts
|
||||
export const migrations = [
|
||||
@ -255,17 +259,19 @@ Follow these steps to add a new database entity:
|
||||
```
|
||||
|
||||
4. **Create Query Module**:
|
||||
|
||||
```typescript
|
||||
// queries/workflow.queries.ts
|
||||
export const WorkflowQueries = {
|
||||
insert: 'INSERT INTO workflows (id, name, agent_id, steps, status, created_at) VALUES (?, ?, ?, ?, ?, ?)',
|
||||
getById: 'SELECT * FROM workflows WHERE id = ?',
|
||||
getByAgentId: 'SELECT * FROM workflows WHERE agent_id = ?',
|
||||
getByAgentId: 'SELECT * FROM workflows WHERE agent_id = ?'
|
||||
// ... other queries
|
||||
}
|
||||
```
|
||||
|
||||
5. **Export Query Module**:
|
||||
|
||||
```typescript
|
||||
// queries/index.ts
|
||||
export { WorkflowQueries } from './workflow.queries'
|
||||
@ -316,16 +322,16 @@ Main migration management class with transaction support.
|
||||
```typescript
|
||||
class Migrator {
|
||||
constructor(database: Client)
|
||||
|
||||
|
||||
// Migration management
|
||||
addMigration(migration: Migration): void
|
||||
addMigrations(migrations: Migration[]): void
|
||||
|
||||
|
||||
// System lifecycle
|
||||
initialize(): Promise<void>
|
||||
migrate(options?: MigrationOptions): Promise<MigrationResult[]>
|
||||
rollbackLast(): Promise<MigrationResult | null>
|
||||
|
||||
|
||||
// Status and validation
|
||||
getMigrationSummary(): Promise<MigrationSummary>
|
||||
validateMigrations(): Promise<ValidationResult>
|
||||
@ -336,10 +342,10 @@ class Migrator {
|
||||
|
||||
```typescript
|
||||
interface MigrationOptions {
|
||||
useTransaction?: boolean // Run in transaction (default: true)
|
||||
validateChecksums?: boolean // Validate migration checksums (default: true)
|
||||
limit?: number // Max migrations to run (default: unlimited)
|
||||
dryRun?: boolean // Preview mode (default: false)
|
||||
useTransaction?: boolean // Run in transaction (default: true)
|
||||
validateChecksums?: boolean // Validate migration checksums (default: true)
|
||||
limit?: number // Max migrations to run (default: unlimited)
|
||||
dryRun?: boolean // Preview mode (default: false)
|
||||
}
|
||||
```
|
||||
|
||||
@ -347,19 +353,19 @@ interface MigrationOptions {
|
||||
|
||||
```typescript
|
||||
interface Migration {
|
||||
id: string // Unique migration identifier
|
||||
description: string // Human-readable description
|
||||
up: string[] // Forward migration SQL statements
|
||||
down?: string[] // Rollback SQL statements (optional)
|
||||
createdAt: Date // Migration creation timestamp
|
||||
id: string // Unique migration identifier
|
||||
description: string // Human-readable description
|
||||
up: string[] // Forward migration SQL statements
|
||||
down?: string[] // Rollback SQL statements (optional)
|
||||
createdAt: Date // Migration creation timestamp
|
||||
}
|
||||
|
||||
interface MigrationResult {
|
||||
migration: Migration // Migration that was executed
|
||||
success: boolean // Execution success status
|
||||
error?: string // Error message if failed
|
||||
executedAt: Date // Execution timestamp
|
||||
executionTime: number // Execution duration in milliseconds
|
||||
migration: Migration // Migration that was executed
|
||||
success: boolean // Execution success status
|
||||
error?: string // Error message if failed
|
||||
executedAt: Date // Execution timestamp
|
||||
executionTime: number // Execution duration in milliseconds
|
||||
}
|
||||
```
|
||||
|
||||
@ -418,7 +424,7 @@ CREATE TABLE sessions (
|
||||
)
|
||||
```
|
||||
|
||||
#### Session Logs Table
|
||||
#### Session Logs Table
|
||||
|
||||
```sql
|
||||
CREATE TABLE session_logs (
|
||||
@ -447,16 +453,16 @@ import { createClient } from '@libsql/client'
|
||||
async function setupDatabase() {
|
||||
// Create database connection
|
||||
const db = createClient({ url: 'file:agents.db' })
|
||||
|
||||
|
||||
// Initialize migration system
|
||||
const migrator = new Migrator(db)
|
||||
migrator.addMigrations(migrations)
|
||||
await migrator.initialize()
|
||||
|
||||
|
||||
// Run pending migrations
|
||||
const results = await migrator.migrate()
|
||||
console.log(`Migrations complete: ${results.length} applied`)
|
||||
|
||||
|
||||
return db
|
||||
}
|
||||
```
|
||||
@ -470,7 +476,7 @@ async function createAgent(db: Client) {
|
||||
const agent = {
|
||||
id: crypto.randomUUID(),
|
||||
type: 'custom',
|
||||
name: 'Code Review Assistant',
|
||||
name: 'Code Review Assistant',
|
||||
description: 'Helps review code for best practices',
|
||||
avatar: null,
|
||||
instructions: 'Review code for security, performance, and maintainability',
|
||||
@ -487,12 +493,12 @@ async function createAgent(db: Client) {
|
||||
created_at: new Date().toISOString(),
|
||||
updated_at: new Date().toISOString()
|
||||
}
|
||||
|
||||
|
||||
await db.execute({
|
||||
sql: AgentQueries.insert,
|
||||
args: Object.values(agent)
|
||||
})
|
||||
|
||||
|
||||
return agent.id
|
||||
}
|
||||
```
|
||||
@ -504,7 +510,7 @@ import { SessionQueries, SessionLogQueries } from './database'
|
||||
|
||||
async function createSession(db: Client, agentId: string) {
|
||||
const sessionId = crypto.randomUUID()
|
||||
|
||||
|
||||
// Create session
|
||||
await db.execute({
|
||||
sql: SessionQueries.insert,
|
||||
@ -516,12 +522,21 @@ async function createSession(db: Client, agentId: string) {
|
||||
'Review the authentication module for security issues',
|
||||
'running',
|
||||
null, // external_session_id
|
||||
null, null, null, null, null, null, null, null, null, null, // config overrides
|
||||
null,
|
||||
null,
|
||||
null,
|
||||
null,
|
||||
null,
|
||||
null,
|
||||
null,
|
||||
null,
|
||||
null,
|
||||
null, // config overrides
|
||||
new Date().toISOString(),
|
||||
new Date().toISOString()
|
||||
]
|
||||
})
|
||||
|
||||
|
||||
// Add initial log entry
|
||||
await db.execute({
|
||||
sql: SessionLogQueries.insert,
|
||||
@ -539,7 +554,7 @@ async function createSession(db: Client, agentId: string) {
|
||||
new Date().toISOString()
|
||||
]
|
||||
})
|
||||
|
||||
|
||||
return sessionId
|
||||
}
|
||||
```
|
||||
@ -552,11 +567,11 @@ async function getSessionWithAgent(db: Client, sessionId: string) {
|
||||
sql: SessionQueries.getSessionWithAgent,
|
||||
args: [sessionId]
|
||||
})
|
||||
|
||||
|
||||
if (result.rows.length === 0) {
|
||||
throw new Error('Session not found')
|
||||
}
|
||||
|
||||
|
||||
const row = result.rows[0]
|
||||
return {
|
||||
// Session data
|
||||
@ -591,7 +606,7 @@ async function getSessionWithAgent(db: Client, sessionId: string) {
|
||||
|
||||
1. **Use Transactions**: Always wrap related operations in transactions
|
||||
2. **Foreign Key Constraints**: Define relationships with proper CASCADE rules
|
||||
3. **Indexes**: Create indexes for foreign keys and frequently queried columns
|
||||
3. **Indexes**: Create indexes for foreign keys and frequently queried columns
|
||||
4. **JSON Columns**: Use JSON for flexible, extensible data structures
|
||||
5. **Timestamps**: Include created_at and updated_at for audit trails
|
||||
|
||||
@ -611,7 +626,7 @@ async function getSessionWithAgent(db: Client, sessionId: string) {
|
||||
4. **Efficient Queries**: Use appropriate indexes and avoid N+1 query problems
|
||||
5. **Documentation**: Comment complex queries and business logic
|
||||
|
||||
### Error Handling
|
||||
### Error Handling
|
||||
|
||||
1. **Transaction Rollback**: Use transactions with proper rollback on errors
|
||||
2. **Validation**: Validate input data before database operations
|
||||
@ -628,6 +643,7 @@ async function getSessionWithAgent(db: Client, sessionId: string) {
|
||||
**Problem**: Migration tries to create a table that already exists.
|
||||
|
||||
**Solution**: Use `IF NOT EXISTS` in all CREATE statements:
|
||||
|
||||
```sql
|
||||
CREATE TABLE IF NOT EXISTS my_table (...);
|
||||
CREATE INDEX IF NOT EXISTS idx_name ON my_table(column);
|
||||
@ -640,6 +656,7 @@ CREATE INDEX IF NOT EXISTS idx_name ON my_table(column);
|
||||
**Cause**: Migration file was modified after being applied.
|
||||
|
||||
**Solutions**:
|
||||
|
||||
1. **Never modify applied migrations** - create a new migration instead
|
||||
2. If in development, reset database and re-run all migrations
|
||||
3. For production, investigate what changed and create corrective migration
|
||||
@ -649,6 +666,7 @@ CREATE INDEX IF NOT EXISTS idx_name ON my_table(column);
|
||||
**Problem**: Cannot insert/update due to foreign key constraint violation.
|
||||
|
||||
**Solutions**:
|
||||
|
||||
1. Ensure referenced record exists before creating relationship
|
||||
2. Check foreign key column names and types match exactly
|
||||
3. Use transactions to maintain referential integrity
|
||||
@ -657,12 +675,14 @@ CREATE INDEX IF NOT EXISTS idx_name ON my_table(column);
|
||||
|
||||
**Problem**: Migration appears to hang during execution.
|
||||
|
||||
**Causes**:
|
||||
**Causes**:
|
||||
|
||||
- Long-running ALTER TABLE operations
|
||||
- Lock contention from other database connections
|
||||
- Complex migration with multiple DDL operations
|
||||
|
||||
**Solutions**:
|
||||
|
||||
1. Break large migrations into smaller chunks
|
||||
2. Ensure no other processes are using database during migration
|
||||
3. Use migration limit option to apply migrations incrementally
|
||||
@ -672,6 +692,7 @@ CREATE INDEX IF NOT EXISTS idx_name ON my_table(column);
|
||||
**Problem**: `SQLITE_BUSY` or `database is locked` errors.
|
||||
|
||||
**Solutions**:
|
||||
|
||||
1. Ensure only one process accesses database at a time
|
||||
2. Close all database connections properly
|
||||
3. Use shorter transactions to reduce lock duration
|
||||
@ -689,7 +710,7 @@ migrator.addMigrations(migrations)
|
||||
|
||||
// Enable verbose migration logging
|
||||
const results = await migrator.migrate({
|
||||
dryRun: true // Preview migrations without applying
|
||||
dryRun: true // Preview migrations without applying
|
||||
})
|
||||
|
||||
console.log('Migration preview:', results)
|
||||
@ -782,4 +803,4 @@ For questions about the database system:
|
||||
3. **Examine Schema**: Use SQLite tools to inspect current database schema
|
||||
4. **Test in Development**: Always test schema changes in development environment first
|
||||
|
||||
This database system is designed to be robust and production-ready. Following the migration-only approach and best practices outlined in this guide will ensure reliable, maintainable database operations for your agent management system.
|
||||
This database system is designed to be robust and production-ready. Following the migration-only approach and best practices outlined in this guide will ensure reliable, maintainable database operations for your agent management system.
|
||||
|
||||
@ -16,7 +16,7 @@ const logger = loggerService.withContext('Migrator')
|
||||
|
||||
/**
|
||||
* Database migration manager with transaction support
|
||||
*
|
||||
*
|
||||
* This class manages database schema evolution through migrations.
|
||||
* All table and index definitions are maintained exclusively in migration files,
|
||||
* providing a single source of truth for the database schema.
|
||||
|
||||
Loading…
Reference in New Issue
Block a user