mirror of
https://github.com/CherryHQ/cherry-studio.git
synced 2026-01-06 05:09:09 +08:00
📝 docs: update validation and refactoring documentation for agents service
This commit is contained in:
parent
e8c94f3584
commit
a4bb82a02d
@ -1,9 +1,11 @@
|
|||||||
# Agents Service Refactoring - Validation Report
|
# Agents Service Refactoring - Validation Report
|
||||||
|
|
||||||
## Overview
|
## Overview
|
||||||
|
|
||||||
This report documents the comprehensive validation of the agents service refactoring completed on September 12, 2025. All tests were performed to ensure the refactored system maintains full functionality while providing improved structure and maintainability.
|
This report documents the comprehensive validation of the agents service refactoring completed on September 12, 2025. All tests were performed to ensure the refactored system maintains full functionality while providing improved structure and maintainability.
|
||||||
|
|
||||||
## Validation Summary
|
## Validation Summary
|
||||||
|
|
||||||
✅ **ALL VALIDATIONS PASSED** - The refactoring has been successfully completed and verified.
|
✅ **ALL VALIDATIONS PASSED** - The refactoring has been successfully completed and verified.
|
||||||
|
|
||||||
---
|
---
|
||||||
@ -11,9 +13,11 @@ This report documents the comprehensive validation of the agents service refacto
|
|||||||
## 1. Build and Compilation Validation
|
## 1. Build and Compilation Validation
|
||||||
|
|
||||||
### Command: `yarn build:check`
|
### Command: `yarn build:check`
|
||||||
|
|
||||||
**Status:** ✅ PASSED
|
**Status:** ✅ PASSED
|
||||||
|
|
||||||
**Results:**
|
**Results:**
|
||||||
|
|
||||||
- TypeScript compilation for Node.js environment: ✅ PASSED
|
- TypeScript compilation for Node.js environment: ✅ PASSED
|
||||||
- TypeScript compilation for Web environment: ✅ PASSED
|
- TypeScript compilation for Web environment: ✅ PASSED
|
||||||
- i18n validation: ✅ PASSED
|
- i18n validation: ✅ PASSED
|
||||||
@ -22,6 +26,7 @@ This report documents the comprehensive validation of the agents service refacto
|
|||||||
**Duration:** 23.12s
|
**Duration:** 23.12s
|
||||||
|
|
||||||
### Key Findings:
|
### Key Findings:
|
||||||
|
|
||||||
- All TypeScript files compile without errors
|
- All TypeScript files compile without errors
|
||||||
- No type definition conflicts detected
|
- No type definition conflicts detected
|
||||||
- Import/export structure is correctly maintained
|
- Import/export structure is correctly maintained
|
||||||
@ -32,9 +37,11 @@ This report documents the comprehensive validation of the agents service refacto
|
|||||||
## 2. Migration System Validation
|
## 2. Migration System Validation
|
||||||
|
|
||||||
### Custom Migration Test
|
### Custom Migration Test
|
||||||
|
|
||||||
**Status:** ✅ PASSED
|
**Status:** ✅ PASSED
|
||||||
|
|
||||||
**Test Coverage:**
|
**Test Coverage:**
|
||||||
|
|
||||||
1. ✅ Migration tracking table creation
|
1. ✅ Migration tracking table creation
|
||||||
2. ✅ Migration indexes creation
|
2. ✅ Migration indexes creation
|
||||||
3. ✅ Migration record insertion/retrieval
|
3. ✅ Migration record insertion/retrieval
|
||||||
@ -47,6 +54,7 @@ This report documents the comprehensive validation of the agents service refacto
|
|||||||
10. ✅ Migration cleanup
|
10. ✅ Migration cleanup
|
||||||
|
|
||||||
### Key Findings:
|
### Key Findings:
|
||||||
|
|
||||||
- Migration system initializes correctly
|
- Migration system initializes correctly
|
||||||
- All migration tables and indexes are created properly
|
- All migration tables and indexes are created properly
|
||||||
- Transaction support works as expected
|
- Transaction support works as expected
|
||||||
@ -58,9 +66,11 @@ This report documents the comprehensive validation of the agents service refacto
|
|||||||
## 3. Service Initialization Validation
|
## 3. Service Initialization Validation
|
||||||
|
|
||||||
### Custom Service Structure Test
|
### Custom Service Structure Test
|
||||||
|
|
||||||
**Status:** ✅ PASSED
|
**Status:** ✅ PASSED
|
||||||
|
|
||||||
**Validated Components:**
|
**Validated Components:**
|
||||||
|
|
||||||
1. ✅ All service files are present and accessible
|
1. ✅ All service files are present and accessible
|
||||||
2. ✅ Migration files are properly organized
|
2. ✅ Migration files are properly organized
|
||||||
3. ✅ Query files are correctly structured
|
3. ✅ Query files are correctly structured
|
||||||
@ -71,6 +81,7 @@ This report documents the comprehensive validation of the agents service refacto
|
|||||||
8. ✅ TypeScript compilation validated
|
8. ✅ TypeScript compilation validated
|
||||||
|
|
||||||
### File Structure Verification:
|
### File Structure Verification:
|
||||||
|
|
||||||
```
|
```
|
||||||
src/main/services/agents/
|
src/main/services/agents/
|
||||||
├── ✅ BaseService.ts
|
├── ✅ BaseService.ts
|
||||||
@ -105,9 +116,11 @@ src/main/services/agents/
|
|||||||
## 4. Database Operations Validation
|
## 4. Database Operations Validation
|
||||||
|
|
||||||
### Comprehensive CRUD Operations Test
|
### Comprehensive CRUD Operations Test
|
||||||
|
|
||||||
**Status:** ✅ PASSED
|
**Status:** ✅ PASSED
|
||||||
|
|
||||||
**Test Scenarios:**
|
**Test Scenarios:**
|
||||||
|
|
||||||
1. ✅ Database schema setup (tables + indexes)
|
1. ✅ Database schema setup (tables + indexes)
|
||||||
2. ✅ Agent CRUD operations
|
2. ✅ Agent CRUD operations
|
||||||
- Create: ✅ Agent creation with JSON field serialization
|
- Create: ✅ Agent creation with JSON field serialization
|
||||||
@ -130,6 +143,7 @@ src/main/services/agents/
|
|||||||
- Data Integrity: ✅ All concurrent operations verified
|
- Data Integrity: ✅ All concurrent operations verified
|
||||||
|
|
||||||
### Performance Metrics:
|
### Performance Metrics:
|
||||||
|
|
||||||
- Agent CRUD operations: < 50ms per operation
|
- Agent CRUD operations: < 50ms per operation
|
||||||
- Migration system: < 100ms initialization
|
- Migration system: < 100ms initialization
|
||||||
- Concurrent operations: Successfully handled 5 parallel operations
|
- Concurrent operations: Successfully handled 5 parallel operations
|
||||||
@ -139,6 +153,7 @@ src/main/services/agents/
|
|||||||
## 5. Backward Compatibility Validation
|
## 5. Backward Compatibility Validation
|
||||||
|
|
||||||
### Compatibility Checks:
|
### Compatibility Checks:
|
||||||
|
|
||||||
- ✅ Export structure maintains backward compatibility
|
- ✅ Export structure maintains backward compatibility
|
||||||
- ✅ Legacy query exports available via `AgentQueries_Legacy`
|
- ✅ Legacy query exports available via `AgentQueries_Legacy`
|
||||||
- ✅ Service singleton instances preserved
|
- ✅ Service singleton instances preserved
|
||||||
@ -150,6 +165,7 @@ src/main/services/agents/
|
|||||||
## 6. Code Quality and Structure
|
## 6. Code Quality and Structure
|
||||||
|
|
||||||
### Improvements Delivered:
|
### Improvements Delivered:
|
||||||
|
|
||||||
1. **Modular Organization**: ✅ Services split into focused, single-responsibility files
|
1. **Modular Organization**: ✅ Services split into focused, single-responsibility files
|
||||||
2. **Migration System**: ✅ Version-controlled schema changes with rollback support
|
2. **Migration System**: ✅ Version-controlled schema changes with rollback support
|
||||||
3. **Query Organization**: ✅ SQL queries organized by entity type
|
3. **Query Organization**: ✅ SQL queries organized by entity type
|
||||||
@ -159,6 +175,7 @@ src/main/services/agents/
|
|||||||
7. **Testing**: ✅ All existing tests continue to pass
|
7. **Testing**: ✅ All existing tests continue to pass
|
||||||
|
|
||||||
### Benefits Realized:
|
### Benefits Realized:
|
||||||
|
|
||||||
- **Maintainability**: Easier to locate and modify specific functionality
|
- **Maintainability**: Easier to locate and modify specific functionality
|
||||||
- **Scalability**: Simple to add new entities without affecting existing code
|
- **Scalability**: Simple to add new entities without affecting existing code
|
||||||
- **Production Readiness**: Atomic migrations with transaction support
|
- **Production Readiness**: Atomic migrations with transaction support
|
||||||
@ -170,6 +187,7 @@ src/main/services/agents/
|
|||||||
## 7. Security and Safety Validation
|
## 7. Security and Safety Validation
|
||||||
|
|
||||||
### Security Measures Verified:
|
### Security Measures Verified:
|
||||||
|
|
||||||
- ✅ SQL injection protection via parameterized queries
|
- ✅ SQL injection protection via parameterized queries
|
||||||
- ✅ Transaction isolation for atomic operations
|
- ✅ Transaction isolation for atomic operations
|
||||||
- ✅ Foreign key constraints prevent orphaned records
|
- ✅ Foreign key constraints prevent orphaned records
|
||||||
@ -181,6 +199,7 @@ src/main/services/agents/
|
|||||||
## 8. Performance Validation
|
## 8. Performance Validation
|
||||||
|
|
||||||
### Database Operations:
|
### Database Operations:
|
||||||
|
|
||||||
- ✅ Index utilization verified for common queries
|
- ✅ Index utilization verified for common queries
|
||||||
- ✅ Foreign key constraints optimized with indexes
|
- ✅ Foreign key constraints optimized with indexes
|
||||||
- ✅ JSON field operations efficient
|
- ✅ JSON field operations efficient
|
||||||
@ -191,6 +210,7 @@ src/main/services/agents/
|
|||||||
## Cleanup
|
## Cleanup
|
||||||
|
|
||||||
The following temporary test files were created for validation and can be safely removed:
|
The following temporary test files were created for validation and can be safely removed:
|
||||||
|
|
||||||
- `/Users/weliu/workspace/cherry-studio/migration-validation-test.js`
|
- `/Users/weliu/workspace/cherry-studio/migration-validation-test.js`
|
||||||
- `/Users/weliu/workspace/cherry-studio/service-initialization-test.js`
|
- `/Users/weliu/workspace/cherry-studio/service-initialization-test.js`
|
||||||
- `/Users/weliu/workspace/cherry-studio/database-operations-test.js`
|
- `/Users/weliu/workspace/cherry-studio/database-operations-test.js`
|
||||||
@ -215,4 +235,4 @@ The agents service refactoring has been successfully completed and thoroughly va
|
|||||||
**Validation completed:** September 12, 2025
|
**Validation completed:** September 12, 2025
|
||||||
**Total validation time:** ~45 minutes
|
**Total validation time:** ~45 minutes
|
||||||
**Tests executed:** 1420 + custom validation tests
|
**Tests executed:** 1420 + custom validation tests
|
||||||
**Overall result:** ✅ SUCCESS
|
**Overall result:** ✅ SUCCESS
|
||||||
|
|||||||
@ -1,9 +1,11 @@
|
|||||||
# Agents Service Refactoring Plan
|
# Agents Service Refactoring Plan
|
||||||
|
|
||||||
## Overview
|
## Overview
|
||||||
|
|
||||||
Restructure the agents service to split database operations into smaller, more manageable files with migration support.
|
Restructure the agents service to split database operations into smaller, more manageable files with migration support.
|
||||||
|
|
||||||
## New Folder Structure
|
## New Folder Structure
|
||||||
|
|
||||||
```
|
```
|
||||||
src/main/services/agents/
|
src/main/services/agents/
|
||||||
├── database/
|
├── database/
|
||||||
@ -36,11 +38,13 @@ src/main/services/agents/
|
|||||||
## Implementation Tasks
|
## Implementation Tasks
|
||||||
|
|
||||||
### Task 1: Create Folder Structure and Migration System Infrastructure
|
### Task 1: Create Folder Structure and Migration System Infrastructure
|
||||||
|
|
||||||
**Status**: ✅ COMPLETED
|
**Status**: ✅ COMPLETED
|
||||||
**Agent**: `general-purpose`
|
**Agent**: `general-purpose`
|
||||||
**Description**: Create all necessary directories and implement the migration system infrastructure
|
**Description**: Create all necessary directories and implement the migration system infrastructure
|
||||||
|
|
||||||
**Subtasks**:
|
**Subtasks**:
|
||||||
|
|
||||||
- [x] Create database/, database/migrations/, database/queries/, database/schema/, services/ directories
|
- [x] Create database/, database/migrations/, database/queries/, database/schema/, services/ directories
|
||||||
- [x] Implement migration types and interfaces in database/migrations/types.ts
|
- [x] Implement migration types and interfaces in database/migrations/types.ts
|
||||||
- [x] Build Migrator class with transaction support in database/migrator.ts
|
- [x] Build Migrator class with transaction support in database/migrator.ts
|
||||||
@ -49,11 +53,13 @@ src/main/services/agents/
|
|||||||
---
|
---
|
||||||
|
|
||||||
### Task 2: Split Database Queries from db.ts
|
### Task 2: Split Database Queries from db.ts
|
||||||
|
|
||||||
**Status**: ✅ COMPLETED
|
**Status**: ✅ COMPLETED
|
||||||
**Agent**: `general-purpose`
|
**Agent**: `general-purpose`
|
||||||
**Description**: Extract and organize queries from the current db.ts file into separate, focused files
|
**Description**: Extract and organize queries from the current db.ts file into separate, focused files
|
||||||
|
|
||||||
**Subtasks**:
|
**Subtasks**:
|
||||||
|
|
||||||
- [x] Move agent queries to database/queries/agent.queries.ts
|
- [x] Move agent queries to database/queries/agent.queries.ts
|
||||||
- [x] Move session queries to database/queries/session.queries.ts
|
- [x] Move session queries to database/queries/session.queries.ts
|
||||||
- [x] Move session log queries to database/queries/sessionLog.queries.ts
|
- [x] Move session log queries to database/queries/sessionLog.queries.ts
|
||||||
@ -65,24 +71,27 @@ src/main/services/agents/
|
|||||||
---
|
---
|
||||||
|
|
||||||
### Task 3: Create Initial Migration Files
|
### Task 3: Create Initial Migration Files
|
||||||
|
|
||||||
**Status**: ✅ COMPLETED
|
**Status**: ✅ COMPLETED
|
||||||
**Agent**: `general-purpose`
|
**Agent**: `general-purpose`
|
||||||
**Description**: Create migration files based on existing schema
|
**Description**: Create migration files based on existing schema
|
||||||
|
|
||||||
**Subtasks**:
|
**Subtasks**:
|
||||||
|
|
||||||
- [x] Create 001_initial_schema.ts with agents table and indexes
|
- [x] Create 001_initial_schema.ts with agents table and indexes
|
||||||
- [x] Create 002_add_session_tables.ts with sessions and session_logs tables
|
- [x] Create 002_add_session_tables.ts with sessions and session_logs tables
|
||||||
- [x] Create database/migrations/index.ts to export all migrations
|
- [x] Create database/migrations/index.ts to export all migrations
|
||||||
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
### Task 4: Update BaseService with Migration Support
|
### Task 4: Update BaseService with Migration Support
|
||||||
|
|
||||||
**Status**: ✅ COMPLETED
|
**Status**: ✅ COMPLETED
|
||||||
**Agent**: `general-purpose`
|
**Agent**: `general-purpose`
|
||||||
**Description**: Integrate migration system into BaseService initialization
|
**Description**: Integrate migration system into BaseService initialization
|
||||||
|
|
||||||
**Subtasks**:
|
**Subtasks**:
|
||||||
|
|
||||||
- [x] Update BaseService.ts to use Migrator on initialize
|
- [x] Update BaseService.ts to use Migrator on initialize
|
||||||
- [x] Keep existing JSON serialization utilities
|
- [x] Keep existing JSON serialization utilities
|
||||||
- [x] Update database initialization flow
|
- [x] Update database initialization flow
|
||||||
@ -90,11 +99,13 @@ src/main/services/agents/
|
|||||||
---
|
---
|
||||||
|
|
||||||
### Task 5: Reorganize Service Files
|
### Task 5: Reorganize Service Files
|
||||||
|
|
||||||
**Status**: ✅ COMPLETED
|
**Status**: ✅ COMPLETED
|
||||||
**Agent**: `general-purpose`
|
**Agent**: `general-purpose`
|
||||||
**Description**: Move service files to services subdirectory and update imports
|
**Description**: Move service files to services subdirectory and update imports
|
||||||
|
|
||||||
**Subtasks**:
|
**Subtasks**:
|
||||||
|
|
||||||
- [x] Move AgentService.ts to services/
|
- [x] Move AgentService.ts to services/
|
||||||
- [x] Move SessionService.ts to services/
|
- [x] Move SessionService.ts to services/
|
||||||
- [x] Move SessionLogService.ts to services/
|
- [x] Move SessionLogService.ts to services/
|
||||||
@ -104,11 +115,13 @@ src/main/services/agents/
|
|||||||
---
|
---
|
||||||
|
|
||||||
### Task 6: Create Export Structure and Clean Up
|
### Task 6: Create Export Structure and Clean Up
|
||||||
|
|
||||||
**Status**: ✅ COMPLETED
|
**Status**: ✅ COMPLETED
|
||||||
**Agent**: `general-purpose`
|
**Agent**: `general-purpose`
|
||||||
**Description**: Create proper export hierarchy and clean up old files
|
**Description**: Create proper export hierarchy and clean up old files
|
||||||
|
|
||||||
**Subtasks**:
|
**Subtasks**:
|
||||||
|
|
||||||
- [x] Create main agents/index.ts with clean exports
|
- [x] Create main agents/index.ts with clean exports
|
||||||
- [x] Create database/index.ts for database exports
|
- [x] Create database/index.ts for database exports
|
||||||
- [x] Ensure backward compatibility for existing imports
|
- [x] Ensure backward compatibility for existing imports
|
||||||
@ -118,11 +131,13 @@ src/main/services/agents/
|
|||||||
---
|
---
|
||||||
|
|
||||||
### Task 7: Test and Validate Refactoring
|
### Task 7: Test and Validate Refactoring
|
||||||
|
|
||||||
**Status**: ✅ COMPLETED
|
**Status**: ✅ COMPLETED
|
||||||
**Agent**: `general-purpose`
|
**Agent**: `general-purpose`
|
||||||
**Description**: Ensure all functionality works after refactoring
|
**Description**: Ensure all functionality works after refactoring
|
||||||
|
|
||||||
**Subtasks**:
|
**Subtasks**:
|
||||||
|
|
||||||
- [x] Run build check: `yarn build:check` ✅ PASSED (1420 tests, TypeScript compilation successful)
|
- [x] Run build check: `yarn build:check` ✅ PASSED (1420 tests, TypeScript compilation successful)
|
||||||
- [x] Run tests: `yarn test` ✅ PASSED (All existing tests continue to pass)
|
- [x] Run tests: `yarn test` ✅ PASSED (All existing tests continue to pass)
|
||||||
- [x] Validate migration system works ✅ PASSED (11 migration tests, transaction support verified)
|
- [x] Validate migration system works ✅ PASSED (11 migration tests, transaction support verified)
|
||||||
@ -130,6 +145,7 @@ src/main/services/agents/
|
|||||||
- [x] Verify database operations work as expected ✅ PASSED (CRUD operations, foreign keys, concurrent operations)
|
- [x] Verify database operations work as expected ✅ PASSED (CRUD operations, foreign keys, concurrent operations)
|
||||||
|
|
||||||
**Additional Validation**:
|
**Additional Validation**:
|
||||||
|
|
||||||
- [x] Created comprehensive validation report (VALIDATION_REPORT.md)
|
- [x] Created comprehensive validation report (VALIDATION_REPORT.md)
|
||||||
- [x] Validated migration system with custom test suite
|
- [x] Validated migration system with custom test suite
|
||||||
- [x] Verified service initialization and file structure
|
- [x] Verified service initialization and file structure
|
||||||
@ -164,6 +180,7 @@ src/main/services/agents/
|
|||||||
**Status**: ✅ **REFACTORING COMPLETED SUCCESSFULLY**
|
**Status**: ✅ **REFACTORING COMPLETED SUCCESSFULLY**
|
||||||
|
|
||||||
All tasks have been completed and thoroughly validated. The agents service refactoring delivers:
|
All tasks have been completed and thoroughly validated. The agents service refactoring delivers:
|
||||||
|
|
||||||
- ✅ Modular, maintainable code structure
|
- ✅ Modular, maintainable code structure
|
||||||
- ✅ Production-ready migration system
|
- ✅ Production-ready migration system
|
||||||
- ✅ Complete backward compatibility
|
- ✅ Complete backward compatibility
|
||||||
@ -171,6 +188,7 @@ All tasks have been completed and thoroughly validated. The agents service refac
|
|||||||
- ✅ Enhanced developer experience
|
- ✅ Enhanced developer experience
|
||||||
|
|
||||||
**Final deliverables:**
|
**Final deliverables:**
|
||||||
|
|
||||||
- 📁 Reorganized service architecture with clear separation of concerns
|
- 📁 Reorganized service architecture with clear separation of concerns
|
||||||
- 🗃️ Database migration system with transaction support and rollback capability
|
- 🗃️ Database migration system with transaction support and rollback capability
|
||||||
- 📋 Comprehensive validation report (VALIDATION_REPORT.md)
|
- 📋 Comprehensive validation report (VALIDATION_REPORT.md)
|
||||||
|
|||||||
211
plan.md
211
plan.md
@ -37,107 +37,124 @@ Phase 1: Database Service Setup
|
|||||||
Phase 2: Agent CRUD Operations
|
Phase 2: Agent CRUD Operations
|
||||||
|
|
||||||
1. Implement service methods:
|
1. Implement service methods:
|
||||||
- createAgent(agent: Omit<AgentEntity, 'id' | 'created_at' | 'updated_at'>)
|
|
||||||
- getAgent(id: string)
|
- createAgent(agent: Omit<AgentEntity, 'id' | 'created_at' | 'updated_at'>)
|
||||||
- listAgents(options?: { limit?: number, offset?: number })
|
- getAgent(id: string)
|
||||||
- updateAgent(id: string, updates: Partial<AgentEntity>)
|
- listAgents(options?: { limit?: number, offset?: number })
|
||||||
- deleteAgent(id: string)
|
- updateAgent(id: string, updates: Partial<AgentEntity>)
|
||||||
|
- deleteAgent(id: string)
|
||||||
|
|
||||||
2. Create API routes:
|
2. Create API routes:
|
||||||
- POST /v1/agents - Create agent
|
|
||||||
- GET /v1/agents - List all agents
|
- POST /v1/agents - Create agent
|
||||||
- GET /v1/agents/:agentId - Get agent by ID
|
- GET /v1/agents - List all agents
|
||||||
- PUT /v1/agents/:agentId - Update agent
|
- GET /v1/agents/:agentId - Get agent by ID
|
||||||
- DELETE /v1/agents/:agentId - Delete agent
|
- PUT /v1/agents/:agentId - Update agent
|
||||||
|
- DELETE /v1/agents/:agentId - Delete agent
|
||||||
|
|
||||||
Phase 3: Session CRUD Operations
|
Phase 3: Session CRUD Operations
|
||||||
|
|
||||||
1. Implement service methods:
|
1. Implement service methods:
|
||||||
- createSession(session: Omit<AgentSessionEntity, 'id' | 'created_at' | 'updated_at'>)
|
|
||||||
- getSession(id: string)
|
- createSession(session: Omit<AgentSessionEntity, 'id' | 'created_at' | 'updated_at'>)
|
||||||
- listSessions(agentId?: string, options?: { status?: SessionStatus, limit?: number,
|
- getSession(id: string)
|
||||||
offset?: number })
|
- listSessions(agentId?: string, options?: { status?: SessionStatus, limit?: number,
|
||||||
- updateSession(id: string, updates: Partial<AgentSessionEntity>)
|
offset?: number })
|
||||||
- updateSessionStatus(id: string, status: SessionStatus)
|
- updateSession(id: string, updates: Partial<AgentSessionEntity>)
|
||||||
- deleteSession(id: string)
|
- updateSessionStatus(id: string, status: SessionStatus)
|
||||||
- getSessionWithAgent(id: string) - Get session with merged agent configuration
|
- deleteSession(id: string)
|
||||||
|
- getSessionWithAgent(id: string) - Get session with merged agent configuration
|
||||||
|
|
||||||
2. Create API routes (RESTful nested resources):
|
2. Create API routes (RESTful nested resources):
|
||||||
- POST /v1/agents/:agentId/sessions - Create session for specific agent
|
|
||||||
- GET /v1/agents/:agentId/sessions - List sessions for specific agent
|
- POST /v1/agents/:agentId/sessions - Create session for specific agent
|
||||||
- GET /v1/agents/:agentId/sessions/:sessionId - Get specific session
|
- GET /v1/agents/:agentId/sessions - List sessions for specific agent
|
||||||
- PUT /v1/agents/:agentId/sessions/:sessionId - Update session
|
- GET /v1/agents/:agentId/sessions/:sessionId - Get specific session
|
||||||
- PATCH /v1/agents/:agentId/sessions/:sessionId/status - Update session status
|
- PUT /v1/agents/:agentId/sessions/:sessionId - Update session
|
||||||
- DELETE /v1/agents/:agentId/sessions/:sessionId - Delete session
|
- PATCH /v1/agents/:agentId/sessions/:sessionId/status - Update session status
|
||||||
|
- DELETE /v1/agents/:agentId/sessions/:sessionId - Delete session
|
||||||
|
|
||||||
Additional convenience endpoints:
|
Additional convenience endpoints:
|
||||||
- GET /v1/sessions - List all sessions (across all agents)
|
|
||||||
- GET /v1/sessions/:sessionId - Get session by ID (without agent context)
|
- GET /v1/sessions - List all sessions (across all agents)
|
||||||
|
- GET /v1/sessions/:sessionId - Get session by ID (without agent context)
|
||||||
|
|
||||||
Phase 4: Session Logs CRUD Operations
|
Phase 4: Session Logs CRUD Operations
|
||||||
|
|
||||||
1. Implement service methods:
|
1. Implement service methods:
|
||||||
- createSessionLog(log: Omit<SessionLogEntity, 'id' | 'created_at' | 'updated_at'>)
|
|
||||||
- getSessionLog(id: number)
|
- createSessionLog(log: Omit<SessionLogEntity, 'id' | 'created_at' | 'updated_at'>)
|
||||||
- listSessionLogs(sessionId: string, options?: { limit?: number, offset?: number })
|
- getSessionLog(id: number)
|
||||||
- updateSessionLog(id: number, updates: { content?: any, metadata?: any })
|
- listSessionLogs(sessionId: string, options?: { limit?: number, offset?: number })
|
||||||
- deleteSessionLog(id: number)
|
- updateSessionLog(id: number, updates: { content?: any, metadata?: any })
|
||||||
- getSessionLogTree(sessionId: string) - Get logs with parent-child relationships
|
- deleteSessionLog(id: number)
|
||||||
- bulkCreateSessionLogs(logs: Array<...>) - Batch insert logs
|
- getSessionLogTree(sessionId: string) - Get logs with parent-child relationships
|
||||||
|
- bulkCreateSessionLogs(logs: Array<...>) - Batch insert logs
|
||||||
|
|
||||||
2. Create API routes (RESTful nested resources):
|
2. Create API routes (RESTful nested resources):
|
||||||
- POST /v1/agents/:agentId/sessions/:sessionId/logs - Create log entry
|
|
||||||
- GET /v1/agents/:agentId/sessions/:sessionId/logs - List logs for session
|
- POST /v1/agents/:agentId/sessions/:sessionId/logs - Create log entry
|
||||||
- GET /v1/agents/:agentId/sessions/:sessionId/logs/:logId - Get specific log
|
- GET /v1/agents/:agentId/sessions/:sessionId/logs - List logs for session
|
||||||
- PUT /v1/agents/:agentId/sessions/:sessionId/logs/:logId - Update log
|
- GET /v1/agents/:agentId/sessions/:sessionId/logs/:logId - Get specific log
|
||||||
- DELETE /v1/agents/:agentId/sessions/:sessionId/logs/:logId - Delete log
|
- PUT /v1/agents/:agentId/sessions/:sessionId/logs/:logId - Update log
|
||||||
- POST /v1/agents/:agentId/sessions/:sessionId/logs/bulk - Bulk create logs
|
- DELETE /v1/agents/:agentId/sessions/:sessionId/logs/:logId - Delete log
|
||||||
|
- POST /v1/agents/:agentId/sessions/:sessionId/logs/bulk - Bulk create logs
|
||||||
|
|
||||||
Additional convenience endpoints:
|
Additional convenience endpoints:
|
||||||
- GET /v1/sessions/:sessionId/logs - Get logs without agent context
|
|
||||||
- GET /v1/session-logs/:logId - Get specific log by ID
|
- GET /v1/sessions/:sessionId/logs - Get logs without agent context
|
||||||
|
- GET /v1/session-logs/:logId - Get specific log by ID
|
||||||
|
|
||||||
Phase 5: Route Organization
|
Phase 5: Route Organization
|
||||||
|
|
||||||
1. Mount routes with proper nesting:
|
1. Mount routes with proper nesting:
|
||||||
// In app.ts
|
// In app.ts
|
||||||
apiRouter.use('/agents', agentsRoutes)
|
apiRouter.use('/agents', agentsRoutes)
|
||||||
// agentsRoutes will handle:
|
// agentsRoutes will handle:
|
||||||
// - /agents/*
|
// - /agents/_
|
||||||
// - /agents/:agentId/sessions/*
|
// - /agents/:agentId/sessions/_
|
||||||
// - /agents/:agentId/sessions/:sessionId/logs/*
|
// - /agents/:agentId/sessions/:sessionId/logs/\*
|
||||||
|
|
||||||
// Convenience routes
|
// Convenience routes
|
||||||
apiRouter.use('/sessions', sessionsRoutes)
|
apiRouter.use('/sessions', sessionsRoutes)
|
||||||
apiRouter.use('/session-logs', sessionLogsRoutes)
|
apiRouter.use('/session-logs', sessionLogsRoutes)
|
||||||
|
|
||||||
2. Use Express Router mergeParams for nested routes:
|
2. Use Express Router mergeParams for nested routes:
|
||||||
// In agents.ts
|
// In agents.ts
|
||||||
const sessionsRouter = express.Router({ mergeParams: true })
|
const sessionsRouter = express.Router({ mergeParams: true })
|
||||||
router.use('/:agentId/sessions', sessionsRouter)
|
router.use('/:agentId/sessions', sessionsRouter)
|
||||||
|
|
||||||
Phase 6: OpenAPI Documentation
|
Phase 6: OpenAPI Documentation
|
||||||
|
|
||||||
1. Add Swagger schemas for new entities:
|
1. Add Swagger schemas for new entities:
|
||||||
- AgentEntity schema
|
|
||||||
- AgentSessionEntity schema
|
- AgentEntity schema
|
||||||
- SessionLogEntity schema
|
- AgentSessionEntity schema
|
||||||
- Request/Response schemas
|
- SessionLogEntity schema
|
||||||
|
- Request/Response schemas
|
||||||
|
|
||||||
2. Document all new endpoints with:
|
2. Document all new endpoints with:
|
||||||
- Clear path parameters (agentId, sessionId, logId)
|
|
||||||
- Request body schemas
|
- Clear path parameters (agentId, sessionId, logId)
|
||||||
- Response examples
|
- Request body schemas
|
||||||
- Error responses
|
- Response examples
|
||||||
- Proper grouping by resource
|
- Error responses
|
||||||
|
- Proper grouping by resource
|
||||||
|
|
||||||
Phase 7: Validation & Error Handling
|
Phase 7: Validation & Error Handling
|
||||||
|
|
||||||
1. Add path parameter validation:
|
1. Add path parameter validation:
|
||||||
- Validate agentId exists before processing session requests
|
|
||||||
- Validate sessionId belongs to agentId
|
- Validate agentId exists before processing session requests
|
||||||
- Validate logId belongs to sessionId
|
- Validate sessionId belongs to agentId
|
||||||
|
- Validate logId belongs to sessionId
|
||||||
|
|
||||||
2. Implement middleware for:
|
2. Implement middleware for:
|
||||||
- Request validation using express-validator
|
|
||||||
- Resource existence checks
|
- Request validation using express-validator
|
||||||
- Permission validation (future consideration)
|
- Resource existence checks
|
||||||
- Transaction support for complex operations
|
- Permission validation (future consideration)
|
||||||
|
- Transaction support for complex operations
|
||||||
|
|
||||||
Phase 8: Testing
|
Phase 8: Testing
|
||||||
|
|
||||||
@ -151,59 +168,59 @@ File Structure
|
|||||||
|
|
||||||
src/
|
src/
|
||||||
├── main/
|
├── main/
|
||||||
│ └── services/
|
│ └── services/
|
||||||
│ └── agents/
|
│ └── agents/
|
||||||
│ ├── index.ts (existing)
|
│ ├── index.ts (existing)
|
||||||
│ ├── db.ts (existing)
|
│ ├── db.ts (existing)
|
||||||
│ └── AgentService.ts (new)
|
│ └── AgentService.ts (new)
|
||||||
├── main/
|
├── main/
|
||||||
│ └── apiServer/
|
│ └── apiServer/
|
||||||
│ └── routes/
|
│ └── routes/
|
||||||
│ ├── agents.ts (new - includes nested routes)
|
│ ├── agents.ts (new - includes nested routes)
|
||||||
│ ├── sessions.ts (new - convenience endpoints)
|
│ ├── sessions.ts (new - convenience endpoints)
|
||||||
│ └── session-logs.ts (new - convenience endpoints)
|
│ └── session-logs.ts (new - convenience endpoints)
|
||||||
└── renderer/
|
└── renderer/
|
||||||
└── src/
|
└── src/
|
||||||
└── types/
|
└── types/
|
||||||
└── agent.ts (existing)
|
└── agent.ts (existing)
|
||||||
|
|
||||||
API Endpoint Summary
|
API Endpoint Summary
|
||||||
|
|
||||||
Agent Endpoints
|
Agent Endpoints
|
||||||
|
|
||||||
- POST /v1/agents
|
- POST /v1/agents
|
||||||
- GET /v1/agents
|
- GET /v1/agents
|
||||||
- GET /v1/agents/:agentId
|
- GET /v1/agents/:agentId
|
||||||
- PUT /v1/agents/:agentId
|
- PUT /v1/agents/:agentId
|
||||||
- DELETE /v1/agents/:agentId
|
- DELETE /v1/agents/:agentId
|
||||||
|
|
||||||
Session Endpoints (RESTful)
|
Session Endpoints (RESTful)
|
||||||
|
|
||||||
- POST /v1/agents/:agentId/sessions
|
- POST /v1/agents/:agentId/sessions
|
||||||
- GET /v1/agents/:agentId/sessions
|
- GET /v1/agents/:agentId/sessions
|
||||||
- GET /v1/agents/:agentId/sessions/:sessionId
|
- GET /v1/agents/:agentId/sessions/:sessionId
|
||||||
- PUT /v1/agents/:agentId/sessions/:sessionId
|
- PUT /v1/agents/:agentId/sessions/:sessionId
|
||||||
- PATCH /v1/agents/:agentId/sessions/:sessionId/status
|
- PATCH /v1/agents/:agentId/sessions/:sessionId/status
|
||||||
- DELETE /v1/agents/:agentId/sessions/:sessionId
|
- DELETE /v1/agents/:agentId/sessions/:sessionId
|
||||||
|
|
||||||
Session Convenience Endpoints
|
Session Convenience Endpoints
|
||||||
|
|
||||||
- GET /v1/sessions
|
- GET /v1/sessions
|
||||||
- GET /v1/sessions/:sessionId
|
- GET /v1/sessions/:sessionId
|
||||||
|
|
||||||
Session Log Endpoints (RESTful)
|
Session Log Endpoints (RESTful)
|
||||||
|
|
||||||
- POST /v1/agents/:agentId/sessions/:sessionId/logs
|
- POST /v1/agents/:agentId/sessions/:sessionId/logs
|
||||||
- GET /v1/agents/:agentId/sessions/:sessionId/logs
|
- GET /v1/agents/:agentId/sessions/:sessionId/logs
|
||||||
- GET /v1/agents/:agentId/sessions/:sessionId/logs/:logId
|
- GET /v1/agents/:agentId/sessions/:sessionId/logs/:logId
|
||||||
- PUT /v1/agents/:agentId/sessions/:sessionId/logs/:logId
|
- PUT /v1/agents/:agentId/sessions/:sessionId/logs/:logId
|
||||||
- DELETE /v1/agents/:agentId/sessions/:sessionId/logs/:logId
|
- DELETE /v1/agents/:agentId/sessions/:sessionId/logs/:logId
|
||||||
- POST /v1/agents/:agentId/sessions/:sessionId/logs/bulk
|
- POST /v1/agents/:agentId/sessions/:sessionId/logs/bulk
|
||||||
|
|
||||||
Session Log Convenience Endpoints
|
Session Log Convenience Endpoints
|
||||||
|
|
||||||
- GET /v1/sessions/:sessionId/logs
|
- GET /v1/sessions/:sessionId/logs
|
||||||
- GET /v1/session-logs/:logId
|
- GET /v1/session-logs/:logId
|
||||||
|
|
||||||
Key Considerations
|
Key Considerations
|
||||||
|
|
||||||
|
|||||||
@ -11,7 +11,7 @@ const logger = loggerService.withContext('BaseService')
|
|||||||
/**
|
/**
|
||||||
* Base service class providing shared database connection and utilities
|
* Base service class providing shared database connection and utilities
|
||||||
* for all agent-related services.
|
* for all agent-related services.
|
||||||
*
|
*
|
||||||
* Uses a migration-only approach for database schema management.
|
* Uses a migration-only approach for database schema management.
|
||||||
* The database schema is defined and maintained exclusively through
|
* The database schema is defined and maintained exclusively through
|
||||||
* migration files, ensuring a single source of truth.
|
* migration files, ensuring a single source of truth.
|
||||||
|
|||||||
@ -5,6 +5,7 @@ A production-ready database management system for Cherry Studio's autonomous age
|
|||||||
## Overview
|
## Overview
|
||||||
|
|
||||||
The Agents Database Module handles persistent storage for:
|
The Agents Database Module handles persistent storage for:
|
||||||
|
|
||||||
- **Agents**: Autonomous AI agents with configurable models, tools, and permissions
|
- **Agents**: Autonomous AI agents with configurable models, tools, and permissions
|
||||||
- **Sessions**: Agent execution sessions with status tracking and configuration overrides
|
- **Sessions**: Agent execution sessions with status tracking and configuration overrides
|
||||||
- **Session Logs**: Hierarchical message and action logs for debugging and audit trails
|
- **Session Logs**: Hierarchical message and action logs for debugging and audit trails
|
||||||
@ -37,7 +38,7 @@ database/
|
|||||||
├── index.ts # Main export file with centralized access
|
├── index.ts # Main export file with centralized access
|
||||||
├── migrator.ts # Core migration engine with transaction support
|
├── migrator.ts # Core migration engine with transaction support
|
||||||
├── migrations/ # Migration files and registry
|
├── migrations/ # Migration files and registry
|
||||||
│ ├── index.ts # Migration registry and utility functions
|
│ ├── index.ts # Migration registry and utility functions
|
||||||
│ ├── types.ts # TypeScript interfaces for migration system
|
│ ├── types.ts # TypeScript interfaces for migration system
|
||||||
│ ├── 001_initial_schema.ts # Initial agents table and indexes
|
│ ├── 001_initial_schema.ts # Initial agents table and indexes
|
||||||
│ └── 002_add_session_tables.ts # Sessions and session_logs tables
|
│ └── 002_add_session_tables.ts # Sessions and session_logs tables
|
||||||
@ -53,12 +54,12 @@ database/
|
|||||||
|
|
||||||
### File Responsibilities
|
### File Responsibilities
|
||||||
|
|
||||||
| Directory | Purpose | Key Files |
|
| Directory | Purpose | Key Files |
|
||||||
|-----------|---------|-----------|
|
| ------------- | ------------------------------------------ | ---------------------------- |
|
||||||
| `/` | Main entry point and core migration engine | `index.ts`, `migrator.ts` |
|
| `/` | Main entry point and core migration engine | `index.ts`, `migrator.ts` |
|
||||||
| `migrations/` | Version-controlled schema changes | `001_*.ts`, `002_*.ts`, etc. |
|
| `migrations/` | Version-controlled schema changes | `001_*.ts`, `002_*.ts`, etc. |
|
||||||
| `queries/` | Pre-built SQL queries by entity | `*.queries.ts` |
|
| `queries/` | Pre-built SQL queries by entity | `*.queries.ts` |
|
||||||
| `schema/` | Migration system infrastructure | `migrations.ts` |
|
| `schema/` | Migration system infrastructure | `migrations.ts` |
|
||||||
|
|
||||||
## Migration System
|
## Migration System
|
||||||
|
|
||||||
@ -73,9 +74,9 @@ graph TD
|
|||||||
E --> F[Record Migration in Tracking Table]
|
E --> F[Record Migration in Tracking Table]
|
||||||
F --> G[Commit Transaction]
|
F --> G[Commit Transaction]
|
||||||
G --> H[Migration Complete]
|
G --> H[Migration Complete]
|
||||||
|
|
||||||
E --> I[Error Occurred]
|
E --> I[Error Occurred]
|
||||||
I --> J[Rollback Transaction]
|
I --> J[Rollback Transaction]
|
||||||
J --> K[Migration Failed]
|
J --> K[Migration Failed]
|
||||||
```
|
```
|
||||||
|
|
||||||
@ -138,7 +139,7 @@ export const migration_003_add_permissions: Migration = {
|
|||||||
export const migrations: Migration[] = [
|
export const migrations: Migration[] = [
|
||||||
migration_001_initial_schema,
|
migration_001_initial_schema,
|
||||||
migration_002_add_session_tables,
|
migration_002_add_session_tables,
|
||||||
migration_003_add_permissions // Add here
|
migration_003_add_permissions // Add here
|
||||||
]
|
]
|
||||||
```
|
```
|
||||||
|
|
||||||
@ -169,12 +170,12 @@ Queries are organized by entity with consistent naming patterns:
|
|||||||
|
|
||||||
```typescript
|
```typescript
|
||||||
// Basic CRUD operations
|
// Basic CRUD operations
|
||||||
AgentQueries.insert // Create new agent
|
AgentQueries.insert // Create new agent
|
||||||
AgentQueries.update // Update existing agent
|
AgentQueries.update // Update existing agent
|
||||||
AgentQueries.getById // Get agent by ID
|
AgentQueries.getById // Get agent by ID
|
||||||
AgentQueries.list // List all agents
|
AgentQueries.list // List all agents
|
||||||
AgentQueries.delete // Delete agent
|
AgentQueries.delete // Delete agent
|
||||||
AgentQueries.count // Count total agents
|
AgentQueries.count // Count total agents
|
||||||
AgentQueries.checkExists // Check if agent exists
|
AgentQueries.checkExists // Check if agent exists
|
||||||
```
|
```
|
||||||
|
|
||||||
@ -182,13 +183,13 @@ AgentQueries.checkExists // Check if agent exists
|
|||||||
|
|
||||||
```typescript
|
```typescript
|
||||||
// Session management
|
// Session management
|
||||||
SessionQueries.insert // Create new session
|
SessionQueries.insert // Create new session
|
||||||
SessionQueries.update // Update session
|
SessionQueries.update // Update session
|
||||||
SessionQueries.updateStatus // Update just status
|
SessionQueries.updateStatus // Update just status
|
||||||
SessionQueries.getById // Get session by ID
|
SessionQueries.getById // Get session by ID
|
||||||
SessionQueries.list // List all sessions
|
SessionQueries.list // List all sessions
|
||||||
SessionQueries.listWithLimit // Paginated session list
|
SessionQueries.listWithLimit // Paginated session list
|
||||||
SessionQueries.getByStatus // Filter by status
|
SessionQueries.getByStatus // Filter by status
|
||||||
SessionQueries.getSessionWithAgent // Join with agent data
|
SessionQueries.getSessionWithAgent // Join with agent data
|
||||||
SessionQueries.getByExternalSessionId // Find by external ID
|
SessionQueries.getByExternalSessionId // Find by external ID
|
||||||
```
|
```
|
||||||
@ -197,13 +198,13 @@ SessionQueries.getByExternalSessionId // Find by external ID
|
|||||||
|
|
||||||
```typescript
|
```typescript
|
||||||
// Log operations
|
// Log operations
|
||||||
SessionLogQueries.insert // Add log entry
|
SessionLogQueries.insert // Add log entry
|
||||||
SessionLogQueries.getBySessionId // Get all logs for session
|
SessionLogQueries.getBySessionId // Get all logs for session
|
||||||
SessionLogQueries.getBySessionIdWithPagination // Paginated logs
|
SessionLogQueries.getBySessionIdWithPagination // Paginated logs
|
||||||
SessionLogQueries.getLatestBySessionId // Most recent logs
|
SessionLogQueries.getLatestBySessionId // Most recent logs
|
||||||
SessionLogQueries.update // Update log entry
|
SessionLogQueries.update // Update log entry
|
||||||
SessionLogQueries.deleteBySessionId // Clear session logs
|
SessionLogQueries.deleteBySessionId // Clear session logs
|
||||||
SessionLogQueries.countBySessionId // Count session logs
|
SessionLogQueries.countBySessionId // Count session logs
|
||||||
```
|
```
|
||||||
|
|
||||||
## Development Workflow
|
## Development Workflow
|
||||||
@ -213,15 +214,17 @@ SessionLogQueries.countBySessionId // Count session logs
|
|||||||
Follow these steps to add a new database entity:
|
Follow these steps to add a new database entity:
|
||||||
|
|
||||||
1. **Create Migration**:
|
1. **Create Migration**:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# Create new migration file
|
# Create new migration file
|
||||||
touch migrations/004_add_workflows.ts
|
touch migrations/004_add_workflows.ts
|
||||||
```
|
```
|
||||||
|
|
||||||
2. **Define Migration**:
|
2. **Define Migration**:
|
||||||
|
|
||||||
```typescript
|
```typescript
|
||||||
export const migration_004_add_workflows: Migration = {
|
export const migration_004_add_workflows: Migration = {
|
||||||
id: '004',
|
id: '004',
|
||||||
description: 'Add workflows table for agent automation',
|
description: 'Add workflows table for agent automation',
|
||||||
createdAt: new Date(),
|
createdAt: new Date(),
|
||||||
up: [
|
up: [
|
||||||
@ -239,13 +242,14 @@ Follow these steps to add a new database entity:
|
|||||||
],
|
],
|
||||||
down: [
|
down: [
|
||||||
'DROP INDEX IF EXISTS idx_workflows_status',
|
'DROP INDEX IF EXISTS idx_workflows_status',
|
||||||
'DROP INDEX IF EXISTS idx_workflows_agent_id',
|
'DROP INDEX IF EXISTS idx_workflows_agent_id',
|
||||||
'DROP TABLE IF EXISTS workflows'
|
'DROP TABLE IF EXISTS workflows'
|
||||||
]
|
]
|
||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
3. **Register Migration**:
|
3. **Register Migration**:
|
||||||
|
|
||||||
```typescript
|
```typescript
|
||||||
// migrations/index.ts
|
// migrations/index.ts
|
||||||
export const migrations = [
|
export const migrations = [
|
||||||
@ -255,17 +259,19 @@ Follow these steps to add a new database entity:
|
|||||||
```
|
```
|
||||||
|
|
||||||
4. **Create Query Module**:
|
4. **Create Query Module**:
|
||||||
|
|
||||||
```typescript
|
```typescript
|
||||||
// queries/workflow.queries.ts
|
// queries/workflow.queries.ts
|
||||||
export const WorkflowQueries = {
|
export const WorkflowQueries = {
|
||||||
insert: 'INSERT INTO workflows (id, name, agent_id, steps, status, created_at) VALUES (?, ?, ?, ?, ?, ?)',
|
insert: 'INSERT INTO workflows (id, name, agent_id, steps, status, created_at) VALUES (?, ?, ?, ?, ?, ?)',
|
||||||
getById: 'SELECT * FROM workflows WHERE id = ?',
|
getById: 'SELECT * FROM workflows WHERE id = ?',
|
||||||
getByAgentId: 'SELECT * FROM workflows WHERE agent_id = ?',
|
getByAgentId: 'SELECT * FROM workflows WHERE agent_id = ?'
|
||||||
// ... other queries
|
// ... other queries
|
||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
5. **Export Query Module**:
|
5. **Export Query Module**:
|
||||||
|
|
||||||
```typescript
|
```typescript
|
||||||
// queries/index.ts
|
// queries/index.ts
|
||||||
export { WorkflowQueries } from './workflow.queries'
|
export { WorkflowQueries } from './workflow.queries'
|
||||||
@ -316,16 +322,16 @@ Main migration management class with transaction support.
|
|||||||
```typescript
|
```typescript
|
||||||
class Migrator {
|
class Migrator {
|
||||||
constructor(database: Client)
|
constructor(database: Client)
|
||||||
|
|
||||||
// Migration management
|
// Migration management
|
||||||
addMigration(migration: Migration): void
|
addMigration(migration: Migration): void
|
||||||
addMigrations(migrations: Migration[]): void
|
addMigrations(migrations: Migration[]): void
|
||||||
|
|
||||||
// System lifecycle
|
// System lifecycle
|
||||||
initialize(): Promise<void>
|
initialize(): Promise<void>
|
||||||
migrate(options?: MigrationOptions): Promise<MigrationResult[]>
|
migrate(options?: MigrationOptions): Promise<MigrationResult[]>
|
||||||
rollbackLast(): Promise<MigrationResult | null>
|
rollbackLast(): Promise<MigrationResult | null>
|
||||||
|
|
||||||
// Status and validation
|
// Status and validation
|
||||||
getMigrationSummary(): Promise<MigrationSummary>
|
getMigrationSummary(): Promise<MigrationSummary>
|
||||||
validateMigrations(): Promise<ValidationResult>
|
validateMigrations(): Promise<ValidationResult>
|
||||||
@ -336,10 +342,10 @@ class Migrator {
|
|||||||
|
|
||||||
```typescript
|
```typescript
|
||||||
interface MigrationOptions {
|
interface MigrationOptions {
|
||||||
useTransaction?: boolean // Run in transaction (default: true)
|
useTransaction?: boolean // Run in transaction (default: true)
|
||||||
validateChecksums?: boolean // Validate migration checksums (default: true)
|
validateChecksums?: boolean // Validate migration checksums (default: true)
|
||||||
limit?: number // Max migrations to run (default: unlimited)
|
limit?: number // Max migrations to run (default: unlimited)
|
||||||
dryRun?: boolean // Preview mode (default: false)
|
dryRun?: boolean // Preview mode (default: false)
|
||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
@ -347,19 +353,19 @@ interface MigrationOptions {
|
|||||||
|
|
||||||
```typescript
|
```typescript
|
||||||
interface Migration {
|
interface Migration {
|
||||||
id: string // Unique migration identifier
|
id: string // Unique migration identifier
|
||||||
description: string // Human-readable description
|
description: string // Human-readable description
|
||||||
up: string[] // Forward migration SQL statements
|
up: string[] // Forward migration SQL statements
|
||||||
down?: string[] // Rollback SQL statements (optional)
|
down?: string[] // Rollback SQL statements (optional)
|
||||||
createdAt: Date // Migration creation timestamp
|
createdAt: Date // Migration creation timestamp
|
||||||
}
|
}
|
||||||
|
|
||||||
interface MigrationResult {
|
interface MigrationResult {
|
||||||
migration: Migration // Migration that was executed
|
migration: Migration // Migration that was executed
|
||||||
success: boolean // Execution success status
|
success: boolean // Execution success status
|
||||||
error?: string // Error message if failed
|
error?: string // Error message if failed
|
||||||
executedAt: Date // Execution timestamp
|
executedAt: Date // Execution timestamp
|
||||||
executionTime: number // Execution duration in milliseconds
|
executionTime: number // Execution duration in milliseconds
|
||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
@ -418,7 +424,7 @@ CREATE TABLE sessions (
|
|||||||
)
|
)
|
||||||
```
|
```
|
||||||
|
|
||||||
#### Session Logs Table
|
#### Session Logs Table
|
||||||
|
|
||||||
```sql
|
```sql
|
||||||
CREATE TABLE session_logs (
|
CREATE TABLE session_logs (
|
||||||
@ -447,16 +453,16 @@ import { createClient } from '@libsql/client'
|
|||||||
async function setupDatabase() {
|
async function setupDatabase() {
|
||||||
// Create database connection
|
// Create database connection
|
||||||
const db = createClient({ url: 'file:agents.db' })
|
const db = createClient({ url: 'file:agents.db' })
|
||||||
|
|
||||||
// Initialize migration system
|
// Initialize migration system
|
||||||
const migrator = new Migrator(db)
|
const migrator = new Migrator(db)
|
||||||
migrator.addMigrations(migrations)
|
migrator.addMigrations(migrations)
|
||||||
await migrator.initialize()
|
await migrator.initialize()
|
||||||
|
|
||||||
// Run pending migrations
|
// Run pending migrations
|
||||||
const results = await migrator.migrate()
|
const results = await migrator.migrate()
|
||||||
console.log(`Migrations complete: ${results.length} applied`)
|
console.log(`Migrations complete: ${results.length} applied`)
|
||||||
|
|
||||||
return db
|
return db
|
||||||
}
|
}
|
||||||
```
|
```
|
||||||
@ -470,7 +476,7 @@ async function createAgent(db: Client) {
|
|||||||
const agent = {
|
const agent = {
|
||||||
id: crypto.randomUUID(),
|
id: crypto.randomUUID(),
|
||||||
type: 'custom',
|
type: 'custom',
|
||||||
name: 'Code Review Assistant',
|
name: 'Code Review Assistant',
|
||||||
description: 'Helps review code for best practices',
|
description: 'Helps review code for best practices',
|
||||||
avatar: null,
|
avatar: null,
|
||||||
instructions: 'Review code for security, performance, and maintainability',
|
instructions: 'Review code for security, performance, and maintainability',
|
||||||
@ -487,12 +493,12 @@ async function createAgent(db: Client) {
|
|||||||
created_at: new Date().toISOString(),
|
created_at: new Date().toISOString(),
|
||||||
updated_at: new Date().toISOString()
|
updated_at: new Date().toISOString()
|
||||||
}
|
}
|
||||||
|
|
||||||
await db.execute({
|
await db.execute({
|
||||||
sql: AgentQueries.insert,
|
sql: AgentQueries.insert,
|
||||||
args: Object.values(agent)
|
args: Object.values(agent)
|
||||||
})
|
})
|
||||||
|
|
||||||
return agent.id
|
return agent.id
|
||||||
}
|
}
|
||||||
```
|
```
|
||||||
@ -504,7 +510,7 @@ import { SessionQueries, SessionLogQueries } from './database'
|
|||||||
|
|
||||||
async function createSession(db: Client, agentId: string) {
|
async function createSession(db: Client, agentId: string) {
|
||||||
const sessionId = crypto.randomUUID()
|
const sessionId = crypto.randomUUID()
|
||||||
|
|
||||||
// Create session
|
// Create session
|
||||||
await db.execute({
|
await db.execute({
|
||||||
sql: SessionQueries.insert,
|
sql: SessionQueries.insert,
|
||||||
@ -516,12 +522,21 @@ async function createSession(db: Client, agentId: string) {
|
|||||||
'Review the authentication module for security issues',
|
'Review the authentication module for security issues',
|
||||||
'running',
|
'running',
|
||||||
null, // external_session_id
|
null, // external_session_id
|
||||||
null, null, null, null, null, null, null, null, null, null, // config overrides
|
null,
|
||||||
|
null,
|
||||||
|
null,
|
||||||
|
null,
|
||||||
|
null,
|
||||||
|
null,
|
||||||
|
null,
|
||||||
|
null,
|
||||||
|
null,
|
||||||
|
null, // config overrides
|
||||||
new Date().toISOString(),
|
new Date().toISOString(),
|
||||||
new Date().toISOString()
|
new Date().toISOString()
|
||||||
]
|
]
|
||||||
})
|
})
|
||||||
|
|
||||||
// Add initial log entry
|
// Add initial log entry
|
||||||
await db.execute({
|
await db.execute({
|
||||||
sql: SessionLogQueries.insert,
|
sql: SessionLogQueries.insert,
|
||||||
@ -539,7 +554,7 @@ async function createSession(db: Client, agentId: string) {
|
|||||||
new Date().toISOString()
|
new Date().toISOString()
|
||||||
]
|
]
|
||||||
})
|
})
|
||||||
|
|
||||||
return sessionId
|
return sessionId
|
||||||
}
|
}
|
||||||
```
|
```
|
||||||
@ -552,11 +567,11 @@ async function getSessionWithAgent(db: Client, sessionId: string) {
|
|||||||
sql: SessionQueries.getSessionWithAgent,
|
sql: SessionQueries.getSessionWithAgent,
|
||||||
args: [sessionId]
|
args: [sessionId]
|
||||||
})
|
})
|
||||||
|
|
||||||
if (result.rows.length === 0) {
|
if (result.rows.length === 0) {
|
||||||
throw new Error('Session not found')
|
throw new Error('Session not found')
|
||||||
}
|
}
|
||||||
|
|
||||||
const row = result.rows[0]
|
const row = result.rows[0]
|
||||||
return {
|
return {
|
||||||
// Session data
|
// Session data
|
||||||
@ -591,7 +606,7 @@ async function getSessionWithAgent(db: Client, sessionId: string) {
|
|||||||
|
|
||||||
1. **Use Transactions**: Always wrap related operations in transactions
|
1. **Use Transactions**: Always wrap related operations in transactions
|
||||||
2. **Foreign Key Constraints**: Define relationships with proper CASCADE rules
|
2. **Foreign Key Constraints**: Define relationships with proper CASCADE rules
|
||||||
3. **Indexes**: Create indexes for foreign keys and frequently queried columns
|
3. **Indexes**: Create indexes for foreign keys and frequently queried columns
|
||||||
4. **JSON Columns**: Use JSON for flexible, extensible data structures
|
4. **JSON Columns**: Use JSON for flexible, extensible data structures
|
||||||
5. **Timestamps**: Include created_at and updated_at for audit trails
|
5. **Timestamps**: Include created_at and updated_at for audit trails
|
||||||
|
|
||||||
@ -611,7 +626,7 @@ async function getSessionWithAgent(db: Client, sessionId: string) {
|
|||||||
4. **Efficient Queries**: Use appropriate indexes and avoid N+1 query problems
|
4. **Efficient Queries**: Use appropriate indexes and avoid N+1 query problems
|
||||||
5. **Documentation**: Comment complex queries and business logic
|
5. **Documentation**: Comment complex queries and business logic
|
||||||
|
|
||||||
### Error Handling
|
### Error Handling
|
||||||
|
|
||||||
1. **Transaction Rollback**: Use transactions with proper rollback on errors
|
1. **Transaction Rollback**: Use transactions with proper rollback on errors
|
||||||
2. **Validation**: Validate input data before database operations
|
2. **Validation**: Validate input data before database operations
|
||||||
@ -628,6 +643,7 @@ async function getSessionWithAgent(db: Client, sessionId: string) {
|
|||||||
**Problem**: Migration tries to create a table that already exists.
|
**Problem**: Migration tries to create a table that already exists.
|
||||||
|
|
||||||
**Solution**: Use `IF NOT EXISTS` in all CREATE statements:
|
**Solution**: Use `IF NOT EXISTS` in all CREATE statements:
|
||||||
|
|
||||||
```sql
|
```sql
|
||||||
CREATE TABLE IF NOT EXISTS my_table (...);
|
CREATE TABLE IF NOT EXISTS my_table (...);
|
||||||
CREATE INDEX IF NOT EXISTS idx_name ON my_table(column);
|
CREATE INDEX IF NOT EXISTS idx_name ON my_table(column);
|
||||||
@ -640,6 +656,7 @@ CREATE INDEX IF NOT EXISTS idx_name ON my_table(column);
|
|||||||
**Cause**: Migration file was modified after being applied.
|
**Cause**: Migration file was modified after being applied.
|
||||||
|
|
||||||
**Solutions**:
|
**Solutions**:
|
||||||
|
|
||||||
1. **Never modify applied migrations** - create a new migration instead
|
1. **Never modify applied migrations** - create a new migration instead
|
||||||
2. If in development, reset database and re-run all migrations
|
2. If in development, reset database and re-run all migrations
|
||||||
3. For production, investigate what changed and create corrective migration
|
3. For production, investigate what changed and create corrective migration
|
||||||
@ -649,6 +666,7 @@ CREATE INDEX IF NOT EXISTS idx_name ON my_table(column);
|
|||||||
**Problem**: Cannot insert/update due to foreign key constraint violation.
|
**Problem**: Cannot insert/update due to foreign key constraint violation.
|
||||||
|
|
||||||
**Solutions**:
|
**Solutions**:
|
||||||
|
|
||||||
1. Ensure referenced record exists before creating relationship
|
1. Ensure referenced record exists before creating relationship
|
||||||
2. Check foreign key column names and types match exactly
|
2. Check foreign key column names and types match exactly
|
||||||
3. Use transactions to maintain referential integrity
|
3. Use transactions to maintain referential integrity
|
||||||
@ -657,12 +675,14 @@ CREATE INDEX IF NOT EXISTS idx_name ON my_table(column);
|
|||||||
|
|
||||||
**Problem**: Migration appears to hang during execution.
|
**Problem**: Migration appears to hang during execution.
|
||||||
|
|
||||||
**Causes**:
|
**Causes**:
|
||||||
|
|
||||||
- Long-running ALTER TABLE operations
|
- Long-running ALTER TABLE operations
|
||||||
- Lock contention from other database connections
|
- Lock contention from other database connections
|
||||||
- Complex migration with multiple DDL operations
|
- Complex migration with multiple DDL operations
|
||||||
|
|
||||||
**Solutions**:
|
**Solutions**:
|
||||||
|
|
||||||
1. Break large migrations into smaller chunks
|
1. Break large migrations into smaller chunks
|
||||||
2. Ensure no other processes are using database during migration
|
2. Ensure no other processes are using database during migration
|
||||||
3. Use migration limit option to apply migrations incrementally
|
3. Use migration limit option to apply migrations incrementally
|
||||||
@ -672,6 +692,7 @@ CREATE INDEX IF NOT EXISTS idx_name ON my_table(column);
|
|||||||
**Problem**: `SQLITE_BUSY` or `database is locked` errors.
|
**Problem**: `SQLITE_BUSY` or `database is locked` errors.
|
||||||
|
|
||||||
**Solutions**:
|
**Solutions**:
|
||||||
|
|
||||||
1. Ensure only one process accesses database at a time
|
1. Ensure only one process accesses database at a time
|
||||||
2. Close all database connections properly
|
2. Close all database connections properly
|
||||||
3. Use shorter transactions to reduce lock duration
|
3. Use shorter transactions to reduce lock duration
|
||||||
@ -689,7 +710,7 @@ migrator.addMigrations(migrations)
|
|||||||
|
|
||||||
// Enable verbose migration logging
|
// Enable verbose migration logging
|
||||||
const results = await migrator.migrate({
|
const results = await migrator.migrate({
|
||||||
dryRun: true // Preview migrations without applying
|
dryRun: true // Preview migrations without applying
|
||||||
})
|
})
|
||||||
|
|
||||||
console.log('Migration preview:', results)
|
console.log('Migration preview:', results)
|
||||||
@ -782,4 +803,4 @@ For questions about the database system:
|
|||||||
3. **Examine Schema**: Use SQLite tools to inspect current database schema
|
3. **Examine Schema**: Use SQLite tools to inspect current database schema
|
||||||
4. **Test in Development**: Always test schema changes in development environment first
|
4. **Test in Development**: Always test schema changes in development environment first
|
||||||
|
|
||||||
This database system is designed to be robust and production-ready. Following the migration-only approach and best practices outlined in this guide will ensure reliable, maintainable database operations for your agent management system.
|
This database system is designed to be robust and production-ready. Following the migration-only approach and best practices outlined in this guide will ensure reliable, maintainable database operations for your agent management system.
|
||||||
|
|||||||
@ -16,7 +16,7 @@ const logger = loggerService.withContext('Migrator')
|
|||||||
|
|
||||||
/**
|
/**
|
||||||
* Database migration manager with transaction support
|
* Database migration manager with transaction support
|
||||||
*
|
*
|
||||||
* This class manages database schema evolution through migrations.
|
* This class manages database schema evolution through migrations.
|
||||||
* All table and index definitions are maintained exclusively in migration files,
|
* All table and index definitions are maintained exclusively in migration files,
|
||||||
* providing a single source of truth for the database schema.
|
* providing a single source of truth for the database schema.
|
||||||
|
|||||||
Loading…
Reference in New Issue
Block a user