Northern_Lights/cherry-studio
1
0
Fork 0
mirror of https://github.com/CherryHQ/cherry-studio.git synced 2025-12-30 07:39:06 +08:00
Code Issues Actions 3 Packages Projects Releases Wiki Activity

docs(data): update README and remove outdated API design guidelines

Browse Source 
- Revised the README files for shared data and main data layers to improve clarity and structure.
- Consolidated documentation on shared data types and API types, removing the now-deleted `api-design-guidelines.md`.
- Streamlined directory structure descriptions and updated links to relevant documentation.
- Enhanced quick reference sections for better usability and understanding of the data architecture.
This commit is contained in:
fullex 2025-12-29 17:15:06 +08:00
parent e4fd1af1b8
commit 819c209821
19 changed files with 2438 additions and 1355 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
.github/CODEOWNERS vendored
View File

@ -9,6 +9,8 @@
/src/main/data/ @0xfullex
/src/renderer/src/data/ @0xfullex
/v2-refactor-temp/ @0xfullex
/docs/en/references/data/ @0xfullex
/docs/zh/references/data/ @0xfullex
/packages/ui/ @MyPrototypeWhat

193
docs/en/references/data/README.md Normal file
View File

@ -0,0 +1,193 @@
# Data System Reference
This is the main entry point for Cherry Studio's data management documentation. The application uses three distinct data systems based on data characteristics.
## Quick Navigation
### System Overview (Architecture)
- [Cache Overview](./cache-overview.md) - Three-tier caching architecture
- [Preference Overview](./preference-overview.md) - User settings management
- [DataApi Overview](./data-api-overview.md) - Business data API architecture
### Usage Guides (Code Examples)
- [Cache Usage](./cache-usage.md) - useCache hooks, CacheService examples
- [Preference Usage](./preference-usage.md) - usePreference hook, PreferenceService examples
- [DataApi in Renderer](./data-api-in-renderer.md) - useQuery/useMutation, DataApiService
- [DataApi in Main](./data-api-in-main.md) - Handlers, Services, Repositories patterns
### Reference Guides (Coding Standards)
- [API Design Guidelines](./api-design-guidelines.md) - RESTful design rules
- [Database Patterns](./database-patterns.md) - DB naming, schema patterns
- [API Types](./api-types.md) - API type system, schemas, error handling
- [V2 Migration Guide](./v2-migration-guide.md) - Migration system
---
## Choosing the Right System
### Quick Decision Table
| Service | Data Characteristics | Lifecycle | Data Loss Impact | Examples |
|---------|---------------------|-----------|------------------|----------|
| **CacheService** | Regenerable, temporary | ≤ App process or survives restart | None to minimal | API responses, computed results, UI state |
| **PreferenceService** | User settings, key-value | Permanent until changed | Low (can rebuild) | Theme, language, font size, shortcuts |
| **DataApiService** | Business data, structured | Permanent | **Severe** (irreplaceable) | Topics, messages, files, knowledge base |
### Decision Flowchart
Ask these questions in order:
1. **Can this data be regenerated or lost without affecting the user?**
- Yes → **CacheService**
- No → Continue to #2
2. **Is this a user-configurable setting that affects app behavior?**
- Yes → Does it have a fixed key and stable value structure?
- Yes → **PreferenceService**
- No (structure changes often) → **DataApiService**
- No → Continue to #3
3. **Is this business data created/accumulated through user activity?**
- Yes → **DataApiService**
- No → Reconsider #1 (most data falls into one of these categories)
---
## System Characteristics
### CacheService - Runtime & Cache Data
Use CacheService when:
- Data can be **regenerated or lost without user impact**
- No backup or cross-device synchronization needed
- Lifecycle is tied to component, window, or app session
**Two sub-categories**:
1. **Performance cache**: Computed results, API responses, expensive calculations
2. **UI state cache**: Temporary settings, scroll positions, panel states
**Three tiers based on persistence needs**:
- `useCache` (memory): Lost on app restart, component-level sharing
- `useSharedCache` (shared): Cross-window sharing, lost on restart
- `usePersistCache` (persist): Survives app restarts via localStorage
```typescript
// Good: Temporary computed results
const [searchResults, setSearchResults] = useCache('search.results', [])
// Good: UI state that can be lost
const [sidebarCollapsed, setSidebarCollapsed] = useSharedCache('ui.sidebar.collapsed', false)
// Good: Recent items (nice to have, not critical)
const [recentSearches, setRecentSearches] = usePersistCache('search.recent', [])
```
### PreferenceService - User Preferences
Use PreferenceService when:
- Data is a **user-modifiable setting that affects app behavior**
- Structure is key-value with **predefined keys** (users modify values, not keys)
- **Value structure is stable** (won't change frequently)
- Data loss has **low impact** (user can reconfigure)
**Key characteristics**:
- Auto-syncs across all windows
- Each preference item should be **atomic** (one setting = one key)
- Values are typically: boolean, string, number, or simple array/object
```typescript
// Good: App behavior settings
const [theme, setTheme] = usePreference('app.theme.mode')
const [language, setLanguage] = usePreference('app.language')
const [fontSize, setFontSize] = usePreference('chat.message.font_size')
// Good: Feature toggles
const [showTimestamp, setShowTimestamp] = usePreference('chat.display.show_timestamp')
```
### DataApiService - User Data
Use DataApiService when:
- Data is **business data accumulated through user activity**
- Data is **structured with dedicated schemas/tables**
- Users can **create, delete, modify records** (no fixed limit)
- Data loss would be **severe and irreplaceable**
- Data volume can be **large** (potentially GBs)
**Key characteristics**:
- No automatic window sync (fetch on demand for fresh data)
- May contain sensitive data (encryption consideration)
- Requires proper CRUD operations and transactions
```typescript
// Good: User-generated business data
const { data: topics } = useQuery('/topics')
const { trigger: createTopic } = useMutation('/topics', 'POST')
// Good: Conversation history (irreplaceable)
const { data: messages } = useQuery('/messages', { query: { topicId } })
// Good: User files and knowledge base
const { data: files } = useQuery('/files')
```
---
## Common Anti-patterns
| Wrong Choice | Why It's Wrong | Correct Choice |
|--------------|----------------|----------------|
| Storing AI provider configs in Cache | User loses configured providers on restart | **PreferenceService** |
| Storing conversation history in Preferences | Unbounded growth, complex structure | **DataApiService** |
| Storing topic list in Preferences | User-created records, can grow large | **DataApiService** |
| Storing theme/language in DataApi | Overkill for simple key-value settings | **PreferenceService** |
| Storing API responses in DataApi | Regenerable data, doesn't need persistence | **CacheService** |
| Storing window positions in Preferences | Can be lost without impact | **CacheService** (persist tier) |
## Edge Cases
- **Recently used items** (e.g., recent files, recent searches): Use `usePersistCache` - nice to have but not critical if lost
- **Draft content** (e.g., unsaved message): Use `useSharedCache` for cross-window, consider auto-save to DataApi for recovery
- **Computed statistics**: Use `useCache` with TTL - regenerate when expired
- **User-created templates/presets**: Use **DataApiService** - user-generated content that can grow
---
## Architecture Overview
```
┌─────────────────┐
│ React Components│
└─────────┬───────┘
│
┌─────────▼───────┐
│ React Hooks │ ← useDataApi, usePreference, useCache
└─────────┬───────┘
│
┌─────────▼───────┐
│ Services │ ← DataApiService, PreferenceService, CacheService
└─────────┬───────┘
│
┌─────────▼───────┐
│ IPC Layer │ ← Main Process Communication
└─────────────────┘
```
## Related Source Code
### Type Definitions
- `packages/shared/data/api/` - API type system
- `packages/shared/data/cache/` - Cache type definitions
- `packages/shared/data/preference/` - Preference type definitions
### Main Process Implementation
- `src/main/data/api/` - API server and handlers
- `src/main/data/CacheService.ts` - Cache service
- `src/main/data/PreferenceService.ts` - Preference service
- `src/main/data/db/` - Database schemas
### Renderer Process Implementation
- `src/renderer/src/data/DataApiService.ts` - API client
- `src/renderer/src/data/CacheService.ts` - Cache service
- `src/renderer/src/data/PreferenceService.ts` - Preference service
- `src/renderer/src/data/hooks/` - React hooks

0
packages/shared/data/api/api-design-guidelines.md → docs/en/references/data/api-design-guidelines.md
View File

238
docs/en/references/data/api-types.md Normal file
View File

@ -0,0 +1,238 @@
# Data API Type System
This directory contains the type definitions and utilities for Cherry Studio's Data API system, which provides type-safe IPC communication between renderer and main processes.
## Directory Structure
```
packages/shared/data/api/
├── index.ts # Barrel export for infrastructure types
├── apiTypes.ts # Core request/response types and API utilities
├── apiPaths.ts # Path template literal type utilities
├── apiErrors.ts # Error handling: ErrorCode, DataApiError class, factory
└── schemas/
├── index.ts # Schema composition (merges all domain schemas)
└── test.ts # Test API schema and DTOs
```
## File Responsibilities
| File | Purpose |
|------|---------|
| `apiTypes.ts` | Core types (`DataRequest`, `DataResponse`, `ApiClient`) and schema utilities |
| `apiPaths.ts` | Template literal types for path resolution (`/items/:id` → `/items/${string}`) |
| `apiErrors.ts` | `ErrorCode` enum, `DataApiError` class, `DataApiErrorFactory`, retryability config |
| `index.ts` | Unified export of infrastructure types (not domain DTOs) |
| `schemas/index.ts` | Composes all domain schemas into `ApiSchemas` using intersection types |
| `schemas/*.ts` | Domain-specific API definitions and DTOs |
## Import Conventions
### Infrastructure Types (via barrel export)
Use the barrel export for common API infrastructure:
```typescript
import type {
DataRequest,
DataResponse,
ApiClient,
PaginatedResponse
} from '@shared/data/api'
import {
ErrorCode,
DataApiError,
DataApiErrorFactory,
isDataApiError,
toDataApiError
} from '@shared/data/api'
```
### Domain DTOs (directly from schema files)
Import domain-specific types directly from their schema files:
```typescript
// Topic domain
import type { Topic, CreateTopicDto, UpdateTopicDto } from '@shared/data/api/schemas/topic'
// Message domain
import type { Message, CreateMessageDto } from '@shared/data/api/schemas/message'
// Test domain (development)
import type { TestItem, CreateTestItemDto } from '@shared/data/api/schemas/test'
```
## Adding a New Domain Schema
1. Create the schema file (e.g., `schemas/topic.ts`):
```typescript
import type { PaginatedResponse } from '../apiTypes'
// Domain models
export interface Topic {
id: string
name: string
createdAt: string
}
export interface CreateTopicDto {
name: string
}
// API Schema - validation happens via AssertValidSchemas in index.ts
export interface TopicSchemas {
'/topics': {
GET: {
response: PaginatedResponse<Topic> // response is required
}
POST: {
body: CreateTopicDto
response: Topic
}
}
'/topics/:id': {
GET: {
params: { id: string }
response: Topic
}
}
}
```
**Validation**: Schemas are validated at composition level via `AssertValidSchemas` in `schemas/index.ts`:
- Ensures only valid HTTP methods (GET, POST, PUT, DELETE, PATCH)
- Requires `response` field for each endpoint
- Invalid schemas cause TypeScript errors at the composition point
> **Design Guidelines**: Before creating new schemas, review the [API Design Guidelines](./api-design-guidelines.md) for path naming, HTTP methods, and error handling conventions.
2. Register in `schemas/index.ts`:
```typescript
import type { TopicSchemas } from './topic'
// AssertValidSchemas provides fallback validation even if ValidateSchema is forgotten
export type ApiSchemas = AssertValidSchemas<TestSchemas & TopicSchemas>
```
3. Implement handlers in `src/main/data/api/handlers/`
## Type Safety Features
### Path Resolution
The system uses template literal types to map concrete paths to schema paths:
```typescript
// Concrete path '/topics/abc123' maps to schema path '/topics/:id'
api.get('/topics/abc123') // TypeScript knows this returns Topic
```
### Exhaustive Handler Checking
`ApiImplementation` type ensures all schema endpoints have handlers:
```typescript
// TypeScript will error if any endpoint is missing
const handlers: ApiImplementation = {
'/topics': {
GET: async () => { /* ... */ },
POST: async ({ body }) => { /* ... */ }
}
// Missing '/topics/:id' would cause compile error
}
```
### Type-Safe Client
`ApiClient` provides fully typed methods:
```typescript
const topic = await api.get('/topics/123') // Returns Topic
const topics = await api.get('/topics', { query: { page: 1 } }) // Returns PaginatedResponse<Topic>
await api.post('/topics', { body: { name: 'New' } }) // Body is typed as CreateTopicDto
```
## Error Handling
The error system provides type-safe error handling with automatic retryability detection:
```typescript
import {
DataApiError,
DataApiErrorFactory,
ErrorCode,
isDataApiError,
toDataApiError
} from '@shared/data/api'
// Create errors using the factory (recommended)
throw DataApiErrorFactory.notFound('Topic', id)
throw DataApiErrorFactory.validation({ name: ['Name is required'] })
throw DataApiErrorFactory.timeout('fetch topics', 3000)
throw DataApiErrorFactory.database(originalError, 'insert topic')
// Or create directly with the class
throw new DataApiError(
ErrorCode.NOT_FOUND,
'Topic not found',
404,
{ resource: 'Topic', id: 'abc123' }
)
// Check if error is retryable (for automatic retry logic)
if (error instanceof DataApiError && error.isRetryable) {
await retry(operation)
}
// Check error type
if (error instanceof DataApiError) {
if (error.isClientError) {
// 4xx - issue with the request
} else if (error.isServerError) {
// 5xx - server-side issue
}
}
// Convert any error to DataApiError
const apiError = toDataApiError(unknownError, 'context')
// Serialize for IPC (Main → Renderer)
const serialized = apiError.toJSON()
// Reconstruct from IPC response (Renderer)
const reconstructed = DataApiError.fromJSON(response.error)
```
### Retryable Error Codes
The following errors are automatically considered retryable:
- `SERVICE_UNAVAILABLE` (503)
- `TIMEOUT` (504)
- `RATE_LIMIT_EXCEEDED` (429)
- `DATABASE_ERROR` (500)
- `INTERNAL_SERVER_ERROR` (500)
- `RESOURCE_LOCKED` (423)
## Architecture Overview
```
Renderer Main
────────────────────────────────────────────────────
DataApiService ──IPC──► IpcAdapter ──► ApiServer
│ │
│ ▼
ApiClient MiddlewareEngine
(typed) │
▼
Handlers
(typed)
```
- **Renderer**: Uses `DataApiService` with type-safe `ApiClient` interface
- **IPC**: Requests serialized via `IpcAdapter`
- **Main**: `ApiServer` routes to handlers through `MiddlewareEngine`
- **Type Safety**: End-to-end types from client call to handler implementation

125
docs/en/references/data/cache-overview.md Normal file
View File

@ -0,0 +1,125 @@
# Cache System Overview
The Cache system provides a three-tier caching architecture for temporary and regenerable data across the Cherry Studio application.
## Purpose
CacheService handles data that:
- Can be **regenerated or lost without user impact**
- Requires no backup or cross-device synchronization
- Has lifecycle tied to component, window, or app session
## Three-Tier Architecture
| Tier | Scope | Persistence | Use Case |
|------|-------|-------------|----------|
| **Memory Cache** | Component-level | Lost on app restart | API responses, computed results |
| **Shared Cache** | Cross-window | Lost on app restart | Window state, cross-window coordination |
| **Persist Cache** | Cross-window + localStorage | Survives app restarts | Recent items, non-critical preferences |
### Memory Cache
- Fastest access, in-process memory
- Isolated per renderer process
- Best for: expensive computations, API response caching
### Shared Cache
- Synchronized across all windows via IPC
- Main process acts as the source of truth
- Best for: window layouts, shared UI state
### Persist Cache
- Backed by localStorage in renderer
- Main process maintains authoritative copy
- Best for: recent files, search history, non-critical state
## Key Features
### TTL (Time To Live) Support
```typescript
// Cache with 30-second expiration
cacheService.set('temp.calculation', result, 30000)
```
### Hook Reference Tracking
- Prevents deletion of cache entries while React hooks are subscribed
- Automatic cleanup when components unmount
### Cross-Window Synchronization
- Shared and Persist caches sync across all windows
- Uses IPC broadcast for real-time updates
- Main process resolves conflicts
### Type Safety
- Schema-based keys for compile-time checking
- Casual methods for dynamic keys with manual typing
## Data Categories
### Performance Cache (Memory tier)
- Computed results from expensive operations
- API response caching
- Parsed/transformed data
### UI State Cache (Shared tier)
- Sidebar collapsed state
- Panel dimensions
- Scroll positions
### Non-Critical Persistence (Persist tier)
- Recently used items
- Search history
- User-customized but regenerable data
## Architecture Diagram
```
┌─────────────────────────────────────────────────────────────┐
│ Renderer Process │
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
│ │ useCache │ │useSharedCache│ │usePersistCache│ │
│ └──────┬──────┘ └──────┬──────┘ └──────┬──────┘ │
│ │ │ │ │
│ └────────────────┼────────────────┘ │
│ ▼ │
│ ┌─────────────────────┐ │
│ │ CacheService │ │
│ │ (Renderer) │ │
│ └──────────┬──────────┘ │
└─────────────────────────┼────────────────────────────────────┘
│ IPC (shared/persist only)
┌─────────────────────────┼────────────────────────────────────┐
│ Main Process ▼ │
│ ┌─────────────────────┐ │
│ │ CacheService │ │
│ │ (Main) │ │
│ └─────────────────────┘ │
│ - Source of truth for shared/persist │
│ - Broadcasts updates to all windows │
└──────────────────────────────────────────────────────────────┘
```
## Main vs Renderer Responsibilities
### Main Process CacheService
- Manages shared and persist cache storage
- Handles IPC requests from renderers
- Broadcasts updates to all windows
- Manages TTL expiration for shared caches
### Renderer Process CacheService
- Manages local memory cache
- Proxies shared/persist operations to Main
- Handles hook subscriptions and updates
- Local TTL management for memory cache
## Usage Summary
For detailed code examples and API usage, see [Cache Usage Guide](./cache-usage.md).
| Method | Tier | Type Safety |
|--------|------|-------------|
| `useCache` / `get` / `set` | Memory | Schema-based keys |
| `getCasual` / `setCasual` | Memory | Dynamic keys (manual typing) |
| `useSharedCache` / `getShared` / `setShared` | Shared | Schema-based keys |
| `getSharedCasual` / `setSharedCasual` | Shared | Dynamic keys (manual typing) |
| `usePersistCache` / `getPersist` / `setPersist` | Persist | Schema-based keys only |

246
docs/en/references/data/cache-usage.md Normal file
View File

@ -0,0 +1,246 @@
# Cache Usage Guide
This guide covers how to use the Cache system in React components and services.
## React Hooks
### useCache (Memory Cache)
Memory cache is lost on app restart. Best for temporary computed results.
```typescript
import { useCache } from '@data/hooks/useCache'
// Basic usage with default value
const [counter, setCounter] = useCache('ui.counter', 0)
// Update the value
setCounter(counter + 1)
// With TTL (30 seconds)
const [searchResults, setSearchResults] = useCache('search.results', [], { ttl: 30000 })
```
### useSharedCache (Cross-Window Cache)
Shared cache syncs across all windows, lost on app restart.
```typescript
import { useSharedCache } from '@data/hooks/useCache'
// Cross-window state
const [layout, setLayout] = useSharedCache('window.layout', defaultLayout)
// Sidebar state shared between windows
const [sidebarCollapsed, setSidebarCollapsed] = useSharedCache('ui.sidebar.collapsed', false)
```
### usePersistCache (Persistent Cache)
Persist cache survives app restarts via localStorage.
```typescript
import { usePersistCache } from '@data/hooks/useCache'
// Recent files list (survives restart)
const [recentFiles, setRecentFiles] = usePersistCache('app.recent_files', [])
// Search history
const [searchHistory, setSearchHistory] = usePersistCache('search.history', [])
```
## CacheService Direct Usage
For non-React code or more control, use CacheService directly.
### Memory Cache
```typescript
import { cacheService } from '@data/CacheService'
// Type-safe (schema key)
cacheService.set('temp.calculation', result)
const result = cacheService.get('temp.calculation')
// With TTL (30 seconds)
cacheService.set('temp.calculation', result, 30000)
// Casual (dynamic key, manual type)
cacheService.setCasual<TopicCache>(`topic:${id}`, topicData)
const topic = cacheService.getCasual<TopicCache>(`topic:${id}`)
// Check existence
if (cacheService.has('temp.calculation')) {
// ...
}
// Delete
cacheService.delete('temp.calculation')
cacheService.deleteCasual(`topic:${id}`)
```
### Shared Cache
```typescript
// Type-safe (schema key)
cacheService.setShared('window.layout', layoutConfig)
const layout = cacheService.getShared('window.layout')
// Casual (dynamic key)
cacheService.setSharedCasual<WindowState>(`window:${windowId}`, state)
const state = cacheService.getSharedCasual<WindowState>(`window:${windowId}`)
// Delete
cacheService.deleteShared('window.layout')
cacheService.deleteSharedCasual(`window:${windowId}`)
```
### Persist Cache
```typescript
// Schema keys only (no Casual methods for persist)
cacheService.setPersist('app.recent_files', recentFiles)
const files = cacheService.getPersist('app.recent_files')
// Delete
cacheService.deletePersist('app.recent_files')
```
## Type-Safe vs Casual Methods
### Type-Safe Methods
- Use predefined keys from cache schema
- Full auto-completion and type inference
- Compile-time key validation
```typescript
// Key 'ui.counter' must exist in schema
const [counter, setCounter] = useCache('ui.counter', 0)
```
### Casual Methods
- Use dynamically constructed keys
- Require manual type specification via generics
- No compile-time key validation
```typescript
// Dynamic key, must specify type
const topic = cacheService.getCasual<TopicCache>(`topic:${id}`)
```
### When to Use Which
| Scenario | Method | Example |
|----------|--------|---------|
| Fixed cache keys | Type-safe | `useCache('ui.counter')` |
| Entity caching by ID | Casual | `getCasual<Topic>(\`topic:${id}\`)` |
| Session-based keys | Casual | `setCasual(\`session:${sessionId}\`)` |
| UI state | Type-safe | `useSharedCache('window.layout')` |
## Common Patterns
### Caching Expensive Computations
```typescript
function useExpensiveData(input: string) {
const [cached, setCached] = useCache(`computed:${input}`, null)
useEffect(() => {
if (cached === null) {
const result = expensiveComputation(input)
setCached(result)
}
}, [input, cached, setCached])
return cached
}
```
### Cross-Window Coordination
```typescript
// Window A: Update shared state
const [activeFile, setActiveFile] = useSharedCache('editor.activeFile', null)
setActiveFile(selectedFile)
// Window B: Reacts to change automatically
const [activeFile] = useSharedCache('editor.activeFile', null)
// activeFile updates when Window A changes it
```
### Recent Items with Limit
```typescript
const [recentItems, setRecentItems] = usePersistCache('app.recentItems', [])
const addRecentItem = (item: Item) => {
setRecentItems(prev => {
const filtered = prev.filter(i => i.id !== item.id)
return [item, ...filtered].slice(0, 10) // Keep last 10
})
}
```
### Cache with Expiration Check
```typescript
interface CachedData<T> {
data: T
timestamp: number
}
function useCachedWithExpiry<T>(key: string, fetcher: () => Promise<T>, maxAge: number) {
const [cached, setCached] = useCache<CachedData<T> | null>(key, null)
const [data, setData] = useState<T | null>(cached?.data ?? null)
useEffect(() => {
const isExpired = !cached || Date.now() - cached.timestamp > maxAge
if (isExpired) {
fetcher().then(result => {
setCached({ data: result, timestamp: Date.now() })
setData(result)
})
}
}, [key, maxAge])
return data
}
```
## Adding New Cache Keys
### 1. Add to Cache Schema
```typescript
// packages/shared/data/cache/cacheSchemas.ts
export interface CacheSchema {
// Existing keys...
'myFeature.data': MyDataType
}
```
### 2. Define Value Type (if complex)
```typescript
// packages/shared/data/cache/cacheValueTypes.ts
export interface MyDataType {
items: string[]
lastUpdated: number
}
```
### 3. Use in Code
```typescript
// Now type-safe
const [data, setData] = useCache('myFeature.data', defaultValue)
```
## Best Practices
1. **Choose the right tier**: Memory for temp, Shared for cross-window, Persist for survival
2. **Use TTL for stale data**: Prevent serving outdated cached values
3. **Prefer type-safe keys**: Add to schema when possible
4. **Clean up dynamic keys**: Remove casual cache entries when no longer needed
5. **Consider data size**: Persist cache uses localStorage (limited to ~5MB)

360
docs/en/references/data/data-api-in-main.md Normal file
View File

@ -0,0 +1,360 @@
# DataApi in Main Process
This guide covers how to implement API handlers, services, and repositories in the Main process.
## Architecture Layers
```
Handlers → Services → Repositories → Database
```
- **Handlers**: Thin layer, extract params, call service, transform response
- **Services**: Business logic, validation, transaction coordination
- **Repositories**: Data access (for complex domains)
- **Database**: Drizzle ORM + SQLite
## Implementing Handlers
### Location
`src/main/data/api/handlers/`
### Handler Responsibilities
- Extract parameters from request
- Delegate to business service
- Transform response for IPC
- **NO business logic here**
### Example Handler
```typescript
// handlers/topic.ts
import type { ApiImplementation } from '@shared/data/api'
import { TopicService } from '@data/services/TopicService'
export const topicHandlers: Partial<ApiImplementation> = {
'/topics': {
GET: async ({ query }) => {
const { page = 1, limit = 20 } = query ?? {}
return await TopicService.getInstance().list({ page, limit })
},
POST: async ({ body }) => {
return await TopicService.getInstance().create(body)
}
},
'/topics/:id': {
GET: async ({ params }) => {
return await TopicService.getInstance().getById(params.id)
},
PUT: async ({ params, body }) => {
return await TopicService.getInstance().replace(params.id, body)
},
PATCH: async ({ params, body }) => {
return await TopicService.getInstance().update(params.id, body)
},
DELETE: async ({ params }) => {
await TopicService.getInstance().delete(params.id)
}
}
}
```
### Register Handlers
```typescript
// handlers/index.ts
import { topicHandlers } from './topic'
import { messageHandlers } from './message'
export const allHandlers: ApiImplementation = {
...topicHandlers,
...messageHandlers
}
```
## Implementing Services
### Location
`src/main/data/services/`
### Service Responsibilities
- Business validation
- Transaction coordination
- Domain workflows
- Call repositories or direct Drizzle
### Example Service
```typescript
// services/TopicService.ts
import { DbService } from '@data/db/DbService'
import { TopicRepository } from '@data/repositories/TopicRepository'
import { DataApiErrorFactory } from '@shared/data/api'
export class TopicService {
private static instance: TopicService
private topicRepo: TopicRepository
private constructor() {
this.topicRepo = new TopicRepository()
}
static getInstance(): TopicService {
if (!this.instance) {
this.instance = new TopicService()
}
return this.instance
}
async list(options: { page: number; limit: number }) {
return await this.topicRepo.findAll(options)
}
async getById(id: string) {
const topic = await this.topicRepo.findById(id)
if (!topic) {
throw DataApiErrorFactory.notFound('Topic', id)
}
return topic
}
async create(data: CreateTopicDto) {
// Business validation
this.validateTopicData(data)
return await this.topicRepo.create(data)
}
async update(id: string, data: Partial<UpdateTopicDto>) {
const existing = await this.getById(id) // Throws if not found
return await this.topicRepo.update(id, data)
}
async delete(id: string) {
await this.getById(id) // Throws if not found
await this.topicRepo.delete(id)
}
private validateTopicData(data: CreateTopicDto) {
if (!data.name?.trim()) {
throw DataApiErrorFactory.validation({ name: ['Name is required'] })
}
}
}
```
### Service with Transaction
```typescript
async createTopicWithMessage(data: CreateTopicWithMessageDto) {
return await DbService.transaction(async (tx) => {
// Create topic
const topic = await this.topicRepo.create(data.topic, tx)
// Create initial message
const message = await this.messageRepo.create({
...data.message,
topicId: topic.id
}, tx)
return { topic, message }
})
}
```
## Implementing Repositories
### When to Use Repository Pattern
Use repositories for **complex domains**:
- ✅ Complex queries (joins, subqueries, aggregations)
- ✅ GB-scale data requiring pagination
- ✅ Complex transactions involving multiple tables
- ✅ Reusable data access patterns
- ✅ High testing requirements
### When to Use Direct Drizzle
Use direct Drizzle for **simple domains**:
- ✅ Simple CRUD operations
- ✅ Small datasets (< 100MB)
- ✅ Domain-specific queries with no reuse
- ✅ Fast development is priority
### Example Repository
```typescript
// repositories/TopicRepository.ts
import { eq, desc, sql } from 'drizzle-orm'
import { DbService } from '@data/db/DbService'
import { topicTable } from '@data/db/schemas/topic'
export class TopicRepository {
async findAll(options: { page: number; limit: number }) {
const { page, limit } = options
const offset = (page - 1) * limit
const [items, countResult] = await Promise.all([
DbService.db
.select()
.from(topicTable)
.orderBy(desc(topicTable.updatedAt))
.limit(limit)
.offset(offset),
DbService.db
.select({ count: sql<number>`count(*)` })
.from(topicTable)
])
return {
items,
total: countResult[0].count,
page,
limit
}
}
async findById(id: string, tx?: Transaction) {
const db = tx || DbService.db
const [topic] = await db
.select()
.from(topicTable)
.where(eq(topicTable.id, id))
.limit(1)
return topic ?? null
}
async create(data: CreateTopicDto, tx?: Transaction) {
const db = tx || DbService.db
const [topic] = await db
.insert(topicTable)
.values(data)
.returning()
return topic
}
async update(id: string, data: Partial<UpdateTopicDto>, tx?: Transaction) {
const db = tx || DbService.db
const [topic] = await db
.update(topicTable)
.set(data)
.where(eq(topicTable.id, id))
.returning()
return topic
}
async delete(id: string, tx?: Transaction) {
const db = tx || DbService.db
await db
.delete(topicTable)
.where(eq(topicTable.id, id))
}
}
```
### Example: Direct Drizzle in Service
For simple domains, skip the repository:
```typescript
// services/TagService.ts
import { eq } from 'drizzle-orm'
import { DbService } from '@data/db/DbService'
import { tagTable } from '@data/db/schemas/tag'
export class TagService {
async getAll() {
return await DbService.db.select().from(tagTable)
}
async create(name: string) {
const [tag] = await DbService.db
.insert(tagTable)
.values({ name })
.returning()
return tag
}
async delete(id: string) {
await DbService.db
.delete(tagTable)
.where(eq(tagTable.id, id))
}
}
```
## Error Handling
### Using DataApiErrorFactory
```typescript
import { DataApiErrorFactory } from '@shared/data/api'
// Not found
throw DataApiErrorFactory.notFound('Topic', id)
// Validation error
throw DataApiErrorFactory.validation({
name: ['Name is required', 'Name must be at least 3 characters'],
email: ['Invalid email format']
})
// Database error
try {
await db.insert(table).values(data)
} catch (error) {
throw DataApiErrorFactory.database(error, 'insert topic')
}
// Invalid operation
throw DataApiErrorFactory.invalidOperation(
'delete root message',
'cascade=true required'
)
// Conflict
throw DataApiErrorFactory.conflict('Topic name already exists')
// Timeout
throw DataApiErrorFactory.timeout('fetch topics', 3000)
```
## Adding New Endpoints
### Step-by-Step
1. **Define schema** in `packages/shared/data/api/schemas/`
```typescript
// schemas/topic.ts
export interface TopicSchemas {
'/topics': {
GET: { response: PaginatedResponse<Topic> }
POST: { body: CreateTopicDto; response: Topic }
}
}
```
2. **Register schema** in `schemas/index.ts`
```typescript
export type ApiSchemas = AssertValidSchemas<TopicSchemas & MessageSchemas>
```
3. **Create service** in `services/`
4. **Create repository** (if complex) in `repositories/`
5. **Implement handler** in `handlers/`
6. **Register handler** in `handlers/index.ts`
## Best Practices
1. **Keep handlers thin**: Only extract params and call services
2. **Put logic in services**: All business rules belong in services
3. **Use repositories selectively**: Simple CRUD doesn't need a repository
4. **Always use `.returning()`**: Get inserted/updated data without re-querying
5. **Support transactions**: Accept optional `tx` parameter in repositories
6. **Validate in services**: Business validation belongs in the service layer
7. **Use error factory**: Consistent error creation with `DataApiErrorFactory`

298
docs/en/references/data/data-api-in-renderer.md Normal file
View File

@ -0,0 +1,298 @@
# DataApi in Renderer
This guide covers how to use the DataApi system in React components and the renderer process.
## React Hooks
### useQuery (GET Requests)
Fetch data with automatic caching and revalidation via SWR.
```typescript
import { useQuery } from '@data/hooks/useDataApi'
// Basic usage
const { data, loading, error } = useQuery('/topics')
// With query parameters
const { data: messages } = useQuery('/messages', {
query: { topicId: 'abc123', page: 1, limit: 20 }
})
// With path parameters (inferred from path)
const { data: topic } = useQuery('/topics/abc123')
// Conditional fetching
const { data } = useQuery(topicId ? `/topics/${topicId}` : null)
// With refresh callback
const { data, mutate } = useQuery('/topics')
// Refresh data
await mutate()
```
### useMutation (POST/PUT/PATCH/DELETE)
Perform data modifications with loading states.
```typescript
import { useMutation } from '@data/hooks/useDataApi'
// Create (POST)
const { trigger: createTopic, isMutating } = useMutation('/topics', 'POST')
const newTopic = await createTopic({ body: { name: 'New Topic' } })
// Update (PUT - full replacement)
const { trigger: replaceTopic } = useMutation('/topics/abc123', 'PUT')
await replaceTopic({ body: { name: 'Updated Name', description: '...' } })
// Partial Update (PATCH)
const { trigger: updateTopic } = useMutation('/topics/abc123', 'PATCH')
await updateTopic({ body: { name: 'New Name' } })
// Delete
const { trigger: deleteTopic } = useMutation('/topics/abc123', 'DELETE')
await deleteTopic()
```
## DataApiService Direct Usage
For non-React code or more control.
```typescript
import { dataApiService } from '@data/DataApiService'
// GET request
const topics = await dataApiService.get('/topics')
const topic = await dataApiService.get('/topics/abc123')
const messages = await dataApiService.get('/topics/abc123/messages', {
query: { page: 1, limit: 20 }
})
// POST request
const newTopic = await dataApiService.post('/topics', {
body: { name: 'New Topic' }
})
// PUT request (full replacement)
const updatedTopic = await dataApiService.put('/topics/abc123', {
body: { name: 'Updated', description: 'Full update' }
})
// PATCH request (partial update)
const patchedTopic = await dataApiService.patch('/topics/abc123', {
body: { name: 'Just update name' }
})
// DELETE request
await dataApiService.delete('/topics/abc123')
```
## Error Handling
### With Hooks
```typescript
function TopicList() {
const { data, loading, error } = useQuery('/topics')
if (loading) return <Loading />
if (error) {
if (error.code === ErrorCode.NOT_FOUND) {
return <NotFound />
}
return <Error message={error.message} />
}
return <List items={data} />
}
```
### With Try-Catch
```typescript
import { DataApiError, ErrorCode } from '@shared/data/api'
try {
await dataApiService.post('/topics', { body: data })
} catch (error) {
if (error instanceof DataApiError) {
switch (error.code) {
case ErrorCode.VALIDATION_ERROR:
// Handle validation errors
const fieldErrors = error.details?.fieldErrors
break
case ErrorCode.NOT_FOUND:
// Handle not found
break
case ErrorCode.CONFLICT:
// Handle conflict
break
default:
// Handle other errors
}
}
}
```
### Retryable Errors
```typescript
if (error instanceof DataApiError && error.isRetryable) {
// Safe to retry: SERVICE_UNAVAILABLE, TIMEOUT, etc.
await retry(operation)
}
```
## Common Patterns
### List with Pagination
```typescript
function TopicListWithPagination() {
const [page, setPage] = useState(1)
const { data, loading } = useQuery('/topics', {
query: { page, limit: 20 }
})
return (
<>
<List items={data?.items ?? []} />
<Pagination
current={page}
total={data?.total ?? 0}
onChange={setPage}
/>
</>
)
}
```
### Create Form
```typescript
function CreateTopicForm() {
const { trigger: createTopic, isMutating } = useMutation('/topics', 'POST')
const { mutate } = useQuery('/topics') // For revalidation
const handleSubmit = async (data: CreateTopicDto) => {
try {
await createTopic({ body: data })
await mutate() // Refresh list
toast.success('Topic created')
} catch (error) {
toast.error('Failed to create topic')
}
}
return (
<form onSubmit={handleSubmit}>
{/* form fields */}
<button disabled={isMutating}>
{isMutating ? 'Creating...' : 'Create'}
</button>
</form>
)
}
```
### Optimistic Updates
```typescript
function TopicItem({ topic }: { topic: Topic }) {
const { trigger: updateTopic } = useMutation(`/topics/${topic.id}`, 'PATCH')
const { mutate } = useQuery('/topics')
const handleToggleStar = async () => {
// Optimistically update the cache
await mutate(
current => ({
...current,
items: current.items.map(t =>
t.id === topic.id ? { ...t, starred: !t.starred } : t
)
}),
{ revalidate: false }
)
try {
await updateTopic({ body: { starred: !topic.starred } })
} catch (error) {
// Revert on failure
await mutate()
toast.error('Failed to update')
}
}
return (
<div>
<span>{topic.name}</span>
<button onClick={handleToggleStar}>
{topic.starred ? '★' : '☆'}
</button>
</div>
)
}
```
### Dependent Queries
```typescript
function MessageList({ topicId }: { topicId: string }) {
// First query: get topic
const { data: topic } = useQuery(`/topics/${topicId}`)
// Second query: depends on first (only runs when topic exists)
const { data: messages } = useQuery(
topic ? `/topics/${topicId}/messages` : null
)
if (!topic) return <Loading />
return (
<div>
<h1>{topic.name}</h1>
<MessageList messages={messages} />
</div>
)
}
```
### Polling for Updates
```typescript
function LiveTopicList() {
const { data } = useQuery('/topics', {
refreshInterval: 5000 // Poll every 5 seconds
})
return <List items={data} />
}
```
## Type Safety
The API is fully typed based on schema definitions:
```typescript
// Types are inferred from schema
const { data } = useQuery('/topics')
// data is typed as PaginatedResponse<Topic>
const { trigger } = useMutation('/topics', 'POST')
// trigger expects { body: CreateTopicDto }
// returns Topic
// Path parameters are type-checked
const { data: topic } = useQuery('/topics/abc123')
// TypeScript knows this returns Topic
```
## Best Practices
1. **Use hooks for components**: `useQuery` and `useMutation` handle loading/error states
2. **Handle loading states**: Always show feedback while data is loading
3. **Handle errors gracefully**: Provide meaningful error messages to users
4. **Revalidate after mutations**: Keep the UI in sync with the database
5. **Use conditional fetching**: Pass `null` to skip queries when dependencies aren't ready
6. **Batch related operations**: Consider using transactions for multiple updates

158
docs/en/references/data/data-api-overview.md Normal file
View File

@ -0,0 +1,158 @@
# DataApi System Overview
The DataApi system provides type-safe IPC communication for business data operations between the Renderer and Main processes.
## Purpose
DataApiService handles data that:
- Is **business data accumulated through user activity**
- Has **dedicated database schemas/tables**
- Users can **create, delete, modify records** without fixed limits
- Would be **severe and irreplaceable** if lost
- Can grow to **large volumes** (potentially GBs)
## Key Characteristics
### Type-Safe Communication
- End-to-end TypeScript types from client call to handler
- Path parameter inference from route definitions
- Compile-time validation of request/response shapes
### RESTful-Style API
- Familiar HTTP semantics (GET, POST, PUT, PATCH, DELETE)
- Resource-based URL patterns (`/topics/:id/messages`)
- Standard status codes and error responses
### On-Demand Data Access
- No automatic caching (fetch fresh data when needed)
- Explicit cache control via query options
- Supports large datasets with pagination
## Architecture Diagram
```
┌─────────────────────────────────────────────────────────────────┐
│ Renderer Process │
│ ┌─────────────────────────────────────────────────────────┐ │
│ │ React Components │ │
│ │ - useQuery('/topics') │ │
│ │ - useMutation('/topics', 'POST') │ │
│ └──────────────────────────┬──────────────────────────────┘ │
│ ▼ │
│ ┌─────────────────────────────────────────────────────────┐ │
│ │ DataApiService (Renderer) │ │
│ │ - Type-safe ApiClient interface │ │
│ │ - Request serialization │ │
│ │ - Automatic retry with exponential backoff │ │
│ │ - Error handling and transformation │ │
│ └──────────────────────────┬──────────────────────────────┘ │
└──────────────────────────────┼───────────────────────────────────┘
│ IPC
┌──────────────────────────────┼───────────────────────────────────┐
│ Main Process ▼ │
│ ┌─────────────────────────────────────────────────────────┐ │
│ │ IpcAdapter │ │
│ │ - Receives IPC requests │ │
│ │ - Routes to ApiServer │ │
│ └──────────────────────────┬──────────────────────────────┘ │
│ ▼ │
│ ┌─────────────────────────────────────────────────────────┐ │
│ │ ApiServer │ │
│ │ - Request routing by path and method │ │
│ │ - Middleware pipeline processing │ │
│ └──────────────────────────┬──────────────────────────────┘ │
│ ▼ │
│ ┌─────────────────────────────────────────────────────────┐ │
│ │ Handlers (api/handlers/) │ │
│ │ - Thin layer: extract params, call service, transform │ │
│ │ - NO business logic here │ │
│ └──────────────────────────┬──────────────────────────────┘ │
│ ▼ │
│ ┌─────────────────────────────────────────────────────────┐ │
│ │ Services (services/) │ │
│ │ - Business logic and validation │ │
│ │ - Transaction coordination │ │
│ │ - Domain workflows │ │
│ └──────────────────────────┬──────────────────────────────┘ │
│ ▼ │
│ ┌─────────────────────┴─────────────────────┐ │
│ ▼ ▼ │
│ ┌───────────────┐ ┌─────────────────────┐ │
│ │ Repositories │ │ Direct Drizzle │ │
│ │ (Complex) │ │ (Simple domains) │ │
│ │ - Query logic │ │ - Inline queries │ │
│ └───────┬───────┘ └──────────┬──────────┘ │
│ │ │ │
│ └────────────────────┬───────────────────┘ │
│ ▼ │
│ ┌─────────────────────────────────────────────────────────┐ │
│ │ SQLite Database (via Drizzle ORM) │ │
│ │ - topic, message, file tables │ │
│ │ - Full-text search indexes │ │
│ └─────────────────────────────────────────────────────────┘ │
└──────────────────────────────────────────────────────────────────┘
```
## Four-Layer Architecture
### 1. API Layer (Handlers)
- **Location**: `src/main/data/api/handlers/`
- **Responsibility**: HTTP-like interface layer
- **Does**: Extract parameters, call services, transform responses
- **Does NOT**: Contain business logic
### 2. Business Logic Layer (Services)
- **Location**: `src/main/data/services/`
- **Responsibility**: Domain logic and workflows
- **Does**: Validation, transaction coordination, orchestration
- **Uses**: Repositories or direct Drizzle queries
### 3. Data Access Layer (Repositories)
- **Location**: `src/main/data/repositories/`
- **Responsibility**: Complex data operations
- **When to use**: Complex queries, large datasets, reusable patterns
- **Alternative**: Direct Drizzle for simple CRUD
### 4. Database Layer
- **Location**: `src/main/data/db/`
- **Technology**: SQLite + Drizzle ORM
- **Schemas**: `db/schemas/` directory
## Data Access Pattern Decision
### Use Repository Pattern When:
- ✅ Complex queries (joins, subqueries, aggregations)
- ✅ GB-scale data requiring optimization and pagination
- ✅ Complex transactions involving multiple tables
- ✅ Reusable data access patterns across services
- ✅ High testing requirements (mock data access)
### Use Direct Drizzle When:
- ✅ Simple CRUD operations
- ✅ Small datasets (< 100MB)
- ✅ Domain-specific queries with no reuse potential
- ✅ Fast development is priority
## Key Features
### Automatic Retry
- Exponential backoff for transient failures
- Configurable retry count and delays
- Skips retry for client errors (4xx)
### Error Handling
- Typed error codes (`ErrorCode` enum)
- `DataApiError` class with retryability detection
- Factory methods for consistent error creation
### Request Timeout
- Configurable per-request timeouts
- Automatic cancellation of stale requests
## Usage Summary
For detailed code examples, see:
- [DataApi in Renderer](./data-api-in-renderer.md) - Client-side usage
- [DataApi in Main](./data-api-in-main.md) - Server-side implementation
- [API Design Guidelines](./api-design-guidelines.md) - RESTful conventions
- [API Types](./api-types.md) - Type system details

199
docs/en/references/data/database-patterns.md Normal file
View File

@ -0,0 +1,199 @@
# Database Schema Guidelines
## Naming Conventions
- **Table names**: Use **singular** form with snake_case (e.g., `topic`, `message`, `app_state`)
- **Export names**: Use `xxxTable` pattern (e.g., `topicTable`, `messageTable`)
- **Column names**: Drizzle auto-infers from property names, no need to specify explicitly
## Column Helpers
All helpers are exported from `./schemas/columnHelpers.ts`.
### Primary Keys
| Helper | UUID Version | Use Case |
|--------|--------------|----------|
| `uuidPrimaryKey()` | v4 (random) | General purpose tables |
| `uuidPrimaryKeyOrdered()` | v7 (time-ordered) | Large tables with time-based queries |
**Usage:**
```typescript
import { uuidPrimaryKey, uuidPrimaryKeyOrdered } from './columnHelpers'
// General purpose table
export const topicTable = sqliteTable('topic', {
id: uuidPrimaryKey(),
name: text(),
...
})
// Large table with time-ordered data
export const messageTable = sqliteTable('message', {
id: uuidPrimaryKeyOrdered(),
content: text(),
...
})
```
**Behavior:**
- ID is auto-generated if not provided during insert
- Can be manually specified for migration scenarios
- Use `.returning()` to get the generated ID after insert
### Timestamps
| Helper | Fields | Use Case |
|--------|--------|----------|
| `createUpdateTimestamps` | `createdAt`, `updatedAt` | Tables without soft delete |
| `createUpdateDeleteTimestamps` | `createdAt`, `updatedAt`, `deletedAt` | Tables with soft delete |
**Usage:**
```typescript
import { createUpdateTimestamps, createUpdateDeleteTimestamps } from './columnHelpers'
// Without soft delete
export const tagTable = sqliteTable('tag', {
id: uuidPrimaryKey(),
name: text(),
...createUpdateTimestamps
})
// With soft delete
export const topicTable = sqliteTable('topic', {
id: uuidPrimaryKey(),
name: text(),
...createUpdateDeleteTimestamps
})
```
**Behavior:**
- `createdAt`: Auto-set to `Date.now()` on insert
- `updatedAt`: Auto-set on insert, auto-updated on update
- `deletedAt`: `null` by default, set to timestamp for soft delete
## JSON Fields
For JSON column support, use `{ mode: 'json' }`:
```typescript
data: text({ mode: 'json' }).$type<MyDataType>()
```
Drizzle handles JSON serialization/deserialization automatically.
## Foreign Keys
### Basic Usage
```typescript
// SET NULL: preserve record when referenced record is deleted
groupId: text().references(() => groupTable.id, { onDelete: 'set null' })
// CASCADE: delete record when referenced record is deleted
topicId: text().references(() => topicTable.id, { onDelete: 'cascade' })
```
### Self-Referencing Foreign Keys
For self-referencing foreign keys (e.g., tree structures with parentId), **always use the `foreignKey` operator** in the table's third parameter:
```typescript
import { foreignKey, sqliteTable, text } from 'drizzle-orm/sqlite-core'
export const messageTable = sqliteTable(
'message',
{
id: uuidPrimaryKeyOrdered(),
parentId: text(), // Do NOT use .references() here
// ...other fields
},
(t) => [
// Use foreignKey operator for self-referencing
foreignKey({ columns: [t.parentId], foreignColumns: [t.id] }).onDelete('set null')
]
)
```
**Why this approach:**
- Avoids TypeScript circular reference issues (no need for `AnySQLiteColumn` type annotation)
- More explicit and readable
- Allows chaining `.onDelete()` / `.onUpdate()` actions
### Circular Foreign Key References
**Avoid circular foreign key references between tables.** For example:
```typescript
// ❌ BAD: Circular FK between tables
// tableA.currentItemId -> tableB.id
// tableB.ownerId -> tableA.id
```
If you encounter a scenario that seems to require circular references:
1. **Identify which relationship is "weaker"** - typically the one that can be null or is less critical for data integrity
2. **Remove the FK constraint from the weaker side** - let the application layer handle validation and consistency (this is known as "soft references" pattern)
3. **Document the application-layer constraint** in code comments
```typescript
// ✅ GOOD: Break the cycle by handling one side at application layer
export const topicTable = sqliteTable('topic', {
id: uuidPrimaryKey(),
// Application-managed reference (no FK constraint)
// Validated by TopicService.setCurrentMessage()
currentMessageId: text(),
})
export const messageTable = sqliteTable('message', {
id: uuidPrimaryKeyOrdered(),
// Database-enforced FK
topicId: text().references(() => topicTable.id, { onDelete: 'cascade' }),
})
```
**Why soft references for SQLite:**
- SQLite does not support `DEFERRABLE` constraints (unlike PostgreSQL/Oracle)
- Application-layer validation provides equivalent data integrity
- Simplifies insert/update operations without transaction ordering concerns
## Migrations
Generate migrations after schema changes:
```bash
yarn db:migrations:generate
```
## Field Generation Rules
The schema uses Drizzle's auto-generation features. Follow these rules:
### Auto-generated fields (NEVER set manually)
- `id`: Uses `$defaultFn()` with UUID v4/v7, auto-generated on insert
- `createdAt`: Uses `$defaultFn()` with `Date.now()`, auto-generated on insert
- `updatedAt`: Uses `$defaultFn()` and `$onUpdateFn()`, auto-updated on every update
### Using `.returning()` pattern
Always use `.returning()` to get inserted/updated data instead of re-querying:
```typescript
// Good: Use returning()
const [row] = await db.insert(table).values(data).returning()
return rowToEntity(row)
// Avoid: Re-query after insert (unnecessary database round-trip)
await db.insert(table).values({ id, ...data })
return this.getById(id)
```
### Soft delete support
The schema supports soft delete via `deletedAt` field (see `createUpdateDeleteTimestamps`).
Business logic can choose to use soft delete or hard delete based on requirements.

144
docs/en/references/data/preference-overview.md Normal file
View File

@ -0,0 +1,144 @@
# Preference System Overview
The Preference system provides centralized management for user configuration and application settings with cross-window synchronization.
## Purpose
PreferenceService handles data that:
- Is a **user-modifiable setting that affects app behavior**
- Has a **fixed key structure** with stable value types
- Needs to **persist permanently** until explicitly changed
- Should **sync automatically** across all application windows
## Key Characteristics
### Fixed Key Structure
- Predefined keys in the schema (users modify values, not keys)
- Supports 158 configuration items
- Nested key paths supported (e.g., `app.theme.mode`)
### Atomic Values
- Each preference item represents one logical setting
- Values are typically: boolean, string, number, or simple array/object
- Changes are independent (updating one doesn't affect others)
### Cross-Window Synchronization
- Changes automatically broadcast to all windows
- Consistent state across main window, mini window, etc.
- Conflict resolution handled by Main process
## Update Strategies
### Optimistic Updates (Default)
```typescript
// UI updates immediately, then syncs to database
await preferenceService.set('app.theme.mode', 'dark')
```
- Best for: frequent, non-critical settings
- Behavior: Local state updates first, then persists
- Rollback: Automatic revert if persistence fails
### Pessimistic Updates
```typescript
// Waits for database confirmation before updating UI
await preferenceService.set('api.key', 'secret', { optimistic: false })
```
- Best for: critical settings (API keys, security options)
- Behavior: Persists first, then updates local state
- No rollback needed: UI only updates on success
## Architecture Diagram
```
┌─────────────────────────────────────────────────────────────┐
│ Renderer Process │
│ ┌─────────────────────────────────────────────────┐ │
│ │ usePreference Hook │ │
│ │ - Subscribe to preference changes │ │
│ │ - Optimistic/pessimistic update support │ │
│ └──────────────────────┬──────────────────────────┘ │
│ ▼ │
│ ┌─────────────────────────────────────────────────┐ │
│ │ PreferenceService (Renderer) │ │
│ │ - Local cache for fast reads │ │
│ │ - IPC proxy to Main process │ │
│ │ - Subscription management │ │
│ └──────────────────────┬──────────────────────────┘ │
└─────────────────────────┼────────────────────────────────────┘
│ IPC
┌─────────────────────────┼────────────────────────────────────┐
│ Main Process ▼ │
│ ┌─────────────────────────────────────────────────┐ │
│ │ PreferenceService (Main) │ │
│ │ - Full memory cache of all preferences │ │
│ │ - SQLite persistence via Drizzle ORM │ │
│ │ - Cross-window broadcast │ │
│ └──────────────────────┬──────────────────────────┘ │
│ ▼ │
│ ┌─────────────────────────────────────────────────┐ │
│ │ SQLite Database (preference table) │ │
│ │ - scope + key structure │ │
│ │ - JSON value storage │ │
│ └─────────────────────────────────────────────────┘ │
└──────────────────────────────────────────────────────────────┘
```
## Main vs Renderer Responsibilities
### Main Process PreferenceService
- **Source of truth** for all preferences
- Full memory cache for fast access
- SQLite persistence via preference table
- Broadcasts changes to all renderer windows
- Handles batch operations and transactions
### Renderer Process PreferenceService
- Local cache for read performance
- Proxies write operations to Main
- Manages React hook subscriptions
- Handles optimistic update rollbacks
- Listens for cross-window updates
## Database Schema
Preferences are stored in the `preference` table:
```typescript
// Simplified schema
{
scope: string // e.g., 'default', 'user'
key: string // e.g., 'app.theme.mode'
value: json // The preference value
createdAt: number
updatedAt: number
}
```
## Preference Categories
### Application Settings
- Theme mode, language, font sizes
- Window behavior, startup options
### Feature Toggles
- Show/hide UI elements
- Enable/disable features
### User Customization
- Keyboard shortcuts
- Default values for operations
### Provider Configuration
- AI provider settings
- API endpoints and tokens
## Usage Summary
For detailed code examples and API usage, see [Preference Usage Guide](./preference-usage.md).
| Operation | Hook | Service Method |
|-----------|------|----------------|
| Read single | `usePreference(key)` | `preferenceService.get(key)` |
| Write single | `setPreference(value)` | `preferenceService.set(key, value)` |
| Read multiple | `usePreferences([...keys])` | `preferenceService.getMultiple([...keys])` |
| Write multiple | - | `preferenceService.setMultiple({...})` |

260
docs/en/references/data/preference-usage.md Normal file
View File

@ -0,0 +1,260 @@
# Preference Usage Guide
This guide covers how to use the Preference system in React components and services.
## React Hooks
### usePreference (Single Preference)
```typescript
import { usePreference } from '@data/hooks/usePreference'
// Basic usage - optimistic updates (default)
const [theme, setTheme] = usePreference('app.theme.mode')
// Update the value
await setTheme('dark')
// With pessimistic updates (wait for confirmation)
const [apiKey, setApiKey] = usePreference('api.key', { optimistic: false })
```
### usePreferences (Multiple Preferences)
```typescript
import { usePreferences } from '@data/hooks/usePreference'
// Read multiple preferences at once
const { theme, language, fontSize } = usePreferences([
'app.theme.mode',
'app.language',
'chat.message.font_size'
])
```
## Update Strategies
### Optimistic Updates (Default)
UI updates immediately, then syncs to database. Automatic rollback on failure.
```typescript
const [theme, setTheme] = usePreference('app.theme.mode')
const handleThemeChange = async (newTheme: string) => {
try {
await setTheme(newTheme) // UI updates immediately
} catch (error) {
// UI automatically rolls back
console.error('Theme update failed:', error)
}
}
```
**Best for:**
- Frequent changes (theme, font size)
- Non-critical settings
- Better perceived performance
### Pessimistic Updates
Waits for database confirmation before updating UI.
```typescript
const [apiKey, setApiKey] = usePreference('api.key', { optimistic: false })
const handleApiKeyChange = async (newKey: string) => {
try {
await setApiKey(newKey) // Waits for DB confirmation
toast.success('API key saved')
} catch (error) {
toast.error('Failed to save API key')
}
}
```
**Best for:**
- Security-sensitive settings (API keys, passwords)
- Settings that affect external services
- When confirmation feedback is important
## PreferenceService Direct Usage
For non-React code or batch operations.
### Get Preferences
```typescript
import { preferenceService } from '@data/PreferenceService'
// Get single preference
const theme = await preferenceService.get('app.theme.mode')
// Get multiple preferences
const settings = await preferenceService.getMultiple([
'app.theme.mode',
'app.language'
])
// Returns: { 'app.theme.mode': 'dark', 'app.language': 'en' }
// Get with default value
const fontSize = await preferenceService.get('chat.message.font_size') ?? 14
```
### Set Preferences
```typescript
// Set single preference (optimistic by default)
await preferenceService.set('app.theme.mode', 'dark')
// Set with pessimistic update
await preferenceService.set('api.key', 'secret', { optimistic: false })
// Set multiple preferences at once
await preferenceService.setMultiple({
'app.theme.mode': 'dark',
'app.language': 'en',
'chat.message.font_size': 16
})
```
### Subscribe to Changes
```typescript
// Subscribe to preference changes (useful in services)
const unsubscribe = preferenceService.subscribe('app.theme.mode', (newValue) => {
console.log('Theme changed to:', newValue)
})
// Cleanup when done
unsubscribe()
```
## Common Patterns
### Settings Form
```typescript
function SettingsForm() {
const [theme, setTheme] = usePreference('app.theme.mode')
const [language, setLanguage] = usePreference('app.language')
const [fontSize, setFontSize] = usePreference('chat.message.font_size')
return (
<form>
<select value={theme} onChange={e => setTheme(e.target.value)}>
<option value="light">Light</option>
<option value="dark">Dark</option>
<option value="system">System</option>
</select>
<select value={language} onChange={e => setLanguage(e.target.value)}>
<option value="en">English</option>
<option value="zh">中文</option>
</select>
<input
type="number"
value={fontSize}
onChange={e => setFontSize(Number(e.target.value))}
min={12}
max={24}
/>
</form>
)
}
```
### Feature Toggle
```typescript
function ChatMessage({ message }) {
const [showTimestamp] = usePreference('chat.display.show_timestamp')
return (
<div className="message">
<p>{message.content}</p>
{showTimestamp && <span className="timestamp">{message.createdAt}</span>}
</div>
)
}
```
### Conditional Rendering Based on Settings
```typescript
function App() {
const [theme] = usePreference('app.theme.mode')
const [sidebarPosition] = usePreference('app.sidebar.position')
return (
<div className={`app theme-${theme}`}>
{sidebarPosition === 'left' && <Sidebar />}
<MainContent />
{sidebarPosition === 'right' && <Sidebar />}
</div>
)
}
```
### Batch Settings Update
```typescript
async function resetToDefaults() {
await preferenceService.setMultiple({
'app.theme.mode': 'system',
'app.language': 'en',
'chat.message.font_size': 14,
'chat.display.show_timestamp': true
})
}
```
## Adding New Preference Keys
### 1. Add to Preference Schema
```typescript
// packages/shared/data/preference/preferenceSchemas.ts
export interface PreferenceSchema {
// Existing keys...
'myFeature.enabled': boolean
'myFeature.options': MyFeatureOptions
}
```
### 2. Set Default Value
```typescript
// Same file or separate defaults file
export const preferenceDefaults: Partial<PreferenceSchema> = {
// Existing defaults...
'myFeature.enabled': true,
'myFeature.options': { mode: 'auto', limit: 100 }
}
```
### 3. Use in Code
```typescript
// Now type-safe with auto-completion
const [enabled, setEnabled] = usePreference('myFeature.enabled')
```
## Best Practices
1. **Choose update strategy wisely**: Optimistic for UX, pessimistic for critical settings
2. **Batch related updates**: Use `setMultiple` when changing multiple related settings
3. **Provide sensible defaults**: All preferences should have default values
4. **Keep values atomic**: One preference = one logical setting
5. **Use consistent naming**: Follow `domain.feature.setting` pattern
## Preference vs Other Storage
| Scenario | Use |
|----------|-----|
| User theme preference | `usePreference('app.theme.mode')` |
| Window position | `usePersistCache` (can be lost without impact) |
| API key | `usePreference` with pessimistic updates |
| Search history | `usePersistCache` (nice to have) |
| Conversation history | `DataApiService` (business data) |

64
docs/en/references/data/v2-migration-guide.md Normal file
View File

@ -0,0 +1,64 @@
# Migration V2 (Main Process)
Architecture for the new one-shot migration from the legacy Dexie + Redux Persist stores into the SQLite schema. This module owns orchestration, data access helpers, migrator plugins, and IPC entry points used by the renderer migration window.
## Directory Layout
```
src/main/data/migration/v2/
├── core/ # Engine + shared context
├── migrators/ # Domain-specific migrators and mappings
├── utils/ # Data source readers (Redux, Dexie, streaming JSON)
├── window/ # IPC handlers + migration window manager
└── index.ts # Public exports for main process
```
## Core Contracts
- `core/MigrationEngine.ts` coordinates all migrators in order, surfaces progress to the UI, and marks status in `app_state.key = 'migration_v2_status'`. It will clear new-schema tables before running and abort on any validation failure.
- `core/MigrationContext.ts` builds the shared context passed to every migrator:
- `sources`: `ConfigManager` (ElectronStore), `ReduxStateReader` (parsed Redux Persist data), `DexieFileReader` (JSON exports)
- `db`: current SQLite connection
- `sharedData`: `Map` for passing cross-cutting info between migrators
- `logger`: `loggerService` scoped to migration
- `@shared/data/migration/v2/types` defines stages, results, and validation stats used across main and renderer.
## Migrators
- Base contract: extend `migrators/BaseMigrator.ts` and implement:
- `id`, `name`, `description`, `order` (lower runs first)
- `prepare(ctx)`: dry-run checks, counts, and staging data; return `PrepareResult`
- `execute(ctx)`: perform inserts/updates; manage your own transactions; report progress via `reportProgress`
- `validate(ctx)`: verify counts and integrity; return `ValidateResult` with stats (`sourceCount`, `targetCount`, `skippedCount`) and any `errors`
- Registration: list migrators (in order) in `migrators/index.ts` so the engine can sort and run them.
- Current migrators:
- `PreferencesMigrator` (implemented): maps ElectronStore + Redux settings to the `preference` table using `mappings/PreferencesMappings.ts`.
- `AssistantMigrator`, `KnowledgeMigrator`, `ChatMigrator` (placeholders): scaffolding and TODO notes for future tables.
- Conventions:
- All logging goes through `loggerService` with a migrator-specific context.
- Use `MigrationContext.sources` instead of accessing raw files/stores directly.
- Use `sharedData` to pass IDs or lookup tables between migrators (e.g., assistant -> chat references) instead of re-reading sources.
- Stream large Dexie exports (`JSONStreamReader`) and batch inserts to avoid memory spikes.
- Count validation is mandatory; engine will fail the run if `targetCount < sourceCount - skippedCount` or if `ValidateResult.errors` is non-empty.
- Keep migrations idempotent per run—engine clears target tables before it starts, but each migrator should tolerate retries within the same run.
## Utilities
- `utils/ReduxStateReader.ts`: safe accessor for categorized Redux Persist data with dot-path lookup.
- `utils/DexieFileReader.ts`: reads exported Dexie JSON tables; can stream large tables.
- `utils/JSONStreamReader.ts`: streaming reader with batching, counting, and sampling helpers for very large arrays.
## Window & IPC Integration
- `window/MigrationIpcHandler.ts` exposes IPC channels for the migration UI:
- Receives Redux data and Dexie export path, starts the engine, and streams progress back to renderer.
- Manages backup flow (dialogs via `BackupManager`) and retry/cancel/restart actions.
- `window/MigrationWindowManager.ts` creates the frameless migration window, handles lifecycle, and relaunch instructions after completion in production.
## Implementation Checklist for New Migrators
- [ ] Add mapping definitions (if needed) under `migrators/mappings/`.
- [ ] Implement `prepare/execute/validate` with explicit counts, batch inserts, and integrity checks.
- [ ] Wire progress updates through `reportProgress` so UI shows per-migrator progress.
- [ ] Register the migrator in `migrators/index.ts` with the correct `order`.
- [ ] Add any new target tables to `MigrationEngine.verifyAndClearNewTables` once those tables exist.

138
packages/shared/data/README.md
View File

@ -1,124 +1,50 @@
# Cherry Studio Shared Data
# Shared Data Types
This directory contains shared type definitions and schemas for the Cherry Studio data management systems. These files provide type safety and consistency across the entire application.
This directory contains shared type definitions for Cherry Studio's data layer.
## Documentation
For comprehensive documentation, see:
- **Overview**: [docs/en/references/data/README.md](../../../docs/en/references/data/README.md)
- **Cache Types**: [cache-overview.md](../../../docs/en/references/data/cache-overview.md)
- **Preference Types**: [preference-overview.md](../../../docs/en/references/data/preference-overview.md)
- **API Types**: [api-types.md](../../../docs/en/references/data/api-types.md)
## Directory Structure
```
packages/shared/data/
├── api/ # Data API type system (see api/README.md)
│ ├── index.ts # Barrel exports for infrastructure types
│ ├── apiTypes.ts # Core request/response types and utilities
│ ├── apiPaths.ts # Path template literal type utilities
│ ├── errorCodes.ts # Error handling utilities
│ ├── schemas/ # Domain-specific API schemas
│ │ ├── index.ts # Schema composition
│ │ └── test.ts # Test API schema and DTOs
│ └── README.md # Detailed API documentation
├── api/ # Data API type system
│ ├── index.ts # Barrel exports
│ ├── apiTypes.ts # Core request/response types
│ ├── apiPaths.ts # Path template utilities
│ ├── apiErrors.ts # Error handling
│ └── schemas/ # Domain-specific API schemas
├── cache/ # Cache system type definitions
│ ├── cacheTypes.ts # Core cache infrastructure types
│ ├── cacheSchemas.ts # Cache key schemas and type mappings
│ └── cacheValueTypes.ts # Cache value type definitions
│ ├── cacheTypes.ts # Core cache types
│ ├── cacheSchemas.ts # Cache key schemas
│ └── cacheValueTypes.ts # Cache value types
├── preference/ # Preference system type definitions
│ ├── preferenceTypes.ts # Core preference system types
│ └── preferenceSchemas.ts # Preference schemas and default values
├── types/ # Shared data types for Main/Renderer
└── README.md # This file
│ ├── preferenceTypes.ts # Core preference types
│ └── preferenceSchemas.ts # Preference schemas
└── types/ # Shared data types
```
## System Overview
## Quick Reference
This directory provides type definitions for four main data management systems:
### Import Conventions
### Types System (`types/`)
- **Purpose**: Shared data types for cross-process (Main/Renderer) communication and database schemas
- **Features**: Database table field types, business entity definitions
- **Usage**: Used in Drizzle ORM schemas via `.$type<T>()` and runtime type checking
### API System (`api/`)
- **Purpose**: Type-safe IPC communication between Main and Renderer processes
- **Features**: RESTful patterns, modular schema design, error handling
- **Documentation**: See [`api/README.md`](./api/README.md) for detailed usage
### Cache System (`cache/`)
- **Purpose**: Type definitions for three-layer caching architecture
- **Features**: Memory/shared/persist cache schemas, TTL support, hook integration
- **Usage**: Type-safe caching operations across the application
### Preference System (`preference/`)
- **Purpose**: User configuration and settings management
- **Features**: 158 configuration items, default values, nested key support
- **Usage**: Type-safe preference access and synchronization
## File Categories
**Framework Infrastructure** - These are TypeScript type definitions that:
- ✅ Exist only at compile time
- ✅ Provide type safety and IntelliSense support
- ✅ Define contracts between application layers
- ✅ Enable static analysis and error detection
## Usage Examples
### API Types
```typescript
// Infrastructure types from barrel export
// API infrastructure types (from barrel)
import type { DataRequest, DataResponse, ApiClient } from '@shared/data/api'
import { DataApiErrorFactory, ErrorCode } from '@shared/data/api'
import { ErrorCode, DataApiError, DataApiErrorFactory } from '@shared/data/api'
// Domain DTOs directly from schema files
import type { TestItem, CreateTestItemDto } from '@shared/data/api/schemas/test'
```
// Domain DTOs (from schema files)
import type { Topic, CreateTopicDto } from '@shared/data/api/schemas/topic'
### Cache Types
```typescript
// Import cache types
// Cache types
import type { UseCacheKey, UseSharedCacheKey } from '@shared/data/cache'
// Preference types
import type { PreferenceKeyType } from '@shared/data/preference'
```
### Preference Types
```typescript
// Import preference types
import type { PreferenceKeyType, PreferenceDefaultScopeType } from '@shared/data/preference'
```
## Development Guidelines
### Adding Shared Types
1. Create or update type file in `types/` directory
2. Use camelCase for field names
3. Reference types in Drizzle schemas using `.$type<T>()`
### Adding Cache Types
1. Add cache key to `cache/cacheSchemas.ts`
2. Define value type in `cache/cacheValueTypes.ts`
3. Update type mappings for type safety
### Adding Preference Types
1. Add preference key to `preference/preferenceSchemas.ts`
2. Define default value and type
3. Preference system automatically picks up new keys
### Adding API Types
1. Create schema file in `api/schemas/` (e.g., `topic.ts`)
2. Define domain models, DTOs, and API schema in the file
3. Register schema in `api/schemas/index.ts` using intersection type
4. See [`api/README.md`](./api/README.md) for detailed guide
### Best Practices
- Use `import type` for type-only imports
- Infrastructure types from barrel, domain DTOs from schema files
- Follow existing naming conventions
- Document complex types with JSDoc
## Related Implementation
### Main Process
- `src/main/data/api/` - API server, handlers, and IPC adapter
- `src/main/data/cache/` - Cache service implementation
- `src/main/data/preference/` - Preference service implementation
### Renderer Process
- `src/renderer/src/services/DataApiService.ts` - API client
- `src/renderer/src/services/CacheService.ts` - Cache service
- `src/renderer/src/services/PreferenceService.ts` - Preference service

242
packages/shared/data/api/README.md
View File

@ -1,238 +1,42 @@
# Data API Type System
This directory contains the type definitions and utilities for Cherry Studio's Data API system, which provides type-safe IPC communication between renderer and main processes.
This directory contains type definitions for the DataApi system.
## Documentation
- **DataApi Overview**: [docs/en/references/data/data-api-overview.md](../../../../docs/en/references/data/data-api-overview.md)
- **API Types**: [api-types.md](../../../../docs/en/references/data/api-types.md)
- **API Design Guidelines**: [api-design-guidelines.md](../../../../docs/en/references/data/api-design-guidelines.md)
## Directory Structure
```
packages/shared/data/api/
├── index.ts # Barrel export for infrastructure types
├── apiTypes.ts # Core request/response types and API utilities
├── apiPaths.ts # Path template literal type utilities
├── apiErrors.ts # Error handling: ErrorCode, DataApiError class, factory
├── index.ts # Barrel exports
├── apiTypes.ts # Core request/response types
├── apiPaths.ts # Path template utilities
├── apiErrors.ts # Error handling
└── schemas/
├── index.ts # Schema composition (merges all domain schemas)
└── test.ts # Test API schema and DTOs
├── index.ts # Schema composition
└── *.ts # Domain-specific schemas
```
## File Responsibilities
## Quick Reference
| File | Purpose |
|------|---------|
| `apiTypes.ts` | Core types (`DataRequest`, `DataResponse`, `ApiClient`) and schema utilities |
| `apiPaths.ts` | Template literal types for path resolution (`/items/:id` → `/items/${string}`) |
| `apiErrors.ts` | `ErrorCode` enum, `DataApiError` class, `DataApiErrorFactory`, retryability config |
| `index.ts` | Unified export of infrastructure types (not domain DTOs) |
| `schemas/index.ts` | Composes all domain schemas into `ApiSchemas` using intersection types |
| `schemas/*.ts` | Domain-specific API definitions and DTOs |
## Import Conventions
### Infrastructure Types (via barrel export)
Use the barrel export for common API infrastructure:
### Import Conventions
```typescript
import type {
DataRequest,
DataResponse,
ApiClient,
PaginatedResponse
} from '@shared/data/api'
// Infrastructure types (via barrel)
import type { DataRequest, DataResponse, ApiClient } from '@shared/data/api'
import { ErrorCode, DataApiError, DataApiErrorFactory } from '@shared/data/api'
import {
ErrorCode,
DataApiError,
DataApiErrorFactory,
isDataApiError,
toDataApiError
} from '@shared/data/api'
```
### Domain DTOs (directly from schema files)
Import domain-specific types directly from their schema files:
```typescript
// Topic domain
import type { Topic, CreateTopicDto, UpdateTopicDto } from '@shared/data/api/schemas/topic'
// Message domain
// Domain DTOs (directly from schema files)
import type { Topic, CreateTopicDto } from '@shared/data/api/schemas/topic'
import type { Message, CreateMessageDto } from '@shared/data/api/schemas/message'
// Test domain (development)
import type { TestItem, CreateTestItemDto } from '@shared/data/api/schemas/test'
```
## Adding a New Domain Schema
1. Create the schema file (e.g., `schemas/topic.ts`):
```typescript
import type { PaginatedResponse } from '../apiTypes'
// Domain models
export interface Topic {
id: string
name: string
createdAt: string
}
export interface CreateTopicDto {
name: string
}
// API Schema - validation happens via AssertValidSchemas in index.ts
export interface TopicSchemas {
'/topics': {
GET: {
response: PaginatedResponse<Topic> // response is required
}
POST: {
body: CreateTopicDto
response: Topic
}
}
'/topics/:id': {
GET: {
params: { id: string }
response: Topic
}
}
}
```
**Validation**: Schemas are validated at composition level via `AssertValidSchemas` in `schemas/index.ts`:
- Ensures only valid HTTP methods (GET, POST, PUT, DELETE, PATCH)
- Requires `response` field for each endpoint
- Invalid schemas cause TypeScript errors at the composition point
> **Design Guidelines**: Before creating new schemas, review the [API Design Guidelines](./api-design-guidelines.md) for path naming, HTTP methods, and error handling conventions.
2. Register in `schemas/index.ts`:
```typescript
import type { TopicSchemas } from './topic'
// AssertValidSchemas provides fallback validation even if ValidateSchema is forgotten
export type ApiSchemas = AssertValidSchemas<TestSchemas & TopicSchemas>
```
### Adding New Schemas
1. Create schema file in `schemas/` (e.g., `topic.ts`)
2. Register in `schemas/index.ts` using intersection type
3. Implement handlers in `src/main/data/api/handlers/`
## Type Safety Features
### Path Resolution
The system uses template literal types to map concrete paths to schema paths:
```typescript
// Concrete path '/topics/abc123' maps to schema path '/topics/:id'
api.get('/topics/abc123') // TypeScript knows this returns Topic
```
### Exhaustive Handler Checking
`ApiImplementation` type ensures all schema endpoints have handlers:
```typescript
// TypeScript will error if any endpoint is missing
const handlers: ApiImplementation = {
'/topics': {
GET: async () => { /* ... */ },
POST: async ({ body }) => { /* ... */ }
}
// Missing '/topics/:id' would cause compile error
}
```
### Type-Safe Client
`ApiClient` provides fully typed methods:
```typescript
const topic = await api.get('/topics/123') // Returns Topic
const topics = await api.get('/topics', { query: { page: 1 } }) // Returns PaginatedResponse<Topic>
await api.post('/topics', { body: { name: 'New' } }) // Body is typed as CreateTopicDto
```
## Error Handling
The error system provides type-safe error handling with automatic retryability detection:
```typescript
import {
DataApiError,
DataApiErrorFactory,
ErrorCode,
isDataApiError,
toDataApiError
} from '@shared/data/api'
// Create errors using the factory (recommended)
throw DataApiErrorFactory.notFound('Topic', id)
throw DataApiErrorFactory.validation({ name: ['Name is required'] })
throw DataApiErrorFactory.timeout('fetch topics', 3000)
throw DataApiErrorFactory.database(originalError, 'insert topic')
// Or create directly with the class
throw new DataApiError(
ErrorCode.NOT_FOUND,
'Topic not found',
404,
{ resource: 'Topic', id: 'abc123' }
)
// Check if error is retryable (for automatic retry logic)
if (error instanceof DataApiError && error.isRetryable) {
await retry(operation)
}
// Check error type
if (error instanceof DataApiError) {
if (error.isClientError) {
// 4xx - issue with the request
} else if (error.isServerError) {
// 5xx - server-side issue
}
}
// Convert any error to DataApiError
const apiError = toDataApiError(unknownError, 'context')
// Serialize for IPC (Main → Renderer)
const serialized = apiError.toJSON()
// Reconstruct from IPC response (Renderer)
const reconstructed = DataApiError.fromJSON(response.error)
```
### Retryable Error Codes
The following errors are automatically considered retryable:
- `SERVICE_UNAVAILABLE` (503)
- `TIMEOUT` (504)
- `RATE_LIMIT_EXCEEDED` (429)
- `DATABASE_ERROR` (500)
- `INTERNAL_SERVER_ERROR` (500)
- `RESOURCE_LOCKED` (423)
## Architecture Overview
```
Renderer Main
────────────────────────────────────────────────────
DataApiService ──IPC──► IpcAdapter ──► ApiServer
│ │
│ ▼
ApiClient MiddlewareEngine
(typed) │
▼
Handlers
(typed)
```
- **Renderer**: Uses `DataApiService` with type-safe `ApiClient` interface
- **IPC**: Requests serialized via `IpcAdapter`
- **Main**: `ApiServer` routes to handlers through `MiddlewareEngine`
- **Type Safety**: End-to-end types from client call to handler implementation

415
src/main/data/README.md
View File

@ -1,405 +1,44 @@
# Main Data Layer
This directory contains the main process data management system, providing unified data access for the entire application.
This directory contains the main process data management implementation.
## Documentation
- **Overview**: [docs/en/references/data/README.md](../../../docs/en/references/data/README.md)
- **DataApi in Main**: [data-api-in-main.md](../../../docs/en/references/data/data-api-in-main.md)
- **Database Patterns**: [database-patterns.md](../../../docs/en/references/data/database-patterns.md)
## Directory Structure
```
src/main/data/
├── api/ # Data API framework (interface layer)
│ ├── core/ # Core API infrastructure
│ │ ├── ApiServer.ts # Request routing and handler execution
│ │ ├── MiddlewareEngine.ts # Request/response middleware
│ │ └── adapters/ # Communication adapters (IPC)
│ ├── handlers/ # API endpoint implementations
│ │ ├── index.ts # Handler aggregation and exports
│ │ └── test.ts # Test endpoint handlers
│ └── index.ts # API framework exports
│
├── api/ # Data API framework
│ ├── core/ # ApiServer, MiddlewareEngine, adapters
│ └── handlers/ # API endpoint implementations
├── services/ # Business logic layer
│ ├── base/ # Service base classes and interfaces
│ │ └── IBaseService.ts # Service interface definitions
│ └── TestService.ts # Test service (placeholder for real services)
│ # Future business services:
│ # - TopicService.ts # Topic business logic
│ # - MessageService.ts # Message business logic
│ # - FileService.ts # File business logic
│
├── repositories/ # Data access layer (selective usage)
│ # Repository pattern used selectively for complex domains
│ # Future repositories:
│ # - TopicRepository.ts # Complex: Topic data access
│ # - MessageRepository.ts # Complex: Message data access
│
├── db/ # Database layer
│ ├── schemas/ # Drizzle table definitions
│ │ ├── preference.ts # Preference configuration table
│ │ ├── appState.ts # Application state table
│ │ ├── topic.ts # Topic/conversation table
│ │ ├── message.ts # Message table
│ │ ├── group.ts # Group table
│ │ ├── tag.ts # Tag table
│ │ ├── entityTag.ts # Entity-tag relationship table
│ │ ├── messageFts.ts # Message full-text search table
│ │ └── columnHelpers.ts # Reusable column definitions
│ ├── seeding/ # Database initialization
│ └── DbService.ts # Database connection and management
│
├── migration/ # Data migration system
│ └── v2/ # v2 data refactoring migration tools
│
├── CacheService.ts # Infrastructure: Cache management
├── DataApiService.ts # Infrastructure: API coordination
└── PreferenceService.ts # System service: User preferences
├── db/ # Database layer
│ ├── schemas/ # Drizzle table definitions
│ ├── seeding/ # Database initialization
│ └── DbService.ts # Database connection management
├── migration/ # Data migration system
├── CacheService.ts # Cache management
├── DataApiService.ts # API coordination
└── PreferenceService.ts # User preferences
```
## Core Components
### Naming Note
Three components at the root of `data/` use the "Service" suffix but serve different purposes:
#### CacheService (Infrastructure Component)
- **True Nature**: Cache Manager / Infrastructure Utility
- **Purpose**: Multi-tier caching system (memory/shared/persist)
- **Features**: TTL support, IPC synchronization, cross-window broadcasting
- **Characteristics**: Zero business logic, purely technical functionality
- **Note**: Named "Service" for management consistency, but is actually infrastructure
#### DataApiService (Coordinator Component)
- **True Nature**: API Coordinator (Main) / API Client (Renderer)
- **Main Process Purpose**: Coordinates ApiServer and IpcAdapter initialization
- **Renderer Purpose**: HTTP-like client for IPC communication
- **Characteristics**: Zero business logic, purely coordination/communication plumbing
- **Note**: Named "Service" for management consistency, but is actually coordinator/client
#### PreferenceService (System Service)
- **True Nature**: System-level Data Access Service
- **Purpose**: User configuration management with caching and multi-window sync
- **Features**: SQLite persistence, full memory cache, cross-window synchronization
- **Characteristics**: Minimal business logic (validation, defaults), primarily data access
- **Note**: Hybrid between data access and infrastructure, "Service" naming is acceptable
**Key Takeaway**: Despite all being named "Service", these are infrastructure/coordination components, not business services. The "Service" suffix is kept for consistency with existing codebase conventions.
## Architecture Layers
### API Framework Layer (`api/`)
The API framework provides the interface layer for data access:
#### API Server (`api/core/ApiServer.ts`)
- Request routing and handler execution
- Middleware pipeline processing
- Type-safe endpoint definitions
#### Handlers (`api/handlers/`)
- **Purpose**: Thin API endpoint implementations
- **Responsibilities**:
- HTTP-like parameter extraction from requests
- DTO/domain model conversion
- Delegating to business services
- Transforming responses for IPC
- **Anti-pattern**: Do NOT put business logic in handlers
- **Currently**: Contains test handlers (business handlers pending)
- **Type Safety**: Must implement all endpoints defined in `@shared/data/api/schemas/`
### Business Logic Layer (`services/`)
Business services implement domain logic and workflows:
#### When to Create a Service
- Contains business rules and validation
- Orchestrates multiple repositories or data sources
- Implements complex workflows
- Manages transactions across multiple operations
#### Service Pattern
Just an example for understanding.
```typescript
// services/TopicService.ts
export class TopicService {
constructor(
private topicRepo: TopicRepository, // Use repository for complex data access
private cacheService: CacheService // Use infrastructure utilities
) {}
async createTopicWithMessage(data: CreateTopicDto) {
// Business validation
this.validateTopicData(data)
// Transaction coordination
return await DbService.transaction(async (tx) => {
const topic = await this.topicRepo.create(data.topic, tx)
const message = await this.messageRepo.create(data.message, tx)
return { topic, message }
})
}
}
```
#### Current Services
- `TestService`: Placeholder service for testing API framework
- More business services will be added as needed (TopicService, MessageService, etc.)
### Data Access Layer (`repositories/`)
Repositories handle database operations with a **selective usage pattern**:
#### When to Use Repository Pattern
Use repositories for **complex domains** that meet multiple criteria:
- ✅ Complex queries (joins, subqueries, aggregations)
- ✅ GB-scale data requiring optimization and pagination
- ✅ Complex transactions involving multiple tables
- ✅ Reusable data access patterns across services
- ✅ High testing requirements (mock data access in tests)
#### When to Use Direct Drizzle in Services
Skip repository layer for **simple domains**:
- ✅ Simple CRUD operations
- ✅ Small datasets (< 100MB)
- ✅ Domain-specific queries with no reuse potential
- ✅ Fast development is priority
#### Repository Pattern
Just an example for understanding.
```typescript
// repositories/TopicRepository.ts
export class TopicRepository {
async findById(id: string, tx?: Transaction): Promise<Topic | null> {
const db = tx || DbService.db
return await db.select()
.from(topicTable)
.where(eq(topicTable.id, id))
.limit(1)
}
async findByIdWithMessages(
topicId: string,
pagination: PaginationOptions
): Promise<TopicWithMessages> {
// Complex join query with pagination
// Handles GB-scale data efficiently
}
}
```
#### Direct Drizzle Pattern (Simple Services)
```typescript
// services/SimpleService.ts
export class SimpleService extends BaseService {
async getItem(id: string) {
// Direct Drizzle query for simple operations
return await this.database
.select()
.from(itemTable)
.where(eq(itemTable.id, id))
}
}
```
#### Planned Repositories
- **TopicRepository**: Complex topic data access with message relationships
- **MessageRepository**: GB-scale message queries with pagination
- **FileRepository**: File reference counting and cleanup logic
**Decision Principle**: Use the simplest approach that solves the problem. Add repository abstraction only when complexity demands it.
## Database Layer
### DbService
- SQLite database connection management
- Automatic migrations and seeding
- Drizzle ORM integration
### Schemas (`db/schemas/`)
- Table definitions using Drizzle ORM
- Follow naming convention: `{entity}Table` exports
- Use `crudTimestamps` helper for timestamp fields
- See [db/README.md](./db/README.md#field-generation-rules) for detailed field generation rules and `.returning()` pattern
### Current Tables
- `preference`: User configuration storage
- `appState`: Application state persistence
- `topic`: Conversation/topic storage
- `message`: Message storage with full-text search
- `group`: Group organization
- `tag`: Tag definitions
- `entityTag`: Entity-tag relationships
- `messageFts`: Message full-text search index
## Usage Examples
### Accessing Services
```typescript
// Get service instances
import { cacheService } from '@/data/CacheService'
import { preferenceService } from '@/data/PreferenceService'
import { dataApiService } from '@/data/DataApiService'
// Services are singletons, initialized at app startup
```
## Quick Reference
### Adding New API Endpoints
1. Create or update schema in `@shared/data/api/schemas/` (see `@shared/data/api/README.md`)
2. Register schema in `@shared/data/api/schemas/index.ts`
3. Implement handler in `api/handlers/` (thin layer, delegate to service)
4. Create business service in `services/` for domain logic
5. Create repository in `repositories/` if domain is complex (optional)
6. Add database schema in `db/schemas/` if required
### Adding Database Tables
1. Create schema in `db/schemas/{tableName}.ts`
2. Generate migration: `yarn run db:migrations:generate`
3. Add seeding data in `db/seeding/` if needed
4. Decide: Repository pattern or direct Drizzle?
- Complex domain → Create repository in `repositories/`
- Simple domain → Use direct Drizzle in service
5. Create business service in `services/`
6. Implement API handler in `api/handlers/`
1. Define schema in `@shared/data/api/schemas/`
2. Implement handler in `api/handlers/`
3. Create business service in `services/`
4. Create repository in `repositories/` (if complex domain)
### Creating a New Business Service
### Database Commands
**For complex domains (with repository)**:
```typescript
// 1. Create repository: repositories/ExampleRepository.ts
export class ExampleRepository {
async findById(id: string, tx?: Transaction) { /* ... */ }
async create(data: CreateDto, tx?: Transaction) { /* ... */ }
}
// 2. Create service: services/ExampleService.ts
export class ExampleService {
constructor(private exampleRepo: ExampleRepository) {}
async createExample(data: CreateDto) {
// Business validation
this.validate(data)
// Use repository
return await this.exampleRepo.create(data)
}
}
// 3. Create handler: api/handlers/example.ts
import { ExampleService } from '@data/services/ExampleService'
export const exampleHandlers: Partial<ApiImplementation> = {
'/examples': {
POST: async ({ body }) => {
return await ExampleService.getInstance().createExample(body)
}
}
}
```bash
# Generate migrations
yarn db:migrations:generate
```
**For simple domains (direct Drizzle)**:
```typescript
// 1. Create service: services/SimpleService.ts
export class SimpleService extends BaseService {
async getItem(id: string) {
// Direct database access
return await this.database
.select()
.from(itemTable)
.where(eq(itemTable.id, id))
}
}
// 2. Create handler: api/handlers/simple.ts
export const simpleHandlers: Partial<ApiImplementation> = {
'/items/:id': {
GET: async ({ params }) => {
return await SimpleService.getInstance().getItem(params.id)
}
}
}
```
## Data Flow
### Complete Request Flow
```
┌─────────────────────────────────────────────────────┐
│ Renderer Process │
│ React Component → useDataApi Hook │
└────────────────┬────────────────────────────────────┘
│ IPC Request
┌────────────────▼────────────────────────────────────┐
│ Infrastructure Layer │
│ DataApiService (coordinator) │
│ ↓ │
│ ApiServer (routing) → MiddlewareEngine │
└────────────────┬────────────────────────────────────┘
│
┌────────────────▼────────────────────────────────────┐
│ API Layer (api/handlers/) │
│ Handler: Thin layer │
│ - Extract parameters │
│ - Call business service │
│ - Transform response │
└────────────────┬────────────────────────────────────┘
│
┌────────────────▼────────────────────────────────────┐
│ Business Logic Layer (services/) │
│ Service: Domain logic │
│ - Business validation │
│ - Transaction coordination │
│ - Call repository or direct DB │
└────────────────┬────────────────────────────────────┘
│
┌──────────┴──────────┐
│ │
┌─────▼─────────┐ ┌──────▼──────────────────────────┐
│ repositories/ │ │ Direct Drizzle │
│ (Complex) │ │ (Simple domains) │
│ - Repository │ │ - Service uses DbService.db │
│ - Query logic │ │ - Inline queries │
└─────┬─────────┘ └──────┬──────────────────────────┘
│ │
└──────────┬─────────┘
│
┌────────────────▼────────────────────────────────────┐
│ Database Layer (db/) │
│ DbService → SQLite (Drizzle ORM) │
└─────────────────────────────────────────────────────┘
```
### Architecture Principles
1. **Separation of Concerns**
- Handlers: Request/response transformation only
- Services: Business logic and orchestration
- Repositories: Data access (when complexity demands it)
2. **Dependency Flow** (top to bottom only)
- Handlers depend on Services
- Services depend on Repositories (or DbService directly)
- Repositories depend on DbService
- **Never**: Services depend on Handlers
- **Never**: Repositories contain business logic
3. **Selective Repository Usage**
- Use Repository: Complex domains (Topic, Message, File)
- Direct Drizzle: Simple domains (Agent, Session, Translate)
- Decision based on: query complexity, data volume, testing needs
## Development Guidelines
- All services use singleton pattern
- Database operations must be type-safe (Drizzle)
- API endpoints require complete type definitions
- Services should handle errors gracefully
- Use existing logging system (`@logger`)
## Integration Points
- **IPC Communication**: All services expose IPC handlers for renderer communication
- **Type Safety**: Shared types in `@shared/data` ensure end-to-end type safety
- **Error Handling**: Standardized error codes and handling across all services
- **Logging**: Comprehensive logging for debugging and monitoring

207
src/main/data/db/README.md
View File

@ -1,199 +1,46 @@
# Database Schema Guidelines
# Database Layer
## Naming Conventions
This directory contains database schemas and configuration.
- **Table names**: Use **singular** form with snake_case (e.g., `topic`, `message`, `app_state`)
- **Export names**: Use `xxxTable` pattern (e.g., `topicTable`, `messageTable`)
- **Column names**: Drizzle auto-infers from property names, no need to specify explicitly
## Documentation
## Column Helpers
- **Database Patterns**: [docs/en/references/data/database-patterns.md](../../../../docs/en/references/data/database-patterns.md)
All helpers are exported from `./schemas/columnHelpers.ts`.
## Directory Structure
### Primary Keys
| Helper | UUID Version | Use Case |
|--------|--------------|----------|
| `uuidPrimaryKey()` | v4 (random) | General purpose tables |
| `uuidPrimaryKeyOrdered()` | v7 (time-ordered) | Large tables with time-based queries |
**Usage:**
```typescript
import { uuidPrimaryKey, uuidPrimaryKeyOrdered } from './columnHelpers'
// General purpose table
export const topicTable = sqliteTable('topic', {
id: uuidPrimaryKey(),
name: text(),
...
})
// Large table with time-ordered data
export const messageTable = sqliteTable('message', {
id: uuidPrimaryKeyOrdered(),
content: text(),
...
})
```
src/main/data/db/
├── schemas/ # Drizzle table definitions
│ ├── columnHelpers.ts # Reusable column definitions
│ ├── topic.ts # Topic table
│ ├── message.ts # Message table
│ └── ... # Other tables
├── seeding/ # Database initialization
└── DbService.ts # Database connection management
```
**Behavior:**
## Quick Reference
- ID is auto-generated if not provided during insert
- Can be manually specified for migration scenarios
- Use `.returning()` to get the generated ID after insert
### Naming Conventions
### Timestamps
- **Table names**: Singular snake_case (`topic`, `message`, `app_state`)
- **Export names**: `xxxTable` pattern (`topicTable`, `messageTable`)
| Helper | Fields | Use Case |
|--------|--------|----------|
| `createUpdateTimestamps` | `createdAt`, `updatedAt` | Tables without soft delete |
| `createUpdateDeleteTimestamps` | `createdAt`, `updatedAt`, `deletedAt` | Tables with soft delete |
### Common Commands
**Usage:**
```bash
# Generate migrations after schema changes
yarn db:migrations:generate
```
### Column Helpers
```typescript
import { createUpdateTimestamps, createUpdateDeleteTimestamps } from './columnHelpers'
import { uuidPrimaryKey, createUpdateTimestamps } from './columnHelpers'
// Without soft delete
export const tagTable = sqliteTable('tag', {
export const myTable = sqliteTable('my_table', {
id: uuidPrimaryKey(),
name: text(),
...createUpdateTimestamps
})
// With soft delete
export const topicTable = sqliteTable('topic', {
id: uuidPrimaryKey(),
name: text(),
...createUpdateDeleteTimestamps
})
```
**Behavior:**
- `createdAt`: Auto-set to `Date.now()` on insert
- `updatedAt`: Auto-set on insert, auto-updated on update
- `deletedAt`: `null` by default, set to timestamp for soft delete
## JSON Fields
For JSON column support, use `{ mode: 'json' }`:
```typescript
data: text({ mode: 'json' }).$type<MyDataType>()
```
Drizzle handles JSON serialization/deserialization automatically.
## Foreign Keys
### Basic Usage
```typescript
// SET NULL: preserve record when referenced record is deleted
groupId: text().references(() => groupTable.id, { onDelete: 'set null' })
// CASCADE: delete record when referenced record is deleted
topicId: text().references(() => topicTable.id, { onDelete: 'cascade' })
```
### Self-Referencing Foreign Keys
For self-referencing foreign keys (e.g., tree structures with parentId), **always use the `foreignKey` operator** in the table's third parameter:
```typescript
import { foreignKey, sqliteTable, text } from 'drizzle-orm/sqlite-core'
export const messageTable = sqliteTable(
'message',
{
id: uuidPrimaryKeyOrdered(),
parentId: text(), // Do NOT use .references() here
// ...other fields
},
(t) => [
// Use foreignKey operator for self-referencing
foreignKey({ columns: [t.parentId], foreignColumns: [t.id] }).onDelete('set null')
]
)
```
**Why this approach:**
- Avoids TypeScript circular reference issues (no need for `AnySQLiteColumn` type annotation)
- More explicit and readable
- Allows chaining `.onDelete()` / `.onUpdate()` actions
### Circular Foreign Key References
**Avoid circular foreign key references between tables.** For example:
```typescript
// ❌ BAD: Circular FK between tables
// tableA.currentItemId -> tableB.id
// tableB.ownerId -> tableA.id
```
If you encounter a scenario that seems to require circular references:
1. **Identify which relationship is "weaker"** - typically the one that can be null or is less critical for data integrity
2. **Remove the FK constraint from the weaker side** - let the application layer handle validation and consistency (this is known as "soft references" pattern)
3. **Document the application-layer constraint** in code comments
```typescript
// ✅ GOOD: Break the cycle by handling one side at application layer
export const topicTable = sqliteTable('topic', {
id: uuidPrimaryKey(),
// Application-managed reference (no FK constraint)
// Validated by TopicService.setCurrentMessage()
currentMessageId: text(),
})
export const messageTable = sqliteTable('message', {
id: uuidPrimaryKeyOrdered(),
// Database-enforced FK
topicId: text().references(() => topicTable.id, { onDelete: 'cascade' }),
})
```
**Why soft references for SQLite:**
- SQLite does not support `DEFERRABLE` constraints (unlike PostgreSQL/Oracle)
- Application-layer validation provides equivalent data integrity
- Simplifies insert/update operations without transaction ordering concerns
## Migrations
Generate migrations after schema changes:
```bash
yarn db:migrations:generate
```
## Field Generation Rules
The schema uses Drizzle's auto-generation features. Follow these rules:
### Auto-generated fields (NEVER set manually)
- `id`: Uses `$defaultFn()` with UUID v4/v7, auto-generated on insert
- `createdAt`: Uses `$defaultFn()` with `Date.now()`, auto-generated on insert
- `updatedAt`: Uses `$defaultFn()` and `$onUpdateFn()`, auto-updated on every update
### Using `.returning()` pattern
Always use `.returning()` to get inserted/updated data instead of re-querying:
```typescript
// Good: Use returning()
const [row] = await db.insert(table).values(data).returning()
return rowToEntity(row)
// Avoid: Re-query after insert (unnecessary database round-trip)
await db.insert(table).values({ id, ...data })
return this.getById(id)
```
### Soft delete support
The schema supports soft delete via `deletedAt` field (see `createUpdateDeleteTimestamps`).
Business logic can choose to use soft delete or hard delete based on requirements.

75
src/main/data/migration/v2/README.md
View File

@ -1,64 +1,33 @@
# Migration V2 (Main Process)
# Data Migration System
Architecture for the new one-shot migration from the legacy Dexie + Redux Persist stores into the SQLite schema. This module owns orchestration, data access helpers, migrator plugins, and IPC entry points used by the renderer migration window.
This directory contains the v2 data migration implementation.
## Directory Layout
## Documentation
- **Migration Guide**: [docs/en/references/data/v2-migration-guide.md](../../../../../docs/en/references/data/v2-migration-guide.md)
## Directory Structure
```
src/main/data/migration/v2/
├── core/ # Engine + shared context
├── migrators/ # Domain-specific migrators and mappings
├── utils/ # Data source readers (Redux, Dexie, streaming JSON)
├── window/ # IPC handlers + migration window manager
└── index.ts # Public exports for main process
├── core/ # MigrationEngine, MigrationContext
├── migrators/ # Domain-specific migrators
│ └── mappings/ # Mapping definitions
├── utils/ # ReduxStateReader, DexieFileReader, JSONStreamReader
├── window/ # IPC handlers, window manager
└── index.ts # Public exports
```
## Core Contracts
## Quick Reference
- `core/MigrationEngine.ts` coordinates all migrators in order, surfaces progress to the UI, and marks status in `app_state.key = 'migration_v2_status'`. It will clear new-schema tables before running and abort on any validation failure.
- `core/MigrationContext.ts` builds the shared context passed to every migrator:
- `sources`: `ConfigManager` (ElectronStore), `ReduxStateReader` (parsed Redux Persist data), `DexieFileReader` (JSON exports)
- `db`: current SQLite connection
- `sharedData`: `Map` for passing cross-cutting info between migrators
- `logger`: `loggerService` scoped to migration
- `@shared/data/migration/v2/types` defines stages, results, and validation stats used across main and renderer.
### Creating a New Migrator
## Migrators
1. Extend `BaseMigrator` in `migrators/`
2. Implement `prepare`, `execute`, `validate` methods
3. Register in `migrators/index.ts`
- Base contract: extend `migrators/BaseMigrator.ts` and implement:
- `id`, `name`, `description`, `order` (lower runs first)
- `prepare(ctx)`: dry-run checks, counts, and staging data; return `PrepareResult`
- `execute(ctx)`: perform inserts/updates; manage your own transactions; report progress via `reportProgress`
- `validate(ctx)`: verify counts and integrity; return `ValidateResult` with stats (`sourceCount`, `targetCount`, `skippedCount`) and any `errors`
- Registration: list migrators (in order) in `migrators/index.ts` so the engine can sort and run them.
- Current migrators:
- `PreferencesMigrator` (implemented): maps ElectronStore + Redux settings to the `preference` table using `mappings/PreferencesMappings.ts`.
- `AssistantMigrator`, `KnowledgeMigrator`, `ChatMigrator` (placeholders): scaffolding and TODO notes for future tables.
- Conventions:
- All logging goes through `loggerService` with a migrator-specific context.
- Use `MigrationContext.sources` instead of accessing raw files/stores directly.
- Use `sharedData` to pass IDs or lookup tables between migrators (e.g., assistant -> chat references) instead of re-reading sources.
- Stream large Dexie exports (`JSONStreamReader`) and batch inserts to avoid memory spikes.
- Count validation is mandatory; engine will fail the run if `targetCount < sourceCount - skippedCount` or if `ValidateResult.errors` is non-empty.
- Keep migrations idempotent per run—engine clears target tables before it starts, but each migrator should tolerate retries within the same run.
### Key Contracts
## Utilities
- `utils/ReduxStateReader.ts`: safe accessor for categorized Redux Persist data with dot-path lookup.
- `utils/DexieFileReader.ts`: reads exported Dexie JSON tables; can stream large tables.
- `utils/JSONStreamReader.ts`: streaming reader with batching, counting, and sampling helpers for very large arrays.
## Window & IPC Integration
- `window/MigrationIpcHandler.ts` exposes IPC channels for the migration UI:
- Receives Redux data and Dexie export path, starts the engine, and streams progress back to renderer.
- Manages backup flow (dialogs via `BackupManager`) and retry/cancel/restart actions.
- `window/MigrationWindowManager.ts` creates the frameless migration window, handles lifecycle, and relaunch instructions after completion in production.
## Implementation Checklist for New Migrators
- [ ] Add mapping definitions (if needed) under `migrators/mappings/`.
- [ ] Implement `prepare/execute/validate` with explicit counts, batch inserts, and integrity checks.
- [ ] Wire progress updates through `reportProgress` so UI shows per-migrator progress.
- [ ] Register the migrator in `migrators/index.ts` with the correct `order`.
- [ ] Add any new target tables to `MigrationEngine.verifyAndClearNewTables` once those tables exist.
- `prepare(ctx)`: Dry-run checks, return counts
- `execute(ctx)`: Perform inserts, report progress
- `validate(ctx)`: Verify counts and integrity

429
src/renderer/src/data/README.md
View File

@ -1,429 +1,40 @@
# Data Layer - Renderer Process
# Renderer Data Layer
This directory contains the unified data access layer for Cherry Studio's renderer process, providing type-safe interfaces for data operations, preference management, and caching.
This directory contains the renderer process data services.
## Overview
## Documentation
The `src/renderer/src/data` directory implements the new data architecture as part of the ongoing database refactoring project. It provides three core services that handle all data operations in the renderer process:
- **Overview**: [docs/en/references/data/README.md](../../../../docs/en/references/data/README.md)
- **Cache**: [cache-overview.md](../../../../docs/en/references/data/cache-overview.md) | [cache-usage.md](../../../../docs/en/references/data/cache-usage.md)
- **Preference**: [preference-overview.md](../../../../docs/en/references/data/preference-overview.md) | [preference-usage.md](../../../../docs/en/references/data/preference-usage.md)
- **DataApi**: [data-api-in-renderer.md](../../../../docs/en/references/data/data-api-in-renderer.md)
- **DataApiService**: RESTful-style API for communication with the main process
- **PreferenceService**: Unified preference/configuration management with real-time sync
- **CacheService**: Three-tier caching system for optimal performance
## Architecture
## Directory Structure
```
┌─────────────────┐
│ React Components│
└─────────┬───────┘
│
┌─────────▼───────┐
│ React Hooks │ ← useDataApi, usePreference, useCache
└─────────┬───────┘
│
┌─────────▼───────┐
│ Services │ ← DataApiService, PreferenceService, CacheService
└─────────┬───────┘
│
┌─────────▼───────┐
│ IPC Layer │ ← Main Process Communication
└─────────────────┘
src/renderer/src/data/
├── DataApiService.ts # User Data API service
├── PreferenceService.ts # Preferences management
├── CacheService.ts # Three-tier caching system
└── hooks/
├── useDataApi.ts # useQuery, useMutation
├── usePreference.ts # usePreference, usePreferences
└── useCache.ts # useCache, useSharedCache, usePersistCache
```
## Quick Start
### Data API Operations
```typescript
// Data API
import { useQuery, useMutation } from '@data/hooks/useDataApi'
// Fetch data with auto-retry and caching
const { data, loading, error } = useQuery('/topics')
// Create/update data with optimistic updates
const { data } = useQuery('/topics')
const { trigger: createTopic } = useMutation('/topics', 'POST')
await createTopic({ title: 'New Topic', content: 'Hello World' })
```
### Preference Management
```typescript
// Preferences
import { usePreference } from '@data/hooks/usePreference'
// Manage user preferences with real-time sync
const [theme, setTheme] = usePreference('app.theme.mode')
const [fontSize, setFontSize] = usePreference('chat.message.font_size')
// Optimistic updates (default)
await setTheme('dark') // UI updates immediately, syncs to database
```
### Cache Management
```typescript
import { useCache, useSharedCache, usePersistCache } from '@data/hooks/useCache'
// Component-level cache (lost on app restart)
const [count, setCount] = useCache('ui.counter')
// Cross-window cache (shared between all windows)
const [windowState, setWindowState] = useSharedCache('window.layout')
// Persistent cache (survives app restarts)
const [recentFiles, setRecentFiles] = usePersistCache('app.recent_files')
```
## Core Services
### DataApiService
**Purpose**: Type-safe communication with the main process using RESTful-style APIs.
**Key Features**:
- Type-safe request/response handling
- Automatic retry with exponential backoff
- Real-time subscriptions
- Request timeout handling
**Basic Usage**:
```typescript
import { dataApiService } from '@data/DataApiService'
// Simple GET request
const topics = await dataApiService.get('/topics')
// POST with body
const newTopic = await dataApiService.post('/topics', {
body: { title: 'Hello', content: 'World' }
})
```
### PreferenceService
**Purpose**: Centralized preference/configuration management with cross-window synchronization.
**Key Features**:
- Optimistic and pessimistic update strategies
- Real-time cross-window synchronization
- Local caching for performance
- Race condition handling
- Batch operations for multiple preferences
**Basic Usage**:
```typescript
import { preferenceService } from '@data/PreferenceService'
// Get single preference
const theme = await preferenceService.get('app.theme.mode')
// Set with optimistic updates (default)
await preferenceService.set('app.theme.mode', 'dark')
// Set with pessimistic updates
await preferenceService.set('api.key', 'secret', { optimistic: false })
// Batch operations
await preferenceService.setMultiple({
'app.theme.mode': 'dark',
'chat.message.font_size': 14
})
```
### CacheService
**Purpose**: Three-tier caching system for different data persistence needs.
**Cache Tiers**:
1. **Memory Cache**: Component-level, lost on app restart
2. **Shared Cache**: Cross-window, lost on app restart
3. **Persist Cache**: Cross-window + localStorage, survives restarts
**Key Features**:
- TTL (Time To Live) support
- Hook reference tracking (prevents deletion of active data)
- Cross-window synchronization
- Type-safe cache schemas
- Automatic default value handling
**Basic Usage**:
```typescript
import { cacheService } from '@data/CacheService'
// Memory cache - Type-safe (schema key, with auto-completion)
cacheService.set('temp.calculation', result, 30000) // 30s TTL
const result = cacheService.get('temp.calculation')
// Memory cache - Casual (dynamic key, requires manual type)
cacheService.setCasual<TopicCache>(`topic:${id}`, topicData)
const topic = cacheService.getCasual<TopicCache>(`topic:${id}`)
// Shared cache - Type-safe (schema key)
cacheService.setShared('window.layout', layoutConfig)
const layout = cacheService.getShared('window.layout')
// Shared cache - Casual (dynamic key)
cacheService.setSharedCasual<WindowState>(`window:${windowId}`, state)
const state = cacheService.getSharedCasual<WindowState>(`window:${windowId}`)
// Persist cache (survives restarts, schema keys only)
cacheService.setPersist('app.recent_files', recentFiles)
const files = cacheService.getPersist('app.recent_files')
```
**When to Use Type-safe vs Casual Methods**:
- **Type-safe** (`get`, `set`, `getShared`, `setShared`): Use when the key is predefined in the cache schema. Provides auto-completion and type inference.
- **Casual** (`getCasual`, `setCasual`, `getSharedCasual`, `setSharedCasual`): Use when the key is dynamically constructed (e.g., `topic:${id}`). Requires manual type specification via generics.
- **Persist Cache**: Only supports schema keys (no Casual methods) to ensure data integrity.
## React Hooks
### useDataApi
Type-safe data fetching with SWR integration.
```typescript
import { useQuery, useMutation } from '@data/hooks/useDataApi'
// GET requests with auto-caching
const { data, loading, error, mutate } = useQuery('/topics', {
query: { page: 1, limit: 20 }
})
// Mutations with optimistic updates
const { trigger: updateTopic, isMutating } = useMutation('/topics/123', 'PUT')
await updateTopic({ title: 'Updated Title' })
```
### usePreference
Reactive preference management with automatic synchronization.
```typescript
import { usePreference } from '@data/hooks/usePreference'
// Basic usage with optimistic updates
const [theme, setTheme] = usePreference('app.theme.mode')
// Pessimistic updates for critical settings
const [apiKey, setApiKey] = usePreference('api.key', { optimistic: false })
// Handle updates
const handleThemeChange = async (newTheme) => {
try {
await setTheme(newTheme) // Auto-rollback on failure
} catch (error) {
console.error('Theme update failed:', error)
}
}
```
### useCache Hooks
Component-friendly cache management with automatic lifecycle handling.
```typescript
// Cache
import { useCache, useSharedCache, usePersistCache } from '@data/hooks/useCache'
// Memory cache (useState-like, but shared between components)
const [counter, setCounter] = useCache('ui.counter', 0)
// Shared cache (cross-window)
const [layout, setLayout] = useSharedCache('window.layout')
// Persistent cache (survives restarts)
const [recentFiles, setRecentFiles] = usePersistCache('app.recent_files')
```
## Best Practices
### When to Use Which Service
The three services map to distinct data categories based on the original architecture design. Use the following guide to choose the right service.
#### Quick Decision Table
| Service | Data Characteristics | Lifecycle | Data Loss Impact | Examples |
|---------|---------------------|-----------|------------------|----------|
| **CacheService** | Regenerable, temporary | ≤ App process or survives restart | None to minimal | API responses, computed results, UI state |
| **PreferenceService** | User settings, key-value | Permanent until changed | Low (can rebuild) | Theme, language, font size, shortcuts |
| **DataApiService** | Business data, structured | Permanent | **Severe** (irreplaceable) | Topics, messages, files, knowledge base |
#### CacheService - Runtime & Cache Data
Use CacheService when:
- Data can be **regenerated or lost without user impact**
- No backup or cross-device synchronization needed
- Lifecycle is tied to component, window, or app session
**Two sub-categories**:
1. **Performance cache**: Computed results, API responses, expensive calculations
2. **UI state cache**: Temporary settings, scroll positions, panel states
**Three tiers based on persistence needs**:
- `useCache` (memory): Lost on app restart, component-level sharing
- `useSharedCache` (shared): Cross-window sharing, lost on restart
- `usePersistCache` (persist): Survives app restarts via localStorage
```typescript
// Good: Temporary computed results
const [searchResults, setSearchResults] = useCache('search.results', [])
// Good: UI state that can be lost
const [sidebarCollapsed, setSidebarCollapsed] = useSharedCache('ui.sidebar.collapsed', false)
// Good: Recent items (nice to have, not critical)
const [recentSearches, setRecentSearches] = usePersistCache('search.recent', [])
```
#### PreferenceService - User Preferences
Use PreferenceService when:
- Data is a **user-modifiable setting that affects app behavior**
- Structure is key-value with **predefined keys** (users modify values, not keys)
- **Value structure is stable** (won't change frequently)
- Data loss has **low impact** (user can reconfigure)
**Key characteristics**:
- Auto-syncs across all windows
- Each preference item should be **atomic** (one setting = one key)
- Values are typically: boolean, string, number, or simple array/object
```typescript
// Good: App behavior settings
const [theme, setTheme] = usePreference('app.theme.mode')
const [language, setLanguage] = usePreference('app.language')
const [fontSize, setFontSize] = usePreference('chat.message.font_size')
// Good: Feature toggles
const [showTimestamp, setShowTimestamp] = usePreference('chat.display.show_timestamp')
```
#### DataApiService - User Data
Use DataApiService when:
- Data is **business data accumulated through user activity**
- Data is **structured with dedicated schemas/tables**
- Users can **create, delete, modify records** (no fixed limit)
- Data loss would be **severe and irreplaceable**
- Data volume can be **large** (potentially GBs)
**Key characteristics**:
- No automatic window sync (fetch on demand for fresh data)
- May contain sensitive data (encryption consideration)
- Requires proper CRUD operations and transactions
```typescript
// Good: User-generated business data
const { data: topics } = useQuery('/topics')
const { trigger: createTopic } = useMutation('/topics', 'POST')
// Good: Conversation history (irreplaceable)
const { data: messages } = useQuery('/messages', { query: { topicId } })
// Good: User files and knowledge base
const { data: files } = useQuery('/files')
```
#### Decision Flowchart
Ask these questions in order:
1. **Can this data be regenerated or lost without affecting the user?**
- Yes → **CacheService**
- No → Continue to #2
2. **Is this a user-configurable setting that affects app behavior?**
- Yes → Does it have a fixed key and stable value structure?
- Yes → **PreferenceService**
- No (structure changes often) → **DataApiService**
- No → Continue to #3
3. **Is this business data created/accumulated through user activity?**
- Yes → **DataApiService**
- No → Reconsider #1 (most data falls into one of these categories)
#### Common Anti-patterns
| Wrong Choice | Why It's Wrong | Correct Choice |
|--------------|----------------|----------------|
| Storing AI provider configs in Cache | User loses configured providers on restart | **PreferenceService** |
| Storing conversation history in Preferences | Unbounded growth, complex structure | **DataApiService** |
| Storing topic list in Preferences | User-created records, can grow large | **DataApiService** |
| Storing theme/language in DataApi | Overkill for simple key-value settings | **PreferenceService** |
| Storing API responses in DataApi | Regenerable data, doesn't need persistence | **CacheService** |
| Storing window positions in Preferences | Can be lost without impact | **CacheService** (persist tier) |
#### Edge Cases
- **Recently used items** (e.g., recent files, recent searches): Use `usePersistCache` - nice to have but not critical if lost
- **Draft content** (e.g., unsaved message): Use `useSharedCache` for cross-window, consider auto-save to DataApi for recovery
- **Computed statistics**: Use `useCache` with TTL - regenerate when expired
- **User-created templates/presets**: Use **DataApiService** - user-generated content that can grow
### Performance Guidelines
1. **Prefer React Hooks**: Use `useQuery`, `usePreference`, `useCache` for component integration
2. **Batch Operations**: Use `setMultiple()` for updating multiple preferences
3. **Cache Strategically**: Use appropriate cache tiers based on data lifetime needs
4. **Optimize Re-renders**: SWR and useSyncExternalStore minimize unnecessary re-renders
### Common Patterns
```typescript
// Loading states with error handling
const { data, loading, error } = useQuery('/topics')
if (loading) return <Loading />
if (error) return <Error error={error} />
// Form handling with preferences
const [fontSize, setFontSize] = usePreference('chat.message.font_size')
const handleChange = (e) => setFontSize(Number(e.target.value))
// Temporary state with caching
const [searchQuery, setSearchQuery] = useCache('search.current_query', '')
const [searchResults, setSearchResults] = useCache('search.results', [])
```
## Type Safety
All services provide full TypeScript support with auto-completion and type checking:
- **API Types**: Defined in `@shared/data/api/`
- **Preference Types**: Defined in `@shared/data/preference/`
- **Cache Types**: Defined in `@shared/data/cache/`
Type definitions are automatically inferred, providing:
- Request/response type safety
- Preference key validation
- Cache schema enforcement
- Auto-completion in IDEs
## Migration from Legacy Systems
This new data layer replaces multiple legacy systems:
- Redux-persist slices → PreferenceService
- localStorage direct access → CacheService
- Direct IPC calls → DataApiService
- Dexie database operations → DataApiService
For migration guidelines, see the project's `.claude/` directory documentation.
## File Structure
```
src/renderer/src/data/
├── DataApiService.ts # User Data API querying service
├── PreferenceService.ts # Preferences management
├── CacheService.ts # Three-tier caching system
└── hooks/
├── useDataApi.ts # React hooks for user data operations
├── usePreference.ts # React hooks for preferences
└── useCache.ts # React hooks for caching
```
## Related Documentation
- **API Schemas**: `packages/shared/data/` - Type definitions and API contracts
- **Architecture Design**: `.claude/data-architecture.md` - Detailed system design
- **Migration Guide**: `.claude/migration-planning.md` - Legacy system migration
- **Project Overview**: `CLAUDE.local.md` - Complete refactoring context