adding freight web and holding

This commit is contained in:
JB
2025-12-24 00:38:53 -05:00
parent 0c8630b8ba
commit f39e9132ad
164 changed files with 5559 additions and 156655 deletions

View File

@@ -1,213 +1,213 @@
# Architecture Overview - Star Kitten
## System Architecture
Star Kitten follows a modular monorepo architecture with four main packages and two applications:
```mermaid
graph TB
subgraph "Applications"
EB[eve-bot<br/>Discord Bot]
EW[eve-web<br/>Brisa Web App]
end
subgraph "Core Packages"
D[@star-kitten/discord<br/>Discord Framework]
E[@star-kitten/eve<br/>EVE Online Integration]
U[@star-kitten/util<br/>Shared Utilities]
end
subgraph "External Services"
DS[Discord API]
ESI[EVE ESI API]
JA[Janice API]
ET[EveTycoon API]
end
subgraph "Data Layer"
DB[(SQLite Database)]
RD[(Reference Data<br/>JSON Files)]
end
EB --> D
EB --> E
EW --> E
EW --> U
D --> DS
E --> ESI
E --> JA
E --> ET
E --> DB
E --> RD
U --> DB
```
## Source Code Paths
### Core Packages
#### [@star-kitten/discord](packages/discord/)
- **Entry Point**: [`packages/discord/src/index.ts`](packages/discord/src/index.ts)
- **Command System**: [`packages/discord/src/commands/`](packages/discord/src/commands/)
- [`import-commands.ts`](packages/discord/src/commands/import-commands.ts) - Auto-discovery of command files
- [`handle-commands.ts`](packages/discord/src/commands/handle-commands.ts) - Command execution handler
- **Localization**: [`packages/discord/src/locales.ts`](packages/discord/src/locales.ts)
#### [@star-kitten/eve](packages/eve/)
- **Entry Point**: [`packages/eve/src/index.ts`](packages/eve/src/index.ts)
- **Database Layer**: [`packages/eve/src/db/`](packages/eve/src/db/)
- [`schema.ts`](packages/eve/src/db/schema.ts) - Drizzle ORM schema definitions
- [`index.ts`](packages/eve/src/db/index.ts) - Database connection and exports
- [`models/`](packages/eve/src/db/models/) - Character and user model helpers
- **ESI Integration**: [`packages/eve/src/esi/`](packages/eve/src/esi/)
- [`auth.ts`](packages/eve/src/esi/auth.ts) - EVE SSO authentication flow
- [`scopes.ts`](packages/eve/src/esi/scopes.ts) - ESI scope definitions and token validation
- [`character.ts`](packages/eve/src/esi/character.ts) - Character API endpoints
- [`fetch.ts`](packages/eve/src/esi/fetch.ts) - ESI API client wrapper
- **Data Models**: [`packages/eve/src/models/`](packages/eve/src/models/)
- [`type.ts`](packages/eve/src/models/type.ts) - EVE item type definitions and utilities
- [`skill.ts`](packages/eve/src/models/skill.ts) - Skill-related models
- **Third-party APIs**: [`packages/eve/src/third-party/`](packages/eve/src/third-party/)
- [`janice.ts`](packages/eve/src/third-party/janice.ts) - Market appraisal integration
- [`evetycoon.ts`](packages/eve/src/third-party/evetycoon.ts) - Market data integration
- **Utilities**: [`packages/eve/src/utils/`](packages/eve/src/utils/)
- [`markdown.ts`](packages/eve/src/utils/markdown.ts) - EVE markup to Discord formatting
#### [@star-kitten/util](packages/util/)
- **Entry Point**: [`packages/util/src/`](packages/util/src/) (no index.ts, exports individual modules)
- **Core Utilities**:
- [`text.ts`](packages/util/src/text.ts) - Text processing and formatting
- [`jsonQuery.ts`](packages/util/src/jsonQuery.ts) - Streaming JSON querying for large files
- [`time.ts`](packages/util/src/time.ts) - Time utilities
- [`logger.ts`](packages/util/src/logger.ts) - Logging utilities
- **Scheduler System**: [`packages/util/src/scheduler/`](packages/util/src/scheduler/)
- [`scheduler.service.ts`](packages/util/src/scheduler/scheduler.service.ts) - Job scheduling service
- [`queue.ts`](packages/util/src/scheduler/queue.ts) - Job queue implementation
- [`workers/`](packages/util/src/scheduler/workers/) - Background worker implementations
### Applications
#### [eve-bot](packages/eve-bot/)
- **Entry Point**: [`packages/eve-bot/src/main.ts`](packages/eve-bot/src/main.ts)
- **Commands**: [`packages/eve-bot/src/commands/`](packages/eve-bot/src/commands/)
- [`appraise.command.ts`](packages/eve-bot/src/commands/appraise.command.ts) - Market appraisal command
- **Types**: [`packages/eve-bot/src/types/global.d.ts`](packages/eve-bot/src/types/global.d.ts)
#### [eve-web](packages/eve-web/)
- **Entry Point**: [`packages/eve-web/src/pages/index.tsx`](packages/eve-web/src/pages/index.tsx)
- **Middleware**: [`packages/eve-web/src/middleware.ts`](packages/eve-web/src/middleware.ts) - EVE SSO integration
- **Components**: [`packages/eve-web/src/components/stats/`](packages/eve-web/src/components/stats/)
- [`skill-queue.tsx`](packages/eve-web/src/components/stats/skill-queue.tsx) - Real-time skill training display
- [`wallet.tsx`](packages/eve-web/src/components/stats/wallet.tsx) - Wallet balance and changes
- **API Routes**: [`packages/eve-web/src/api/auth/`](packages/eve-web/src/api/auth/) - Authentication endpoints
- **Utilities**: [`packages/eve-web/src/utils/cookies.ts`](packages/eve-web/src/utils/cookies.ts)
## Key Technical Decisions
### Command System Architecture
- **Auto-discovery**: Commands are automatically discovered using glob patterns (`**/*.command.{js,ts}`)
- **Convention-based**: Each command exports a default Discord command definition
- **Event-driven**: Commands handle their own interaction events using global event listeners
- **Localized**: Full internationalization support with description translations
### Database Architecture
- **ORM**: Drizzle ORM with SQLite for local data storage
- **Schema**: [`packages/eve/src/db/schema.ts`](packages/eve/src/db/schema.ts) defines all tables
- **Models**: Active Record pattern with helper classes for complex operations
- **Migrations**: Drizzle Kit handles schema migrations
### EVE Online Integration
- **OAuth Flow**: Complete EVE SSO implementation with token management
- **Scope Management**: Comprehensive ESI scope definitions and validation
- **Token Refresh**: Automatic token refresh with scope preservation
- **Data Caching**: Smart caching of EVE reference data and API responses
### Web Interface Architecture
- **Framework**: Brisa for server-side rendering with progressive enhancement
- **Authentication**: EVE SSO integrated at middleware level
- **Components**: Server-side components with suspense for async data loading
- **Styling**: Tailwind CSS with DaisyUI components
## Design Patterns
### Package Exports
Each package uses structured exports for different concerns:
```typescript
// @star-kitten/eve exports
export * from "./esi"; // ESI API integration
export * as CharacterAPI from "./esi/character";
export * as models from "./models"; // Data models
export * as db from "./db"; // Database access
export * from "./third-party"; // External APIs
```
### Error Handling
- **API Errors**: Consistent error wrapping for external API calls
- **Token Validation**: Graceful handling of expired/invalid tokens
- **Database Errors**: Transaction rollback and error logging
### Data Processing
- **Streaming**: Large JSON files processed via streaming for memory efficiency
- **Caching**: Multi-level caching (in-memory, database, file system)
- **Type Safety**: Full TypeScript coverage with strict type checking
## Component Relationships
### Discord Bot Flow
1. [`main.ts`](packages/eve-bot/src/main.ts) initializes Dysnomia client
2. [`importCommands()`](packages/discord/src/commands/import-commands.ts) discovers command files
3. Commands register interaction handlers on global client
4. User interactions trigger command-specific handlers
### Web Authentication Flow
1. [`middleware.ts`](packages/eve-web/src/middleware.ts) intercepts requests
2. EVE SSO redirect initiated for unauthenticated users
3. OAuth callback validates tokens and creates/updates user records
4. Character data synchronized from ESI API
5. Dashboard components display real-time character statistics
### Data Synchronization
1. Reference data downloaded from external sources
2. JSON files processed using streaming query utilities
3. Character data fetched from ESI API
4. Local database updated with character and user information
5. Web components display cached data with real-time updates
## Critical Implementation Paths
### Adding New Commands
1. Create `*.command.ts` file in [`packages/eve-bot/src/commands/`](packages/eve-bot/src/commands/)
2. Export Discord command definition as default
3. Add interaction handlers to global client event listeners
4. Commands automatically discovered and registered
### EVE API Integration
1. Define required scopes in [`scopes.ts`](packages/eve/src/esi/scopes.ts)
2. Implement API client in appropriate ESI module
3. Add character model methods for data access
4. Create web components for data display
### Database Schema Changes
1. Update [`schema.ts`](packages/eve/src/db/schema.ts) with new tables/columns
2. Generate migration using Drizzle Kit
3. Update model classes with new methods
4. Add database exports to [`db/index.ts`](packages/eve/src/db/index.ts)
# Architecture Overview - Star Kitten
## System Architecture
Star Kitten follows a modular monorepo architecture with four main packages and two applications:
```mermaid
graph TB
subgraph "Applications"
EB[eve-bot<br/>Discord Bot]
EW[eve-web<br/>Brisa Web App]
end
subgraph "Core Packages"
D[@star-kitten/discord<br/>Discord Framework]
E[@star-kitten/eve<br/>EVE Online Integration]
U[@star-kitten/util<br/>Shared Utilities]
end
subgraph "External Services"
DS[Discord API]
ESI[EVE ESI API]
JA[Janice API]
ET[EveTycoon API]
end
subgraph "Data Layer"
DB[(SQLite Database)]
RD[(Reference Data<br/>JSON Files)]
end
EB --> D
EB --> E
EW --> E
EW --> U
D --> DS
E --> ESI
E --> JA
E --> ET
E --> DB
E --> RD
U --> DB
```
## Source Code Paths
### Core Packages
#### [@star-kitten/discord](packages/discord/)
- **Entry Point**: [`packages/discord/src/index.ts`](packages/discord/src/index.ts)
- **Command System**: [`packages/discord/src/commands/`](packages/discord/src/commands/)
- [`import-commands.ts`](packages/discord/src/commands/import-commands.ts) - Auto-discovery of command files
- [`handle-commands.ts`](packages/discord/src/commands/handle-commands.ts) - Command execution handler
- **Localization**: [`packages/discord/src/locales.ts`](packages/discord/src/locales.ts)
#### [@star-kitten/eve](packages/eve/)
- **Entry Point**: [`packages/eve/src/index.ts`](packages/eve/src/index.ts)
- **Database Layer**: [`packages/eve/src/db/`](packages/eve/src/db/)
- [`schema.ts`](packages/eve/src/db/schema.ts) - Drizzle ORM schema definitions
- [`index.ts`](packages/eve/src/db/index.ts) - Database connection and exports
- [`models/`](packages/eve/src/db/models/) - Character and user model helpers
- **ESI Integration**: [`packages/eve/src/esi/`](packages/eve/src/esi/)
- [`auth.ts`](packages/eve/src/esi/auth.ts) - EVE SSO authentication flow
- [`scopes.ts`](packages/eve/src/esi/scopes.ts) - ESI scope definitions and token validation
- [`character.ts`](packages/eve/src/esi/character.ts) - Character API endpoints
- [`fetch.ts`](packages/eve/src/esi/fetch.ts) - ESI API client wrapper
- **Data Models**: [`packages/eve/src/models/`](packages/eve/src/models/)
- [`type.ts`](packages/eve/src/models/type.ts) - EVE item type definitions and utilities
- [`skill.ts`](packages/eve/src/models/skill.ts) - Skill-related models
- **Third-party APIs**: [`packages/eve/src/third-party/`](packages/eve/src/third-party/)
- [`janice.ts`](packages/eve/src/third-party/janice.ts) - Market appraisal integration
- [`evetycoon.ts`](packages/eve/src/third-party/evetycoon.ts) - Market data integration
- **Utilities**: [`packages/eve/src/utils/`](packages/eve/src/utils/)
- [`markdown.ts`](packages/eve/src/utils/markdown.ts) - EVE markup to Discord formatting
#### [@star-kitten/util](packages/util/)
- **Entry Point**: [`packages/util/src/`](packages/util/src/) (no index.ts, exports individual modules)
- **Core Utilities**:
- [`text.ts`](packages/util/src/text.ts) - Text processing and formatting
- [`jsonQuery.ts`](packages/util/src/jsonQuery.ts) - Streaming JSON querying for large files
- [`time.ts`](packages/util/src/time.ts) - Time utilities
- [`logger.ts`](packages/util/src/logger.ts) - Logging utilities
- **Scheduler System**: [`packages/util/src/scheduler/`](packages/util/src/scheduler/)
- [`scheduler.service.ts`](packages/util/src/scheduler/scheduler.service.ts) - Job scheduling service
- [`queue.ts`](packages/util/src/scheduler/queue.ts) - Job queue implementation
- [`workers/`](packages/util/src/scheduler/workers/) - Background worker implementations
### Applications
#### [eve-bot](packages/eve-bot/)
- **Entry Point**: [`packages/eve-bot/src/main.ts`](packages/eve-bot/src/main.ts)
- **Commands**: [`packages/eve-bot/src/commands/`](packages/eve-bot/src/commands/)
- [`appraise.command.ts`](packages/eve-bot/src/commands/appraise.command.ts) - Market appraisal command
- **Types**: [`packages/eve-bot/src/types/global.d.ts`](packages/eve-bot/src/types/global.d.ts)
#### [eve-web](packages/eve-web/)
- **Entry Point**: [`packages/eve-web/src/pages/index.tsx`](packages/eve-web/src/pages/index.tsx)
- **Middleware**: [`packages/eve-web/src/middleware.ts`](packages/eve-web/src/middleware.ts) - EVE SSO integration
- **Components**: [`packages/eve-web/src/components/stats/`](packages/eve-web/src/components/stats/)
- [`skill-queue.tsx`](packages/eve-web/src/components/stats/skill-queue.tsx) - Real-time skill training display
- [`wallet.tsx`](packages/eve-web/src/components/stats/wallet.tsx) - Wallet balance and changes
- **API Routes**: [`packages/eve-web/src/api/auth/`](packages/eve-web/src/api/auth/) - Authentication endpoints
- **Utilities**: [`packages/eve-web/src/utils/cookies.ts`](packages/eve-web/src/utils/cookies.ts)
## Key Technical Decisions
### Command System Architecture
- **Auto-discovery**: Commands are automatically discovered using glob patterns (`**/*.command.{js,ts}`)
- **Convention-based**: Each command exports a default Discord command definition
- **Event-driven**: Commands handle their own interaction events using global event listeners
- **Localized**: Full internationalization support with description translations
### Database Architecture
- **ORM**: Drizzle ORM with SQLite for local data storage
- **Schema**: [`packages/eve/src/db/schema.ts`](packages/eve/src/db/schema.ts) defines all tables
- **Models**: Active Record pattern with helper classes for complex operations
- **Migrations**: Drizzle Kit handles schema migrations
### EVE Online Integration
- **OAuth Flow**: Complete EVE SSO implementation with token management
- **Scope Management**: Comprehensive ESI scope definitions and validation
- **Token Refresh**: Automatic token refresh with scope preservation
- **Data Caching**: Smart caching of EVE reference data and API responses
### Web Interface Architecture
- **Framework**: Brisa for server-side rendering with progressive enhancement
- **Authentication**: EVE SSO integrated at middleware level
- **Components**: Server-side components with suspense for async data loading
- **Styling**: Tailwind CSS with DaisyUI components
## Design Patterns
### Package Exports
Each package uses structured exports for different concerns:
```typescript
// @star-kitten/eve exports
export * from "./esi"; // ESI API integration
export * as CharacterAPI from "./esi/character";
export * as models from "./models"; // Data models
export * as db from "./db"; // Database access
export * from "./third-party"; // External APIs
```
### Error Handling
- **API Errors**: Consistent error wrapping for external API calls
- **Token Validation**: Graceful handling of expired/invalid tokens
- **Database Errors**: Transaction rollback and error logging
### Data Processing
- **Streaming**: Large JSON files processed via streaming for memory efficiency
- **Caching**: Multi-level caching (in-memory, database, file system)
- **Type Safety**: Full TypeScript coverage with strict type checking
## Component Relationships
### Discord Bot Flow
1. [`main.ts`](packages/eve-bot/src/main.ts) initializes Dysnomia client
2. [`importCommands()`](packages/discord/src/commands/import-commands.ts) discovers command files
3. Commands register interaction handlers on global client
4. User interactions trigger command-specific handlers
### Web Authentication Flow
1. [`middleware.ts`](packages/eve-web/src/middleware.ts) intercepts requests
2. EVE SSO redirect initiated for unauthenticated users
3. OAuth callback validates tokens and creates/updates user records
4. Character data synchronized from ESI API
5. Dashboard components display real-time character statistics
### Data Synchronization
1. Reference data downloaded from external sources
2. JSON files processed using streaming query utilities
3. Character data fetched from ESI API
4. Local database updated with character and user information
5. Web components display cached data with real-time updates
## Critical Implementation Paths
### Adding New Commands
1. Create `*.command.ts` file in [`packages/eve-bot/src/commands/`](packages/eve-bot/src/commands/)
2. Export Discord command definition as default
3. Add interaction handlers to global client event listeners
4. Commands automatically discovered and registered
### EVE API Integration
1. Define required scopes in [`scopes.ts`](packages/eve/src/esi/scopes.ts)
2. Implement API client in appropriate ESI module
3. Add character model methods for data access
4. Create web components for data display
### Database Schema Changes
1. Update [`schema.ts`](packages/eve/src/db/schema.ts) with new tables/columns
2. Generate migration using Drizzle Kit
3. Update model classes with new methods
4. Add database exports to [`db/index.ts`](packages/eve/src/db/index.ts)