238 lines
5.5 KiB
Markdown
238 lines
5.5 KiB
Markdown
# Technical Context
|
|
|
|
## Development Environment
|
|
- macOS development platform
|
|
- Xcode IDE
|
|
- SwiftUI for user interface
|
|
- Swift 5.x language features
|
|
|
|
## Core Technologies
|
|
|
|
### Metal Resource Management
|
|
- Shared CIContext pattern:
|
|
* Static shared instance to prevent command queue exhaustion
|
|
* Used across BoardDetector and ViewModel
|
|
* Proper cleanup and resource management
|
|
- Performance considerations:
|
|
* Reduced Metal command queue creation
|
|
* Efficient resource utilization
|
|
* Support for long-running captures
|
|
|
|
### ScreenCaptureKit
|
|
- System framework for screen capture
|
|
- Implemented features:
|
|
* Window detection using SCShareableContent
|
|
* iOS app window capture support
|
|
* Real-time frame capture
|
|
* Proper error handling
|
|
- Key components:
|
|
* SCShareableContent: Window and display access
|
|
* SCContentFilter: Window-specific capture
|
|
* SCStream: Frame capture management
|
|
* SCStreamOutput: Frame processing
|
|
|
|
### Vision Framework (Planned)
|
|
- Will be used for board and coordinate detection
|
|
- Key components to implement:
|
|
* VNRecognizeTextRequest: Chess coordinate detection
|
|
* VNDetectRectanglesRequest: Board boundary detection
|
|
- Planned configuration:
|
|
* Text recognition level: accurate
|
|
* Language correction: disabled
|
|
* Rectangle aspect ratio: 0.3-0.5
|
|
* Minimum size: 0.4
|
|
* Maximum observations: 1
|
|
|
|
### Coordinate Systems
|
|
1. Vision Framework
|
|
- Origin: Bottom-left (0,0)
|
|
- Y-axis: Upward positive
|
|
- Normalized coordinates (0-1)
|
|
- Used in: VNRectangleObservation, VNTextObservation
|
|
|
|
2. NSImage/CGImage
|
|
- Origin: Bottom-left (0,0)
|
|
- Y-axis: Upward positive
|
|
- Pixel coordinates
|
|
- Used in: Image cropping, processing
|
|
|
|
3. SwiftUI
|
|
- Origin: Top-left (0,0)
|
|
- Y-axis: Downward positive
|
|
- Point coordinates
|
|
- Used in: View layout, rendering
|
|
|
|
4. Transformations
|
|
- Vision → Screen: Flip Y coordinate
|
|
- Screen → Image: Scale to pixel coordinates
|
|
- Image → View: SwiftUI handles automatically
|
|
|
|
### SwiftUI
|
|
- Modern declarative UI framework
|
|
- Handles view lifecycle
|
|
- State management via @Published properties
|
|
- Environmental object propagation
|
|
|
|
## Technical Constraints
|
|
|
|
### Window Capture System
|
|
1. Window Detection
|
|
- Using SCShareableContent for window access
|
|
- Multiple validation criteria:
|
|
* Bundle ID verification
|
|
* Window visibility check
|
|
* Size validation
|
|
- Error handling for missing windows
|
|
|
|
2. Frame Capture
|
|
- Window-specific capture configuration
|
|
- Frame dimension matching
|
|
- Proper delegate handling
|
|
- Resource cleanup
|
|
|
|
3. Performance
|
|
- Main thread safety for UI updates
|
|
- Efficient image conversion
|
|
- Proper task cancellation
|
|
- Memory management
|
|
- Shared CIContext for Metal efficiency
|
|
|
|
4. Error Handling
|
|
- Clear error types
|
|
- User-friendly messages
|
|
- State recovery
|
|
- Resource cleanup
|
|
|
|
### System Requirements
|
|
- macOS 12.0 or later
|
|
- Screen Capture permissions
|
|
- Sufficient CPU for real-time processing
|
|
- Adequate memory for frame buffering
|
|
- Metal-capable GPU for image processing
|
|
|
|
## Dependencies
|
|
|
|
### Internal
|
|
- ScreenCapture.swift: Core capture logic
|
|
- ScreenCaptureViewModel.swift: State management
|
|
- BoardDetector.swift: Pattern recognition
|
|
- ContentView.swift: User interface
|
|
|
|
### External
|
|
- ScreenCaptureKit.framework
|
|
- Vision.framework
|
|
- SwiftUI.framework
|
|
- CoreImage.framework
|
|
- Metal.framework (via CIContext)
|
|
|
|
## Development Guidelines
|
|
|
|
### Code Organization
|
|
- MVVM architecture
|
|
- Protocol-oriented design
|
|
- Clear separation of concerns
|
|
- Comprehensive error handling
|
|
- Resource sharing patterns
|
|
|
|
### Performance Optimization
|
|
- Shared CIContext for Metal efficiency
|
|
- Efficient frame processing
|
|
- Memory management
|
|
- Resource cleanup
|
|
- Background queue usage
|
|
|
|
### Error Handling
|
|
- Custom error types
|
|
- Comprehensive error cases
|
|
- User-friendly error messages
|
|
- Proper error propagation
|
|
|
|
## Testing Requirements
|
|
|
|
### Unit Tests
|
|
- Board detection accuracy
|
|
- Coordinate transformations
|
|
- Error handling
|
|
- State management
|
|
- Resource management
|
|
|
|
### Integration Tests
|
|
- End-to-end workflows
|
|
- Component interaction
|
|
- Event propagation
|
|
- Resource sharing
|
|
|
|
### UI Tests
|
|
- User interaction flows
|
|
- Error state handling
|
|
- Visual feedback
|
|
- Performance monitoring
|
|
|
|
## Documentation Requirements
|
|
|
|
### Code Documentation
|
|
- Function documentation
|
|
- Parameter descriptions
|
|
- Return value documentation
|
|
- Error documentation
|
|
- Resource usage documentation
|
|
|
|
### Architecture Documentation
|
|
- System overview
|
|
- Component interaction
|
|
- Data flow diagrams
|
|
- State management
|
|
- Resource management patterns
|
|
|
|
## Current Challenges
|
|
|
|
### Resource Management
|
|
1. Metal Efficiency
|
|
- Command queue management
|
|
- Shared context patterns
|
|
- Resource cleanup
|
|
- Performance monitoring
|
|
|
|
2. Memory Usage
|
|
- Frame buffer management
|
|
- Image processing optimization
|
|
- Resource pooling
|
|
- Cleanup strategies
|
|
|
|
### Board Detection
|
|
1. Full Capture
|
|
- Complete board visibility
|
|
- Proper positioning
|
|
- Consistent results
|
|
- Coordinate accuracy
|
|
|
|
2. Performance
|
|
- Processing efficiency
|
|
- Memory usage
|
|
- Resource management
|
|
- Error recovery
|
|
|
|
## Future Considerations
|
|
|
|
### Planned Features
|
|
1. ML Model Integration
|
|
- Piece detection
|
|
- Position analysis
|
|
- Move validation
|
|
|
|
2. Engine Integration
|
|
- Stockfish analysis
|
|
- Move evaluation
|
|
- Position scoring
|
|
|
|
3. Visual Overlay
|
|
- Move suggestions
|
|
- Analysis visualization
|
|
- Interactive elements
|
|
|
|
### Technical Debt
|
|
- Refactor coordinate handling
|
|
- Optimize frame processing
|
|
- Improve error recovery
|
|
- Enhanced permission handling
|
|
- Resource usage monitoring
|