ChessPrism/cline_docs/techContext.md

Technical Context

Development Environment

• macOS development platform
• Xcode IDE
• SwiftUI for user interface
• Swift 5.x language features

Core Technologies

Metal Resource Management

• Shared CIContext pattern:
  • Static shared instance to prevent command queue exhaustion
  • Used across BoardDetector and ViewModel
  • Proper cleanup and resource management
• Performance considerations:
  • Reduced Metal command queue creation
  • Efficient resource utilization
  • Support for long-running captures

ScreenCaptureKit

• System framework for screen capture
• Implemented features:
  • Window detection using SCShareableContent
  • iOS app window capture support
  • Real-time frame capture
  • Proper error handling
  • Configurable cursor visibility
• Key components:
  • SCShareableContent: Window and display access
  • SCContentFilter: Window-specific capture
  • SCStream: Frame capture management
  • SCStreamOutput: Frame processing
  • SCStreamConfiguration: Capture settings including cursor control

Vision Framework (Planned)

• Will be used for board and coordinate detection
• Key components to implement:
  • VNRecognizeTextRequest: Chess coordinate detection
  • VNDetectRectanglesRequest: Board boundary detection
• Planned configuration:
  • Text recognition level: accurate
  • Language correction: disabled
  • Rectangle aspect ratio: 0.3-0.5
  • Minimum size: 0.4
  • Maximum observations: 1

Coordinate Systems

1. Vision Framework

   • Origin: Bottom-left (0,0)
   • Y-axis: Upward positive
   • Normalized coordinates (0-1)
   • Used in: VNRectangleObservation, VNTextObservation

2. NSImage/CGImage

   • Origin: Bottom-left (0,0)
   • Y-axis: Upward positive
   • Pixel coordinates
   • Used in: Image cropping, processing

3. SwiftUI

   • Origin: Top-left (0,0)
   • Y-axis: Downward positive
   • Point coordinates
   • Used in: View layout, rendering

4. Transformations

   • Vision → Screen: Flip Y coordinate
   • Screen → Image: Scale to pixel coordinates
   • Image → View: SwiftUI handles automatically

SwiftUI

• Modern declarative UI framework
• Handles view lifecycle
• State management via @Published properties
• Environmental object propagation

Technical Constraints

Window Capture System

1. Window Detection

   • Using SCShareableContent for window access
   • Multiple validation criteria:
     • Bundle ID verification
     • Window visibility check
     • Size validation
   • Error handling for missing windows

2. Frame Capture

   • Window-specific capture configuration
   • Frame dimension matching
   • Proper delegate handling
   • Resource cleanup
   • Cursor visibility control:
     • Configurable via SCStreamConfiguration
     • State preservation between captures
     • Clean snapshot support

3. Performance

   • Main thread safety for UI updates
   • Efficient image conversion
   • Proper task cancellation
   • Memory management
   • Shared CIContext for Metal efficiency

4. Error Handling

   • Clear error types
   • User-friendly messages
   • State recovery
   • Resource cleanup

System Requirements

• macOS 12.0 or later
• Screen Capture permissions
• Sufficient CPU for real-time processing
• Adequate memory for frame buffering
• Metal-capable GPU for image processing

Dependencies

Internal

• ScreenCapture.swift: Core capture logic
• ScreenCaptureViewModel.swift: State management
• BoardDetector.swift: Pattern recognition
• ContentView.swift: User interface

External

• ScreenCaptureKit.framework
• Vision.framework
• SwiftUI.framework
• CoreImage.framework
• Metal.framework (via CIContext)

Development Guidelines

Code Organization

• MVVM architecture
• Protocol-oriented design
• Clear separation of concerns
• Comprehensive error handling
• Resource sharing patterns

Performance Optimization

• Shared CIContext for Metal efficiency
• Efficient frame processing
• Memory management
• Resource cleanup
• Background queue usage

Error Handling

• Custom error types
• Comprehensive error cases
• User-friendly error messages
• Proper error propagation

Testing Requirements

Unit Tests

• Board detection accuracy
• Coordinate transformations
• Error handling
• State management
• Resource management
• Cursor control functionality

Integration Tests

• End-to-end workflows
• Component interaction
• Event propagation
• Resource sharing
• Snapshot system

UI Tests

• User interaction flows
• Error state handling
• Visual feedback
• Performance monitoring
• Snapshot visualization

Documentation Requirements

Code Documentation

• Function documentation
• Parameter descriptions
• Return value documentation
• Error documentation
• Resource usage documentation

Architecture Documentation

• System overview
• Component interaction
• Data flow diagrams
• State management
• Resource management patterns

Current Challenges

Resource Management

1. Metal Efficiency

   • Command queue management
   • Shared context patterns
   • Resource cleanup
   • Performance monitoring

2. Memory Usage

   • Frame buffer management
   • Image processing optimization
   • Resource pooling
   • Cleanup strategies

Board Detection

1. Full Capture

   • Complete board visibility
   • Proper positioning
   • Consistent results
   • Coordinate accuracy

2. Performance

   • Processing efficiency
   • Memory usage
   • Resource management
   • Error recovery

Future Considerations

Planned Features

1. ML Model Integration

   • Piece detection
   • Position analysis
   • Move validation

2. Engine Integration

   • Stockfish analysis
   • Move evaluation
   • Position scoring

3. Visual Overlay

   • Move suggestions
   • Analysis visualization
   • Interactive elements

Technical Debt

• Refactor coordinate handling
• Optimize frame processing
• Improve error recovery
• Enhanced permission handling
• Resource usage monitoring