ChessPrism/cline_docs/systemPatterns.md

System Patterns

Window Capture Architecture

Window Detection Pattern

1. SCShareableContent Access

   • Async/await pattern for content access
   • Proper error propagation
   • Permission handling

2. Window Identification

   • Multiple validation criteria:

     let bundleID = window.owningApplication?.bundleIdentifier ?? ""
     let isChessApp = bundleID == "com.chess.iphone"
     let hasValidSize = window.frame.width > 100 && window.frame.height > 100
     return isChessApp && window.isOnScreen && hasValidSize
     

   • Fail-fast approach with guard statements
   • Clear error states

Capture System Pattern

1. Stream Configuration

   • Window-specific capture setup
   • Frame dimension matching
   • Proper delegate handling
   • Cursor visibility control:
     • Configurable cursor display
     • Clean snapshot support
     • State preservation

2. Frame Processing

   • Main thread safety for UI updates
   • Efficient image conversion pipeline
   • Resource cleanup

Snapshot System Pattern

1. Cursor-Free Capture

   • Temporary capture session:
     • Disables cursor visibility
     • Takes clean snapshot
     • Restores normal capture
   • Error handling:
     • Session cleanup
     • State recovery
     • Capture restoration

2. Process Flow

   • Stop current capture
   • Start cursor-free capture
   • Wait for stabilization
   • Take snapshot
   • Process image
   • Restore normal capture

Error Handling Pattern

1. Task Management

   • Proper cancellation points
   • Clean state management
   • Resource cleanup

2. Error States

   • Clear error types
   • User-friendly messages
   • State recovery

UI Architecture

MVVM Implementation

1. ViewModel

   • @MainActor for thread safety
   • Published properties for state
   • Clear separation of concerns

2. View Layer

   • SwiftUI declarative UI
   • State-driven updates
   • Error presentation

Async Operations

1. Task Management

   • Structured concurrency
   • Proper cancellation
   • State synchronization

2. State Updates

   • Main thread safety
   • Clear state transitions
   • Error recovery

Core Architecture

Resource Management Patterns

1. Shared CIContext Pattern
   • Static shared instance:

     private static let shared = CIContext()
     private var context: CIContext { Self.shared }
     

   • Benefits:
     • Prevents Metal command queue exhaustion
     • Reduces resource overhead
     • Enables long-running captures
   • Implementation:
     • Used in BoardDetector and ViewModel
     • Proper cleanup on task completion
     • Thread-safe access

Screen Capture System

• Uses ScreenCaptureKit for efficient screen capture
• Implements SCStreamOutput protocol for frame processing
• Handles capture session lifecycle and cleanup
• Manages permissions and error handling
• Optimized resource usage
• Configurable cursor visibility

Board Detection System

Two implemented approaches:

1. Pattern Recognition Approach (Primary)

   • Rectangle detection with Vision framework
   • Aspect ratio-based filtering (0.3-0.5 for taller rectangles)
   • Size-based filtering (0.4 minimum for larger areas)
   • Single observation for precision
   • Board extraction from upper portion
   • Width-based square calculation

2. Coordinate Detection (Fallback)

   • Text recognition for board coordinates
   • Rectangle detection with Vision framework
   • Grid-based validation
   • Coordinate-based refinement

3. Common Infrastructure

   • Asynchronous frame processing
   • Dedicated processing queue
   • Efficient memory management
   • Performance monitoring

Coordinate Systems

• Vision framework: Bottom-left origin (0,0)
• NSImage/CGImage: Bottom-left origin (0,0)
• SwiftUI: Top-left origin (0,0)
• Transformations needed between systems:
  1. Vision → Screen: Flip Y coordinate
  2. Screen → Image: Direct mapping
  3. Image → View: SwiftUI handles automatically

Notification System

• Uses NotificationCenter for event propagation
• Key notifications:
  • boardDetected: Sends detected board rectangle and confidence score
  • captureStateChanged: Updates capture status
  • capturedFrame: Delivers processed frames
  • boardCoordinatesDetected: Reports coordinate detection
  • detectionStats: Reports performance metrics

Design Patterns

MVVM Architecture

• ScreenCapture: Model layer handling capture logic
• ScreenCaptureViewModel: View model managing UI state
• ContentView: SwiftUI view for user interface

Observer Pattern

• NotificationCenter for loose coupling
• Enables modular component communication
• Supports async event handling

Error Handling

• Custom ScreenCaptureError enum
• Comprehensive error cases
• Proper error propagation

Technical Decisions

Vision Framework

• Primary tool for board detection
• Provides rectangle and text detection
• Handles various board orientations
• Requires coordinate system transformation

Pattern Recognition

• Focus on larger detection areas
• Use width as reference measurement
• Extract square board from top portion
• Maintain aspect ratio constraints

Performance Considerations

• Dedicated dispatch queue for frame processing
• Efficient memory management
• Proper resource cleanup
• Single observation optimization

Future Patterns

Planned Implementations

1. Board Position Analysis

   • ML model integration
   • Piece detection system
   • Position validation

2. Move Analysis

   • Stockfish integration
   • Real-time evaluation
   • Visual overlay system

3. State Management

   • Game state tracking
   • Move history
   • Analysis persistence

Testing Patterns

Unit Testing

• ScreenCapture functionality
• Board detection accuracy
• Coordinate recognition

Integration Testing

• End-to-end capture workflow
• Vision framework integration
• Notification system

UI Testing

• SwiftUI interface validation
• User interaction flows
• Error state handling