ChessPrism/cline_docs/techContext.md

Technical Context

Development Environment

• macOS development platform
• Xcode IDE
• SwiftUI for user interface
• Swift 5.x language features

Core Technologies

ScreenCaptureKit

• System framework for screen capture
• Requires user permissions
• Supports window/display filtering
• Real-time frame capture capabilities

Vision Framework

• Used for board and coordinate detection
• Key components:
  • VNRecognizeTextRequest: Chess coordinate detection
  • VNDetectRectanglesRequest: Board boundary detection
• Configuration:
  • Text recognition level: accurate
  • Language correction: disabled
  • Rectangle aspect ratio: 0.3-0.5 (for taller rectangles)
  • Minimum size: 0.4
  • Maximum observations: 1

Coordinate Systems

1. Vision Framework

   • Origin: Bottom-left (0,0)
   • Y-axis: Upward positive
   • Normalized coordinates (0-1)
   • Used in: VNRectangleObservation, VNTextObservation

2. NSImage/CGImage

   • Origin: Bottom-left (0,0)
   • Y-axis: Upward positive
   • Pixel coordinates
   • Used in: Image cropping, processing

3. SwiftUI

   • Origin: Top-left (0,0)
   • Y-axis: Downward positive
   • Point coordinates
   • Used in: View layout, rendering

4. Transformations

   • Vision → Screen: Flip Y coordinate
   • Screen → Image: Scale to pixel coordinates
   • Image → View: SwiftUI handles automatically

SwiftUI

• Modern declarative UI framework
• Handles view lifecycle
• State management via @Published properties
• Environmental object propagation

Technical Constraints

Board Detection

1. Pattern Recognition

   • Detect taller rectangles (0.3-0.5 aspect ratio)
   • Extract square board from upper portion
   • Use width as reference measurement
   • Handle coordinate system transformations

2. Coordinate Recognition

   • Must detect a-h and 1-8 coordinates
   • Handles both light and dark themes
   • Requires clear coordinate visibility
   • Minimum text size requirements

3. Board Boundaries

   • Square aspect ratio (1:1)
   • Tolerance: 0.3 for dimensions
   • Extract from detected area
   • Proper coordinate transformations

4. Performance

   • Frame processing on dedicated queue
   • Asynchronous Vision requests
   • Memory management for capture session
   • Resource cleanup requirements

System Requirements

• macOS 12.0 or later
• Screen Capture permissions
• Sufficient CPU for real-time processing
• Adequate memory for frame buffering

Dependencies

Internal

• ScreenCapture.swift: Core capture logic
• ScreenCaptureViewModel.swift: State management
• BoardDetector.swift: Pattern recognition
• ContentView.swift: User interface

External

• ScreenCaptureKit.framework
• Vision.framework
• SwiftUI.framework
• CoreImage.framework

Development Guidelines

Code Organization

• MVVM architecture
• Protocol-oriented design
• Clear separation of concerns
• Comprehensive error handling

Performance Optimization

• Efficient frame processing
• Memory management
• Resource cleanup
• Background queue usage

Error Handling

• Custom error types
• Comprehensive error cases
• User-friendly error messages
• Proper error propagation

Testing Requirements

Unit Tests

• Board detection accuracy
• Coordinate transformations
• Error handling
• State management

Integration Tests

• End-to-end workflows
• Component interaction
• Event propagation

UI Tests

• User interaction flows
• Error state handling
• Visual feedback

Documentation Requirements

Code Documentation

• Function documentation
• Parameter descriptions
• Return value documentation
• Error documentation

Architecture Documentation

• System overview
• Component interaction
• Data flow diagrams
• State management

Current Challenges

Coordinate Systems

1. Understanding

   • Different origin points
   • Axis directions
   • Coordinate spaces
   • Transformation requirements

2. Implementation

   • Proper transformations
   • Consistent handling
   • Validation methods
   • Error checking

Board Detection

1. Full Capture

   • Complete board visibility
   • Proper positioning
   • Consistent results
   • Coordinate accuracy

2. Performance

   • Processing efficiency
   • Memory usage
   • Resource management
   • Error recovery

Future Considerations

Planned Features

1. ML Model Integration

   • Piece detection
   • Position analysis
   • Move validation

2. Engine Integration

   • Stockfish analysis
   • Move evaluation
   • Position scoring

3. Visual Overlay

   • Move suggestions
   • Analysis visualization
   • Interactive elements

Technical Debt

• Refactor coordinate handling
• Optimize frame processing
• Improve error recovery
• Enhanced permission handling