ChessPrism/cline_docs/problem.md

Chess Board Detection Problem

Issue Description

The chess board detection system is currently only displaying the bottom 2-3 rows of the chess board in the "Chessboard" preview window, while the "Full Capture" window shows the complete chess board. The width of the detected board is correct, but the height is truncated.

Visual Evidence

• Full Capture: Shows complete chess board with all pieces
• Chessboard Preview: Shows only bottom portion (approximately 2.5 rows) of the board
• Width appears correct in both views
• Height is significantly truncated in Chessboard Preview

Technical Analysis

Detection Pipeline

1. Screen capture works correctly (evidenced by Full Capture view)
2. Initial board detection appears to find correct width
3. Problem occurs during either:
   • Rectangle detection phase
   • Coordinate transformation
   • Image cropping stage

Coordinate System Complexity

1. Multiple coordinate systems involved:
   • Vision framework (bottom-left origin)
   • NSImage/CGImage (bottom-left origin)
   • SwiftUI (top-left origin)
2. Current transformations may be:
   • Incorrectly mapping between coordinate spaces
   • Losing vertical positioning information
   • Miscalculating crop region

Detection Parameters

1. Current approach:
   • Using 0.3-0.5 aspect ratio for detection
   • 0.4 minimum size requirement
   • Single observation limit
2. These parameters may be:
   • Causing partial detection of board
   • Missing full vertical extent
   • Incorrectly identifying board boundaries

Code Areas to Investigate

BoardDetector.swift

1. detectBoard() function:

   • Board extraction logic
   • Coordinate calculations
   • Rectangle positioning

2. Rectangle Detection:

   request.minimumAspectRatio = 0.3
   request.maximumAspectRatio = 0.5
   request.minimumSize = 0.4
   

   • May need adjustment for full board capture

ScreenCaptureViewModel.swift

1. Image cropping logic:
   • Coordinate transformation
   • Crop region calculation
   • Final image generation

Potential Solutions to Explore

1. Detection Approach

   • Adjust aspect ratio constraints
   • Modify detection parameters
   • Consider alternative detection methods

2. Coordinate Handling

   • Review all coordinate transformations
   • Validate coordinate space conversions
   • Ensure proper origin point handling

3. Image Processing

   • Verify crop region calculations
   • Review image scaling operations
   • Validate final image generation

4. Alternative Approaches

   • Use full-frame detection
   • Implement grid-based detection
   • Consider machine learning approach

Impact

• Critical functionality issue
• Blocks accurate board analysis
• Affects user experience
• Prevents proper game state detection

Priority

HIGH - This issue blocks core functionality of the chess analysis system.

Next Steps

1. Verify coordinate system transformations
2. Test different aspect ratio parameters
3. Implement logging for detection boundaries
4. Add visualization of detected regions
5. Consider alternative detection approaches

Additional Notes

• The issue appears consistent across different games
• Width detection is working correctly
• Height truncation is consistent (showing ~2.5 rows)
• Full board is visible in capture, suggesting screen capture is working