windows-cua

A high-performance, native C++ port of Meta’s macOS computer-use agent (originally named metacua).

This implementation replaces the original Swift/Python macOS architecture with a single, dependency-free Win32 executable. It drives Meta's muse-spark-1.1 model (via the Responses API) to interact with your actual Windows desktop. The agent operates in a classic look-act-look loop: capturing the screen, evaluating goals, planning, executing actions via native input synthesis, and repeating until the goal is complete or stopped.

---

Architectural & System-Level Highlights

Unlike sandboxed variants, this agent drives your actual physical machine. The C++ codebase integrates directly with native Windows subsystem APIs:

1. Pixel-Perfect Coordinate Mapping & DPI Awareness

To ensure that the model's normalized 0-1000 coordinates map accurately to physical coordinates on any screen:

• The application calls SetProcessDpiAwarenessContext(DPI_AWARENESS_CONTEXT_PER_MONITOR_AWARE_V2) at startup (falling back to SetProcessDPIAware()).
• This prevents Windows from virtualizing or scaling the display coordinates for the process, ensuring GDI screenshot dimensions align exactly with Win32 input dimensions.

2. High-Performance GDI+ Screen Capture

Screenshots are captured natively without shell process dependencies:

• The primary display is copied into a 32bpp DIB section via a Win32 BitBlt with the CAPTUREBLT flag enabled (ensuring layered/transparent windows are captured).
• Downsampling is handled in memory via Gdiplus::Bitmap using High-Quality Bicubic interpolation and half-pixel offsets to prevent alias distortion.
• The resulting image is encoded directly to PNG base64 using GDI+ COM encoders streamed via IStream.

3. Native Win32 Input Synthesis (SendInput)

Input execution matches the performance and timing requirements of interactive environments:

• Mouse Glide: Mouse movements are interpolated using a cubic ease-in-out curve over dynamic step counts based on travel distance to mimic human actions.
• Absolute Coordinates: Mouse positions are scaled from normalized logical coordinate spaces to the Win32 0..65535 absolute coordinate system.
• The Focus Theft Prevention Bypass: Windows modern focus-theft protection can block non-active processes from calling SetForegroundWindow. `wind