Local Computer Use Skill

This skill provides local Mac desktop control - screenshots, mouse, and keyboard automation.

When to Use

• Taking screenshots of the local Mac screen
• Clicking at coordinates on the local screen
• Typing text locally
• Pressing keyboard shortcuts
• Moving the mouse cursor
• Scrolling
• Checking accessibility permissions

Available Tools

• local_screenshot - Capture the local Mac screen
• localcomputeraction - Perform mouse/keyboard actions:

- leftclick - Left click at coordinates - rightclick - Right click at coordinates - doubleclick - Double click - tripleclick - Triple click - mousemove - Move cursor to position - leftclickdrag - Click and drag - type - Type text - key - Press keyboard key with modifiers - scroll - Scroll up/down/left/right - cursorposition - Get current cursor position

• localgetdisplay_info - Get display resolution, scale factor
• localcheckpermissions - Check accessibility permissions status

Requirements

The local computer use feature requires Accessibility permissions in macOS: System Settings > Privacy & Security > Accessibility

Example Usage

User: Take a screenshot of my screen
Assistant: [Uses local_screenshot]

User: Click on the Dock at position 500, 900
Assistant: [Uses local_computer_action with action="left_click", x=500, y=900]

User: Type "Hello World" in the current app
Assistant: [Uses local_computer_action with action="type", text="Hello World"]

User: Press Cmd+C to copy
Assistant: [Uses local_computer_action with action="key", key="c", modifiers=["cmd"]]