AI Mouse Project

The AI Mouse:
Your Private AI, At Your Fingertips.

Seamlessly integrate Large Language Models into your workflow, directly from your mouse – even when copy/paste is disabled.

Explore the Build
AI Mouse Project in action

The Challenge: Seamless & Private AI Interaction

In an age of powerful AI, interacting with large language models often involves clunky copy-pasting, exposing sensitive data to cloud services, or breaking workflow. We envisioned a more direct, private, and intuitive way to tap into AI intelligence.

The ultimate test? Building a system that can capture your typed thoughts from any text editor, even when traditional copy/paste methods (Ctrl+C, Ctrl+V, right-click context menus) are intentionally disabled. This forces a truly integrated and custom input solution, making the AI Mouse not just convenient, but essential.

The Solution: How It Works

The AI Mouse is a symphony of integrated hardware and custom software, working together to bring local AI processing directly to your desktop.

Key Components & Product Links:

  • The Mouse: Your custom-modified Logitech G502 HERO (Wired Version). Logitech G502 HERO Product Page
  • The Brain (Orange Pi Zero 2W): A tiny single-board computer inside the mouse. Orange Pi Zero 2W on Amazon
  • The Muscle (LattePanda Sigma): A powerful local server. LattePanda Sigma Product Page
  • The AI (Ollama + Meta-Llama-3-8B-Instruct): The local LLM server.
  • The Smart Input (PC Companion App): A custom application on your PC that silently captures typed input, even when copy/paste is disabled, and communicates with the Orange Pi.

Deep Dive: Hardware Integration

The heart of the AI Mouse lies in its meticulous hardware modification. We carefully integrated the Orange Pi Zero 2W into the chassis of a Logitech G502 HERO, leveraging its internal space and robust build.

Key Hardware Modifications:

  • USB Power Integration: Tapping into the mouse's existing USB cable (5V and GND) to provide reliable power directly to the Orange Pi Zero 2W.
  • Middle Button GPIO Wiring: Soldering direct connections from the scroll wheel click button to a GPIO input pin on the Orange Pi, enabling precise trigger control.
  • Space Optimization: Careful cable management to ensure the Orange Pi fits snugly without obstructing any mouse functions.

Deep Dive: Software Intelligence

The software stack orchestrates the entire AI interaction, from capturing your thoughts to delivering AI responses.

Orange Pi Zero 2W: The Client Brain

Running Armbian OS, the Orange Pi hosts a Python script that monitors the middle mouse button. Upon activation, it communicates with the PC Companion App to initiate text capture and, subsequently, sends the full prompt to the Ollama server.

Python Client Script (Excerpt):


# GPIO Setup (Conceptual)
import gpiod
# ...
chip = gpiod.Chip('gpiochip0')
ai_trigger_line = chip.get_line(17) # Your specific GPIO pin
ai_trigger_line.request(consumer="ai_mouse_button", type=gpiod.LINE_REQ_DIR_IN, flags=gpiod.LINE_REQ_FLAG_BIAS_PULL_UP)

# ...
# Ollama API Call (from Orange Pi)
import requests
# ...
response = requests.post(OLLAMA_API_URL, headers=headers, json=payload, timeout=300)
response.raise_for_status()
llm_response = response.json().get("response", "").strip()
                

PC Companion App: The Input Interceptor

Developed in Python, this application runs on your main PC. It features a custom keylogger (using `pynput`) that captures individual keypresses, bypassing traditional copy/paste restrictions. It acts as a bridge, sending the buffered text to the Orange Pi when commanded by the mouse button.

PC Companion App (Keylogger Excerpt):


import pynput.keyboard
# ...
typed_buffer = []
# ...
def on_press(key):
    global typed_buffer
    if recording_active: # Activated by Orange Pi command
        try:
            if hasattr(key, 'char') and key.char is not None:
                typed_buffer.append(key.char)
            elif key == pynput.keyboard.Key.backspace:
                if typed_buffer: typed_buffer.pop()
            # ... handle other keys ...
        except AttributeError: pass
# ...
listener = pynput.keyboard.Listener(on_press=on_press)
listener.start()
# ... TCP/IP communication to Orange Pi
                

LattePanda Sigma: The AI Server Hub

Running Ubuntu Server, the LattePanda hosts Ollama, providing a powerful and efficient environment for running the Meta-Llama-3-8B-Instruct model. Ollama handles the heavy lifting of LLM inference, responding to requests from the Orange Pi via its API.

Ollama Setup (Terminal Commands):


# Install Ollama
curl -fsSL https://ollama.com/install.sh | sh

# Download Llama 3 model
ollama pull llama3

# Example API interaction (from Orange Pi)
# POST http://YOUR_LATTEPANDA_IP:11434/api/generate
# { "model": "llama3", "prompt": "Your captured text here" }
                

Future Enhancements & Learnings

This project is just the beginning. The AI Mouse serves as a robust platform for further innovation in personal AI interaction.

Potential Future Features:

  • **Integrated Display:** A tiny OLED screen on the mouse to show brief AI responses or status updates.
  • **Multi-Modal AI:** Expand beyond text to include voice input or image analysis capabilities.
  • **Haptic Feedback:** Provide tactile responses from the mouse when AI processing is complete or a new thought is generated.
  • **Customizable Hotkeys:** Allow users to define custom AI actions linked to specific mouse gestures or button combinations.

Key Learnings:

This journey underscored the power of open-source hardware and software, the challenges of embedded systems integration, and the immense potential of local, private AI solutions.