Voice Mouse for Mac OS X

This voice mouse for Mac OS X is a work in progress. It allows those with mobility issues to interact with the mouse cursor using only their voice.

You may already be making use of the superb speech recognition technology that’s built into Mac OS X, especially if you have any mobility problems. Out of the box you can switch applications, scroll pages, navigate the menu bar and much more. That’s great for basic interaction with your system, but what if you could control the mouse with your voice? Wouldn’t that be great!?
VoiceMouseX uses the same speech recognition technology that’s built into your Mac computer, giving you complete control over the mouse cursor.
If you have mobility issues, you may rely heavily on tabbing through elements on a webpage. This can be a slow process and many elements created by javascript, such as drop down menus are inaccessible. Interacting with Flash or Shockwave elements is usually not possible. With VoiceMouseX you can use your voice to move the mouse and interact with those elements that were previously inaccessible.

How does it work?

For moving the mouse a long distance, you bring up the main grid by saying “Show Grid”. This will then show a 100-cell grid over your screen. You specify which cell you’d like the mouse to move to by saying the cell number, i.e “50”. The mouse will be moved to the center of the selected cell and the selected cell will then be split into a 9-cell grid, allowing you to more accurately place the mouse. Again you select from this grid by saying the name of the cell, i.e “5”.
At any time you can issue a compass command, such as “North” or “South East”, this moves the mouse a short distance in the corresponding direction, allowing pin point positioning.
To click the mouse you simply say “click left” or “click right”, except for Flash objects (such as Youtube videos) where you’ll say “click flash”.


This software is not yet ready for release but a demo video is shown below. The screen recording software, used to record the demo video, was interfering with the mouse click commands, as such, some commands had to be issued more than once. Under normal operation, issuing a click command will work on the first try.

10 Responses to “Voice Mouse for Mac OS X”

  1. […] very far, but that is entirely too taxing. [Stephen] came up with a really neat way to control a mouse with your voice, a project that is sure to find its way onto the desktops of those with mobility issues very […]

  2. Iddles says:

    Very good. I wonder how hard it’d be to incorporate tone-of-voice to allow clicking precisely within the grid cell in a single utterance. Or just by humming a couple of notes.

    • admin says:

      Thanks for the feedback Iddles. That’s a great idea and could definitely serve as a quick way to move the mouse around. errrrrrrrrrr could move the mouse up X pixels for every second of audio and owwwwww could move the mouse down X pixels for every second and so on for each direction. Is that close to what you had in mind?

  3. Joeing says:

    Looks awesome. It’s probably the easy to use voice mouse I’ve seen. Although to be fair there aren’t many options out there for this kind of thing, especially for a Mac! How much do you intend to charge for the software?

    • admin says:

      Thanks Joeing, I’m glad you like it. And yeah you’re right, there’s not many options on Mac for those with mobility problems. The software will be free :) I just want to make sure it’s quick and stable enough before putting out the first version. Everything’s looking good but I like to be extra sure.

  4. kidna says:

    make it accessable via keyboard and this is a must-have for even potent people.

    • admin says:

      What a great idea, I can’t believe I hadn’t thought of that! Quick mouse movements by voice, for those with mobility issues and then quick mouse movements by keyboard for those without mobility issues but simply don’t want to use a mouse!

  5. jchr says:

    Windows 7 actually has something similar, although I’ve never found it very accurate and it’s a bit more complicated to use.

    • admin says:

      I haven’t used Windows since back in the XP days so I’m probably a bit behind on the feature list! I’ll do some research and see if I can get any tips to improve VoiceMouseX :)

  6. Durgmin3 says:

    have you looked at dragon dictate? they have something like this but its a harder to use cos its a cross grid. plus it costs over $150 !

Leave a Reply