How to fix a USB audio device that interferes with mouse clicks in Linux

(This post includes an overview of how to solve similar problems to the one I encountered.  If you're looking for just the solution?  Skip to "The solution" below.)

I recently got a Jabra USB headset.  It's a very nice headset with active noise cancelling, a decent microphone, and a USB audio controller that includes a few buttons to control the audio volume.  These buttons work directly with the OS by presenting as a USB keyboard and sending the same keystrokes that the volume up/down buttons on a real keyboard would.

Unfortunately, it also presents itself as a USB mouse with 12 buttons and takes precedence over any actual pointing devices attached to the system.  Why a USB audio device needs to present itself to the system as a mouse is beyond me.  (The only theory I've heard that makes any sense at all is that there's some Windows-specific driver that would intercept mouse events and do some sort of deeper OS integration.)  Whatever the reason, it's a terrible "feature" and an abuse of the USB HID specification.

But rant over. Fortunately because X (the graphics engine of desktop Linux systems) allows the user fine control over the system, this is pretty easy to fix.  I'm including the steps I used to troubleshoot and fix this so that if anyone runs into similar issues in the future they know what to do.

Narrowing the issue down to X

As soon as I started using this headset I noticed something strange: I couldn't click the mouse anywhere outside of the window that I had on top when I plugged it in.  This persisted until I unplugged the USB cable, when suddenly mouse clicks worked again.  When I plugged the USB cable back in, mouse clicks stopped working again.  Weird, but at least consistent!

Knowing that it had something to do with the USB device I started looking at a few system logs.  I ran 'dmesg', which shows the device driver logs for the system, but didn't seem anything unusual.  So it wasn't a Linux issue, which meant it had to be in X.

Diagnosing in X

The first thing I did was to see what X saw when I had the device plugged in.  I ran 'xinput', which shows a list of all of the input devices attached to the system.  Before plugging in the device I saw:

⎡ Virtual core pointer                     id=2 [master pointer  (3)]
⎜   ↳ Virtual core XTEST pointer               id=4 [slave  pointer  (2)]
⎜   ↳ ELAN Touchscreen                         id=10 [slave  pointer  (2)]
⎜   ↳ DLL0665:01 06CB:76AD Touchpad           id=12 [slave  pointer  (2)]
⎣ Virtual core keyboard                   id=3 [master keyboard (2)]
    ↳ Virtual core XTEST keyboard             id=5 [slave  keyboard (3)]
    ↳ Power Button                             id=6 [slave  keyboard (3)]
    ↳ Video Bus                               id=7 [slave  keyboard (3)]
    ↳ Power Button                             id=8 [slave  keyboard (3)]
    ↳ Sleep Button                             id=9 [slave  keyboard (3)]
    ↳ Integrated_Webcam_HD                     id=11 [slave  keyboard (3)]
    ↳ AT Translated Set 2 keyboard             id=13 [slave  keyboard (3)]
    ↳ Dell WMI hotkeys                         id=14 [slave  keyboard (3)]

and afterwards I saw:

⎡ Virtual core pointer                     id=2 [master pointer  (3)]

⎜   ↳ Virtual core XTEST pointer               id=4 [slave  pointer  (2)]

⎜   ↳ ELAN Touchscreen                         id=10 [slave  pointer  (2)]

⎜   ↳ DLL0665:01 06CB:76AD Touchpad           id=12 [slave  pointer  (2)]

⎜   ↳ GN Netcom A/S Jabra EVOLVE LINK MS       id=15 [slave  pointer  (2)]

⎣ Virtual core keyboard                   id=3 [master keyboard (2)]

    ↳ Virtual core XTEST keyboard             id=5 [slave  keyboard (3)]

    ↳ Power Button                             id=6 [slave  keyboard (3)]

    ↳ Video Bus                               id=7 [slave  keyboard (3)]

    ↳ Power Button                             id=8 [slave  keyboard (3)]

    ↳ Sleep Button                             id=9 [slave  keyboard (3)]

    ↳ Integrated_Webcam_HD                     id=11 [slave  keyboard (3)]

    ↳ AT Translated Set 2 keyboard             id=13 [slave  keyboard (3)]

    ↳ Dell WMI hotkeys                         id=14 [slave  keyboard (3)]

The highlighted line shows the change: this was the device I had plugged in.  (It appears that GN Netcom A/S is the actual manufacturer and Jabra the brand.)

This told me I was on the right track: the USB headset was showing up as an input device.  This could be why it was interfering with the mouse!  But I needed more information.  Fortunately, xinput has a more verbose mode, 'xinput list --long':

⎜   ↳ GN Netcom A/S Jabra EVOLVE LINK MS       id=15 [slave  pointer  (2)]

Reporting 16 classes:

Class originated from: 15. Type: XIButtonClass

Buttons supported: 12

Button labels: "Button 0" "Button 1" "Button 2" "Button Wheel Up" "Button Wheel Down" "Button Horiz Wheel Left" "Button Horiz Wheel Right" "Button 3" "Button 4" "Button 5" "Button 6" "Button 7"

Button state:

Class originated from: 15. Type: XIKeyClass

Keycodes supported: 248

Class originated from: 15. Type: XIValuatorClass

Detail for Valuator 0:

 Label: Abs X

 Range: 0.000000 - 1000.000000

 Resolution: 0 units/m

 Mode: absolute

 Current value: 1600.000000

...

(I've trimmed this down quite a bit; 'xinput list --long' really is long.)

From this I saw that it was reporting itself as a mouse device.  (Why does a USB headset need to pretend to the OS to be a mouse?  I still don't know.)  Something in that list of mouse buttons was messing with my actual mouse.  So I needed to disable that functionality while leaving the rest unchanged.

Fine-grained device control in X

The X device control system allows you to make a lot of tweaks to how devices work.  You can change how a keyboard or mouse work, how quickly the pointer moves when you move the mouse, how it accelerates based on the direction you are moving... all sorts of things.  But for my purposes I needed to use the functionality to change the order of mouse buttons.  This is normally for doing things like making a mouse more comfortable for left-handed users (e.g. changing the button order on the mouse from "1 2 3" to "3 2 1" so the left-hand index finger is the primary button).  But you can also map physical buttons to "button 0", which means disabled.  So I just disabled all 12 (why 12‽) mouse buttons.

I did this with the help of the X device control manual here: www.x.org/archive/X11R7.5/doc/man/man4/evdev.4.html

The device control system looks in a special location for files that contain instructions  to change how devices work.  On Ubuntu this location is /usr/share/X11/xorg.conf.d/ but may be different on other Linux distributions.

The solultion

I created /usr/share/X11/xorg.conf.d/50-jabra.conf with the following contents:

Section "InputClass"

Identifier "Jabra"

        MatchProduct "GN Netcom A/S Jabra EVOLVE LINK MS"

Option "ButtonMapping" "0 0 0 0 0 0 0 0 0 0 0 0"

EndSection

I then restarted the X system to make sure the changes would take effect.  (A reboot is the simplest way to do this if you're unsure how.)

Once that was finished I could plug and unplug my USB headset with no issues!

Followup

I submitted a bug report to the Ubuntu bug tracking system with my fix attached.  Hopefully it will be picked up and added to the distribution.  In the meantime I'll leave this post up in case anyone else has similar issues.

Exploring the Onion Omega

Some backstory (or the importance of being engaged)

I'll be honest, I felt a bit burned out on new small Internet of Things devices after my previous experience with the Belleds Q.  The Q shipped, but was immediately abandoned by the development team and left to die.  I even tried to help keep things moving by creating some projects that I documented here and setting up a forum called IoTTalk to bring the folks hacking on the Q together, but it wasn't enough without the support of the project's founders who moved with no forwarding address.

But a few months back, I joined in on the Kickstarter for the Onion Omega, a platform for prototyping and building Internet of Things devices.  I'm happy to say that my experience with Onion couldn't be more different!  The founders have been incredibly involved and responsive, interacting with the community on a daily basis.  Seeing what people are already doing with this new platform and being part of the community is truly exciting!

Now onto the Onion Omega

The Omega is a tiny development platform based on an ARM System On a Chip (SOC).  It includes a 400MHz processor, 64MB of RAM, 16MB of onboard flash storage, WiFi, and a number of expansion pins that can be used for a multitude of purposes, all costing about $19.  It barely sips power, being able to run for several days off of a common cell phone external battery via its USB power input.  Some existing expansions provide USB ports, Ethernet networking, relays, and even a tiny OLED display!

The Omega itself is about 44mm/1.75" along its longest axis, or about 12mm longer than a standard SD card. 

The best part for me is that it runs Linux natively.  I've been hacking on Linux for a couple of decades at this point and am usually more comfortable with a command line than a soldering iron, so having a platform that I can start using from a familiar environment is huge for me.

The Omega provides a WiFi access point out of the box and if docked to an expansion with a USB port also provides a serial console and provides both ssh/console access and a web interface.  This makes it easy to start using with no special tools or setup required.  It runs OpenWRT, a Linux distribution commonly used in wireless routers, meaning it has access to a wide array of software already packaged.  (At the moment, there's still some ongoing effort to recompile all those packages for the Omega, but even lacking existing ones the Onion team has great instructions and support for cross-compiling and pushing files over to the Omega.)

So what can it do?

Toy projects

I've done a couple of toy projects with the Omega already.  To play with the platform, the first thing I did was set up a script to poll the wireless networks within its range every 5 seconds and write them to a file.  I then plugged the Omega into an external cell phone battery and took a brief walk around my neighborhood to see what I would see.  The code for that and the simple (and ugly!) parser is in a repository on Github.  (It can even display the current summary status on the OLED expansion!)

I've written up a few more tutorials (mostly aimed at people with limited Linux experience) over at the Onion Community site.  For example, by installing a single package and using simple shell commands, you can blink the onboard LED in a Morse code message.

Save some money

Slightly more usefully, I used some shell scripts and an Omega to act as a WiFi-enabled remote control for the serial port in my home projector.  This is hundreds of dollars cheaper than any existing wireless serial system I could find and will allow me to connect my projector to a home automation system.

An Onion Omega and USB to serial adapter plugged into my video projector allowed me to duplicate a wireless serial adapter that sells for hundreds of dollars.

The real beauty of this is that I didn't have to do any hardware hacking.  All of the functionality is exposed through Linux, meaning a few shell scripts or a Python program is all I need to use the functionality of these real-world devices.

What's next?

After I mount my projector on the ceiling and have a single button to dim the lights, turn on the stereo receiver and projector, and load up Netflix, I'll need a new project.  I wonder if anyone is looking at porting Snappy Ubuntu to the Omega...

(There were over 500 unique unsecured access points in range during my 20 minute walk, and an even larger number still using the long-deprecated WEP security system that would be basically effortless to break.  Upgrade your routers, people!)

The Q API

I wrote about the Q and my thoughts about it at a general level recently.  I've also been tinkering with making it actually do interesting things via the API that it exposes.

The API is part of a beta firmware update, and it shows.  For example, right now the command to get a list of connected light bulbs locks up the device and requires a reboot.  I'm sure that will get fixed, and fortunately I can just read the serial numbers off of the physical bulbs themselves in the meantime.  But it does mean that it's basically impossible to write a consumer application to the API right now.

The API

The API that the Q exposes is pretty simple: it listens on a UDP port for messages in JSON.  It then returns nothing, a string, or JSON data.  It's a bit warty; anything communicating with it needs to expect different formatted data depending on the action.  I'd prefer to see it always return JSON, including when it doesn't understand the request.  Unfortunately, it never returns anything if there is an error and some commands don't get a success confirmation or even acknowledgement.  So it's possible that you think something has failed when it has succeeded or vice-versa.   I really hope that changes.

It's also not as well documented yet as I'd like.  There are fields that are entirely explained right now.  For example, the API to control the lights themselves contains fields like "effect", "matchValue", and "iswitch" which are undocumented.  So far I've determined that setting effect to "9" makes the bulb do what you say, a few other values (e.g. "8") makes it turn white no matter what color is specified, and anything else seems to have no effect at all but receives an "ok cmd!" response from the Q station.

Tinkering quickly

While it's possible to write a C, Python, Perl, etc. program as a harness to play with the Q, that's more work than I wanted to go through to just poke at it.  Plus I'm lazy and want to just do things from Bash wherever I can.

Since the Q just expects JSON messages on a given UDP port, it's possible to use netcat to do this without writing anything extra.  As an example, this will simply ping the unit and determine if it is working:

$ echo '{ ping" }' | nc -u 10.1.10.182 11600 -q 1
belled,MQ1AC43700000669,172.16.0.1,1

(This is using my Q's IP address.  You'll want to replace 10.1.10.182 with your unit's IP address.)

The "-q 1" parameter to nc tells it to disconnect after nothing is sent for one second.  Once I had this in my command history, it was a bit more convenient than hitting ^C to end the connection.

I've also used this to change a bulb's color:

$ echo '{ "cmd":"light_ctrl", "r":"155", "g":"20", "b":"255", "bright":"100", "sn_list":[ { "sn":"MD1AC44200001127" } ], "iswitch":"1", "matchValue":"0", "effect":"9"}' | nc -u 10.1.10.182 11600 -q 1

ok cmd!

(Again, my Q's IP address and my light bulb's serial number won't work for you, so be sure to change those.)

netcat allows simple interaction with the Q to interactively play with the API without having to bother with writing a full script and deal with sockets yourself.

Adding hardware support to MAAS

MAAS and power

MAAS, or Metal as a Service, is a tool for treating physical servers similarly to cloud resources.  It lets you take a pile of hardware and assign workloads to it without worrying about all the support infrastructure underneath.  In effect, you can plug a bunch of systems into a network and have a Hadoop or OpenStack cluster up, running, and properly configured inside of half an hour.  It's really cool stuff.

But MAAS is targeted at enterprise-level hardware.  It expects things like IPMI to be available to control the hardware at a basic level.  (If your tool is going to be installing an operating system onto a computer, it needs to be able to control that computer at a level below the OS.)  I have a stack of older desktops sitting around gathering dust that I wanted to use as a cluster for experimenting at home.  They still function quite well, but being desktops they don't have enterprise features like IPMI.

Fortunately most desktop BIOSes have a setting to control what they do after a power outage.  Crucially, they can be set to turn the system on when electrical power is restored, regardless of whether the system was on or off when power was lost.  This means I could use an externally controlled power switch in place of IPMI to allow MAAS to power my desktops on and off.

Power on after power failure.  (Helpfully hidden under a random submenu in the BIOS rather than under the 'POWER SETTINGS' heading.)

I used the Web Power Switch 7 from Digital Loggers for this project.  (Warning: almost every page on their site has autoplaying audio and/or video.  Ugh.)  I'll be completely honest here -- price was the reason I chose this switch.  It allows control of up to 8 outlets and cost me just under $140 from Amazon.

My shiny new power switch.  (What do you mean, your datacenter doesn't have an oriental rug?)

Power templates

There are a few bits of MAAS that I had to touch to fully enable support for my DLI switch.  The first and most obvious was creating a power control template to tell MAAS how to talk to the switch.

When MAAS is installed, the templates that tell it how to talk to power management systems are put in /etc/maas/templates/power.  These are shell scripts with a bit of Python evaluation and substitution (surrounded by {{double braces}})that MAAS uses at runtime to build a command to control a specific system.  There are several examples to use there already, and these can be modified to suit your environment.  For example, the SeaMicro SM15000 hardware template looks like this:

# -*- mode: shell-script -*-
#
# Control a system via ipmipower, sending the seamicro specific hex codes
#
{{py: power_mode = 1 if power_change == 'on' else 6 }}

{{ipmitool}} -I lanplus \
    -H {{power_address}} -U {{power_user}} \
    -P {{power_pass}} raw 0x2E 1 0x00 0x7d 0xab \
    {{power_mode}} 0 {{system_id}}

I started by simply replacing one of these templates with my own code.  Because the DLI Web Power Switch can be controlled via HTTP, I just needed to build a GET request for the outlet I wanted to control and send it to the switch, like this:

#!/bin/sh
# -*- mode: shell-script -*-
#
# Control a system via the Digital Loggers, Inc. RESTful interface
#
{{py: power_mode = 'ON' if power_change == 'on' else 'OFF' }}

wget --auth-no-challenge -O /dev/null \
    http://{{power_user}}:{{power_pass}}@{{power_address}}/outlet?{{system_id}}={{power_mode}}

Once I had this template replacing the original sm15k template, I was able to control my switch by telling MAAS I had SeaMicro hardware.  Success!

Better integration

I could have stopped there with a working system.  But it bothered me a bit to lie to MAAS about my hardware.  And what if someday I or someone else wanted to mix the DLI switch with actual SeaMicro hardware?  Replacing the sm15k template with my DLI template wouldn't work then.

It turns out that there are two places in the MAAS code that need to be modified to make it aware of a new power switch.  In the MAAS source, these are src/maasserver/power_parameters.py and src/provisioningserver/enum.py.  power_parameters.py is where the real definition of a new power switch is; enum.py just needs an entry in two lists to make the UI show the switch as an option.  (There's a comment in enum.py to the effect that adding the switch name in it will not be necessary at some point in the future, but at least as of March 2014 it still is.)

The code needed to add a power switch is straightforward.  Provide the variables the switch will need to operate a system (e.g. outlet number, IP address of the switch, credentials, etc.) and give them names that can be referenced in the power template later.

Success!

The code I added can be seen in Launchpad (click "expand all" to see the diffs).

Once I added the code there, I was able to rebuild MAAS and see my new switch as an option.  It all works, and now my old desktops have new life!

Configuring NFS on Ubuntu in Amazon EC2

(Quick note: if you're looking for the port ranges you need for NFS in EC2, check Step 4 below.)

When Libboo migrated from hosting everything ourselves to using Amazon's Elastic Compute Cloud (EC2), we decided to do a bit of rearchitecting at the same time to make scaling easier in the future.  Part of this was removing any assumptions that everything was running on a single server, so we wanted to put our data in one place that could be shared by any number of webservers.

As we are running everything on Linux, the obvious solution for this was the Network File System (NFS) protocol.  NFS is established and well understood in the Unix world... which means that there are a number of tools built for it and most of the bugs are worked out (or at least well understood) already.  We're using Ubuntu Server at Libboo, so that's what my examples use.  But this should work identically on any Debian-based distribution and be similar anywhere else.

It turns out to be easy to run NFS in EC2, but I didn't see any good documentation about exactly how to do it.  So to save others' time, here's what you need to do to set up NFS on Ubuntu 12.04 in EC2:

Step 1 - Install the NFS server

This is trivial on Ubuntu Server:

sudo apt-get update && sudo apt-get install nfs-kernel-server

This will download the latest version of the NFS server and set it up.

Step 2 - Configure the shared directories ("exports")

Getting to the file you need to work on is simple:

sudo nano /etc/exports

Actually configuring the exports is a bit more involved and unfortunately isn't a 'one size fits all' solution -- it just depends on what you want to share to whom and with what permissions.  But on the bright side there is a lot of good information out there about configuring NFS exports:

• The NFS HOWTO has good, clear detail
• The Ubuntu Help Wiki also has good information, though it's a bit verbose for my taste

Of course, searching Google for "NFS exports" will return a huge amount of help too.

Step 3 - Tell the NFS server about the exports

Once you've configured the exports, you need to tell the NFS server that you've done it:

sudo service nfs-kernel-server reload

This will make the NFS server load the configuration you've done and start using it.

Step 4 - Configure EC2 security

This is the magic bit that has to be done for everything else to work.  You need to tell Amazon to allow other systems to connect to your server on the ports that NFS expects to use.

Go to the EC2 dashboard and select "Security Groups" under "NETWORK & SECURITY".  Choose the security group you've put your NFS server in and add the following rules:

Inbound TCP ports 111, 2049, 44182, 54508

Inbound UDP ports 111, 2049, 32768, 32770 - 32800

 You should also be sure to limit these to a specific IP address (or range if you must).  Leaving these at the default of 0.0.0.0/0 will allow anyone on the Internet to connect to your server.  (You can -- and should -- also restrict this in the NFS configuration, but there's no sense in leaving ports open to anywhere you don't have to.)

Step 5 - You're done!

At this point the server should be working!  There is a lot more that you can configure but the defaults should be enough to get you running.  Now it's just a matter of configuring the clients to mount the shares.