Showing posts with label code. Show all posts
Showing posts with label code. Show all posts

Saturday, February 13, 2016

Building a home automation system the easy way

In a previous post I talked about the idea of "cheap code" to go with cheap hardware.  Not every program has to be useful to the entire world or fit for general purpose -- it's fine to have a simple, ad-hoc script to do what you need to do quickly.

I've been using this idea to build a simple home automation system for the connected devices in my house.  The code isn't elegant.  There's no user interface to add new devices or change settings.  It's not even secure in any way.  But it works and I have it now, and if I were aiming to make something I could box up and ship as a real product, I'd still be working on it.  (Actually, that's not true.  I'd lose interest and never finish it.)  But even if the code I've written isn't generally useful, the structure and ideas could be.

So with that in mind, here's a sketch of how all the pieces fit together.

Overview

Logical overview of my home automation remote system
At a high level the architecture is pretty simple.  At the center running the whole show is an Onion Omega, a cheap wifi-enabled computer made for prototyping and projects like these.

The Omega is running a webserver and PHP to serve up the remote control interface and process commands for the projector, which is connected via a USB serial interface.  My stereo and lighting system are on the local network and can be controlled through HTTP requests.

The interface

The remote control interface is just a simple HTML page, not even processed through PHP.  All of the actual control is done through Javascript, which means almost everything is actually executing on the device I access the remote control interface through (e.g. a phone, tablet, or laptop).  This gives me a few advantages.  I don't need a lot of processing power or memory on the Omega, as it's really doing very little as far as actual control goes.  But it also lets me control devices and have a dynamic interface with no wait for a round-trip request to the Omega's webserver.  The entire interface is loaded on the first connection, and then parts are simply displayed or hidden in reaction to what the user presses, so there's no frustrating lag after controlling one device before selecting another.

One other advantage that doing the whole thing in Javascript gave me was a complete accident.  I realized after I'd started writing the remote control that I could have different looks or skins for it but share the same underlying code.  This lets me develop and test in a bare-bones HTML interface but then get really silly for running it on the tablet in my living room.

Yes, that is a Star Trek LCARS interface.  And yes, I am a geek.  The CSS is available on Github.

Javascript for REST devices

Both my Denon stereo and Philips Hue lights are controlled by sending an HTTP request to the device.  Since I'm executing the control through Javascript, the easiest way to do this is to abuse AJAX, the functionality built into Javascript to allow it to make asynchronous requests to a webserver.  Usually this is used to get data from a remote server and update something on a page in the browser.  But in my case, all I want to do is send a bit of information to a device and I don't care what the device sends back.  In fact, if it sends me anything, I'm just going to ignore it.

(Remember, this is all about doing things cheaply and quickly!  If my stereo gets confused by something I send it, no one will get hurt or lose money.  The worst that will happen is that I have to walk across the room and turn it on by hand.  So why fret too much about checking for errors?)

The stereo is the easy one.  It just requires an HTTP GET of a specific URL to set it to what I want.  It does return some status information, but again I just ignore that.


function setStereo(status)
{
    var xhr = new XMLHttpRequest();
    var url;
    
    switch (status)
    {
        case 'on':
            //url = 'MainZone/index.put.asp?cmd0=PutSystem_OnStandby%2FON';
            url = 'MainZone/index.put.asp?cmd0=PutZone_OnOff%2FON';
            break;
        case 'off':
            url = 'MainZone/index.put.asp?cmd0=PutSystem_OnStandby%2FSTANDBY';
            break;
        case 'mute':
            url = 'MainZone/index.put.asp?cmd0=PutVolumeMute%2Fon';
            break;
...
    }
    
    xhr.open('GET', 'http://0005CD3A5CE1.udlunder/' + url);
    xhr.send(null);
}
The Philips Hue system is only slightly more complex.  It requires an HTTP PUT rather than a GET, and expects a JSON document.  While you can control color and brightness of individual bulbs, I just made some presets stored in the Hue itself for different lighting scenes and so only need a very simple JSON document to recall them.


function setScene(scene)
{
    var xhr = new XMLHttpRequest();
    
    xhr.open('PUT',
    encodeURI('http://philips-hue.udlunder/api/openHABRuntime/groups/0/action'));
    xhr.send('{"scene":"' + scene + '"}');
}

Controlling the projector

Probably the ugliest part of the whole thing is the projector.  It doesn't have a REST interface like the other devices, but it does have a serial port.  This is intended for industrial applications or control of the projector when it's mounted somewhere inaccessible, but it works just as well for people like me who are too lazy to keep track of yet another remote control.

This is the only place where the Omega is doing anything more than serving static web pages.  The PHP server on the Omega is used only to receive a projector command and pass it to a shell script that runs it.  The PHP script is truly ugly, doing no error control or security checks -- it's a great way for someone to take control of my Omega or run malicious code there.  But frankly there's nothing of value stored on it and to attack it they'd have to be on my local network anyway, meaning I have bigger problems.  So I'm okay with this.


<?php
    system("stty -F /dev/ttyUSB0 115200");
    system("/root/projector.sh " . $_GET["c"]);
?>

(That's it -- the entire PHP script is two lines of code.)

Putting it all together

These individual pieces are all simple and useful, but combined they're fantastic!  I can reuse those Javascript functions all over my remote control and have one button do many things.  It looks more complicated than it really is when you diagram out what's going on inside:

Flow to set up multiple devices for a "movie" preset from a single remote control button press


There are ways to build dynamic Javascript code, but again I took the easy way out.  I made a short function for every preset that I wanted and just call the appropriate functions.  For example, when I select the "Movie" button, the lights are dimmed, the stereo and projector are turned on and both are set to the proper input:


function scene_lr_movietime()
{
    setProjector('on');
    setProjector('srchdmi');
    setStereo('on');
    setStereo('inputdefault');
    lights_lr_movie();
}

(I broke the light settings into a separate function just so that I could have another button on the remote that would change the lights without turning on the projector or stereo and not have to write more code.)

Wrapping up

That's really it.  Nothing here is what I would call robust or beautiful, but it all works and was pretty easy to do.  It's a great sort of project to tackle if you want to learn more about the Internet of Things or find a use for some of the cheap hardware that's available today.  And these components can be further built on as you add new devices or want to add new functionality.

Friday, January 23, 2015

Building and running the Alljoyn 'About' sample application

I got some nifty color-changing networked light bulbs via a Kickstarter campaign recently.  Hopefully I'll have more to say about those later when I have a bit more time (as I took some unboxing shots and poked around in the router they shipped with it and found some curious things).  But in the meantime, I've been poking around at the Allseen Alliance's Alljoyn framework.

Alljoyn is a framework that allows all sorts of nifty communication and collaboration between devices.  Imagine getting a new window-mounted air conditioner and having it and your thermostat immediately start talking to each other and getting your house to the right temperature, or your lights dimming in response to you starting a movie on your television.  There are all sorts of cool things this sort of communication could allow.

But for now I'm just trying to get it built and working with my light bulbs.  Tonight I spent a little bit of time getting it built and running the simple 'About' sample application that ships with it.  I thought the notes I took might be useful if anyone else starts playing with it. And unfortunately some of the docs about running the 'About' samples are a bit wrong, so maybe this will help someone out.

And if you do play with Alljoyn, please let me know!  Double bonus points if you can point me at some tutorials that don't start with the assumption that you've read the whole codebase and every document ever produced.  I just want to make some light bulbs blink!

Set up the environment

I used the following:

  • Virtual machine (VirtualBox) with 2x x86_64 CPUs, 1GB RAM, 16 GB disk, bridged network
  • Ubuntu 14.04 Server
  • Default packageset plus OpenSSH server
  • Applied full updates (apt-get update ; apt-get upgrade) as of 23 Jan 2015 and reboot

I initially started with libvirt but after a few minutes of trying to convince it to bridge the network device, I decided to just go with VirtualBox rather than spend all night tinkering with that.

Install prerequisites

git:

sudo apt-get install git
git config --global user.email "email@server.tld"
git config --global user.name "User Name"

Repo:

(per http://source.android.com/source/downloading.html#installing-repo)

mkdir ~/bin
PATH=~/bin:$PATH
curl https://storage.googleapis.com/git-repo-downloads/repo > ~/bin/repo
chmod a+x ~/bin/repo

Build tools:

(per https://allseenalliance.org/developers/develop/building/linux)
[NB: ia32-libs was not available in the package archives but didn't seem to be necessary]

sudo apt-get install build-essential libgtk2.0-dev libssl-dev xsltproc libxml2-dev scons libssl-dev #ia32-libs

Grab sources

Per https://wiki.allseenalliance.org/develop/downloading_the_source

mkdir -p src/alljoyn
cd src/alljoyn

# Uncomment other versions below if you want something other than 14.06
#repo init -u https://git.allseenalliance.org/gerrit/devtools/manifest
#repo init -u https://git.allseenalliance.org/gerrit/devtools/manifest -b refs/tags/v14.06 -m versioned.xml
repo init -u https://git.allseenalliance.org/gerrit/devtools/manifest -b RB14.06
repo sync

Build samples

export AJ_ROOT=`pwd`
cd core/alljoyn
scons BINDINGS=cpp WS=off BT=off ICE=off SERVICES="about,notification,controlpanel,config,onboarding,sample_apps"

Run 'about' sample

(per https://allseenalliance.org/developers/develop/run-sample-apps/about/linux)

cd $AJ_ROOT
export TARGET_CPU=x86_64

# NB: The following are in the docs but are wrong.  Corrections below.
#export LD_LIBRARY_PATH=$AJ_ROOT/core/alljoyn/build/linux/$TARGET_CPU/debug/dist/cpp/lib:$LD_LIBRARY_PATH
#$AJ_ROOT/core/alljoyn/build/linux/$TARGET_CPU/debug/dist/cpp/bin/samples/AboutService

export LD_LIBRARY_PATH=$AJ_ROOT/core/alljoyn/build/linux/$TARGET_CPU/debug/dist/cpp/lib:$AJ_ROOT/core/alljoyn/build/linux/$TARGET_CPU/debug/dist/about/lib:$LD_LIBRARY_PATH
$AJ_ROOT/core/alljoyn/build/linux/$TARGET_CPU/debug/dist/about/bin/AboutClient

Friday, July 20, 2012

Altering ebooks -- adding pages to an existing epub


I've been working with ebooks (specifically epub files) lately, particularly with modifying them.  There are some great tools out there to help with this work.

Calibre is fantastic at converting between formats, and better yet it has a command-line interface, so it can be part of an automated script.  It can also do some simple editing of metadata, allowing you to update the author, title, etc. of a book.  But there's no functionality for editing the content of an ebook.

Sigil is another awesome tool.  This is the go-to program for editing the content of an ebook.  It has one major drawback for me though -- it's not scriptable.  There's no way to do something like adding an informational page into an existing ebook without doing it by hand.

I did some research but wasn't able to really find anything that would let me add a page to an ebook non-interactively, so I did it myself.  I thought this might be useful to other people looking to modify epub files, I've included it below.

A couple of notes to keep in mind:

  • I wrote this in PHP because I needed to interface with a large existing PHP codebase.  This would be even easier to do in Python, but the logic here is pretty straightforward and should be easily adapted.
  • The epub format is really simple.  At heart it's some XML, some HTML (or XHTML), maybe a few images, all wrapped up in a zip container.  That's fortunate in that there are a lot of libraries out there to work on exactly these formats.
  • That said, there are some tricky bits to how the epub zip has to be structured.  You can't just throw everything into a zip and rename it, which means the built-in PHP zip libraries don't work for it.
The code is here, and a quick text overview follows after.

<?php
    /*** Helper function from php.net ***/
    // This allows the delete of a directory and its contents
    function rrmdir($dir)
    { 
        if (is_dir($dir))
        { 
            $objects = scandir($dir); 
            foreach ($objects as $object)
            { 
                if ($object != "." && $object != "..")
                     if (filetype($dir."/".$object) == "dir")
                         rrmdir($dir."/".$object);
                     else
                         unlink($dir."/".$object);  
            } 
        reset($objects); 
        rmdir($dir); 
       }
     }


    /*** Setup ***/
    // Since this is an example, we can hard-code some things...
    $loc = '/home/fader/Projects/Libboo/epub-test'; // Where epubs live
    $epub = 'test.epub'; // The epub we will modify
    $newepub = 'new.epub'; // The new epub we will generate
    $added_page = 'newpage.xhtml'; // The new page we're going to insert into it


    // Allocate a directory to work in
    $temp_path = sys_get_temp_dir() . "/" . uniqid("epub-");
    mkdir($temp_path) or die("Couldn't create temporary path.");


    /*** Let's do this thing! ***/
    // Open the epub archive
    $zip = new ZipArchive;
    $res = $zip->open($epub);
    if ($res !== TRUE)
        die("Couldn't open epub as a zip.");


    // Unzip the epub into a temporary location
    $zip->extractTo($temp_path);


    // *** Dig into the ebook container
    // The path is defined by the epub spec, so as long as this is a compliant
    // epub file, we should be able to find fit at this location
    $container_path = $temp_path . "/META-INF/container.xml";
    $container_xml = file_get_contents($container_path);
    if ($container_xml === FALSE)
        die("Couldn't open container XML file.");


    // Look in the container to find the spine
    $container = new SimpleXMLElement($container_xml);
    $spine_path = $temp_path . "/" . $container->rootfiles[0]->rootfile["full-path"];


    // Pull up the spine
    $spine_xml = file_get_contents($spine_path);
    if ($spine_xml === FALSE)
        die("Couldn't open the table of contents.");


    // Copy the new page into the correct location
    if (!copy($added_page, dirname($spine_path) . "/" . basename($added_page)))
        die("Unable to copy new page into temporary location.");


    // *** Decide where to insert a node
    // For this example, we'll just plug it in as the third element
    // Unfortunately, SimpleXML is too... simple to let us insert a node into
    // an arbitrary position, so we use the DOM object
    $dom = new DOMDocument;
    $dom->loadXML($spine_xml);
    // Fortunately the structure for an epub spine is pretty simple.  So we can
    // just get the list of pages ("item"s) and run down the tree a bit.
    $items = $dom->getElementsByTagName("item");
    $itemrefs = $dom->getElementsByTagName("itemref");


    // Let'ss grab the third element
    // (NB: Pretty much any epub should have at least 3 items.
    // (ncx, css, title, pages...)  But boundary checks are always a Good Thing.)
    if ($items->length < 3)
        die("Book is ridiculously short.");


    // *** Create and insert the new nodes
    // We'll need two nodes here -- one for the "item" and one for the "itemref".
    // Both need to be present for the new page to be found by the reader.
    $newitem = $dom->createElement("item");
    $newitem->setAttribute("id", "newpageid0");
    $newitem->setAttribute("href", basename($added_page));
    $newitem->setAttribute("media-type", "application/xhtml+xml");
    $insert_point_item = $items->item(3);
    $insert_point_item->parentNode->insertBefore($newitem, $insert_point_item);


    $newitemref = $dom->createElement("itemref");
    $newitemref->setAttribute("idref", "newpageid0");
    $newitemref->setAttribute("linear", "yes");
    $insert_point_itemref = $itemrefs->item(3);
    $insert_point_itemref->parentNode->insertBefore($newitemref, $insert_point_itemref);


    // *** Write it out
    $newxml = $dom->saveXML();
    $result = file_put_contents($spine_path, $newxml);
    if ($result === FALSE)
        die("Unable to write new XML file.");


    // *** Zip everything back up again
    // The mimetype needs to be stored, not compressed.  Unfortunately I have not
    // seen a way to do this with the PHP ZipArchive object.
    // This is the quick, dirty, nonportable, ugly way to do it:
    system("zip -q0Xj $temp_path/$newepub " . $temp_path . "/mimetype");
    // Since we're already calling the system zip binary, this is about 30 lines smaller
    // than using the PHP zip object to accomplish the same thing:
    system("cd $temp_path ; zip -q0Xj $newepub mimetype ; zip -qXr $newepub * -x mimetype");


    /*** Clean up after ourselves ***/
    // Move the new epub file to the working directory
    if (!rename($temp_path . "/" . $newepub, $loc . "/" . $newepub))
        die("Unable to move new epub file to $loc.");
    // Delete the temporary path
    rrmdir($temp_path);
?>

In short, here's what the above does:

  • Sets up a convenience function for cleaning up later
  • Extracts the contents of the epub file into a temporary location
  • Reads the container XML file (specified by the epub spec) to find the index of files (which could be in an arbitrary location inside the epub)
  • Copies in the new page to be added
  • Creates two XML nodes
    • One is the location of the file containing the new page
    • The other is a referent indicating where in the book that page should fall
  • Adds these nodes to the index
  • Zips everything back up
  • Moves the new epub to a specified location
  • Cleans up the temporary files created
It's pretty straightforward, all told.  The tricky bit is in zipping the files up -- epub requires that the mimetype file (specifying that it is an epub) must be the first file in the archive and stored rather than compressed.  This bit's tricky in PHP, so I copped out and just called the native system binary.

If anyone has any questions I'm happy to discuss this... it's a fun toy problem!