Sunday 29 April 2012

ProBee Based Home ZigBee Network - Part 3 - Coordinator Software Display

Welcome to part 3 of the ProBee Based Home ZigBee Network series. In this entry I will be looking at how to display the temperature reported from our ProBee based sensor node onto a webpage using a custom Python script and Open Energy Monitor's emoncms software. "Emoncms is a powerful open-source web-app for processing, logging and visualising energy, temperature and other environmental data."
Will will need to have a working sensor node network comprising of a co-ordinator and at least one sensor node, so make sure you have followed the part 2 of this series!
Additional things you will need:
  • A Web Server: A Linux based PC/server with Apache, MySQL Server and Python installed. I'll be using my Beagleboard running Ubuntu Server, but any major Linux distribution should be suitable (including running one from a virtual machine assuming your virtual machine has access to the ZigBee coordinator's USB serial interface).
  • Emoncms  : The latest version of emoncms installed on the server. You can get it here (at time of writing version 3 is that latest). Installing it is quite trivial once you have your Linux server setup, just follow the installation guide here.
This tutorial should work just as well with a Windows based server, but I will leave it up to the reader to investigate.

Flow of temperature data


So we need to do the following:
  1. Verify sensor node and network - Check what serial device our coordinator is connected to and verify we are receiving temperature data from the sensor node through the co-ordinator's serial interface.
  2. Verify  emoncms  Installation - Check that emoncms is working correctly and receiving sample commands from the browser.
  3. Create the serial data parsing script - Write a Python script to parse the serial data  outputted by the ProBee coordinator and send this data to emoncms for storage and display. The script is very basic, so if you don't know Python you could easily port it to your preferred language.
  4. Integration - Integrate it all so our temperature data is displayed on our web server.
  5. Run at start-up (optional) - I will show you how to get the serial data parsing script running automatically when your Ubuntu server is powered on.
Verify Sensor Node and Network
Plug your pre-configured ProBee coordinator (ProBee USB dongle) into your server and power on your ProBee sensor node (you should have followed part 2 to get these ready). 
We now need to connect to the ProBee USB dongle's USB serial port using a terminal application.
To find the device name for your USB dongle, use the dmesg command:
donal@server:~$ dmesg | grep FTDI
[ 62.209960] USB Serial support registered for FTDI USB Serial Device
[ 62.219909] ftdi_sio 1-2.2:1.0: FTDI USB Serial Device converter detected
[ 62.246856] usb 1-2.2: FTDI USB Serial Device converter now attached to ttyUSB0
[ 62.252075] ftdi_sio: v1.6.0:USB FTDI Serial Converters Driver
The Probee dongle uses an FTDI USB to serial interface chip, which in my case is attached to ttyUSB0.
Now use minicom (Linux equivalent of hyperterminal) to view the serial data (containing our sensor node temperature) coming from the co-ordinator. If you haven't use minicom, there is a good tutorial here.
You should get output similar to this:




Verify Emoncms Installation
Html post requests are used to send input data to the emoncms server. Our script will be creating  requests to send the temperature data to the server each time we receive new data from our temperature sensor.
Now to verify the emoncms installation: Browse to your emoncms installation (in my case http://mylocalserver/emoncms3/user/view), and click on the Account tab. Then click on the "try me" link which will send a sample html post request to the server. Then click on the Inputs tab, and you should see two new feeds in there that were updated several seconds ago.
Emoncms validation

Create Serial Data Parsing Script
The script will have to do the following:
  1. Open the serial port to the ProBee dongle.
  2. Wait for and read a line of data from the serial port.
  3. Parse that line and generate the html post request to the emoncms server.
  4. go back to step 

#! /usr/bin/python
"""
Simple script for parsing probee co-ordinator data and passing it on to Emoncms.
Donal Morrissey
http://donalmorrissey.blogspot.com/
"""
import re       # Used for parsing the data string.
import sys      # For exit
import serial   # Serial port interfacing
import requests # To send html post requests to Emoncms

PORT = '/dev/ttyUSB1'
BAUD_RATE = 115200

MY_EMONCMS_API_KEY = 'your_emoncms3_api_key'
URL_TO_MY_EMONCMS = 'http://your_server_address/emoncms3'

# Open serial port
try:
        ser = serial.Serial(PORT, BAUD_RATE)
        ser.open()
except:
        print "Could not open serial port: ", sys.exc_info()[0]
        sys.exit(2)


# Continuously read and print packets
while True:
    try:
        line = ser.readline();

        #Check the line is valid
        if not "++" in line: continue
        line = line.rstrip()

        # Split the line up, we know our raw temperature data will be located in location 2.
        r = re.compile('[|,]+')
        split_line =  r.split(line)

        # Extract the raw value as an int (our data is in location 2).
        raw_value =  int(split_line[2], 16)

        #Get the measured voltage from the raw ADC value.
        vout_mV = raw_value / 10

        temperature_degC = (vout_mV - 500) / 10

        #print temperature_degC

        # Now there is where the magic happens...
        # We send the new value to emoncms as a post request.
        requests.post(URL_TO_MY_EMONCMS+"/api/post?apikey=" + MY_EMONCMS_API_KEY +"&json={temperature_probee:"+str(temperature_degC)+"}")

    except KeyboardInterrupt:
        break

ser.close()


Integration
We are now ready to test the system. Open a browser and browse to your emoncms installation and click on the Inputs tab. Plug in your ZigBee co-ordinator (ProBee dongle), run your serial data parsing script and power on your sensor node.
You should now see the temperature input updating every 5 seconds on the Inputs tab. You can now add widgets to your dashboard and display various graphs, etc of this data.


Run at Start-Up
As the root user, save the following script to /etc/init.d/home_monitoring.sh, not forgetting to add in the full path to your script.
#! /bin/sh
# /etc/init.d/home_monitoring.sh

# Carry out specific functions when asked to by the system
case "$1" in
  start)
    echo "Starting receive_and_post_emon3_d.py "
    /<full path to this script>/receive_and_post_emon3_d.py &
    ;;
  stop)
    echo "Stopping receive_and_post_emon3_d.py"
    killall receive_and_post_emon3_d.py
    ;;
  *)
    echo "Usage: /etc/init.d/home_monitoring.sh {start|stop}"
    exit 1
    ;;
esac

exit 0


Make the script executable: by running the command: "chmod 755 /etc/init.d/home_monitoring.sh".
Now register the script so it will be called at startup/shutdown: "update-rc.d  home_monitoring.sh defaults".
You can test your script runs at startup by restarting your server, your emoncms dashboard should be continuously updated by your sensor node.


Conclusion
You should now have a basic home temperature monitoring system using ProBee based Sensor Node and Co-ordinator and emoncms.


This series:

Sunday 15 April 2012

ProBee Based Home ZigBee Network - Part 2 - Temperature Reporting End Node

Overview
Welcome to part 2 of the ProbBee based home Zigbee network. In this part of the series I will be detailing how to setup, configure and test a temperature reporting end-node. We will follow these steps:
  1. Design and build the circuit.
  2. Configure the ProBee module to periodically report the temperature and enter a low-power sleep mode.
  3. Configure the network co-ordinator.
  4. Test the system using a terminal app, such as putty or hyperterminal.
and require this hardware:
  1. Breadboard with jump-wires.
  2. ProBee-ZE20S module for the end-node.
  3. A ProBee-ZU10 usb dongle as the co-ordinator.
  4. The ProBee manager software, to configure the end-node and co-ordinator.
  5. A ProBee expansion board, which will allow us to mount the module onto a breadboard. Please contact me if you need an expansion board as I have some spare ones.
  6. A TMP35 temperature sensor in DIP package, used for sensing the room temperature. These are extremely easy to use, for more info please look at ladyada's great overview here.
  7. A DIP LED used for indicating the status of the ProBee module.
  8. A power circuit for regulating our battery power supply to the 3.3 volts required by the ProBee module and TMP36. Alternatively, the easiest thing to do would be to get a Breadboard Power Supply like this from Sparkfun: Breadboard Power Supply 5V/3.3V, it won't be the most efficient approach, but will be fine for what we need.
Circuit Design
The following is my 'back-of-cigarette-packet' style circuit diagram:
Temperature Sensor Circuit Diagram
You can see the main components: the ProBee module, TMP36 temperature sensor and status LED. For an excellent tutorial on the TMP36, please see LadyAda's page here. Note that I have connected the output of the temperature sensor to PB_5/ADC0 pin of the ProBee module.

My ZigBee Temperature Monitoring Node
We can see from the above picture I have chosen to use a regulator circuit on the board. If you are very new to electronics I'd suggest to make life easier and purchase the previously mentioned Breadboard Power Supply 5V/3.3V.
Also from the picture above you can see that I have the ProBee module pugged into an expansion board so the board can be plugged into a breadboard. I have several of these boards spare and will happily post them to any reader for only the cost of postage.

Configuring The ProBee Modules
First of all make sure your ProBees are using firmware v1.5 or higher.
Secondly, I'm not going to delve too deeply into the config of the sensor node and co-ordinator. Programming the ProBee modules is very straight forward, Sena have some good documentation on their website here. I would highly recommend reading the ZigBee Network Configuration section of the ProBee-ZE20S User Guide!

I have created two configuration files that can be programmed into the ProBee modules using Sena's ProBeeManager application:

  • Our Zigbee temperature sensor node. This configures the ProBee module to sleep for 5 seconds, wake up, take samples of it's digital/analog I/O ports, transmit this to the co-ordinator and go back to sleep (see section
    3.4 Setting up ZE20S as a Sleepy End-Device of the user manual)Probee_SED_Blog_150412_115200baud
  • The ZigBee co-ordinator (program your ProBee usb dongle with this): Probee_ZC_Blog_150412_115200baud

Note the serial interface baud rate for both configurations is set to 115200 baud.

Testing
Connect the Co-ordinator (ProBee usb dongle) into a usb port on your computer and open a serial connection to it using your favourite terminal application. If you are using my config files above, ensure you have the baud rate set to 115200.

Now power on your sensor node. You will notice that the status led flashes on briefly every 5 seconds. This is the ProBee waking from sleep node and transmitting the sample of the analog and digital IO.

Looking at the Co-ordinator's terminal output, you can see the messages coming from the sensor node:
Messages received by the co-ordinator from the sensor node

Each line represents a single message received by the co-ordinator from the sensor node. We are interested in the field reading 1C3F, this is the hexadecimal value for the Analog sample of ADC0 port (which the TMP36 is connected to) on our sensor node.

Decoding the Temperature
There are two steps to the decoding of the actual temperature. First we need to get the measured voltage (TMP36 output voltage):
Vadc = AdcValue * 0.1

Then to get the temperature:
Temperature (Celsius): (Vadc - 500) * 0.1

Therefore:
Vadc = 0x1c3f * 0.1 = 723 mV
Temperature = (723 - 500) * 0.1 = 22 Deg Celsius

Note that the ProBee devices can read in a voltage range of 0 to 800mV!

Next
So, we now have a working ZigBee temperature sensor node! Next up is to do something with the raw text that is outputted by the co-ordinator over the serial interface. In Part 3 we will be using emoncms to display the temperature data on a web page.

An example Emoncms widget
This series: