Steve Spigarelli

Webpage Monitoring Lua/Shell Script for Openwrt

I retired my Linode a while back and decided that I wanted to bring back a website monitor that I had written in ruby. I didn't feel like putting up a heroku instance so I used my openwrt flashed router. I installed Ruby since that is my language of choice for scripting and I love using things like Nokogiri. However, this time I decided to use Lua. I've been meaning to write something in it, and this was the perfect opportunity, although I would have used xmllint if that was available since that would have made my job much easier.

There are two parts to the setup. First a script that checks for changes and then another that parses the XML. The script that checks for changes is written in shell/bash/ash. The other script is where the Lua skills were put to use.

The shell script consists of getting a website using curl, and then taking an md5 hash of the output from the Lua script and storing it in a file. The files are compared and if they don't match, an email is sent notifying of changes. Some of the diff code was helped along by a post on Steve's Blog although it is changed a little from the original.

#!/bin/sh

URL=$1
MD5=`echo $URL |md5sum|cut -d\  -f1`

EMAIL="From: email@example.com
To: email@example.com
Subject: $URL has been updated

$URL"

mv "/user/tmp/${MD5}-new.md5" "/user/tmp/${MD5}-old.md5" 2> /dev/null
curl $URL 2>/dev/null | lua /user/parse-xml.lua $2 | md5sum > /user/tmp/${MD5}-new.md5

/usr/bin/diff /user/tmp/${MD5}-new.md5 /user/tmp/${MD5}-old.md5 >/dev/null 2&>/dev/null

DIFF_OUTPUT=$?

if [[ ! $DIFF_OUTPUT -eq 0 ]] ; then
  echo "$EMAIL" | ssmtp email@example.com
fi

The Lua script consists of making an Expat XML parser using the lxp library. One argument is passed in that signifies the name of the id field for the div that is being watched on the page. For instance, if the change you're watching for on the page is in a div called 'content', you'd pass 'content' as the first parameter to the script. The output of the script is simply the content of the text-nodes of the XML.

#!/usr/bin/lua
require"lxp"

local count = 0
local startcounting = false
callbacks = {
  StartElement = function (parser, name, attrs)
    for i=1,table.getn(attrs) do
      if name == 'div' and attrs[i] == 'id' and attrs[attrs[i]] == arg[1] then
        startcounting = true
      end
    end
    if startcounting then
      count = count + 1
    end
  end,
  EndElement = function (parser, name)
    if startcounting then
      count = count - 1
    end
    if count == 0 then startcounting = false end
  end,
  Default = function(parser, string)
    if startcounting then
      print(string)
    end
  end
}

p = lxp.new(callbacks)

for l in io.lines() do  -- iterate lines
    p:parse(l)          -- parses the line
    p:parse("\n")       -- parses the end of line
end
p:parse()               -- finishes the document
p:close()               -- closes the parser

Steve Spigarelli

Other articles

How to Use mongodump and mongorestore Commands on Heroku

Install Marklogic 7.x on CentOS

Launch nginx At Startup

Really removing the OS X sleepimage

Testing pull requests in Stash

First use of XCC

Other articles

social