Epicserve

Using Incron to Monitor Newly Uploaded Files

December 13, 2011 | 3:34 p.m. CST

At the Wenatchee World where I work on all of the in-house Django sites and utility scripts, we have decided to migrate away from the OSX Server platform to using Ubuntu. Because of this decision, I needed to find a replacement for OSX's native launchd, which we use to monitor directories for newly uploaded files in order to launch scripts to process those files in various ways. As an example, we currently have AP news stories that get uploaded as XML files to a specified directory on a server. Launchd then triggers a script that imports the news stories in the XML file into our website.

Needing to find a replace for launchd on Ubuntu, I did some Google searches for things like "Upstart WatchPath, Ubuntu Launchd watchpath, etc.", all which didn't turn up much. That's when I decided to ask some friends on an IRC channel what they would recommend and that's where I received the recommendation from Jacob Kaplan-Moss to use Incron. Also, after showing him the documentation I wrote for myself, he suggested I share it with everyone - hence the reason for the blog post.

So without any further introduction, here are the steps I used to install and test a workable Incron setup.

Step 1: Install and Setup Incron


  $ sudo aptitude install incron

Add root to the config file /etc/incron.allow to allow incron to run as root.


  $ sudo echo "root" >> /etc/incron.allow

Step 2: Setup an Example Script

Since I'm a Python developer, I created a quick little script to test Incron. Open up your favorite editor and the following script somewhere on the computer computer you installed Incron. I saved my script in /usr/local/scripts/event_log.py.


  #!/usr/bin/env python

  from datetime import datetime
  import sys

  LOG_FILE = '/tmp/log_file.log'


  def main():
      f = open(LOG_FILE, 'ab')
      f.write('%s - Event Triggered - %s\n' % (datetime.now().strftime("%Y-%m-%d %H:%M:%S"), ", ".join(sys.argv)))
      f.close

  if __name__ == '__main__':
      main()

Make the script executable:


  $ chmod 744 /usr/local/scripts/event_log.py

Step 3: Setup Incron to Watch a Directory and Execute the Test Script

Run the following to edit the list of directories or files Incron is watching:


  $ sudo incrontab -e

Add something similar to following and save the changes:


  /home/upload_test IN_CLOSE_WRITE /usr/local/scripts/event_log.py $@/$#

Step 4: Test

Upload some test files to /home/upload_test. I uploaded some spare XML files I had laying around. After the files have uploaded you should be to run the following and get a similar output. If you get a similar output, congratulations, it worked!


  $ cat /tmp/log_file.log

  2011-11-11 10:54:50 - Event Triggered - /usr/local/scripts/event_log.py, /home/upload_test/feed_30119_2011-07-06T15-50-23.708Z.xml
  2011-11-11 10:54:50 - Event Triggered - /usr/local/scripts/event_log.py, /home/upload_test/feed_30119_2011-07-06T15-55-47.541Z.xml
  2011-11-11 10:54:50 - Event Triggered - /usr/local/scripts/event_log.py, /home/upload_test/feed_30119_2011-07-06T16-00-49.412Z.xml
  ...

Related tags: ubuntu

Post Your Comment

(Optional & Not Displayed)

(Optional)


(allowed tags - strong, em, cite, & code)

Post Guidelines

Please be considerate of others. Keep comments relevant. Content deemed inappropriate or offensive may be edited and/or deleted. Email addresses are never displayed.

Line breaks and paragraphs are automatically converted — no need to use p or br/. Quotes, apostrophes, and double-dashes are automatically converted to smart punctuation. Be careful when copying and pasting portions of entries or other comments.

Links can be created using the standard <a href="http://url">urlName</a>. The following inline HTML elements may also be used: strong, em, cite, & code. The title attribute is allowed within any element. All other code will get removed before posting.