Walker Blogs

New Media Initiatives

Continuous Deployment with Fabric

Posted February 22, 2012 at 9:25 am — Filed under:

textured brown fabric
We have been using Fabric to deploy changes to walkerart.org. Fabric is a library that enables a string of commands to be run on multiple servers. Though similar things could be done with shell scripts, we enjoy staying in one language as much as possible. In Fabric, strings are composed and sent to remote servers as commands over an SSH connection. Our Fabric scripts have been evolving over time with the project using the mentality: “If you know you are going to be doing something more than twice, script it!”

With Fabric we can tailor our deployments precisely. We deploy often with one of two commands:
fab production simple_deploy or fab production deploy.
simple_deploy simply pulls new code from the repo and restarts the web server.
deploy does many things, each of which can be executed independently, and is explained below.

The scripts we run go both ways, code goes up to the server and data comes back to the workstation. We have fab sync_with_production, which pulls the database and images. The images arrive locally in a directory specified by an environment variable or the default directory. Conventional naming schemes simplify variables across systems such as the database name. Except for some development settings, our workstation environments are identical to the production environment, which means we can replicate a bug or feature locally and immediately.

We have been collecting all of the commands we normally run on the servers into our fabfile. And then we can group them by calling tasks from other tasks. Our deployment consists of 12 tasks. With this Fabric task, one can deploy to the production or staging server with this one command:
fab production deploy.

This makes it incredibly simple to put code that is written on developer workstations into production in as safe and secure way. Here is our deployment in Fabric:

def deploy():
with cd(env.project):
run('git pull')
get_settings()
install_requirements()
production_templates()
celerybeat('stop')
celeryd('stop')
synccompress()
migrate()
gunicorn('restart')
celeryd('start')
celerybeat('start')

First the “with” blocks put us onto the remote server, into the right directory and within Python’s virtual environment. From there “git_pull” gets the new code which contains the settings files, and “get_settings” copies any new settings into place. The task called “install_requirements” calls on pip to validate our virtual environment’s packages against the setting file called requirements. All third party packages are locked to a version so we aren’t surprised by new “features” that have adverse effects. We use celery to harvest data from other sites so we make sure they are running with fresh config files. The task “syncompress” does our compressing of css and js, “migrate” alters the database per our migration files and gunicorn is the program that is running django.

It takes about 60 seconds for a new version of the website to get into production. From there it takes 0-10 minutes for the memcached values to expire before the public changes are visible. We are deploying continuously so watch closely for updates!

Creating a community calendar using Google Apps and WordPress

Posted July 9, 2010 at 12:05 pm — Filed under:

For Walker Open Field, we wanted a way to collect community submitted events and display them on our site. We have our own calendar and we discussed whether adding the events to our internal Calendar CMS was the best way, or if using an outside calendar solution was the direction to go. In the end, we decided to do both, using Google Calendar for community events and our own calendar CMS for Walker-programmed events.

The Open Field website is based on the lovely design work of Andrea Hyde, and the site is built using WordPress, which we use for this blog and a few other portions of our website. WordPress is relatively easy to template once, so it makes for quick development. WordPress also has a load of useful plug-ins and built-in features that saved us a lot of time. Here’s how we used it an put it together:

Collecting Events
To accept event ideas from community members, we used the WordPress Cforms II plugin, which makes it very easy to build otherwise complex forms and process them. You can create workflows with the submissions, but we simply have Cforms submit the form to us over email. A real person (Shaylie) reviews each of the event submissions and adds the details to…

Google Calendar
We use Google’s Calendar app to contain the calendar for the community events. When Shaylie gets the email about a new event, she reviews it, follows up on any conflicts or issues, and then manually adds it to google calendar. We toyed with the idea of using the Calendar API to create a form that would allow users to create events directly in the calendar, but decided against it for two reasons. First, it seemed overly complicated for what we thought would amount to less than 100 events. Secondly, we would still have to review every submission and it would be just as cumbersome to do it after the fact rather than beforehand.

We also use Google Calendar to process our own calendar internal calendar feed. The Walker Calendar can spit out data as XML and ICAL. We have our own proprietary XML format that can be rather complex, but the ICAL format is widely understood and Google Calendar can import it as a subscription.

Getting data out of Google Calendar
We now have two calendars in google calendar: Walker Events and Community Events. Google provides several ways to get data out of google calendar, and the one we use is the ATOM format with a collection of google name-spaced elements. The calendar API is quite robust, but there are a few things worth noting:

  • You must ask for the full feed to get all the date info
  • Make sure you set the time zone, both on the feed request but add it to the events when you link tot hem (using the
    ctz

    paramater)

  • Asking for only
    futureevents

    and

    singleevents

    (as paramaters) makes life easier, since you don’t have to worry about complexities of figuruing out the repeating logic, which is complicated

This is our feed for Open Field Community Events.

Calendar data into WordPress
Since version 2.8, WordPress has included the most excellent SimplePie RSS/ATOM parsing library. As the name would have you believe, it is pretty simple to use. To pull the data out of the Google Calendar items with simplePie, you extend the SimplePie_Item class with some extra methods to get that gd:when data.

Combining two feeds in SimplePie is not hard. By default, SimplePie will sort them by the modified or published date, which in the Google Calendar API is the date the event was edited, not when it happens. Instead, we want to sort them by the

gd:date

field. There are probably a few ways to do this, but the way that I set up was to simply loop through all the data, put it into an array with the timestamp as the key, and then sort that array by key. Here’s the code:

[php]
<?php

//include the rss/atom feed classes
include_once(ABSPATH . WPINC . ‘/feed.php’);
require_once (ABSPATH . WPINC . ‘/class-feed.php’);

// Get a SimplePie feed object from the specified feed source.
$calFeedsRss = array(
#walker events
‘http://www.google.com/calendar/feeds/95g83qt1nat5k31oqd898bj2ako1phq1%40import.calendar.google.com/public/full?orderby=starttime&ctz=America/Chicago&sortorder=a&max-results=1000&futureevents=true’,

#public events
‘http://www.google.com/calendar/feeds/walkerart.org_cptqutet6ou4odcg6n2mvk4f44%40group.calendar.google.com/public/full?orderby=starttime&ctz=America/Chicago&sortorder=a&max-results=1000&futureevents=true&singleevents=true’
);

$feed = new SimplePie();
$feed->set_item_class(‘SimplePie_Item_Extras’);
$feed->set_feed_url($calFeedsRss);
$feed->set_cache_class(‘WP_Feed_Cache’);
$feed->set_file_class(‘WP_SimplePie_File’);
$feed->set_cache_duration(apply_filters(‘wp_feed_cache_transient_lifetime’, 3600)); //cache things for an hour
$feed->init();
$feed->handle_content_type();

if ( $feed->error() )
printf (‘There was an error while connecting to Feed server, please, try again!’);

$count = 0; // hack, but we’re going to count each loop and use it as a little offset on the sort val
foreach ($feed->get_items() as $item){

if (strtolower($item->get_title()) != ‘walker open field’){
$eventType = ‘walker’;
$related = $item->get_links ( ‘related’ );
$related = $related[0];
if ( strpos($related,’walkerart.org’) === False ){
$related = $item->get_link();
// if it’s a google calendar event, make sure we set the tiem zone
$related .= "&ctz=America%2FChicago";
$eventType = ‘community’;
}

#we offset the actual starttime a little bit in case two events have the same start time, they would overwrite in t he array
$sortVal = $item->get_gcal_starttime(‘U’) + $count;
$myData = array(
‘title’ => $item->get_title(),
‘starttime’ => $item->get_gcal_starttime(‘U’),
‘endtime’ => $item->get_gcal_endtime(‘U’),

‘link’ => $related,
‘eventType’ => $eventType,
‘text’ => $item->get_content(),
‘date’ => $item->get_gcal_starttime(‘U’)
);
$cals[ $sortVal ] = $myData;
}
$count++;
}
//sort the array by keys
ksort($cals);
// $cals now contains all the event info we’ll need

?>
[/php]

Once this is done, you can simply take the

$cals

array and loop through it in your theme as needed. Parsing the google calendar feeds is not an inexpensive operation, so you may wish to use the Transients API in WordPress to cache this information.

Caveats and Issues
Overall, this approach has worked well for us. We have run into some issues where the Google Calendar ATOM feed would show events that had been deleted. Making sure to set the

futureevents

and

singleevents

paramaters fixed this. We also ran into some issues using the

signleevents

, so we ended up manually creating occurrences for events that would have otherwise had a repeating structure.

Simple iTunes U stats aggregation with python and xlrd

Posted May 19, 2010 at 11:53 am — Filed under:

Like many institutions, we put numbers for our various online presences in our annual report and other presentations: YouTube views, Twitter followers, Facebook fans, etc. For most services, this is very easy to do: log in, go to the stats page, write the big number down. We also want to include the iTunes U numbers, but for iTunes U, there is no centralized stats reporting outside of the Microsoft Excel file Apple sends iTunes U administrators every week. Tabulating the stats by hand would be time consuming and error-prone, so I wrote a short python script to automate it. Here is is:

[python]
#!/usr/bin/env python
# encoding: utf-8
"""
itunesUstats.py
Created by Justin Heideman on 2010-05-19.
"""

import sys, os, glob, xlrd

def main():
#change this to the path where your stats are
path = ‘itunes_U_stats/all/’
totalDownloads = 0

for infile in glob.glob( os.path.join(path, ‘*.xls’) ):
# open the file
wb = xlrd.open_workbook(infile)

# get the most recent days’s tracks
sh = wb.sheet_by_index(1)

# get the downloads from that day
downloads = sh.col_values(1)

#first entry is u’Count’, whcih we don’t want
downloads.pop(0)

#sum it up
totalDownloads += sum(downloads)

# show a little progress
print sum(downloads)

#done, output results
print "—————————————————-"
print "Total downloads: %d" % totalDownloads

if __name__ == ‘__main__’:
main()
[/python]

This script uses the excellent xlrd python module to read the Excel files (simple xlrd tutorial here), which is roughly 27.314 times easier than trying to use an Excel macro to do the same thing. To use this, simply change the path on line 13 to a directory containing all your iTunes stats files, and run the script from the command line. You’ll get output like this:

929.0
732.0
779.0
854.0
1000.0
987.0
765.0
812.0
1275.0
1333.0
1114.0
1581.0
1278.0
1568.0
1854.0
2102.0
1108.0
1078.0
----------------------------------------------------
Total downloads: 21149

Do note that the script is tied to the excel format Apple has been using, so if they change it, this will break. Apple explains the iTunes report fields here. This script tells you ” all the iTunes U tracks users downloaded that week through the DownloadTrack, DownloadTracks, and SubscriptionEnclosure actions”.

Build a bridging firewall (cheap!)

Posted June 22, 2009 at 1:07 pm — Filed under:

New Media has a number of development servers located in-house where we get stuff done before releasing it out into the wild.  Until last week these were protected by an aging OpenBSD firewall running packet filter and all was well until midweek when the motherboard failed.  Not having a spare on hand, I was scrambling for a solution.

Linksys wireless router

Linksys wireless router

Being familiar with the dd-wrt project, I was pretty sure I could build a firewall out of a Linksys router.  We went with the WRT54GL, currently as cheap as $50 on Amazon.  (We bought local so we’d have it sooner, and it was a bit more).

The first step after flashing the firmware with the latest dd-wrt build (v24-sp2) was to take off the antennas and turn off the radio.  The last thing I want for the firewall is to be broadcasting an SSID and allow wireless associations.  This actually requires a startup script on the router, with a line to remove the wireless module so it won’t try to reenable itself:

wl radio off
wl down
rmmod wl

Good start.  Next I needed to bridge the WAN port with the LAN ports, which ended up being a struggle until I found the easy options in the dd-wrt GUI.  First, set the LAN to use a static IP and make sure you can connect via another machine to configure it.  You’ll also need to enable SSH access and remote configuration – but be sure to lock this down once the firewall is running!

Once you have the LAN configured, you need to set the WAN connection type to “disabled”.  This will give you a checkbox to bridge the LAN and WAN:  “Assign WAN port to switch”.  Lastly, under Advanced Routing set the Operating Mode to “Router” so it stops trying to do NAT.  Apply these settings, and you’ll basically have an expensive dumb switch – all traffic shows up on every port, and there’s no logic at all.  We’re halfway there.

Being unfamiliar with iptables (we use OpenBSD and pf for firewalls around here), I was under the impression that iptables rules would work in a bridging environment.  This is not the case: bridged packets don’t reach iptables at all!  The best I could do was block everything (manual restart needed), or otherwise blow up the configuration (manual restart needed) as I tried to mess with the bridge.  This was an incredibly frustrating learning curve as everything I could find made it sound like this was the way to configure a firewall in Linux, but it just wasn’t working.

Note to keep you sane: don’t do any of this testing in the startup scripts or you’ll brick your router, guaranteed.  Do it all from the command line with a known-good startup.  That way it’s a simple (but annoying) power cycle to get things back up.

The trick, it turns out, is a kernel module called ebtables.  Luckily, this is included in the dd-wrt build, but it’s not turned on by default!  Add this to your startup script:

insmod ebtables
insmod ebtable_filter
insmod ebt_ip.o

And, ta-da, all your iptables rules will start impacting packets!  Now it’s just a matter of configuring the firewall rules.  We’re using something like this:  (vlan0 represents the LAN ports, and vlan1 is the WAN port)

# drop everything by default:
iptables -P FORWARD DROP
# clear the old rules:
iptables -F FORWARD
# forward stuff that's established already
iptables -A FORWARD -m state --state ESTABLISHED,RELATED -j ACCEPT
# let connections out:
iptables -A FORWARD -i vlan0 -m state --state NEW -j ACCEPT

# firewall access rules
iptables -F INPUT
# WAC ips can get to fw:
iptables -A INPUT -p tcp -d 1.2.3.4 -s 4.3.2.1/24 -j ACCEPT
# drop everything else!
iptables -A INPUT -p tcp -d 1.2.3.4 -j DROP

# ... snipped all the actual access rules and packet flood protection ...

The only trick here is the last few lines which limit access to the firewall machine itself.  We can’t use the FORWARD rules since these packets are destined for the internal hardware and not forwarded, but we do need to limit access via the INPUT chain.  In this example the firewall has IP 1.2.3.4 and the network I want to access it from has 4.3.2.x.  That way I can leave the firewall’s remote access turned on and limit it to our network.  (because there’s no terminal access you can’t make it a truly transparent bridge or you’d never be able to change the config!)

I admit I’m a bit nervous posting some of this in case there’s a glaring security hole, but it seems good to me.  Anyone see anything they’d like to warn me about before we get hacked?

And there you have it!  For the cost of a cheap router and some time (not much, since you can just follow these notes!) you have a full-featured bridging firewall running on dedicated hardware.  With a little extra work it would be easy to get VPN running and much more…  I’m hoping for years of service from this little guy!

( Hat tip another DIY firewall solution that I’d really like to try someday. )

Facebook app for My Yard Our Message

Posted May 29, 2008 at 1:54 pm — Filed under:

My Yard Our Message Facebook AppTuesday I posted some of the technical details for My Yard Our Message. Since then, I’ve been working on putting together a Facebook App to let people show the signs on their profiles. It is done, or done enough to be used.

I’ve played around with building a Facebook app before, but never had a clear strategic need to build one. I found this tutorial very helpful on getting my feet wet. For the moment, the app is written and PHP and talks to the actual My Yard Our Message site via json, rather than rss.

One of the things I am evaluating is whether or not I can set up a system to let people vote on signs directly from Facebook. Obviously, this would tremendously expand our pool of eligible voters, and would eliminate the need to force people to register for an account just to vote. During the building of the app, I went back and forth on the necessity of registration to vote, but ultimately decided it was necessary and laid the structural groundwork. However, I think I can get enough information about Facebook users to know they’re unique, track them, and prevent them from voting more than once.

This would require re-writing the Facebook app at that point, most likely in python and django for closer integration with the authentication and verification processing. There is some initial work done on a Facebook client API for python and django that looks promising as well.

Aside about Facebook pages

I’ve always been frustrated with Facebook apps that don’t work on Pages (as opposed to Profiles). After building an app, I have a new found appreciation for the applications that don’t work. Getting an app to work with pages isn’t really that hard, but it sure is confusing, mostly because of the lack of documentation. The only real documentation is a chat log in the Facebook developer wiki (no, I am not making this up):

 

(03:02:02 AM) swombat: ok, so basically, "Facebook pages is all transparent uses fbml blah blah blah"

                       "Oh but btw you need to build a completely different piece of code to handle this new type of user"

(03:02:18 AM) swombat: (not angry at you btw :-P)

(03:02:43 AM) fiveofoh: Yeah pretty much

(03:02:46 AM) swombat: this sounds really quite tedious and error-prone though

(03:02:50 AM) fiveofoh: Yeah it does

(03:02:56 AM) fiveofoh: Which is why I haven't done it on my app yet :P

So instead of just this:

 

$facebook->api_client->profile_setFBML('', $user);

$facebook->api_client->fbml_refreshRefUrl($process_url);

you use this:

 

if (isset($_GET['fb_page_id'])){

	$facebook->api_client->profile_setFBML('', $_GET['fb_page_id']);

} else {

	$facebook->api_client->profile_setFBML('', $user);

}

$facebook->api_client->fbml_refreshRefUrl($process_url);

Where $process_url is the page that spits out the markup to be shown on Facebook.

Party People Tech Details

Posted February 23, 2007 at 12:19 pm — Filed under:

Buoyant Sultry Party People

Party People Photos was back in action at After Hours last week, and I promised more technical details of the new features. The big change this time around was automatic uploading to flickr. After each photo was taken, it was transfered, processed, uploaded and finally displayed on-screen in the lounges. I ended up re-writing all the transfer and processing scripts to work better, and they all functioned without problem.

After a photo was taken, it was saved to a folder on the capture iMac. A script started by launchd watched the folder and transfered the file via rsync to my workstation. The files were transfered to my workstation rather than directly to the display computers for two reasons.

Photoshop

Photoshop Events Manager Once the files got to my workstation, another script watched the incoming folder and sent all incoming jpg files to Photoshop. We wanted the process all the images to make sure they looked their best. Cameron, one of our photographers, developed a handy action for the photos that would give them more contrast and punch. The easiest way to automate this in photoshop is to use the script events manager to run an action on every file that is opened ( File > Scripts > Scripts Event Manager… ). The easiest way on OS X to get a particular application to open a file is to use the open command with the -a argument. The -a lets you specify the binary that you want to use to open the file. Otherwise, you’re at the mercy of whatever program has associated itself with jpeg files. Here’s the script:

<pre>

#! /bin/sh



# this script takes files from the 1_incoming directory and tells photoshop to open them

# should be called by launchd which will be watching the 1_incoming folder

# photoshop should be set to perform the action on open new document



while true; do



myls=`ls /Volumes/Patience/_after_hours/1_incoming/`



    if [ "$myls" != '' ] ; then

        for myFile in /Volumes/Patience/_after_hours/1_incoming/*.JPG

        do

            /usr/bin/open -a /Applications/Adobe\ Photoshop\ CS2/Adobe\ Photoshop\

                CS2.app/Contents/MacOS/Adobe\ Photoshop\ CS2 $myFile

            sleep 2

            mv $myFile /Volumes/Patience/_after_hours/2_incame/  2&gt;&amp;1 &gt; /dev/null

        done

    fi

    sleep 10

    #take care of pesky .DS_Store files, which can pop up from the Finder

    find /Volumes/Patience/_after_hours/ -name .DS_Store -exec rm -f {} \;

done



# added a return on the photoshop open line for clarity in the browser

</pre>

Every 10 seconds, if there is a new file, and it is a jpeg, it will get sent to Photoshop. Photoshop runs the action, and saves the file to a new folder, again as a jpeg.

I used my workstation for this because it already had Photoshop installed, and I didn’t want to deal with licensing issues on another machine. I created another account that had photoshop set up just right and the launchd jobs start onload. When we set up PPP again for Picasso show, I’d love to be able to do post-processing via imagemagick, since it is command-line, easy to install and unencumbered by licensing issues.

Flickr Uploading

Photoshop has now saved the jpeg to a different folder. This folder is being watched by yet another script, similar to the last. This script does two things: copies the jpegs to our projection machines and uploads them to flickr. The copy process is mostly unchanged, though I used scp instead of rsync, since the I want to copy files one at a time. As with all the rsync or scp transfers, I just made sure I had my keys set up and authorized, and it worked fine.

Uploading to flickr was the trickiest part of the operation, but thankfully flickr provides a great API and there are a lot of libraries that simplify the process. I ended up using phpflickr. I am pretty familiar with php, and phpflickr only requires php4, which is the version of php-cli in OS X. In order to get it working, you have to apply for an API key, giving it write permission. Using your API key and the secret, you generate a token that you can use to upload with (you need all three to upload). Phpflickr provides the scripts necessary to provide the callback URL that flickr needs, even if you’re not uploading things directly from the web. It is somewhat confusing, and I’m not entirely sure I need the callback url, but it works all the same.

With the tokens all set, uploading the photo is as simple as upload, add to group, add to set. To get the ID of the group and the set, I just used the API explorer, which lists the groups and sets I visited. I had to create the set before I started uploading, though it is possible to create a set through the APIs. I also created a function in the upload script that used a word list to generate a title for each photo as it was uploaded to flickr. I am not a fan of seeing photos with names like IMG_4097.JPG, and this solved that and created some fun and funny juxtapositions.

Other notes, future

I enabled printing of all our photos on flickr, so you can now get prints if you want. In the US, flickr and yahoo do the printing through Target, which also happens to be a major sponsor of After Hours. Perhaps that is an opportunity for the future.

I still had some trouble with the camera not acting the way I wanted. Sometimes it would get into a state where it was totally locked, and the only way to reset it was to pull the power by removing the battery adapter; turning it off and on again wouldn’t do it. Due to the way that gphoto2 talks to the 10D, the capture command doesn’t fully finish, so I kill it after a few seconds. I think what was happening was that in some focus situations, the autofocus was taking too long and the camera hadn’t finished capturing when I kill the command. In the future, I am going to experiment with using a manual focus, which will eliminate that problem.

Gphoto2 was also recently updated to 2.3.0, and it now compiles without too much trickery on OS X. However, it hasn’t fixed my problems with capture on the Canon 10D. I might experiment with modifying the Canon class for the 10D a bit to see if I can get it to work.

We also had some problems with the flash not always firing when the camera went. Unfortunately, the flash we’re using doesn’t have an input for a power adapter, so we were running on batteries. It also doesn’t indicate low batteries, leaving us, literally, in the dark. For the Picasso After Hours, we’re planning on jerryrigging a wired power adapter to provide the 6V it requires.

I’ll also be setting up Party People Photos for the Free First Saturday on March 3rd. I think kids will get a kick out of it. We won’t add the photos to the After Hours Group Pool, but we will put them on our Flickr. Watch for it (or attend, it is free).

More on using Quartz Composer for digital signage

Posted January 25, 2007 at 4:28 pm — Filed under:

As has been mentioned here in the past, I have been tinkering with quartz composer for use as dynamic, digital signage. It is a good fit: extremely fast, can talk to the internet, usable on a lot of different systems. There are a growing number of locations within the museum where we’d like to apply dynamic signage, but off the shelf systems to do it are often convoluted and proprietary, not to mention expensive. Currently in the Walker Cinema, we use a DVD that I render in After Effects and update periodically. This affords a lot of control, but also takes a fair amount of labor to update.

It is this kind of an application where Quartz Composer can work well. Any quartz composer movie can be saved as a quicktime movie, but there are some limitations:

  • no mouse and keyboard events
  • no contents download from Internet (RSS feeds, images…)
  • edition of the input parameters of the compositions

Notice that second one? That’s the doozy if you want your quartz comp quicktime movie to use an RSS feed to get the text.

There is a simple workaround, though, and that is to simply download the RSS feed to the local machine before you open the movie in quickitime. You simply build the composition (before saving it as a movie) to look for that file on the local drive. Here’s a quick command to grab our RSS feed and save it:

/usr/bin/curl http://calendar.walkerart.org/news/today.wac > /tmp/today.html

And then your path for the RSS feed inside quartz is:

file://localhost/tmp/today.html

Problem one solved. This lets us manually open up the quicktime movie and export it to any format quicktime can export to. Once you have it in that format, you can transform it, play it or transfer it with much more ease.

I’ll post about how to automate the whole process in the future, and the problems that occur when you try to deal with HD resolution screens. In the meantime, here is a short demo of what I have been able to achieve with quartz composer and our identity system (a work in progress).

qtz_sign_sm_demo.jpg

Testing regular expressions

Posted December 6, 2006 at 3:42 pm — Filed under:

Today I’ve got two good tools for web developers.

Lately I’ve had to write a number of regular expressions for the upcoming mnartists.org calendar – most in Java, and a few in Javascript. In theory a regexp is a regexp no matter the language, but in practice that’s rarely the case. Between these subtle differences and the maddening wait for compiling or reloading a page, it’s clear some sort of live testing environment is useful:

  • Javascript tester – allows replacement testing as well
  • Java tester – really nice in that it gives accurate feedback on your regexp errors and even helps you format the matching text as a java String

If you’re a developer messing with Java or Javascript regular expressions, IMHO it’s worth bookmarking those two pages.

Here’s a Java one – looks complicated, actually pretty straightforward. Anyone care to take a stab at what it does? :)

line = line.replaceAll(“\[([bi])\]([^\[]*)\[/\1\]“,”<$1>$2</$1>”);

(or can you do it better? I get by, but I know my regexps are sometimes clunky at best…)

Still images triggered by a computer

Posted September 25, 2006 at 4:55 pm — Filed under:

For an upcoming project, we want to capture high resolution digital still images onto a computer from a camera. This is a technique used quite often in stop motion animation. A digital still camera is a very inexpensive way to capture images that are many, many times higher resolution than the HD video they will eventually be a part of. A simple method would be to capture a large amount of images onto a camera’s storage card, download them, and them import and compose them in a compositing program such as After Effects.

However, for this project, we need the image capture to be triggered not by the button on the camera, but by the computer the camera is connected to. I initially looked at both FrameTheif and iStopMotion because they support digital still cameras and have an applescript library. However, both have spotty support for remote capture and would want to grab onto any other cameras connected to the system (such as a dv cam). Eventually, I found the gphoto project on sourceforge.

More

Accelerated Development: Art on Call

Posted February 28, 2005 at 9:47 am — Filed under:

Last time we checked in on Art on Call, the Walker’s upcoming cellphone-based audio tour project, we were still fleshing out the API and locking down features – and dreaming of iPod integration. Last week I started more intensive design on the actual classes, data structure, and the web-based app to enter data into the system. Coding began Thursday morning, and I’m happy to announce I have working versions (I think) of all the basic classes and data abstraction layer. Because of the need for speed I seriously considered some pre-packaged data modeler tool like Alzabo, which the Walker has previously used with good success, but I was unfamiliar with.

I opted instead to write the classes largely from scratch – I felt good enough about my design work and data structures that I wanted to streamline things and really push the inheritence aspect of the classes. (Probably this could have been done with Alzabo, but I didn’t think I had time to learn a new tool.)

Sure, there’s a million pieces still to write and test, but a big chunk of the code is written…