New Media Initiatives

New Media has a number of development servers located in-house where we get stuff done before releasing it out into the wild.  Until last week these were protected by an aging OpenBSD firewall running packet filter and all was well until midweek when the motherboard failed.  Not having a spare on hand, I was scrambling for a solution.

Linksys wireless router

Being familiar with the dd-wrt project, I was pretty sure I could build a firewall out of a Linksys router.  We went with the WRT54GL, currently as cheap as $50 on Amazon.  (We bought local so we’d have it sooner, and it was a bit more).

The first step after flashing the firmware with the latest dd-wrt build (v24-sp2) was to take off the antennas and turn off the radio.  The last thing I want for the firewall is to be broadcasting an SSID and allow wireless associations.  This actually requires a startup script on the router, with a line to remove the wireless module so it won’t try to reenable itself:

wl radio off
wl down
rmmod wl

Good start.  Next I needed to bridge the WAN port with the LAN ports, which ended up being a struggle until I found the easy options in the dd-wrt GUI.  First, set the LAN to use a static IP and make sure you can connect via another machine to configure it.  You’ll also need to enable SSH access and remote configuration – but be sure to lock this down once the firewall is running!

Once you have the LAN configured, you need to set the WAN connection type to “disabled”.  This will give you a checkbox to bridge the LAN and WAN:  “Assign WAN port to switch”.  Lastly, under Advanced Routing set the Operating Mode to “Router” so it stops trying to do NAT.  Apply these settings, and you’ll basically have an expensive dumb switch – all traffic shows up on every port, and there’s no logic at all.  We’re halfway there.

Being unfamiliar with iptables (we use OpenBSD and pf for firewalls around here), I was under the impression that iptables rules would work in a bridging environment.  This is not the case: bridged packets don’t reach iptables at all!  The best I could do was block everything (manual restart needed), or otherwise blow up the configuration (manual restart needed) as I tried to mess with the bridge.  This was an incredibly frustrating learning curve as everything I could find made it sound like this was the way to configure a firewall in Linux, but it just wasn’t working.

Note to keep you sane: don’t do any of this testing in the startup scripts or you’ll brick your router, guaranteed.  Do it all from the command line with a known-good startup.  That way it’s a simple (but annoying) power cycle to get things back up.

The trick, it turns out, is a kernel module called ebtables.  Luckily, this is included in the dd-wrt build, but it’s not turned on by default!  Add this to your startup script:

insmod ebtables
insmod ebtable_filter
insmod ebt_ip.o

And, ta-da, all your iptables rules will start impacting packets!  Now it’s just a matter of configuring the firewall rules.  We’re using something like this:  (vlan0 represents the LAN ports, and vlan1 is the WAN port)

# drop everything by default:
iptables -P FORWARD DROP
# clear the old rules:
iptables -F FORWARD
# forward stuff that's established already
iptables -A FORWARD -m state --state ESTABLISHED,RELATED -j ACCEPT
# let connections out:
iptables -A FORWARD -i vlan0 -m state --state NEW -j ACCEPT

# firewall access rules
iptables -F INPUT
# WAC ips can get to fw:
iptables -A INPUT -p tcp -d 1.2.3.4 -s 4.3.2.1/24 -j ACCEPT
# drop everything else!
iptables -A INPUT -p tcp -d 1.2.3.4 -j DROP

# ... snipped all the actual access rules and packet flood protection ...

The only trick here is the last few lines which limit access to the firewall machine itself.  We can’t use the FORWARD rules since these packets are destined for the internal hardware and not forwarded, but we do need to limit access via the INPUT chain.  In this example the firewall has IP 1.2.3.4 and the network I want to access it from has 4.3.2.x.  That way I can leave the firewall’s remote access turned on and limit it to our network.  (because there’s no terminal access you can’t make it a truly transparent bridge or you’d never be able to change the config!)

I admit I’m a bit nervous posting some of this in case there’s a glaring security hole, but it seems good to me.  Anyone see anything they’d like to warn me about before we get hacked?

And there you have it!  For the cost of a cheap router and some time (not much, since you can just follow these notes!) you have a full-featured bridging firewall running on dedicated hardware.  With a little extra work it would be easy to get VPN running and much more…  I’m hoping for years of service from this little guy!

( Hat tip another DIY firewall solution that I’d really like to try someday. )

Tuesday I posted some of the technical details for My Yard Our Message. Since then, I’ve been working on putting together a Facebook App to let people show the signs on their profiles. It is done, or done enough to be used.

I’ve played around with building a Facebook app before, but never had a clear strategic need to build one. I found this tutorial very helpful on getting my feet wet. For the moment, the app is written and PHP and talks to the actual My Yard Our Message site via json, rather than rss.

One of the things I am evaluating is whether or not I can set up a system to let people vote on signs directly from Facebook. Obviously, this would tremendously expand our pool of eligible voters, and would eliminate the need to force people to register for an account just to vote. During the building of the app, I went back and forth on the necessity of registration to vote, but ultimately decided it was necessary and laid the structural groundwork. However, I think I can get enough information about Facebook users to know they’re unique, track them, and prevent them from voting more than once.

This would require re-writing the Facebook app at that point, most likely in python and django for closer integration with the authentication and verification processing. There is some initial work done on a Facebook client API for python and django that looks promising as well.

Aside about Facebook pages

I’ve always been frustrated with Facebook apps that don’t work on Pages (as opposed to Profiles). After building an app, I have a new found appreciation for the applications that don’t work. Getting an app to work with pages isn’t really that hard, but it sure is confusing, mostly because of the lack of documentation. The only real documentation is a chat log in the Facebook developer wiki (no, I am not making this up):

(03:02:02 AM) swombat: ok, so basically, "Facebook pages is all transparent uses fbml blah blah blah"

                       "Oh but btw you need to build a completely different piece of code to handle this new type of user"

(03:02:18 AM) swombat: (not angry at you btw :-P)

(03:02:43 AM) fiveofoh: Yeah pretty much

(03:02:46 AM) swombat: this sounds really quite tedious and error-prone though

(03:02:50 AM) fiveofoh: Yeah it does

(03:02:56 AM) fiveofoh: Which is why I haven't done it on my app yet :P

So instead of just this:

$facebook->api_client->profile_setFBML('', $user);

$facebook->api_client->fbml_refreshRefUrl($process_url);

you use this:

if (isset($_GET['fb_page_id'])){

	$facebook->api_client->profile_setFBML('', $_GET['fb_page_id']);

} else {

	$facebook->api_client->profile_setFBML('', $user);

}

$facebook->api_client->fbml_refreshRefUrl($process_url);

Where $process_url is the page that spits out the markup to be shown on Facebook.

Party People Photos was back in action at After Hours last week, and I promised more technical details of the new features. The big change this time around was automatic uploading to flickr. After each photo was taken, it was transfered, processed, uploaded and finally displayed on-screen in the lounges. I ended up re-writing all the transfer and processing scripts to work better, and they all functioned without problem.

After a photo was taken, it was saved to a folder on the capture iMac. A script started by launchd watched the folder and transfered the file via rsync to my workstation. The files were transfered to my workstation rather than directly to the display computers for two reasons.

Photoshop

Once the files got to my workstation, another script watched the incoming folder and sent all incoming jpg files to Photoshop. We wanted the process all the images to make sure they looked their best. Cameron, one of our photographers, developed a handy action for the photos that would give them more contrast and punch. The easiest way to automate this in photoshop is to use the script events manager to run an action on every file that is opened ( File > Scripts > Scripts Event Manager… ). The easiest way on OS X to get a particular application to open a file is to use the open command with the -a argument. The -a lets you specify the binary that you want to use to open the file. Otherwise, you’re at the mercy of whatever program has associated itself with jpeg files. Here’s the script:

#! /bin/sh

# this script takes files from the 1_incoming directory and tells photoshop to open them

# should be called by launchd which will be watching the 1_incoming folder

# photoshop should be set to perform the action on open new document

while true; do

myls=`ls /Volumes/Patience/_after_hours/1_incoming/`

	if [ "$myls" != '' ] ; then

		for myFile in /Volumes/Patience/_after_hours/1_incoming/*.JPG

		do

			/usr/bin/open -a /Applications/Adobe\ Photoshop\ CS2/Adobe\ Photoshop\

				CS2.app/Contents/MacOS/Adobe\ Photoshop\ CS2 $myFile

			sleep 2

			mv $myFile /Volumes/Patience/_after_hours/2_incame/  2>&1 > /dev/null

		done

	fi

	sleep 10

	#take care of pesky .DS_Store files, which can pop up from the Finder

	find /Volumes/Patience/_after_hours/ -name .DS_Store -exec rm -f {} \;

done

# added a return on the photoshop open line for clarity in the browser

Every 10 seconds, if there is a new file, and it is a jpeg, it will get sent to Photoshop. Photoshop runs the action, and saves the file to a new folder, again as a jpeg.

I used my workstation for this because it already had Photoshop installed, and I didn’t want to deal with licensing issues on another machine. I created another account that had photoshop set up just right and the launchd jobs start onload. When we set up PPP again for Picasso show, I’d love to be able to do post-processing via imagemagick, since it is command-line, easy to install and unencumbered by licensing issues.

Flickr Uploading

Photoshop has now saved the jpeg to a different folder. This folder is being watched by yet another script, similar to the last. This script does two things: copies the jpegs to our projection machines and uploads them to flickr. The copy process is mostly unchanged, though I used scp instead of rsync, since the I want to copy files one at a time. As with all the rsync or scp transfers, I just made sure I had my keys set up and authorized, and it worked fine.

Uploading to flickr was the trickiest part of the operation, but thankfully flickr provides a great API and there are a lot of libraries that simplify the process. I ended up using phpflickr. I am pretty familiar with php, and phpflickr only requires php4, which is the version of php-cli in OS X. In order to get it working, you have to apply for an API key, giving it write permission. Using your API key and the secret, you generate a token that you can use to upload with (you need all three to upload). Phpflickr provides the scripts necessary to provide the callback URL that flickr needs, even if you’re not uploading things directly from the web. It is somewhat confusing, and I’m not entirely sure I need the callback url, but it works all the same.

With the tokens all set, uploading the photo is as simple as upload, add to group, add to set. To get the ID of the group and the set, I just used the API explorer, which lists the groups and sets I visited. I had to create the set before I started uploading, though it is possible to create a set through the APIs. I also created a function in the upload script that used a word list to generate a title for each photo as it was uploaded to flickr. I am not a fan of seeing photos with names like IMG_4097.JPG, and this solved that and created some fun and funny juxtapositions.

Other notes, future

I enabled printing of all our photos on flickr, so you can now get prints if you want. In the US, flickr and yahoo do the printing through Target, which also happens to be a major sponsor of After Hours. Perhaps that is an opportunity for the future.

I still had some trouble with the camera not acting the way I wanted. Sometimes it would get into a state where it was totally locked, and the only way to reset it was to pull the power by removing the battery adapter; turning it off and on again wouldn’t do it. Due to the way that gphoto2 talks to the 10D, the capture command doesn’t fully finish, so I kill it after a few seconds. I think what was happening was that in some focus situations, the autofocus was taking too long and the camera hadn’t finished capturing when I kill the command. In the future, I am going to experiment with using a manual focus, which will eliminate that problem.

Gphoto2 was also recently updated to 2.3.0, and it now compiles without too much trickery on OS X. However, it hasn’t fixed my problems with capture on the Canon 10D. I might experiment with modifying the Canon class for the 10D a bit to see if I can get it to work.

We also had some problems with the flash not always firing when the camera went. Unfortunately, the flash we’re using doesn’t have an input for a power adapter, so we were running on batteries. It also doesn’t indicate low batteries, leaving us, literally, in the dark. For the Picasso After Hours, we’re planning on jerryrigging a wired power adapter to provide the 6V it requires.

I’ll also be setting up Party People Photos for the Free First Saturday on March 3rd. I think kids will get a kick out of it. We won’t add the photos to the After Hours Group Pool, but we will put them on our Flickr. Watch for it (or attend, it is free).

As has been mentioned here in the past, I have been tinkering with quartz composer for use as dynamic, digital signage. It is a good fit: extremely fast, can talk to the internet, usable on a lot of different systems. There are a growing number of locations within the museum where we’d like to apply dynamic signage, but off the shelf systems to do it are often convoluted and proprietary, not to mention expensive. Currently in the Walker Cinema, we use a DVD that I render in After Effects and update periodically. This affords a lot of control, but also takes a fair amount of labor to update.

It is this kind of an application where Quartz Composer can work well. Any quartz composer movie can be saved as a quicktime movie, but there are some limitations:

• no mouse and keyboard events
• no contents download from Internet (RSS feeds, images…)
• edition of the input parameters of the compositions

Notice that second one? That’s the doozy if you want your quartz comp quicktime movie to use an RSS feed to get the text.

There is a simple workaround, though, and that is to simply download the RSS feed to the local machine before you open the movie in quickitime. You simply build the composition (before saving it as a movie) to look for that file on the local drive. Here’s a quick command to grab our RSS feed and save it:

/usr/bin/curl http://calendar.walkerart.org/news/today.wac > /tmp/today.html

And then your path for the RSS feed inside quartz is:

file://localhost/tmp/today.html

Problem one solved. This lets us manually open up the quicktime movie and export it to any format quicktime can export to. Once you have it in that format, you can transform it, play it or transfer it with much more ease.

I’ll post about how to automate the whole process in the future, and the problems that occur when you try to deal with HD resolution screens. In the meantime, here is a short demo of what I have been able to achieve with quartz composer and our identity system (a work in progress).

Today I’ve got two good tools for web developers.

Lately I’ve had to write a number of regular expressions for the upcoming mnartists.org calendar – most in Java, and a few in Javascript. In theory a regexp is a regexp no matter the language, but in practice that’s rarely the case. Between these subtle differences and the maddening wait for compiling or reloading a page, it’s clear some sort of live testing environment is useful:

• Javascript tester – allows replacement testing as well
• Java tester – really nice in that it gives accurate feedback on your regexp errors and even helps you format the matching text as a java String

If you’re a developer messing with Java or Javascript regular expressions, IMHO it’s worth bookmarking those two pages.

Here’s a Java one – looks complicated, actually pretty straightforward. Anyone care to take a stab at what it does? :)

line = line.replaceAll(”\[([bi])\]([^\[]*)\[/\1\]“,”<$1>$2</$1>”);

(or can you do it better? I get by, but I know my regexps are sometimes clunky at best…)

For an upcoming project, we want to capture high resolution digital still images onto a computer from a camera. This is a technique used quite often in stop motion animation. A digital still camera is a very inexpensive way to capture images that are many, many times higher resolution than the HD video they will eventually be a part of. A simple method would be to capture a large amount of images onto a camera’s storage card, download them, and them import and compose them in a compositing program such as After Effects.

However, for this project, we need the image capture to be triggered not by the button on the camera, but by the computer the camera is connected to. I initially looked at both FrameTheif and iStopMotion because they support digital still cameras and have an applescript library. However, both have spotty support for remote capture and would want to grab onto any other cameras connected to the system (such as a dv cam). Eventually, I found the gphoto project on sourceforge.

(more…)

Last time we checked in on Art on Call, the Walker’s upcoming cellphone-based audio tour project, we were still fleshing out the API and locking down features – and dreaming of iPod integration. Last week I started more intensive design on the actual classes, data structure, and the web-based app to enter data into the system. Coding began Thursday morning, and I’m happy to announce I have working versions (I think) of all the basic classes and data abstraction layer. Because of the need for speed I seriously considered some pre-packaged data modeler tool like Alzabo, which the Walker has previously used with good success, but I was unfamiliar with.

I opted instead to write the classes largely from scratch – I felt good enough about my design work and data structures that I wanted to streamline things and really push the inheritence aspect of the classes. (Probably this could have been done with Alzabo, but I didn’t think I had time to learn a new tool.)

Sure, there’s a million pieces still to write and test, but a big chunk of the code is written…

While i was cleaning up some of my XSL for the new homepage i documented a couple of the neat XSL things I used on our the WIKI.