Photo by k0a1a.net.

Minneapolis-based Northern Lights.mn has announced the second year of Ar(ists) on the Verge:

Northern Lights announces a second round of Art(ists) on the Verge commissions (AOV2). AOV2 is an intensive, mentor-based fellowship program for 5 Minnesota-based, emerging artists or artist groups working experimentally at the intersection of art,  technology, and digital culture with a focus on network-based practices that are interactive and/or participatory. AOV2 is generously supported by the Jerome Foundation.

Northern Lights was founded by former Walker New Media Curator Steve Dietz. The grants this year will be juried by Dietz, along with Kathleen Forde, Curator for Time-Based Arts at the Experimental Media and Performing Arts Center (EMPAC) in Troy, NY, and the Walker’s chief curator, Darsie Alexander.

The resulting show  show at the Weisman Art Museum from last years grantees was worth checking out. It is good to see work being done to create our own new media art structures here in Minnesota, rather than watching cool things like Eyebeam happen from afar.

And by the way, Northern Lights’ blog, Public Address, has become one of my favorite reads for neat artwork being made around the world. I confess I find a lot of art blogs rather dry and esoteric, but not Public Address. And, this may seem somewhat mundane and obvious, but near every post has an interesting image, which is nice for an art blog.

If you’re visiting town and are out and about, getting info on the Walker and other cultural institutions in the city via the web just got easier. Minneapolis’ city-wide wireless network now lets users access walkerart.org without being a subscriber. Here’s how it works:

On your computer, select the “City of Minneapolis Public WiFi” network.

Open your browser and point yourself to walkerart.org. That should do it. You may be directed to a user agreement log in screen and then the “walled garden” of Minneapolis city information and lists of other accessible community sites. The Walker is listed under Area Arts & Culture > Arts & Museums > Art Museums.

Wireless Log In Screen

Minneapolis Dowtown Area Walled Garden Portal

A brief history of Minneapolis Municipal WiFi

Several years ago, the City of Minneapolis joined with USI Wireless to build out a city-wide network. The goal was to provide access for city government and citizens. The city would be a core tenant, paying USI, and USI would sell access to citizens. The city required USI to build a community portal and USI must provide grants out of it’s profits to non-profits working to bridge the digital divide.

Over the last several years, the network has slowly been built out. Right now there are some problem areas, which include Loring Park and the Minneapolis Sculpture Garden. My understanding is that these areas should see service sometime soon, though I’m not sure of any exact plans on the Sculpture Garden.

There are a couple things I have really liked about the network:

• We’re doing it. A lot of cities have talked about building municipal wifi, and then discover large problems and things don’t work well. There have been some issues with in Minneapolis, it is taking longer to build the network than originally thought, but my impression is that it has worked fairly well.
• It’s network neutral. The agreement between the city and USI specifically requires USI to not hinder any type of traffic over another.
• Parts of it are free. This is how you can get to our site for free.
• It’s low cost. The cost for being a subscriber is pretty low, compared to other wire-based providers.
• It’s local. USI is a local company.

For more information on the network and the history, Peter Fleck has been blogging about Minneapolis WiFi for some time.

On September 1, 2009 the new ArtsConnectEd became available at ArtsConnectEd.org.  The new site provides access to more than 100,000 museum resources, including audio, video, images, and information about works of art, all of which can be saved and presented with the more powerful Art Collector.

This project was at least three years in the making, with the last two of those being the technical work of research, design, and development.  In this series of posts I’d like to present some of the decisions we struggled with and the process we went through in developing the new site.  I’ll start with the Art Finder, followed by a post on the Art Collector and presentations, and finish with a post about some of the more technical aspects including the data and harvesting technologies we’re using.

Art Finder

The Art Finder is the guts of the site, a portal into our thousands and thousands of objects, text records, and more.  I don’t think it’s an exaggeration to say designing and building this component was the biggest challenge we faced in the entire process.  We’ve redesigned the interface many times, often significantly, and are still not certain it’s right.  We’ve changed the underlying technology from a SQL / Lucene hybrid to a straight-up Solr search engine.  We’ve debated (endlessly) what fields to include, and what subset of our data to present in those fields.  We’ve gone back and forth over tab titles, and even whether to use tabs.  A rocky road, to say the least.

The big idea

What if we could start with everything and narrow it down from there?  Offer the user the entire collection and let them whittle away at it until they found what they wanted?

It’s all browse.  Keyword is just another filter.

To me this is the big breakthrough of the ArtsConnectEd interface.  We don’t hide the content behind a search box, or only show filters after you try a keyword.  We don’t have a separate page for “Advanced Search”, but we offer the same power through filters.  There is still a keyword field for those who know exactly what they’re looking for, but we get to use our metadata in a more powerful way than simple text.  That is, since know the difference between the word “painting” appearing in the description and something that is a painting, we can present that to the user through filters.

How we Got here

We wanted many ways for the user to explore the collection, with the idea we might hopefully mimic some of the serendipity of exploring a gallery.  The tech committee felt early on that we’d need, in addition to a robust search, some way to freely browse.  Our initial attempt was to split the Art Finder into a Browse interface (left) and a Search interface (right).

After forcing users to choose a content type to browse (Object, Text, etc), we exposed facets (fields) to allow filtering, e.g. by Medium or Style.  These facets were hidden by default in the Search interface, where instead you started with a keyword and content type as tabs — but could then click to reveal the same browse filters!  The more we played with these two ideas, the more we realized they were essentially the same thing, the only difference being a confusing first step and then having to learn two interfaces.  The real power of the site was in combining them, committing fully to Browse, and adding the keyword search as a filter.

Lastly, as we harvested more of our collections we realized pushing filters to the front offered a better way to drill down when many of our records are not text-heavy and thus less findable via keyword search.  In many ways browse leveled the playing field of our objects between those with healthy wall labels and those with more sparse metadata.

What works

(In my humble opinion!)  A good browse has to do a few things:

• Be fast. Studies have shown that slow search (or browse) results derail a user’s chain of thought and makes it difficult to complete tasks.  We went one step further and did away with the “Go” button for everything but keyword – making a change to a pulldown automatically updates your result set.  (It’s not instant, but it’s fast enough the action feels connected to the results)
• Reduce complex fields to an intuitive subset. We have a huge range of unique strings for the Medium field, but we’ve broadly grouped them to present a reasonable-sized pulldown.  Likewise for the Culture pulldown.  (We manually reduce the terms for Medium, and have a automated Bayesian filter for the Culture field)
• Have good breadcrumbs. Users need to know what options are in effect and be able to backtrack easily.
• Avoid dead ends. With many interfaces it’s entirely too easy to browse yourself into an empty set.  By showing numbers next to our filter choices, we can help users avoid these “dead ends”.
• Expose variety. Type “Jasper Johns” in the artist field, and check out the Medium pulldown: it shows the bulk of his work is in Prints, but we also have a few sculptures, some mixed media, etc.  A nice way to see the variety of an artist’s work at-a-glance.
• Autocomplete complicated fields. If a search box is targeted to a field (like our Artist box), it needs to autocomplete.  Leaving a field like this open to free text is asking for frustration as people get 0 results for “Claes Oldenberg“. (Auto-suggest “did you mean” should also work!)
• Have lots of sort options. One of my favorite features of the new Art Finder is the ability to sort by size.  Super cool.  (check out the Scale tab in the detail view for more fun!)

I’m biased after this project, but I’m fairly convinced combining faceted browsing with keyword search is absolutely the way to go for collection search.  It gives the best of both worlds, powerful but still intuitive.

What could be better

… but is it really intuitive?  People seem to still be looking for a big inviting search box to start with.  The interface is crowded, and the number of options looks intimidating.  We’ve ended up avoiding using the words “Search” and “Browse” because they were loaded and causing confusion.  We’ve tried many versions of the tab bar to try to clarify what filters apply globally (e.g. Institution) and which only effect that tab (Works of Art have an Artist, for instance), but I don’t believe we’ve solved it.

I think the two components of the interface that give us the most trouble and confusion are actually the “Has Image” checkbox and the “Reset All” button.  These are consistently missed by people in testing, and we have tried almost everything we can think of.  Oh, and the back button.  The back button is “broken” in dynamic search like this.

Also, while I really like the look of the tiles in the results panel, we’ve had to heavily overload the rollover data to show fields we can sort by since there’s no more room in the tiles.  We also intended to create alternative result formats, such as text bars, etc, which could show highlights on matching keywords, but this item was pushed back for other features.

We’ve defaulted to sorting alphabetically by title when a user first reaches the page, and I’m no longer sure this is best.  As we’ve populated the collections in ArtsConnectEd we’ve ended up with a bunch of works that have numbers for titles, make the alpha sort less obvious.

You tell me!  Give the site a spin and post a comment – what works, and what could be better?

Resources:

• Designing for Faceted Search (http://www.uie.com/articles/faceted_search/)
• Faceted Search: Designing Your Content, Navigation, and User Interface (http://www.uie.com/events/virtual_seminars/facets/FacetedSearchVS35Handout.pdf)
• Faceted Search (http://en.wikipedia.org/wiki/Faceted_search)
• Best Practices for Designing Faceted Search Filters (http://www.uxmatters.com/mt/archives/2009/09/best-practices-for-designing-faceted-search-filters.php)
• V&A Collections (beta) (http://www.vam.ac.uk/cis-online/search/?q=blue&commit=Search&category%5B%5D=5&narrow=1&offset=0&slug=0)
  • Their facets aren’t as up front as I’d like (you have to start with a keyword), but they’re done really well once they show up.
  • You can also cheat and leave keyword blank to get a full browse and go right to the facets…  Maybe start here?
• MOMA Collections (http://www.moma.org/collection/search.php)
  • Nice presentation of facets, but I wish two things: show me a number next to all constraints, not just artists, and let me add a keyword.  (I got a dead end looking for on-view film from the 20s or 2000s)  I also like that it’s a true browse – leaving everything at “All” seems to give me the whole collection.

• Eric at Adapted Studio put together this sweet little demo of HTML5 and Canvas in action, in the form of the Game of life. Source code is included, too, if you want to learn a few nifty things.
• Color me surprised, but Microsoft is actually purporting to work together on at least some of the HTML5 spec. This could be good. Using <video> would be much easier if everyone would do it. But there still is the nasty issue of codecs, which is even more thorny than W3C specs.
• This is from about a year ago, but John Resig (of jQuery fame) posted a very nice tutorial for Learning Advanced Javascript. It clears up a lot of confusion about seemingly advanced techniques.
• Also worth perusing is Mark Pilgrim’s Gentle introduction to video formatting. If you’re a video geek, you might know some of this, but there’s detail that might fill in some gaps. The slides are also slightly amusing. I had no idea the .mkv format came from a bunch of guys in Russia that decided to opensource it.

HTML 5 image form here.

One of the trending topics on Twitter currently is “IE6 Must Die“, which are mainly retweets to a blog post entitled “IE6 Must Die for the Web to Move On“. This is certainly true, IE6 has many rendering bugs and lacks support for so many things that it is simply a nightmare to work with. The amount of time and money wasted in supporting this browser across the web is staggering.

In fact a few months ago the New Media department decided to drop support for IE6 on all future websites we create. The last website we built with full IE6 support was the new ArtsConnectEd, mainly because teachers tend to have little say in what browsers they can use on school computers. However, moving forward we’re phasing out support for IE6. It simply costs us too much time and resources for the dwindling number of users it has on our sites (currently under 10%, which is down 45% from last year and falling fast). We’re not alone, many other sites are doing this as well.

However calling for the killing of IE6 ignores a bit of history as well as new problems to come. There was a time not so long ago when all web developers wanted to be using IE6. The goal back then was to kill off IE5. You see, IE5 had an incorrect box model. Padding and margins were included in a boxes width and height instead of adding to it like in standards compliant browsers.

This caused all sorts of layout errors, and meant hacks (like the Simplified Box Model Hack) had to be used to get content to align correctly. These hacks were so widely used that Apple was going to allow them to be used in the first version of Safari until I convinced Dave Hyatt (lead Safari dev) to take out support for it. IE6 fixed this bug and everyone was happy (for a while anyway).

Going back further, IE5, even with its broken box model, was at one time the browser of choice back when IE4 was killing Javascript programmers because it didn’t support document.getElementById(). IE4 only supported the proprietary document.all leading to a horrible fracturing of Javascript, whereas IE5 added in the JS standard we still use today. Before people embraced IE5, cross platform JS on the web was almost non-existent, a fact I attempted to rectify by building my Assembler site in 1999.

The reason I bring this up is because we have a history of this behavior with regards to IE. We yearn for the more modern versions, only to end up hating those same versions later on. This will not change with the death of IE6. Soon, it will be IE7 that we are trashing, and then IE8 will be the bane of our existence.

This only becomes more clear as we move to HTML5. IE8 doesn’t support it, nor does it support any CSS3. While IE8 does support many of the older standards it had been ignoring for so long, having just recently been released it is already out of date. All of the other browsers do support these advanced web technologies, but IE is the lone browser to ignore them. Once again IE is two steps behind where the web is going, and severely limits our ability to push web technology forward to everyone for many years to come.

So while we celebrate the death of IE6, let us not forget that there will be a new thorn in our side to take its place in short order. IE7, you’re next.

We’re in the process of retiring our last production server running NT and ColdFusion (whew!), and this means we needed to get a few old projects ported to our newer Linux machines.  The main site, http://aen.walkerart.org/, is marginally database-driven: that is, it pulls random links and projects from a database to make the pages different each time you load.  The admin at the time was nice enough to include MDB dump files from the Microsoft Access(!) project database, and the free mdbtools software was able to extract the schema and generate import scripts.  Most of this page works as-is, but I had to tweak the schema by hand.

After the database was ported to MySQL, it was time to convert the ColdFusion to PHP.  (Note: the pages still say .cfm so we don’t break links or search engines – it’s running php on the server)  Luckily the scripts weren’t doing anything terribly complicated, mostly just selects and loops with some “randomness” thrown in.  I added a quick database-abstraction file to handle connections and errors and sanitize input, and things were up and running quickly.

… sort of.  The site is essentially a repository of links to other projects, and was launched in February 2000.  As you might imagine there’s been some serious link rot, and I’m at a bit of loss on how to approach a solution.  Steve Dietz, former New Media curator here at the Walker, has an article discussing this very issue here (ironically mentioning another Walker-commissioned project that’s suffered link rot.  Hmm.).

One strategy Dietz suggests is to update the links by hand as the net evolves.  This seems resource-heavy, even if a link-validating bot could automate the checking — someone would have to curate new links and update the database.  I’m not sure we can make that happen.

It also occurred to me to build a proxy using the wayback machine to try to give the user a view of the internet in early 2000.  There’s no API for pulling pages, but archive.org allows you to build a URL to get the copy of a page closest to a specific date, so it seems possible.  But this is tricky for other reasons – what if the site actually still exists?  Should we go to the live copy or the copy from 2000?  Do we need to pull the header on the url and only go to archive.org if it’s a 404 to 500?  And what if the domain is now owned by a squatter who returns a 200 page of ads?  Also, archive.org respects robots.txt, so a few of our links have apparently never been archived and are gone forever.  Rough.

In the end, the easy part was pulling the code to a new language and server – it works pretty much exactly like it did before, broken links and all.  The hard part is figuring out what to do with the rest of the web…  I do think I’ll try to build that archive.org proxy someday, but for now the fact it’s running on stable hardware is good enough.

Thoughts?  Anyone already built that proxy and want to share?

New Media has a number of development servers located in-house where we get stuff done before releasing it out into the wild.  Until last week these were protected by an aging OpenBSD firewall running packet filter and all was well until midweek when the motherboard failed.  Not having a spare on hand, I was scrambling for a solution.

Linksys wireless router

Being familiar with the dd-wrt project, I was pretty sure I could build a firewall out of a Linksys router.  We went with the WRT54GL, currently as cheap as $50 on Amazon.  (We bought local so we’d have it sooner, and it was a bit more).

The first step after flashing the firmware with the latest dd-wrt build (v24-sp2) was to take off the antennas and turn off the radio.  The last thing I want for the firewall is to be broadcasting an SSID and allow wireless associations.  This actually requires a startup script on the router, with a line to remove the wireless module so it won’t try to reenable itself:

wl radio off
wl down
rmmod wl

Good start.  Next I needed to bridge the WAN port with the LAN ports, which ended up being a struggle until I found the easy options in the dd-wrt GUI.  First, set the LAN to use a static IP and make sure you can connect via another machine to configure it.  You’ll also need to enable SSH access and remote configuration – but be sure to lock this down once the firewall is running!

Once you have the LAN configured, you need to set the WAN connection type to “disabled”.  This will give you a checkbox to bridge the LAN and WAN:  “Assign WAN port to switch”.  Lastly, under Advanced Routing set the Operating Mode to “Router” so it stops trying to do NAT.  Apply these settings, and you’ll basically have an expensive dumb switch – all traffic shows up on every port, and there’s no logic at all.  We’re halfway there.

Being unfamiliar with iptables (we use OpenBSD and pf for firewalls around here), I was under the impression that iptables rules would work in a bridging environment.  This is not the case: bridged packets don’t reach iptables at all!  The best I could do was block everything (manual restart needed), or otherwise blow up the configuration (manual restart needed) as I tried to mess with the bridge.  This was an incredibly frustrating learning curve as everything I could find made it sound like this was the way to configure a firewall in Linux, but it just wasn’t working.

Note to keep you sane: don’t do any of this testing in the startup scripts or you’ll brick your router, guaranteed.  Do it all from the command line with a known-good startup.  That way it’s a simple (but annoying) power cycle to get things back up.

The trick, it turns out, is a kernel module called ebtables.  Luckily, this is included in the dd-wrt build, but it’s not turned on by default!  Add this to your startup script:

insmod ebtables
insmod ebtable_filter
insmod ebt_ip.o

And, ta-da, all your iptables rules will start impacting packets!  Now it’s just a matter of configuring the firewall rules.  We’re using something like this:  (vlan0 represents the LAN ports, and vlan1 is the WAN port)

# drop everything by default:
iptables -P FORWARD DROP
# clear the old rules:
iptables -F FORWARD
# forward stuff that's established already
iptables -A FORWARD -m state --state ESTABLISHED,RELATED -j ACCEPT
# let connections out:
iptables -A FORWARD -i vlan0 -m state --state NEW -j ACCEPT

# firewall access rules
iptables -F INPUT
# WAC ips can get to fw:
iptables -A INPUT -p tcp -d 1.2.3.4 -s 4.3.2.1/24 -j ACCEPT
# drop everything else!
iptables -A INPUT -p tcp -d 1.2.3.4 -j DROP

# ... snipped all the actual access rules and packet flood protection ...

The only trick here is the last few lines which limit access to the firewall machine itself.  We can’t use the FORWARD rules since these packets are destined for the internal hardware and not forwarded, but we do need to limit access via the INPUT chain.  In this example the firewall has IP 1.2.3.4 and the network I want to access it from has 4.3.2.x.  That way I can leave the firewall’s remote access turned on and limit it to our network.  (because there’s no terminal access you can’t make it a truly transparent bridge or you’d never be able to change the config!)

I admit I’m a bit nervous posting some of this in case there’s a glaring security hole, but it seems good to me.  Anyone see anything they’d like to warn me about before we get hacked?

And there you have it!  For the cost of a cheap router and some time (not much, since you can just follow these notes!) you have a full-featured bridging firewall running on dedicated hardware.  With a little extra work it would be easy to get VPN running and much more…  I’m hoping for years of service from this little guy!

( Hat tip another DIY firewall solution that I’d really like to try someday. )

Now that Don’t Sleep on It is over and everyone has caught up on some sleep, I thought I’d share a bit more on the technical setup and a lesson learned. Witt told me he thought that it was one of the best, if not THE best, event that WACTAC has ever done. I tend to agree.

As I explored in a previous post, we used a digital still camera to take our single frame images, then stitch them together in quicktime as a longer move. For the event itself, we used two cameras. The primary camera, a Canon 10D, was equipped with a 16mm wide-angle lens that gave us a really good shot of the entire space. The second camera, a Canon G9, wasn’t quite as wide-angle, but would be a good backup camera in case something happened to the 10D. A sample of the space:

We taped off sight lines, just out of frame, so the artists would know what was in frame and what was not.

Our events & media production team set up a very nice mount for the cameras, as you can sort of see in this blurry, hastily snapped iPhone shot:

Unfortunately, every good plan has it’s own particular achilies heel. In this case, that heel was electronics’ desire for an uninterrupted flow of electricity. Midway through the evening on Friday night, the circuit breaker that powered the computer and cameras was tripped. Power was quickly restored, and the computers were turned back on. However, the startup procedure to get the time-lapse running was not something that could be scripted or automated, so the capture did not start again until 9 AM the next morning when I cam to check on things.

The lesson here: Time lapse is awesome, but next time, use an uninterpretable power supply. Preferably one that has a loud audible warning. I probably should have thought of this, but it really didn’t occur to me how chaotic and crazy the event would actually be (I mean that in the most endearing way possible).

The fact that we lost 12 hours of the time-lapse does stink, but it also means we still captured 12 hours of the event. I’ve assembled the video, and it has been posted to YouTube, but the quailty is not as good as a quicktime file. Here is a higher-quality quicktime MP4:

Click to play, or download the original file.

To fill some of the 12-hour gap, we hastily collected photos from whoever was available and had taken photos. They’ve been put together as a short slideshow filling a portion of the 12-hour missing period.

WACTAC has an event next week called Don’t Sleep on It, taking place during Art-a-Whirl. The gist of the event: over the course of 24 hours different groups of artists will transform a gallery space, destroying and re-building the art many times over the period. At the end of the event, they want to show a time-lapse video of the transformation.

Making a time-lapse movie is not hard. While it can be done using a video camera, it’s easier to use a digital still camera. You take a series of images at predefined intervals and stitch them together using software like After Effects, or, even simpler, Quicktime Pro. We’re using a Canon G10 and the Canon Remote Capture software to take photos every 10 seconds. I set up a test in our office just to make sure it would run correctly and without incident. Here’s the result:

Taking one photo every 10 seconds over 24 hours generates 8640 frames, creating a video just under 10 minutes long. We may end up dropping every other frame to create a shorter movie. The nice thing about using a digital still camera for this is that it produces a video well beyond even 1080P HD resolution.

In the above video, you can enjoy watching me look up documentation on Django, read a book about symfony, and my be mesmerized by a screensaver.

… blog about it in May!

Museums and the Web 2009 wrapped up with a challenge to all the inspired delegates: use the energy and ideas generated here to get one thing done in April.  (The idea being that many small steps build momentum, and it’s too easy to ignore the small upgrades we should constantly be pushing out.)

Yesterday I pushed out a few small upgrades to our aging collection site:

You can now limit your search to objects that are On View

What works by Dan Flavin can you come see right now?

OpenSearch capable

Can’t get enough of our collection?  Add it to your browser’s built-in search box!  When you’re on the Collection site, you should be able to pull down your browser’s search field and add “Walker Art Center”.

Developers (Piotr!): you can now use the Walker collection in your Yahoo Pipes tool without having to scrape the results!  Not an API (yet), but a good step.  Check out the XML for ideas.

Bring it all together:

You’re a busy person.  You’d love to come see Chuck Close’s Big Self-Portrait, and you know the Walker’s got it in their collection, but you see it’s not on view.  You don’t have time to check our website every day, so how will you ever know when it goes on display?  Easy:  build a search that finds Big Self-Portrait, then turn on the “On View” flag.  The object disappears (not on view), but you can subscribe to the OpenSearch RSS feed for this query (click the icon).  Now, when Big Self-Portrait is available to see in the galleries, the object will show up in your RSS reader!  (note: I picked this painting randomly.  I make no guarantee about seeing it in the galleries any time soon.  :)

So, baby steps.  Get one things done that opens more doors.

#didonethinginapril (I tag Andrew at the MIA to get one thing done in May!)