New Media Initiatives Blog

Technology at the Walker Art Center

Part of: blogs.walkerart.org

 
by Justin Heideman at 9:08 am 2007-08-14
Filed under:
0 Comments

I’ve seen a couple of new mutlitouch related things come down the pipes recently, so here’s an unordered list of morning multitouch links:

 

Fish dolphin.jpg calamari.jpg newsbox.jpg

WebWalker is getting touchy feely all over in this edition with some computer interface goodness.

  • This one might be a bit old, but it seems Panasonic demonstrated some sort of multi-touch table a last year, as well as a gigantic interactive video wall. I don’t really know how to describe the table. The video looks very nice in 720P glory, but the narration that goes with it is worth a chuckle and the interface is just a bit weird. Translucent fish?
  • Another big multitouch screen, this time from DAHAN T&S (via nuigroup via engadget). This time we get dolphins instead of fish, but my questions still remain, why so many creatures of the sea on multitouch screens? There’s no video, so I can’t tell if their dolphin talks like ours.
  • Speaking of sea creatures, did someone say calamari? The iPhone is certainly putting some pressure onto the demand for multitouch, we should remember multitouch is not actually that new. Case in point, Powerbook trackpads have been multitouch for years, giving users that lovely two-fingered scrolling. Apple even owns a multitouch patent. The iPhone is taking the idea and coupling it with a screen, which is really the important part. I’m rather curious to know how it works and what kind of tech they’re using to make it happen. None of us in NMI plan on getting an iPhone for various reasons, so who’s going to be the first person to take apart their iPhone?
  • And while it is not multitouch, this is a neat project: The digital newsstand. It is basically a newspaper box with a computer screen replacing the window showing todays issue. It is not entirely practical, but I certainly appreciate consistency of the visual language and presentation. If you were going to show newspapers, you might as well do it in the right box. (via Paul)
 
by Justin Heideman at 3:52 pm 2007-06-13
Filed under:
0 Comments

Interesting method to create a multitouch surface:

I guess most of the people reading this will have seen some of the multi-touch demos by Jeff Han, Apple and Tactiva. I wanted to play around with some ideas that required a multi-touch pad, but there aren’t any devices available (Tactiva aren’t shipping…)

Long story short, I made a simple one from a plastic bag, some dye and a camera.

This is interesting, but there are a couple problems with it. First, it is just a multitouch surface, not a screen, making it a lot different from Jeff Han or Apple. There is no projection onto the back, and I can’t see an easy way to ever project onto or into water or other liquids. Secondly, $2 is pretty cheap, but you get what you pay for. You might want to spend $3 to get the heavy duty freezer ziploc so that it would last a week of touching rather than an afternoon. A true FTIR screen made of plexi or glass will be more expensive, but the screen itself is never the expensive part. A fancy FTIR screen in an enclosure might cost $300, but that is still nothing next to the computer, projector and software needed. And a plexi FTIR screen will probably give better blobs, since not the whole surface is going to morph when pressed on.

That said, it is still an interesting exploration and use of what I am assuming is touchlib.

[Via Daily Irrelevant]

 
by Justin Heideman at 8:30 am 2007-05-30
Filed under:
6 Comments

It is not often I find exciting Microsoft products on Slashdot, but there are exceptions to the rule. /. linked up a Popular Mechanics article on Microsoft Surface. Surface is a fancy multitouch table that Microsoft has been working on in semi-secrecy for a while. In addition to being multitouch, it also features proximity detection so it can talk to your phone (bluetooth) and camera (wifi) and other devices when you put them on the table.

Gattis took out a digital camera and placed it on the Surface. Instantly, digital pictures spilled out onto the tabletop. As Gattis touched and dragged each picture, it followed his fingers around the screen. Using two fingers, he pulled the corners of a photo and stretched it to a new size. Then, Gattis put a cellphone on the surface and dragged several photos to it -- just like that, the pictures uploaded to the phone. It was like a magic trick. He was dragging and dropping virtual content to physical objects. I’m not often surprised by new technology, but I can honestly say I’d never seen anything like it.

It looks and feels a lot like CityWall and some of Jeff Han’s work, but it does not appear that Han has worked with Microsoft. One of the notable differences between Han’s tables (which are pricey) and Surface is price. Surface is supposedly going to be on sale this year for $5,000 to $10,000. The article mentions commercial applications, but I would think that education and museums would also have a huge interest. While I wouldn’t be able to afford one for my home, $10,000 is a very accessible price point for a museum.

Whiz-bang aside, the table also reminds me a bit of Pac Man. I also wonder if it can run linux. I’m actually sure that when this comes out, there will be a linux distro that includes a Pac Man knock-off.

EDIT: Create Digital Music has some commentary, as does Chris O’Shea.

 
by Justin Heideman at 10:41 am 2007-05-24
Filed under:
2 Comments

City Wall

Continuing my research into multitouch environments, I came across a project I had not seen before. It is called CityWall, and it is located in Helsinki, Finland:

The content displayed on the CityWall is periodically organized into themes or events that are currently taking place in the city such as festivals, carnivals or sports events. The CityWall is designed to support the navigation of media, specifically annotated photos and videos which are continuously gathered in realtime from public sources such as Flickr and YouTube. To contribute content to the CityWall please send pictures and videos via MMS or email to post@citywall.org. Alternatively, tag your media on YouTube or Flickr with ‘Helsinki’ and we will pick up your media and display it here on the CityWall.

It looks like a combination of some of the social networking work the Brooklyn Museum is doing and a beautiful multitouch environment. The video notes that the screen works both during the day and the night. I am a little foggy on how that works with a FTIR screen, but if UIx has worked around the daylight problem, it would be extremely useful. I will just have to go to Helsinki to find out.

 
by Justin Heideman at 2:57 pm 2007-05-23
Filed under:
1 Comment

A Graphic Language for RFIDRFID Form Factors

I found an interesting blog today: Touch. According to the about:

Touch is a research project looking at the intersections between the digital and the physical. Its aim is to explore and develop new uses for RFID, NFC and mobile technology in areas such as retail, public services, social and personal communication.

NFC, or Near Field Communication, in a nutshell is the technology that will some day let us pay for a Coke or pump a parking meeter with our mobile phone. Or, perhaps, wave our phone at a piece of art and hear the Art On Call stop and an image on our phone’s screen. If you’re wondering why a blog about wireless communication is called touch, it is because NFC generally requires very close proximity, often requiring the access card or phone to touch the receiver.

Dig back through the archives, there are some great posts, such as RFID Form Factors and A Graphic Language for RFID. This one is definitely going in my RSS reader.

Photos borrowed from Touch

 
by Justin Heideman at 8:49 am 2007-04-20
Filed under:
0 Comments

Circuit Bending MinneDemo 2006

There are two great events happening this weekend in the Twin Cities if you’re a hardware hacking or development geek. The first is Bentfest MN. The three day festival (starting tonight) consists of demos, workshops and concerts all centered around circuit bending, happening right down the street at Intermedia Arts. What is circuit bending? This youtube video describes better than I ever could:

The second thing to check out this weekend is MinneBar. What is MinneBar?

minnēbar is an ad-hoc gathering of technology enthusiasts born from the desire for people to share and learn in an open environment. Participants work together and try to create something exciting by being in close proximity to lots of smart people. Each person contributes in some way by leading discussions, demos, asking questions, or volunteering.

The conference/gathering is going on in St. Paul tomorrow, is totally free, and will feature many sweet demos, workshops and networking opportunities. Brent and I are both planning on attending.

Photo Credits:
Circuit Bending Photo from salimfadhley.
Minnebar Photo from alt text.

 
by Justin Heideman at 10:36 am 2007-03-14
Filed under:
2 Comments

Chris O’Shea of Pixelsumo sent me a note letting me know he has been collecting info on mutli-touch screens as well and has a wealth of links to check out. The most impressive of which is the Perceptive Pixel screen. Check out the video to see it in action. We’ve previously covered O’Shea’s Sonicforms project.

I also recalled hearing about some big multitouch screens at CES this year, but google is not helping me. I did find a mention of a large screen from Sharp that was shown at IFA 2006. Watch this clip at about 1:37 for a short view. I can’t find any more details on it, so if someone knows more, I’d love to hear it.

Accenture also has produced some large-scale mutli-touch screen recently. Here’s the press release. The scale is certainly impressive (10×7):
Accenture multitouch screen at O
And some tech details:

Accenture’s patent-pending touch sensing system has the ability to distinguish between touches from multiple simultaneous users. Additionally high-resolution cameras are leveraged to provide touch capabilities for simultaneous usage. The screen consists of a series of nine rear-projection DLP screens fastened together to display cohesive images at a clarity of 2100 x 1200 pixels/resolution. The network is managed and updated from a remote location to allow for content to be adjusted regularly.

Here’s a video of what I think is the same tech being used in a military application, though it doesn’t show off the multi-touch capabilities, and some information on Accenture’s site.

 
by Justin Heideman at 1:18 pm 2007-03-01
Filed under:
1 Comment

“Multitouch” seems to be all the rage lately, mostly because Apple hyped it with the iPhone, though the idea has been around for a while. Make lets us know about a way to create a do it yourself multitouch system that seems to work pretty well. See the video to see for yourself:

The system uses a camera, mirrors, and a computer, just like our Dialog Table.

ftirschematic.jpg

The difference is that the software for this, VVVV, is freely available for non-commercial use. I’d love to see their patch to know how they did it. I bet that a similar setup is achievable with Jitter or Processing, too, the latter of which is free as well.

Linked in the comments for that post is also a blog from a grad student in Holland, who goes by the handle Gravano, doing some work with multitouch as well. He has many pictures and progress updates on his work posted, in addition to helping run a wiki and a forum. The opening post on sums up the contents of the blog fairly well:

In January this year I co-founded, a community called NUIgroup (Natural User Interface) Open Source Multi Touch Community, with a couple of other guys. Since there wasn't alot of information online related to FTIR and DIY multitouch, we decided to centralize all the information of all our individual projects, so other people can learn and contribute to it as well.

Currently I'm working on a multitouch related project with 3 other guys from school. I'm soon going to develop a second multitouch display prototype. I will be posting my developments on this blog, as well as some usability thingies from time to time.

Looks like something worth keeping an eye on.

 
by Brent Gustafson at 3:09 pm 2006-05-05
Filed under:
13 Comments

In my last post about using iPods for gallery tours with Art on Call, I talked about ordering serveral iPod Nano’s that we were going to load up and lend out to the public. I also mentioned how this had its own set of unique problems to overcome. That actually turned out to be a bit of an understatement.

Lending iPods out to patrons is much more involved than just the simple question of how you clean them, or avoiding theft (those items of business are handled by our Visitors Services department). In the New Media world, we care more about answering the question, “how do we make them easy to use?”

Ease of use really comes in two forms. One for the user of the device, and the other for those of us having to update the content on the device itself. When there are budgetary constraints, you’re always looking for the best bang for the buck, while not overly hindering the experience because of it. So what do we do?

The default iPod OS is not good enough.

When you’re using an iPod in a normal sense with MP3s as music, the tags for each song make sense, like artist, album, and genre. When you’re dealing with physical objects, the relationship doesn’t always make sense. Sure, Artist makes senes, but Album? What does that mean to someone looking for audio on “Spoonbridge and Cherry” who doesn’t know who the artist is? Where do they look with the default iPod interface? It’s obvious the iPod interface needs some changes to have it make a bit more sense for museum goers.

iPod Notes (aka “Museum Mode”)

Apple put a feature on the iPod called Notes, which is also referred to as “Museum Mode”. These are files you can put on the iPods data section that point to other files or audio. They’re very simplified HTML and allow you to basically customize the interface. You’re also able to default to “notes mode” so that you don’t see the normal iPod OS choices or are you able to fiddle with the iPod’s settings. Sounds like a great solution right? Well, almost.

One of the biggest problems with Notes is that it breaks the second rule of ease of use. Since they’re just flat HTML text files, creating the directory structure you want takes a lot of hand coding. Notes does not really take advantage of the ID3 tags in the MP3 files. While you can link to a list of songs in genre “X”, you can’t link to a list of all genres and go from there (and if you can, it’s not covered in Apple’s API docs). This means having to rewrite a lot of the basic functionality of the normal iPod through notes mode. It’s time consuming, and every update of audio means an update to the Notes files.

Even if this did work nicely there are other issues. One is that special characters don’t seem to render properly, if at all, in Notes mode, even with the correct text encodings set. The other for me was a real deal breaker. Half the reason to restrict users to Notes mode was to keep people out of the normal iPod OS. However, if you hold the Menu button down for 2 seconds (like you would to go from song info to the main menu in one click), it doesn’t go back to the Notes menu, it goes to the iPod menu! Imagine the confusion of someone holding the menu button down just a tad too long and now staring at a screen that looks nothing like they had seen before! Now we’re back to the same old problems we had in the beginning. So what do we do?

Hack the iPod

Before I tried Notes mode, I had thought about just hacking the iPod firmware to change the menu options in the OS. However, I figured “Museum Mode” would be much easier and then went that route. After my dissatisfaction with that, it was back to researching iPod hacks. This would prove to work well, with only a few drawbacks, and also allowed us to add something that we certainly could not do in Notes mode (at least not easily).

First things first, how do you hack an iPod? It’s actually a bit easier than it sounds, though not without a lot of risks. If you’re not confident with machine code, and don’t follow the directions to the letter you can easily turn your iPod into a 6oz paper weight.

That said, there is a great little program called iPod Wizard, that really helps in the process. The basic idea is you download the latest iPod firmware and load it into iPod Wizard. The Wizard helps you find and change various parts of the firmware, like text, fonts, even graphics. Once your changes are made you save the new firmware and then update your iPod with it. If all goes well you’ve just hacked your iPod and it works great!

The results

In our case we mostly just wanted to change the text. Remember how I said earlier how “Album” doesn’t make much sense in a museum audio tour? What if I changed it to “Artwork”? Then we’d have something people could relate to. We’d just have to make sure each artwork title was mapped into the Album ID3 tag in the MP3 and it would be seamless to the user. Do this for other tags, and you get the following, which is our new iPod interface:

Lets start at the top. “Art on Call” is now the name of the iPod. Next are Artist and Artwork, followed by Location which lists each gallery in the museum, and then Code, which is the Art on Call number code on an artwork. As you can see there are a number of ways to drill down to get to the same content, and it all uses the build in ID3 tagging of the MP3s. Just tag them correctly, upload them and they fall into place. Much easier to make sense of and update!

You may have also noticed “Walker Calendar” in the options. This was what I was talking about earlier in regards to extra features that would be hard or impossible to do in Notes mode. iPod allows you to sync with iCal, and we have already made an iCal feed of our online calendar. So why not put it in the iPod? Here’s what it looks like:

Calendar grid is on the left. The red flags represent days when events take place. Click on a day and get the list of events for that day followed by a detail of the event which is shown in the right image. The great thing about this is because it works with iCal, it’s an auto update. When you plug your iPod in it auto syncs with our calendar feed and updates as we recharge. There’s literally no work to do to add this feature on our part.

Lastly you see an “Information” item on the main menu. This is actually Notes mode. We’ll still use notes for things like info about Art on Call, or perhaps various other info we want to push there.

Downfalls

While this does sound nice there is one main downfall. While you can change the text of the items that appear on the OS, you can’t outright remove some of the options. For example, “Settings” always appears on the main menu, meaning anyone could go in and change them if they wanted. However this was also a possibility in Notes mode as well, given the problems I wrote above.

To try to solve this we used a bit of social engineering that will hopefully help at least keep a few people from mucking around in the options. One was to label Settings to “iPod Settings” to alert people that this is perhaps something they want to avoid. If they do happen to click on it though they’re greated with this message:

Most honest people will recognize the mistake at this point and back out. Those who really feel like fiddling of course can, though resetting the defaults is pretty easy and will be done after each iPod is brought back to the counter after use.

Overall this isn’t the greatest solution in the world, but I feel it works a lot better than the default OS menu or the Museum Mode. Time will tell how patrons feel about it, which is the only real thing that matters in the end. Hopefully we’ve covered our bases.

 
by Brent Gustafson at 1:43 pm 2005-12-22
Filed under:
13 Comments

I’ve recently been working on creating what we’re calling a “docking station” for iPods in the galleries. The idea is that people who bring their iPods to the Walker can dock it at this station and download various audio tours from Art On Call to it. Then they can cruise through the galleries and listen to artists and curators talk about said work on their own iPod. It’s really an alternative means to receiving the same info that AOC has. Plus it saves on your cell phone minutes and the recordings sound much better. Choice is good!

I was able to wrangle an old iMac from the IT dept to create a prototype of the station. It’s just one of those old, slow, first gen iMacs with the CRT monitors. Unlike the speedy new G5 iMacs we have in our lobby for web surfing, there’s no “kiosk” verion of iTunes. On the lobby kiosks we use wKiosk, which basically locks the entire system down for us, and, other than a few bugs in the program, works pretty well in this regard. But for an iPod docking station, we need to use iTunes, and as I’ve found out, what we’re attempting to do is not what iTunes, or the iPod for that matter, were built to do.

In essence I had to start from scratch when building this kiosk, as I couldn’t use any of the tricks I did on the lobby kiosks. I’ve got a lot of things covered at this point. I’ve created a user who has permission only to run iTunes. This means they can’t screw up the system or start launching other programs. That’s good! I’ve also been able to turn off things like the music store or music sharing with the “parental” prefs in iTunes. Double good! However, the bad news is they still have most of the control over iTunes as any normal user would. They can edit most of the prefs, quit the app, or even delete all the tracks in iTunes. Not good.

I somewhat have a solution to this. Quitting the app is ok, because iTunes will relaunch automatically when a user connects their iPod. For the other two, I think the solution is to have a master prefs and library file backed up on the machine. If for some reason someone is sneaking around and changes something (which will happen), all you need to do is revert to the master prefs or library. This requires a bit of baby sitting, to check up on the machine every once and a while to make sure it’s running properly, but this would be the case regardless of the tech or how bulletproof it is.

The thing that actually concerns me more than this though is how iPods dock with iTunes when the machine isn’t the user’s own. I need to test this out more, but so far the results seem to be sporadic. It looks like there are a couple of options as to what happens when an iPod is connected. Either there’s an automatic update of content, meaning the iPod will just download whatever is in the library and fill itself up, or it will be set to manual transfer, where you can drag and drop tracks to the iPod manually.

Automatic downloading is perhaps the most concerning for a few reasons. One is that it doesn’t give people the option to select which audio tour they want, and it just gives them everything, something I’d like to avoid. Again, choice is good! However, perhaps worse, each iPod is tethered to a specific libary (usually the users library on their home machine), and when you connect an iPod to a rogue machine, it gives you an alert saying as much, and asks if you want to delete the contents of the iPod and marry the iPod to the new machine.

That’s not exactly a great idea, especially for a user who’s on a trip from out of town and brought their iPod for things other than museum audio tours. There is however the option to cancel this overwrite, in which case, you can then set the iPod up for manual transfer, but it’s not totally clear how to do this to the user (it’s in the iTunes prefs), and you must unmount and then remount your iPod for this change to take place before you can actually begin the manual transfer (too complicated!).

The best thing that could happen is to somehow force the connection to be manual transfer when people connect their iPods. If it is, people can add whatever they want to the iPod without the worry of overwriting whatever they already have on it (assuming the iPod is not full already). The key is going to be testing this out with many iPods to see what sorts of things work and what do not with this set up. Right now it’s the main thing that’s worrisome.

I suppose that’s to be expected. I doubt anyone has really ever thought about using iTunes with Podcasts as a physical delivery method for audio tours to the public, and it’s obvious they were not created to be used in this way. Thus the quest continues on how to make this work in the most seemless way possible.

If anyone has any comments, questions, or ideas (!), please pass them along, I would love to hear about them.

 

Powered by WordPress