keeping it “real” in usabilityland
so yesterday we did usability for our pervasive computing project, which involves an RFID scanner by your front door and a little screen that tells you when you forget things.
That’s a very loose description.
I’m tired.
Anyway, we had two versions of our system for the test - a “working” version with a C#-based front end that showed what items the system saw and generated alerts by talking to the RFID reader using Elvin; and a “simulated” version that was basically a walkthrough of the item adding/editing processes, implemented in Flash.
We did this because we didn’t have time to make all of the C# screens and we didn’t have the resources to hook the Flash up directly to the database. This was fine, but having these two competing versions of our system’s reality made me think a few things.
First off, the “working” system didn’t totally work - the RFID scanner is crappy and has a short range so that you basically have to stand there and hold your items in front of it before it detects them, which totally defeats the purpose of the system working in the background so that you don’t have to think about what you’re carrying. Secondly, there were just the normal bugs and oversights that come with the package of a new system, but in this case that meant that sometimes the wrong screens came up, or nothing happened when a button was clicked, or something unexpected would appear (like at one point it displayed that 2/0 items were detected - perfectly accurate, actually, but not quite what were were looking for…)
All of this is absolutely to be expected, and I’m not trying to slam the CS side of our project in the slightest. The CS half of our team worked admirably, and I was not immune to the “it works!” thrills that we all shared when we saw this thing that we had been talking about all semester actually start to take shape. The working version was just as it should have been for the time we have had to spend on it, and perhaps even better than that.
It is also the case, however, that I found myself thinking a lot of things like “oh, why is it being so picky?!” “why can’t we just tell it to preTend for a freaking minute so that it won’t seem so confusing?!”
the Flash prototype wasn’t perfect either, but I could conceive of ways to make it behave how I wanted to just for the sake of the test, whereas that wasn’t really very easy with the “real” code, and that made me stop and think a bit about the nature of reality (woah, doesn’t Someone sound profound… yeesh). i mean, we were testing the potential of an idea in a certain context, and in that light, which representation of the idea is more real - a system that simulates whatever behavior is necessary to the idea? or a system that follows a strict set of rules that were generated ahead of time based on an incomplete understanding of the idea in context?
now, it’s true there are some things that can’t be simulated, and i am Definitely not saying that there isn’t a place for testing at the implementation level as well. i guess i’m really just saying that we got there too quickly in my pervasive computing project. the pressure to create the final “working” version made us prefer to begin implementation rather than test more simulations, and i wish i had resisted that a bit more. i wish i had done more with the paper prototype i made. i wish i had gone out and done more of the side usability research that i thought up even though my other team members didn’t seem that enthusiastic about it. i wish i had done yesterday’s usability test a month ago entirely in Flash so that my recommendations could have been that much stronger.
but whatever, the pressure to cut those corners is realistic, it was my first time being the only Interaction Design folk on a team, and we ended up doing pretty well in the end as it is. it has been a good experience.
still, i am struck by this idea that, in some cases, simulation really is the best way to get a realistic glimpse of the heart of the system. it’s like saying that, when it comes to design, simulation is sometimes more real than reality.
i think that’s important to take to heart.
it’s easy to take the criticisms that we don’t really Make anything too seriously, just because our contribution tends to come somewhat early in the process. i feel like i hear HCIDers apologize all the time when they run into things that their prototypes can’t do by dismissing them with “oh, it isn’t real” or ” yeah, sorry, it doesn’t work”
and i’m wondering if we should stop…
i’m wondering if we should just announce that the jig is up and make some t-shirts that say:
Ideas Are Real