Blood-Powered Multi-touch Sensing

My summer research at the Human Media Lab involves large investigations into multitouch sensing. For large interfaces, this involves the user touching the surface under some sort of illumination while a video camera captures the entire surface, and then filtering is applied to determine the touch points of the finger. There are often issues with large amounts of noise. The particular technique I'm working with is Frustrated Total Internal Reflection, but there are many other ways to do it; Microsoft uses Diffused Illumination as well. For these surfaces, it is also convenient if you can tell the difference between a human body part touching the sensor, or if someone put a mug down on top of it.

Someone brought a Plasma Ball in last week, and it looks like it makes a great multi-touch sensor too:


If you've ever played with a Plasma Ball, you'll know that the strings of energy (I have absolutely no idea what the proper name is) dissipate out from the central sphere evenly, until one touches the edge of the outer glass sphere with a finger or other body part. Then, the point of contact glows bright pink and a much brighter strand attaches that point and the central sphere. This effect only works with objects that are properly grounded, such as a human body, which is even better.

Finally, I was playing with our acrylic surface, and noticed another technique that could work even better, and would also only work body parts: blood! Look at the pair of images below:


The point of contact between my fingers and the plexiglass goes from healthy red to pale white as the pressure makes the blood gush away! With colour filtering, this effect would actually be incredibly easy to pick up, given ideal skin and light conditions. I was not even applying that much pressure. The rings of frosting around my fingers in the second picture are from condensation.

3 Days left in Kingston...

Scientific Results

An excellent quote from a paper I'm reading:

"A final thought before moving on. Science has one methodology, art and design have another. Are we surprised that art and design are remarkable for their creativity and innovation? While we pride our rigorous stance, we also bemoan the lack of design and innovation. Could there be a correlation between methodology and results?"

Greenberg, S. and Buxton, B. 2008. Usability evaluation considered harmful (some of the time). In Proceeding of the Twenty-Sixth Annual SIGCHI Conference on Human Factors in Computing Systems (Florence, Italy, April 05 - 10, 2008). CHI '08.

Wizard of Oz in South Park

In Human-Computer Interaction studies, we often simulate new interfaces in experimental trials before actually putting the effort into making them. Sometimes this simulation is interactive, like Human Media Lab's Display Objects, and sometimes it's a crappy non-interactive piece of styrofoam (like the first prototype for the iPod). Although the device is non-interactive, we can pretend it IS interactive, by either the user imagining that it is, or the designer standing next to the user making various noises or even grabbing the device intermittently and drawing it. When this happens, this is called a Wizard of Oz experiment, from the Wizard of Oz where a men pretends to be the powerful wizard by amplifying his voice and other effects. It has fooled everyone in The Land of Oz for a very long time, until Toto the dog (presumably not as easily fooled by the pyrotechnics) simply pulls aside a curtain.



The point is that simulation works, as long as we do not look behind the curtain. This is a very useful concept, and it is also pretty funny. This came up in a recent episode of South Park, Over-logging, where the internet goes down and panic breaks out. In one particularly hilarious scene, one of the characters, looking for online porn, finds someone that can perform a Wizard of Oz for him. However, it turns out to be disappointing.



The best part is that he automatically says click for the person drawing to hear him.

A conversation with the Google Earth API

Dustin: Oh hey there, Google Earth, it looks like you have an API so that we can actually talk to each other.
GE: Yeah, hello! It's really useful.
Dustin: Well, we'll see about that. So, I've turned you on. Can you tell me what layers you have?
GE: Of course! I've got the Geographic Web, Roads, 3D Buildings, Street View.. [goes on a long list]
Dustin: Oh that's great. So, can you tell me what is in each layer?
GE: ...
Dustin: Hello?
GE: ...
Dustin: Oh. I see. Well if I zoom in, I can see that panoramas from Street View are now rendered as spheres floating in space where the picture was taken. That's really cool! But, I guess you can't tell me where they are. I'm looking at a few now. Which ones are they?
GE: ...
Dustin: Okay. What about this one? 482 Avenue of the Americas in NY, NY.
GE: Oh yes. I've memorized that one now.
Dustin: Okay. What do you see now?
GE: 482 Avenue of the Americas in NY, NY. That's all.
Dustin: Oh. Well, can you memorize that street view locations for 478, 486, 490 too?
GE: Yep! Done.
Dustin: Okay, what do you see?
GE: 478, 482, 486, 490 Avenue of the Americas in NY, NY.
Dustin: So you'll only tell me you see something if I specifically tell you to look for it?
GE: Yes.
Dustin: Well what about that panorama over there? 94 W 12th Street?
GE: Sorry I don't know what you're talking about.
Dustin: Really. Memorize 94 W 12th Street.
GE: Ah, yes. So now I see 478, 482, 486, 490 Avenue of the Americas and 94 W 12th Street in NY, NY.
Dustin: Okay fine, you're young so it's okay. Now, can you tell me in latitude and longitude where 482 Avenue of the Americas is?
GE: ...
Dustin: No? But its right there on the screen. I can click on it to make you fly to it!
GE: Oh, I can fly there!
Dustin: Where is 482 Avenue of the Americas?
GE: ...
Dustin: I said where is 482 Avenue of the Americas?
GE: ...
Dustin: Please fly to 482 Avenue of the Americas.
GE: Done.
Dustin: Okay, where are you now?
GE: [gives a series of latitude and longitude coordinates of where it's looking]
Dustin: I see. So, you can't tell me where 482 Avenue of the Americas is, but I can ask you to fly right to it and then you'll tell me where you are. It seems like you're trying to make my life difficult.
GE: [shrug]
Dustin: Okay, now that I'm sitting at 482 Avenue of the Americas and looking towards the horizon like I'm driving. Cool! So, let's turn to the left to look over there.
GE: [turns slightly to the left]
Dustin: Wait, hold on, I'm not in the sphere anymore. I've moved over to the right of the road somehow.
GE: Yes, I turned you around the Focus point!
Dustin: What? What is that?
GE: It's where you're looking at.
Dustin: I thought I was looking at the panorama at 482 Avenue of the Americas.
GE: Actually, that's what you told me to do, but I decided to fake it by making you look at a point 60 meters to the north at a really shallow angle. You weren't actually looking at the panorama at all.
Dustin: Can I look directly at the panorama at all?
GE: ...
Dustin: ...
GE: When I look at something, its defined by the point on the earth itself, and then the compass angle, vertical tilt, and distance I am from it. The panorama is just floating above the Earth, so you can never truly look at it.
Dustin: But there's little controls I have on you that let me move you around whatever way I want. However, when I actually want to talk to you I can only tell you to move in certain ways.
GE: Yes.
Dustin: Well okay. Let me think. I know! How about, instead of looking at a point 60 meters to the north of the panorama, I tell you to look at a point 60 meters to the northwest, and then make the compass angle so that the panorama is in the center of your view.
GE: Like this?
Dustin: Wow! It's like I rotated 45 degrees around the panorama.
GE: No it isn't. Not exactly. That's not what I did.
Dustin: Shut up, it looks like I did, and that's all that matters.
GE: Okay.
Dustin: So, in order to rotate around a panorama as if I'm viewing it, I need to exactly tell you the panorama I want (I can't ask which panoramas are available). Then, I have to tell you to fly to it. Then, I have to ask you where I am, and the answer you give back to me is some point 60 meters away from the panorama. Then, I have to do some clever trigonometry to figure out where on earth the actual panorama is. Then, in order to rotate around the panorama to look at it from different angles I have to make up different points on the Earth in latitude and longitude which are on the opposite side of the panorama I want to look at. Then I can tell you to fly to those points, which don't explicitly look at the panorama but panorama happens to exactly be in the way of the point on the ground I told you to look at.
GE: ...
Dustin: Right, you're not much help here are you?

Wierd times on the internet

Keywords: advertising, user-generated content.

So I just got an email from youtube about a music video I made a long time ago. Here it is below:

--------------------------------------------------------------------------------------------

YouTube | Broadcast Yourself™

Dear YouTube Member:

UMG has claimed some or all audio content in your video Golden Videotape (part 1). This claim was made as part of the YouTube Content Identification program.

Your video is still live because UMG has authorized the use of this content on YouTube. As long as UMG has a claim on your video, they will receive public statistics about your video, such as number of views. Viewers may also see advertising on your video's page.

Claim Details:

Copyright owner:	UMG
Content claimed:	Some or all of the audio content
Policy:	Allow this content to remain on YouTube. Place advertisements on this video's watch page. Applies to these locations: Everywhere

UMG claimed this content as a part of the YouTube Content Identification program. YouTube allows partners to review YouTube videos for content to which they own the rights. Partners may use our automated video / audio matching system to identify their content, or they may manually review videos.

If you believe that this claim was made in error, or that you are otherwise authorized to use the content at issue, you can dispute this claim with UMG and view other options in the Video ID Matches section of your YouTube account. Please note that YouTube does not mediate copyright disputes between YouTube owners. Learn more about video identification disputes.

Sincerely,
The YouTube Content Identification Team

--------------------------------------------------------------------------------------------

Anyway, way back in high school I made this weird impromptu video with two close friends of mine, and we happened to stick a bunch of Blink 182 tracks over it. I'm way too lazy to look up the proper legal way of ensuring fair use, but we gave credit at the end of the video (part 2). Now it seems that Universal Media Group (UMG) has discovered this. When I first saw this email I got freaked out because I thought I was getting sued, or my account was going to be banned. However, it seems that UMG just asserts the right to "advertise" and, sure enough, I've now got a little advertisement box next to my youtube video's page. This is actually really smart. I don't get sued (which is a waste of their time anyway) and they get to profit of me and my friend's remix of their owned works. I'm not sure what the long-term implications this are, but I think it's a pretty sweet deal for all concerned.

Meta-blogging

My friend and fellow improvisor Monica Heisey interviewed me about blogging for the Queen's Unversity Journal (student-run paper). Link!

Where I work

Here is where I work at the Human Media Lab. I've got a great view overlooking the construction on the Queen's Centre, and I'm sporting two computers, and a bunch of random other techno-gear, which makes me excited.



The giant blue ball under the desk is a prototype I'm working on.

1001 Books to read

Here's a list of great books one is apparently supposed to have read. Sadly, great non-fiction works are omitted. Here's how I add up:

Book - Author (Reason)

Read:
Choke – Chuck Palahniuk (last year, I think)
Life of Pi – Yann Martel (borrowed from Dad two years ago)
Cryptonomicon – Neal Stephenson (read two years ago, must read again)
The Long Dark Teatime of the Soul – Douglas Adams
Dirk Gently’s Holistic Detective Agency – Douglas Adams
The Name of the Rose – Umberto Eco (read after an undergrad class in Semiotics. Great.)
The Hitchhiker’s Guide to the Galaxy – Douglas Adams (all Douglas Adams is excellent. I have read each of three multiple times)
Watchmen – Alan Moore & David Gibbons (A graphic novel. Excellent.)
Breakfast of Champions – Kurt Vonnegut, Jr. (Finished recently. I tried reading when I was 11, but I was too young and gave up.)
Slaughterhouse-five – Kurt Vonnegut, Jr. (in Southeast Asia last summer)
Do Androids Dream of Electric Sheep? – Philip K. Dick (read in an anthology a few years ago)
Stranger in a Strange Land – Robert Heinlein (read a few years ago)
Catch-22 – Joseph Heller (my favourite novel. ever.)
To Kill a Mockingbird – Harper Lee (for high school english)
The Lord of the Rings – J.R.R. Tolkien (Long, but imaginative. My opinion is mixed on whether books should have multiple appendices.)
Lord of the Flies – William Golding (for high school english)
The Catcher in the Rye – J.D. Salinger (during high school, outside class)
A Modest Proposal – Jonathan Swift (not sure why this is on the list, as its an essay. But I have read it)
Robinson Crusoe – Daniel Defoe (during high school, outside class)
Nineteen Eighty-Four – George Orwell (during high school, outside class)
Animal Farm – George Orwell (during high school, outside class)
The Hobbit – J.R.R. Tolkien (I think when I was younger than 12)
Brave New World – Aldous Huxley (during high school, outside class)
The Great Gatsby – F. Scott Fitzgerald (for high school english)
Heart of Darkness – Joseph Conrad Defoe (during high school, outside class)
The Time Machine – H.G. Wells (when I was much younger)
The Adventures of Huckleberry Finn – Mark Twain (in Southeast Asia last summer)
Little Women – Louisa May Alcott (Mom gave it to me?)
Great Expectations – Charles Dickens (for high school english)

29!

In Progress:
Ulysses – James Joyce

Saw as a movie:
Atonement – Ian McEwan (recent)
Trainspotting – Irvine Welsh (twice)
American Psycho – Bret Easton Ellis (on David's recommendation)
The Cider House Rules – John Irving (few years ago)
Schindler’s Ark – Thomas Keneally (motivation for Schindler's List, which I saw on Sunday.)
Fear and Loathing in Las Vegas – Hunter S. Thompson
The Godfather – Mario Puzo
2001: A Space Odyssey – Arthur C. Clarke (I actually really don't like this film. It does nothing new for me and is horribly paced.)
Do Androids Dream of Electric Sheep? – Philip K. Dick (Bladerunner)
Flew Over the Cuckoo’s Nest – Ken Kesey
A Clockwork Orange – Anthony Burgess
Doctor Zhivago – Boris Pasternak (Mom's favourite movie)
Gone With the Wind – Margaret Mitchell (This was playing a few times around the Freeman household in my youth)
Heart of Darkness – Joseph Conrad (As the excellent adaptation Apocalypse Now)
Ben-Hur – Lew Wallace (Dad's favourite movie)
The Hunchback of Notre Dame – Victor Hugo (yay Disney!)

Gave up:
Foucault’s Pendulum – Umberto Eco (A book profoundly stupid in detail. I realized I wasn't gaining by reading it)

Want to read:

Fear and Loathing in Las Vegas – Hunter S. Thompson
The Unbearable Lightness of Being – Milan Kundera
Neuromancer – William Gibson
Trainspotting – Irvine Welsh
Naked Lunch – William Burroughs
The Island of Dr. Moreau – H.G. Wells
The Adventures of Sherlock Holmes – Sir Arthur Conan Doyle
The Picture of Dorian Gray – Oscar Wilde
The Strange Case of Dr. Jekyll and Mr. Hyde – Robert Louis Stevenson
The Brothers Karamazov – Fyodor Dostoevsky
Through the Looking Glass, and What Alice Found There – Lewis Carroll
A Tale of Two Cities – Charles Dickens
Tristram Shandy – Laurence Sterne
Day of the Triffids – John Wyndham
Foundation – Isaac Asimov


Originally taken from John August's blog.

Can you copyright a brain?

I just got back from an interesting (although limited) session on Canadian copyright for artists at Kingston's Artel, which got some mental juices going from something I heard about when I was visiting the Dynamic Graphics Project (where I'm doing my Master's)



There is currently ongoing research in augmenting participant's memory and other cognitive abilities using recording devices, for both Alzheimer's and Autism. From what I have heard, the results of using a recording device is incredibly valuable. In one case where a camera would hang around an Alzheimer's patient's neck and intermittently take pictures throughout the day, memory performance was compared to a control group and the use of a diary. It was found that not only did reviewing the recorded pictures perform better in event recall but that, whereas other memory-recording techniques would fade over time, the use of pictures which were reviewed once ensured that the memories created were permanent. Unfortunately, I can't seem to find the original paper to link to.

Now, let's return to copyright. Recently, there has been an unfortunate movement to ban public photography, frequently on the basis of copyright as a guise to prevent terrorism. While I think that this is clearly ridiculous, not to mention simply impractical, it raises the question of where the information is going (and how can we wrangle it!). I mean, should my friend with photographic memory not be allowed out in public, unless she is equipped with some sort of distraction device so she can't concentrate on creating a "copy" of the surroundings? I think we all can agree that we cannot control the existence of a "copy" in a human brains, except perhaps that which falls under a non-disclosure agreement. This is illustrated humorously above in the ingenious xkcd.

So then, if we cannot copyright brain contents, yet we can copyright pictures, then what about the Alzheimer's patients who could use photography to augment their daily lives. In fact, they would not be able to function as normal human beings without it. You could say, then, that the camera is, in fact, an extension of and therefore is a part of their brain and similar laws should apply. I'm on the patient's side.

Weekend Roundup

This past weekend:

- My best friend Dave came up from Guelph.
- I volunteered for Admitted Students Day at Queen's.
- I saw Iron Man with Dave, and my other friend Daniel and housemate Jack.
- Visited Free Comic Books Day, saw nothing I really liked.
- Watched three famous movies I had not seen before: Shawshank Redemption, Vertigo by Alfred Hitchcock and Amadeus.
- Started playing with some Image Processing stuff on paper.
- Went for a bike ride.
- Had a recording session for the radio play I'm in.

Generally, I had a pretty sweet weekend. Let there be more!