Context is Next

Looking at the photo above, you see a person on a tennis court, wielding a tennis racket and chasing a...lemon. Right?

Wrong. You don’t think it’s a lemon. You know it's a tennis ball.

A computer might not be so perceptive. A computer with the latest image labeling algorithms would have no problem making the following list of objects for the photo above: person, tennis racket, tennis court, lemon.

The only lemon I can imagine on this tennis court is in the water bottle of the line judge.

Computer scientists at UC San Diego and UCLA are looking to give automated image labeling systems a little more common sense. And that common sense comes in the form of context. And they are squeezing some of that common sense out of a little-known widget from Google Labs called Google Sets.

“We think our paper is the first to bring external semantic context to the problem of object recognition,” said computer science professor Serge Belongie from UC San Diego's Jacobs School of Engineering.

Belongie and his students (including Carolina Galleguillos -- the lead singer for the band Audition Lab) are presenting their "lemon blaster" this week at ICCV 2007 – the 11th IEEE International Conference on Computer Vision in Rio de Janeiro. The computer scientists show that the Google Sets can be used to provide external contextual information to automated object identifiers. The context is added in a post-processing step that comes after the image is split up into parts and labeled by a computer.





A full press release will be available here, on the Jacobs School Web site.

A copy of the paper is available here

Check out the write up on the Wired Science blog

Qualcomm Innovator Challenge

The Voracious Savants: (L-R) David Wong, David Swartz, Andrew Smith, Aaron Swartz
photo credit: Chris Morrow


The biggest challenge for one of the four winners of last weekend's QUALCOMM Innovator Challenge came in a surprising moment: AFTER his team had won first prize and $5,000.

David Wong (far left) had to convince his parents to hand over his social security number so he could fill out the necessary tax-related paperwork to get his check. In the end, after a lot of convincing by the project manager at UCSD, the parents decided it was okay to share the sensitive information.

The first prize winners called themselves the Voracious Savants. They envisioned a portable device that lets you watch TV while you do your homework won first prize and $5,000 last night at the QUALCOMM Innovator Challenge. With BookPal you can also take digital notes, page through hundreds of pounds of text books and cruise the Internet. Turn BookPal on its side and you can start typing as if it were a laptop.

The Voracious Savants (majors: mechanical engineering (2), electrical engineering and undeclared) – was one of 17 teams that presented their designs for how you could use QUALCOMM’s new, ultra powerful Snapdragon platform. Three of the four winners were high school friends from Carlsbad.

The event was organized by the Jacobs School’s Corporate Affiliates Program and QUALCOMM.

Second prize and $3,000 went to another all-freshman team: Athena. The four bioengineering majors envisioned a portable electronic device with a screen that rolls up and fits in your pocket when you’re not using it. You unroll it when you want to watch Internet videos or do anything else you can do with today’s PDAs.

Athena had a sweet video in their presentation. I think you need to be a member of facebook to see it...below...here they about to roll-out (literally) their organic light emitting diode screen.

Team Athena rolls out its design concept. photo credit: Chris Morrow

Third prize and $2,000 went to the five seniors (four engineering majors and one Cog Sci major) who make up team Greek Fire. They designed “MediBoard.” This digital clipboard is supposed to help doctors and other health care professionals work together better – both within the same hospital and across the world. MediBoard will also help doctors share their medical expertise and provide better support to doctors working in areas with little or no medical infrastructure.

Which came first, the chicken genome or the egg genome?

Looks kinda like a chicken, right?


I thought so too. This is genomic duplication data from figure 2 from a Nature Genetics paper published online today.


The title of the press release is: "Which came first, the chicken genome or the egg genome?"


But the paper is really about a different "which came first" question facing scientists? and one that -- if answered fully -- will help to explain exactly how we modern humans came to be. And it will give us a whole lot of new insights on genomic disease as well. The human genome is full of duplicated chunks of DNA that have played important roles in evolution, and which are involved in disease. The new research provides tons of new data on which of the copies of thousands of DNA segmental duplications in the human genome are the originals and which are the copies.


“Identifying the original duplications is a prerequisite to understanding what makes the human genome unstable,” said Pavel Pevzner a UCSD computer science professor who modified an algorithmic genome assembly technique in order to deconstruct the mosaics of repeated stretches of DNA and identify the original sequences. “Maybe there is something special about the originals, some clue or insight into what causes this colonization of the human genome,” said Pevzner.

You can read the press release here. If you're a journalist and would like a PDF of the paper, send me an email at dbkane AT ucsd DOT edu


Below if the full figure 2 from the paper: This colorful image (figure 2 in the paper) illustrates the process of ancestral-state determination for one 750-kb duplication block on human chromosome 2p11. In this example, 15 of 16 ancestral loci were accurately predicted by the computational method.

Do you see the chicken? What shapes do you see?

Login for a greater good

When you want to post a comment to a blog or send an email from a relatively new email account, you have to type in a series of distorted letters and numbers (a CAPTCHA) to prove you're a person and not a computer looking to add comment spam to a blog.

What if – instead of wasting your time and energy on typing ‘SGO9DXG’ you could label an image that will help someone who is visually impaired go shopping?

That’s exactly what computer scientists from UC San Diego led by computer science professor Serge Belongie are working on. They are presenting this work at a computer vision conference in October in Rio (ICCV 2007).

The image describes one of the many useful tasks that could be done by people who are proving that they are live humans and not computers trying to put spam comments on a blog. These kinds of labeling tasks are time consuming but important for Belongie's Grozi project -- which is an effort to create a grocery shopping assistant for the visually impaired (2006 press release).

You can download the PDF of the paper here.

Here is the press release.

Parlez-vour francais? Read the story in on the French tech site: L'Atelier

This work is related the work from Luis von Ahn that has been described by the BBC, TechCrunch and others.

"Google for Music"

When engineers put the words "indie" and "Backstreet Boys" in the same sentence in an academic paper, you got to take notice.

"For example, a pre-teen girl might consider a Backstreet Boys song to be ‘touching and powerful’ whereas a dj at an indie radio station may consider it ‘abrasive and pathetic’."

-from Identifying Words that are Musically Meaningful, by David Torres, Doug Turnbull, Luke Barrington and Gert Lanckriet from UC San Diego’s Jacobs School of Engineering.

The UC San Diego electrical engineers and computer scientists used the Backstreet Boys to highlight the subjective nature of music. This subjectivity makes it tricky to teach a computer to label songs with words. Should the computer label a Backstreet Boys song as if it were a pre-teen girl? or a dj at an indie radio station? If you want to find out, check out the paper. It is one of three papers the UCSD group is presenting at a music information retrieval conference (ACM ISMIR) this week in Vienna.

Beyond the Backstreet Boys, the researchers are working on a music search engine that lets you type in words and get acutal songs in return. One of the main things they are reporting at the conference in Vienna is that their online game (it's called Listen Game) is good for collecting data that they need to train their computer system to automatically label songs (with no input from humans, except for the initial training.) Read the full press release here.

They are now working on new versions of the games. The original version, Listen Game, works well, but, as the authors told me, "it was made my a bunch of engineers." (Read: it looks like an old-school video game and it doesn't exactly measure up to today's coolest online games.)

You can check out the old version of Listen Game here.

If you want to get an email (no spam, I promise) when the new version of the game comes out, send an email to gert@ece.ucsd.edu

Papers presented at ISMIR 2007:
A Game-Based Approach for Collecting Semantic Annotations of Music
http://cosmal.ucsd.edu/~gert/papers/MusicGame_short.pdf

Identifying Words that are Musically Meaningful
http://cosmal.ucsd.edu/~gert/papers/MusicVocab_ISMIR07.pdf

A Supervised Approach for Detecting Boundaries in Music using Difference Features and Boosting
http://cosmal.ucsd.edu/~gert/papers/MusicalBoundryDetection.pdf

UCSD at TechCrunch

App2You, a UCSD startup that spun out of the computer science department landed on the TechCrunch 40 list this week. I'm not suprised. Their Web 2.0 app is super cool. It lets you build a database driven Web site without doing any coding. I'm especially fond of this idea since I actually tried to learn all the client-server and Web coding necesseary to do this kind of thing -- back in 2001 -- right before the first Internet bubble burst. Anywhow, I wrote about App2You for the alumni magazine for UCSD's Jacobs School of Engineering. I will probably put up a story on the Jacobs site about them being named to the TechCrunch list...but you heard it here first!

By the way, if you check out the app2you site, the two guys who are on the front cover are actually two of the computer science students who helped UCSD computer science professor Yannis Papakonstantinou develop the core technology behind this company.

Computer Graphics Research featured in Science

Henrik Wann Jensen, a computer science professor at UC San In the news section of the 17 August, 2007 issue of the journal Science.

If you have access to the content behind the Science wall, you can read the story here.

For visual-effects creators, rendering objects like the mist-emitting Pensieve in Harry Potter is no mean feat. Even creating a realistic glass of milk can take computer artists hours of tedious work. But a new image-generating technique may accurately replicate many substances given only the type and amount of their ingredients.

"If we know what it's made of, we can say what it looks like," says computer scientist Henrik Jensen of the University of California, San Diego, who outlined the technique last week at a computer-graphics conference in San Diego.

Jensen, who won a 2004 Academy Award for a novel method of rendering skin that was used to create The Lord of the Rings' Gollum, worked with colleagues from the Technical University of Denmark in Lyngby to broaden a 100-year-old model of optical scattering called Lorenz-Mie theory. The team extended it to include irregularly shaped particles like the constituents of milk, seawater, and other light-absorbing substances.

The technique can also be run in reverse to derive a substance's composition from digital photographs. Commercially, this could make it possible to spot spoiled or contaminated food. A Danish company, Danisco, is interested in using it to check the freshness of milk and ice cream.