Friday, April 03, 2009

Visual and Context-Cued Semiotic Search Opportunity

Want a hardcore problem to work on? to fund? to stay-up-nights-only-to-see-Google-do-it? or maybe get-bought-by-Google?

We need a search engine that searches based on visual and contextual clues about the appearance of objects -- especially of signs and symbols -- rather than just based on words that (perhaps) describe them.

For example, if I see a bumper sticker around town, with a green star on a blue field, I might want to see if this represents some well known organization or cause. I could search for “green star" and “bumper sticker” or something similar. But I probably won’t find anything.

Moreover, when the elements of the design don’t have names (“star”, “stripe,” “field”), properly describing a complex design in a single search gets difficult. Imagine you saw the new Pepsi logo:

 

You don’t know what it is; for the sake of the argument, imagine you don’t have any cultural Pepsi associations to work from either. What do you type in to the search box? Circle? red? stripe?

Good luck.

How do we solve this problem?

I envision a search that consists of several stages. At the first stage, you can add descriptive words, or you can import a similar image, or even draw/sketch some cues right on the page. That may sound unlikely for less design-oriented folks, but many unknown visual designs consist of largely straight lines, simple geometry, etc. So it’s not unreasonable that I could sketch in a simple design, or even take a swing at the Pepsi logo above, with just a circle and 3 straight lines using an AJAX or Flash inline drawing tool.

From these inputs, the search engine draws a set of possible results – but it also generates a set of context-narrowing options that I can use.

It presents options to choose where I saw this design: web, billboard, tv, clothing, museum, public building (e.g., a capitol or courthouse), manhole cover, etc.

Perhaps knowing material is useful: was this printed? embroidered? leather? denim? engraved metal?

This is a challenging but eminently creatable piece of software.

I’ve actually had a lot of instances where I would have liked to use something like this – but, if it’s never happened to you, consider: when computer vision progresses beyond working with the local environment, objects and known patterns (people), the machine will need to take the next step. It will want to dereference symbols to find data and meanings in order to solve problems. And, in order to do this, it will need benefit from this kind of visual-semiotic search heuristic, which starts with a visual-context search like the one we are discussing.

7 comments:

Anonymous said...

The holidays are a time ed hardy of getting together with friends ed hardy shoes and family, attending elaborate ed hardy clothing parties, and other exciting events ed hardy clothes that involves dressing up in stunning ed hardy store wardrobes. If you ed hardy Bikini are pregnant during ed hardy swimsuits the holidays, it does not ed hardy Caps mean that you are unable buy ed hardy to look fabulous and ed hardy swimwear stylish. Now, an expectant ed hardy sale mother has many styles of chic ed hardy glasses maternity clothing that allows cheap ed hardy her to show off her baby bump Christian audigier while looking spectacular.

Runescape gold said...

I love wow gold, she like buy RS Gold.And then we all like Tera gold, so we are friends now

RS Gold said...

Not long came upon a person's article and also have already been version along. I wish to verbalise the wonder within your written material acquirement and ability to defecate readers study right away towards conclusion. I would really like to learn more sophisticated content and also to part our opinions on hand.

Cheap RS Gold
Sell Runescape Gold

Anonymous said...

The easy fix -- assuming you don't have dependencies dog carrier that mean you need x64 -- is to create a build configuration for your main app that specifies a target platform of x86 (instead of "Any CPU"). The resulting app will run in WoW (the 32-bit shim for 64-bit Windows) and works fine dog boots with these libraries.

Anonymous said...

Way it is noteworthy all around you sunglasses wholesale, additionally, the stylish exploring rarely leave you the only one. Who has a two of awesome fashion accessory drinking glasses, you are the shining an individual who stalls in the center from the fashion world. >

You have eyeglasses frames to land which will people that you're a beautiful an individual who has actually different mastering in regards to the street fashion? We should appear like the far more very popular glasses frames person who has this named flavor involving trendy? In basic terms. The perfect means spectacles is really effective acceptable to help teach all these whizzes.

Robert Welain said...

Check out some more great articles on a topic at this site right now, guys!

Abby Taylor said...

Hello, I would like to thank you for sharing this interesting information. The topic uncovered here will be useful for a lot of people based on semiotic search. I will actually use information mentioned herein my 100 yen sushi house task.