| Frequently Asked Questions
In what areas is research occurring?
We are looking at (1) extensive testing of the different sketch to image matching
algorithms; (2) better color edge detection; (3) fast sketch matching methods;
and (4) object recognition.
Who was involved with this project?
The principal investigator was Michael Lew. The lead for the initial version
of the java sketch interface was Kim Lempinen. Significant enhancements were
made regarding image coding/decoding (a 100:1 decrease in transmitted bytes)
the sketch algorithms and debugging by Kim Lempinen and Michael Lew. Improvements
to the Java client interface, an automatic testing program, and combining neural
networks with the sketch engine are being worked on currently by Kim Lempinen.
The web search engine for the thumbnail database, the object recognition programs,
and the integration with the server side of the image query engine are being
improved by Michael Lew.
How does the sketch matching work?
Sketch matching works on the assumption that the sketches made by people are
similar to the edge maps found by computers. The user sketch is sent from the
user's browser to our server at Leiden University. Then the sketch is compared
to the edge maps from our database based on similarity in shape. Thus the database
images which have edge maps which are most similar to the user sketch are selected.
In order to achieve scalable results for very large image databases, we focussed
on multi-scale approaches. This means that the original image is reduced in
size to smaller or low resolution copies. The actual matching is first performed
on the smaller copies and then the set of best results is refined by using the
higher resolution copies. The final set is given to the user.
How does the object detection/recognition work?
Our approach stems from our work on human face detection in complex backgrounds.
We use information theory to determine which features are the best for classification
and then apply principal component analysis for the detection engine. In addition
we perform color and texture analysis for color images. For instance, we also
use skin detection by color for detecting faces. See the link for Leiden University
below for more details. Note that detection should not be interchanged with
recognition. In the case of human faces, face detection refers to locating where
the human faces are within an image. Face recognition would refer to identifying
who the person is. Face recognition algorithms would be of minimal usefulness
for detecting faces and vice-versa.
What research institutions have similar ongoing research?
The most relevant research is on detecting human faces in complex backgrounds.
Three representative research institutions which have been publishing in the
scientific world are: Leiden University (M. Lew and N. Huijsmans), Carnegie
Mellon University (H. Rowley and T. Kanade), and MIT (K. Sung and T. Poggio).
In particular you can find working WWW pages, demos or executables at
Leiden
University: Information Theory (M. Lew and N. Huijsmans)
Carnegie Mellon University:
Neural Networks (H. Rowley and T. Kanade)
We aren't aware of a web page at MIT for this topic, but numerous technical
reports describe their methods.
A special purpose skin detector was created at
Berkeley (D. Forsyth). However, its not clear if their method can be easily
generalized to detect other objects.
How effective are the sketch and object recognition
methods?
We are currently performing benchmarking all of the methods. The sketch methods
are particularly difficult to evaluate since the ground truth is not always
clear. Suppose the user draws a circle which he intends to be a basketball.
Then suppose the algorithm shows the user a picture of the planet Jupiter. Is
the algorithm wrong? Thus, the problem is that one sketch may have multiple
correct answers.
The object recognition methods are designed to have a low "false alarm" rate.
This means that we would prefer to have the algorithm err on the side of missed
objects as opposed to objects which are misclassified. In other words, the user
is being shown objects which the algorithm is "confident" about. Fewer hits,
but less noise!
Try our prototype Java sketch interface and indexing engine. You must be using
Netscape 3.0 or 4.0
ImageScape Visual WebSearching Demo
|
