TOP-SURF is an image descriptor that combines interest points
with visual words, resulting in a high performance yet compact
descriptor that is designed with a wide range of content-based
image retrieval applications in mind. TOP-SURF offers the
flexibility to vary descriptor size and supports very fast image
matching. In addition to the source code for the visual word
extraction and comparisons, we also provide a high level API and
very large pre-computed codebooks targeting web image content
for both research and teaching purposes.
Authors
Bart Thomee,
Erwin M. Bakker and Michael S. Lew
Licenses
The TOP-SURF descriptor is completely open source, although the libraries it depends
on use different licenses. As the original SURF descriptor is closed source, we
used the open source alternative called OpenSURF,
which is released under the GNU GPL version 3 license. OpenSURF itself is dependent
on OpenCV that is released under the
BSD license. Furthermore we used
FLANN for approximate nearest neighbor matching, which is also released
under the BSD license. Our own code is licensed under the GNU GLP version 3 license,
and also under the Creative Commons Attribution version 3 license. The latter license
simply asks you give us credit*) whenever you use our library. All the
aforementioned open source licenses are compatible with each other.
*) Please cite the following article: B. Thomee, E.M. Bakker, and M.S. Lew, "TOP-SURF: a visual words toolkit", in Proceedings of the 18th ACM International Conference on Multimedia, pp. 1473-1476, Firenze, Italy, 2010.
API
The following functions are accessible through the API:
TopSurf_Initialize
Initialize the library, preparing it for use.
TopSurf_Terminate
Terminate the library, letting it release resources.
TopSurf_LoadDictionary
Tell the library which visual words dictionary you want to use.
TopSurf_CreateDictionary
Create a completely new visual words dictionary. This is particularly useful when
the dictionaries that we have provided are not suitable for the application you
have in mind.
TopSurf_SaveDictionary
Save a newly created dictionary to disk.
TopSurf_ExtractDescriptor
Extract the descriptor of an image, which can be a filename or an RGB pixel array.
The descriptor consists of one or more detected visual words and the locations in
the image where they are detected.
TopSurf_VisualizeDescriptor
Display the locations of the detected visual words of an image. In Figure 1 we show
an example where the visual words are visualized.
TopSurf_CompareDescriptors
Compare two descriptors and return the distance between them, either using cosine
normalized difference or absolute difference. Note that the library does not have
to be initialized for this to be called, because descriptors can be compared without
needing to lookup any visual words in the dictionary. In Figure 2 we show an example
where we compare the descriptors of several images.
TopSurf_LoadDescriptor
Load a descriptor from disk.
TopSurf_SaveDescriptor
Save a descriptor to disk.
TopSurf_ReleaseDescriptor
Release the memory used by a descriptor.
Figure 1. Visualizing the visual words.
Figure 2. Comparing the descriptors of several images using cosine normalized difference, which ranges between 0 (identical) and 1 (completely different). The first image is the original image, the second is the original image but significantly changed in saturation, the third image is the original image but framed with black borders and the fourth image is a completely different one. Using a dictionary of 10,000 words, the distance between the first and second images is 0.42, the distance between the first and the third is 0.64 and the distance between the first and the fourth is 0.98. We have noticed that a (seemingly high) threshold of around 0.80 appears to be able to separate the near-duplicates from the non-duplicates, although this value requires more validation.
Programming environments
Our source code targets both Microsoft Windows and Unix-based platforms, so it should
be able to compile on most systems. The code was originally developed for the Microsoft
Windows platform using Visual Studio 2008, and we have since then extended the compatibility
to also cover the 2010 edition.
Microsoft Visual Studio
If you don't yet have a recent version of the Visual Studio C++ compiler, please follow the next steps.
Downloading the C++ compiler
This has been tested on Windows XP.
- Go to this URL.
- Click on the black link that says "Visual Studio 2010 Express".
- Then, click on the link that says "Visual C++ 2010 Express".
- Select 'English' from the dropbox. This will prompt you to save a file called
"vc_web.exe".
- Run the vc_web.exe file, and accept all default option settings.
- The installer should say that it is installing the "Microsoft Visual C++
2010 Express Edition".
Eclipse
The TOP-SURF source code can now also be compiled using Eclipse, which is an open source programming environment available for Apple OS X, and Linux. If you don't yet have a version of the Eclipse C++ compiler, please follow the next steps.
Downloading the C++ compiler
This has been tested on Apple OS X 10.6, and Ubuntu 10.04.
- Go to this URL.
- Find the "Eclipse IDE for C/C++ Developers" package, and select the
one suitable for your platform.
- On the next page start the download by selecting the mirror closest to you.
- Unpack the archive to a directory of your choice.
- In that directory, double-click the 'Eclipse' icon to launch the programming environment.
Source code
All the libraries the TOP-SURF descriptor depends on have been bundled into a single
project, using their original source code, making compiling an easy task. On our
workstations it is simply a matter of opening the project file from within the programming
environment and hitting the compile button to make it work. If you are using a Visual
Studio edition, download the file marked as 'Visual Studio', otherwise choose 'Eclipse'.
Get the source code: Visual Studio | Eclipse
Please refer to the enclosed README for additional information.
Binaries and examples
As the TOP-SURF library only contains a collection of functions, it requires another
project to actually use these functions. The Visual Studio-based TOP-SURF project
above is configured to compile as a dynamic link library (DLL), and can generate
suitable versions for both 32-bit and 64-bit machines. To obtain these DLLs without
requiring you to compile the source code yourself, click on the link below. We also
provide the dynamic libraries for Apple OS X and Ubuntu, compiled using Eclipse, although
due to the multitude of system configurations and platforms that the source code can
run on, we cannot guarantee they will work on your particular system.
Get the DLLs: Visual Studio | Eclipse
Please refer to the enclosed README for information on how to incorporate the TOP-SURF library into your own project, so you can access its functionality through the API. To demonstrate how the TOP-SURF library can be called from a project when it is compiled as a DLL, please refer to the example project below that showcase how an executable links to the dynamic TOP-SURF library and accesses its functions. Once you compile and run the example project it will extract the descriptors of several test images and compare their descriptors with each other.
DLL example project source code: Visual Studio | Eclipse
The TOP-SURF library does not have to be compiled as a dynamic library, but can also be compiled as a static library (LIB). Additionally, by directly embedding the code in another project, it no longer has to be compiled separately, and can be bundled with the other project as a single executable. We have done this in the example project below. This project produces an executable that can be passed certain options, e.g. to extract the descriptor of an image, or to visualize where in an image the visual words were detected. Please refer to the README for additional information. At present we do not provide the precompiled executables for Apple OS X and Ubuntu.
EXE example project source code:
Visual Studio |
Eclipse
EXE example for 32-bit machines:
Windows Setup |
Windows Zip
EXE example for 64-bit machines:
Windows Setup |
Windows Zip
For easy installation of the example on Windows, download the 'Setup' version, since that will contain all required Microsoft libraries to make the application run. However, you may run into issues when installing the example on machines where you are not the administrator, so in that case please try downloading the 'Zip' version instead. The latter version does not contain any of the Microsoft libraries, making it nice and portable, but this also means that it may not run on your system if you don't have these libraries installed. When you choose the setup version, please make sure to run the setup.exe and not the other file, otherwise the application may not work! Also ensure to download the appropriate 32-bit or 64-bit version for your operating system.
GUI
For a user-friendly approach to using our library, we have created a Microsoft Windows-based
graphical user interface for TOP-SURF, which supports all functionality of the library,
see Figure 3. With our GUI you can load and create dictionaries with ease, and the
interface makes it also straightforward to extract, save and visualize TOP-SURF
descriptors. At the moment we don't yet have an Eclipse version of the GUI.
Figure 3. The TOP-SURF GUI.
Our GUI is also developed using Visual Studio 2008, but using the C# language rather than C++. If you would like to compile the GUI yourself, but do not have Visual Studio yet, please follow the next steps.
Downloading the C# compiler
This has been tested on Windows XP.
- Go to this URL.
- Click on the black link that says "Visual Studio 2010 Express".
- Then, click on the link that says "Visual C# 2010 Express".
- Select 'English' from the dropbox. This will prompt you to save a file called
"vc_web.exe".
- Run the vcs_web.exe file, and accept all default option settings.
- The installer should say that it is installing the "Microsoft Visual C# 2010
Express Edition".
Downloading the TOP-SURF GUI
GUI project source code:
Visual Studio
GUI for 32-bit machines:
Windows Setup |
Windows Zip
GUI for 64-bit machines:
Windows Setup |
Windows Zip
Just like is the case with our executable examples above, we have bundled the GUI into an application installer. Please run the setup.exe and not the other file, because otherwise the GUI might not be properly installed.
Dictionaries
Our visual word dictionaries are rather large, and therefore we have stored each
dictionary in a separate zip file. Please download one or more dictionaries, since
without them the TOP-SURF library cannot be used.
You don't have to download all dictionaries, but only those you want, e.g. the one containing 200,000 visual words. Whenever you use the code/binaries, you only need to inform the API of the particular dictionary you are using (if you look at the api.h file you'll see this is straightforward to do).
Version history
Version 1.4.2 - 11 June 2011
Fixed bug in dictionary creation.
Added note to READMEs for how to deal with paths starting with a tilde on Apple OS X
and Ubuntu.
Version 1.4.1 - 6 June 2011
Improved documentation for compilation with Eclipse.
Added example code for Eclipse.
Added binaries for Apple OS X and Ubuntu.
Version 1.4.0 - 23 April 2011
Replaced the CxImage image processing library with the one present in OpenCV.
Added compatibility with the Eclipse programming environment, so TOP-SURF is now
supported on most platforms.
Version 1.3.2 - 6 March 2011
Added compatibility with Visual Studio 2010 Professional and Visual Studio 2010
Express editions.
Version 1.3.1 - 11 July 2010
Fixed a small issue where SURF descriptors were incorrectly compared. Note that
this has no effect on extracting or matching TOP-SURF descriptors.
Version 1.3.0 - 7 June 2010
Release of the GUI application.
Fixed an issue where the visual words detected in a previous image could linger
around and influence those detected in a subsequent image.
Version 1.2 - 26 May 2010
Fixed memory leak that would only occur during a failed attempt to load a TOP-SURF
descriptor from disk.
More efficient loading/saving of a TOP-SURF descriptor.
Version 1.1 - 21 May 2010
Fixed memory leak that would occur during the extraction of a TOP-SURF descriptor.
Version 1.0 - 7 May 2010
Initial release.
Acknowledgements
Leiden University and NWO BSIK/BRICKS supported this research under grant #642.066.603.
We would like to thank Stephan Gammeter from the Computer Vision Laboratory at ETH
Zürich for his critical suggestions.