How does Image-Based search works?

276 views Asked by At

I have been working on a eCommerce project and now I am trying to implement search based on image .I have searched the web for possible solution.I came to know google and yahoo has stopped its support for image search API.I would like to know what needs to be extracted from an image and based on what i need to search in my db.Any suggestions will be helpful.Thanks

1

There are 1 answers

1
Jeffrey On BEST ANSWER

If you want a brute force method, you can calculate the hash of each image, store it in a database and calculate the hash of the file to search for, match it with the database and well... now you found the exact match of the image.

That may be useful in some situations, but in most you would want to find "similar" images. You could extract meta data from the image, like date taken, filename, etc. If you want to search your own fotoalbum, it is likely that images taken around the same time, are near the same location and thus contain the same content.

Google uses an (as far as I know) unknown method to take a part of the image and search using that information. For example: if you split the image in a X by Y grid and calculate the mean colour value, you can search the database for a match (obviously, you'll have to do this for every image and store the result in the database). If you allow a certain difference in value between the search image and database values, you are likely to find another image that is similar. Searching for only a part of the image, in the database, allows you to find pictures that look the same, but are moved.

Microsoft has created photoDNA, a method that finds the "edges" of the objects in the picture, turning it in a black-white image. Than they resize it to a small resolution and calculate a has. Using this method, you can find photo's that are near to the same, but slightly differ. Ideal if you want to find edited images and resized images.

Another method is to calculate the colour spectrum of the image, normalise it and search for that (with small variations) in a database. Than you'll get images that have near to the same colours, yet the content can be entirely different!

Deep learning could also be an option, if you have allot of images of the same object. By training the computer (e.g. with nVidea cuda), you can make the model recognise objects. If you than search with a photo with a dog on it, your result might be other images with a dog on it.

In summary: there are allot of different methods, each and every one has its own strength and weaknesses, but one they have in common: it's not easy to make!