TechNayak(mirror)

For quite some time now, I am using social media sites to do my searching. Its not that traditional search results are bad, just that for most of the results that I am trying to get to, social media sites are doing a far better job. Take for example a search on accessibility or cognitive psychology. It’s painful to get through the clutter and get to results that actually pertain to sites that describe accessibility and information on it. But a couple of searches on Digg and Delicious and I have tons of results at my disposal. Traditional web searches work well for certain types of queries, like word lookups, product lookups, news etc. Non trivial query results have a tough time gaining page rank and will usually fail to show up on the results screen. The central point here is that, for certain searches, you just need the wisdom of the crowds.

I know when I look at sites like Digg, Reddit and Delicious people have gone through these links and painfully tagged and saved these links. Which means that with a very high probability its not marketing gibberish or spam. Folksonomy should definitely be given the credit for making life more organized. While searching for certain tags(delicious), I also discover other related tags and then run more filtered searches to improve the relevance of the results. The web, at least for the moment, is said to be partial to content on computers (technology in general, iPhone in particular J ) and for the zillion other domains that the web doesn’t do justice to, traditional ranking methods do little to improve relevance. Human in the loop is definitely better for such queries.

One of the projects that I have worked on is targeting this very need for non trivial searches. Silverfish, is a semantics extraction engine for academic documents and courses. The indexed results and the social aspect of the site are used to update researchers on the latest in their fields of interest and also recommend fresh material. When more and more people start using the internet there will be an increased demand for searches not related to technology and in such cases human in the loop searches will definitely take front stage. Future of search is definitely going to be more interesting.