~ ~

JSON for the masses

Ask, and it shall be given you; seek, and ye shall find; knock, and it shall be opened unto you.


Jason and the golden fleece.  
         to essays   

First published @ Searchlores in June 2007 | Version 0.01 | By Nemo


Although search engines often have the data you want, that very same data is not presented in the way you need. For instance search engines show a snapshot of each document on search results, but do not show frequency data about words appearing on those snapshots. In some cases engines' search syntax is not sufficiently versatile. For instance you do not have a way of excluding dynamic documents (those having a '?' in the URL) or of sorting search results by date to see who was the first to have a given idea, even if last modified date is shown on search results... Post processing search results may save the day by presenting / sifting them in a more adequate way. In other cases the information you want is scattered among several services and you need to combine them to do your research.

Thanks to Yahoo! APIs, which offer JSON access with a callback function, and Yahoo! pipes, which offer JSON access with a callback function and can interface with foreign APIs, you can do these tasks browser side using javascript and share those script with others.

src="images/bulletr.gif" Introduction src="images/bulletr.gif" Hello World! src="images/bulletr.gif" Example src="images/bulletr.gif" Security src="images/bulletr.gif"


With the advent of web 2.0, more and more sites offer APIs (Application Programming Interface) to access some services. Those services range from plain vanilla RSS feeds (a sortof of live bookmarks, which sites update when they have new content and recent browsers automatically check) to more sophisticated ones such as Google or Yahoo! APIs. Until the end of 2005 all but one of those APIs (del.icio.us, bought by Yahoo!) required a server side proxy due to XMLHTTPRequest' cross domain hassles, but everything have changed, because Yahoo! now offers JSON (JavaScript Object Notation) access to its APIs and all you need is a not too old javascript enabled browser (IE 5.01+, Netscape 7.2+, Opera 7.54+, Firefox 1.06+ or Safari 1.3+).

Hello World!

Lets see an easy example to get started on JSON wizardry. The main ingredients to do the magic are the following:


With the previous ingredients we can build our tool [Seekers' Oracle], which add two tricks: multiple calls to Yahoo!'s JSON APIs and the remotion of unneeded script tags as we do not want to push memory limits of browsers due to continual use. What this tool do is building a list of words appearing in search results, be it web or images search results, by consulting Yahoo!'s web and image search APIs (in the later case, what images search API is showing you is an image's thumbnail and text from points 1, 3 and 4 as explained here). For those interested in knowing how the tool was made, use viewsource; for the others, just use it.

The purpose of this tool is twofold:

  1. Getting the list of most frequent and unrelated keywords clinging to your search terms. To use the tool in this mode, enter your keywords, one by one. To learn what to do with this, consult my essays Search Engines Anti-Optimization and Yahoo!/AllTheWeb's image search syntax to see examples of how you can easily write powerful queries to quickly get what you want.
  2. Giving a context to your search query. To use the tool on this mode, enter two related keywords and you'll get a list with an hight density of keywords from your context. To learn what to do with this, see my essay Yahoo!/AllTheWeb's image search syntax.


Patient: Doctor, it hurts when I do this way...
Doctor: Then, don't do that way!

JSON and the dynamic script tag have been bad mouthed for its security issues, but those problems are easy to handle:


(c) Nemo 2007    nemo vitam meam regit@yahoo.com    replace white spaces by underscores.

Petit image

(c) III Millennium: [fravia+], all rights reserved, reversed, reviled and revealed