Having the ability to combine data silos and reduce their contents to a single search box is the Holy Grail of computing, and like the Holy Grail it remains allusive. Google has made huge progress in integrating data from multiple sources for Internet users, however, Google can’t get behind company firewalls and it can only index what it can find. This means only a fraction of the data relevant to you has been indexed by Google or the like. The rest is locked in company databases and storage systems safe and sound from the prying eyes of YOU and Google. This means you still open your email client to search emails. You still go to the internal company blog to find that important message from HR. There is no Holy Grail of Integration that pulls everything together to make it easy for you to find and use or reuse. Or is there?
The current trend in search is to index things using a web crawler that constantly scans a system for updated files and then indexes the content of those files (if it can parse them). This is a fabulous technology for indexing the World Wide Web where standards like HTTP, HTML, and XML exist, but what about inside the Ivory Tower of Corporate America where we live off IMAP and SQL? Something we’ve talked about in the Kablink project to help with this problem is the use of OpenSearch (http://www.opensearch.org/Home) and RSS (http://en.wikipedia.org/wiki/Rss) to bring things together in a meaningful way for users. What if each system in the corporate infrastructure supported an OpenSearch query and could return its results as an RSS feed? Now the steady diet of IMAP and SQL protocols you’ve been using inside the firewall has a standard mechanism for producing query results that can be displayed by numerous tools on your desktop. Nearly everyone has an RSS reader installed on their computer somewhere, and if not probably use an online one like Bloglines. Well, what if you could subscribe to a database table or what if you could run a web search that also integrates results from your internal email system? I think we are at a point where these things are possible and just around the corner. What do you think? Let me know your thoughts on whether you think this will work or what your ideas for a fix are.by