David Davies' Radio Weblog

Yesterday I talked about the problem with RSS overload. Today it's search engines.

What are some of he problems with search engines and searching databases in general?

I'm glad you asked. Things I don't like:

1. Each search engine has its own query syntax
2. Each search engine returns results in a slightly different HTML format
3. You have to search each engine via its own web form. OK, some site offer to search a number of engines for you through a single interface but problems 1. And 2. Still apply.
4. I can't incorporate search results into my own content management system. Don't even mention page scrapes. What's the point.

What do all search engines do?

They search their own databases for internet resources that match a keyword you specify. They return the resulting 'hit's as an HTML page of resources, usually pages, comprising title, link and a brief description of the page culled from the page itself.

That result format, remind you of anything?

What if search engines returned their results as an RSS file. It's be structured in a way you could expect and rely upon so that you could incorporate the resulting data into whatever system you liked. Repurpose it using your own content management system for example.

Why limit this critique to internet search engines?

What if all database searches returned results in this way? What if you could bolt together search interfaces in a plug-and-play kind of way to any database and have the results all returned, aggregated even, in a consistent portable format that can be incorporated into content management systems? You wouldn't have to worry about what platform they were running on, what database was being used at the back end, what query structure they used and more importantly, what format the results would come back as.

Show us the way David, what are you on about?

Here's how I've adapted my RSS aggregator to search for teaching and learning resources. These resources, or rather their descriptors and place holders, held in databases that search engines couldn't index, interoperate with whatever content management system you're using just so long as it's XML-friendly.

http://medweb5.bham.ac.uk/databases/interop/mcqs

More later.

Tidied up the RSS feeds aggregator demo for Manila. You can now add your own feed to the aggregator.