webSSearchy - general Web Service AJAX consumer, for MT 3.3
The webSSearchy, the plugin for Movable Type 3.3, enables you to embed AJAX web service consumer into your entries. There are many web service API open, Google, Yahoo, Technorati, and so on. As time passes, more and more web service will be usable. But benefiting these web services was a pie in the sky to normal users, bloggers because it needs some technical knowledge and there has been not so many usable web service consumer tool.
Blog is a very good platform to consume such web services systematically. Movable Type has supported Google search. I think, however, this has not been popular to MT bloggers. Why? First MT is a mainly static publishing platform, although dynamic publishing also can be possible. As a result, using Google Search MT tags and saving an entry will take more building time, Moreover the response of Google's web service is the slowest compared other web service. I guess Google doesn't focus system resource on that. Secondly, the embeded Google search results will be outdated and broken as time passes.
I think that if web services have to be connected with blogs, then its form of connection may be AJAX, decoupling of entry building and consuming web service. The webSSearchy is such a thing.
In previous and the first version, many customizings have to be done in CGI source level, and its customzing only affect system wide, no blog wide. This upgraded 3.3.01 version reinfored full customzing in a level of each blogs, each web service, each service's targets. So this entry has to be a relatively long, but you can use it well by default without knowing customzing parts and just following install steps.
If you are new to webSSearchy plugin, please see previous version's intoduction part shortly.
Screenshots

* Other screenshot is in previous version entry(See above link)
Requirement
- Movable Type 3.3 (It uses MT 3.3's newly Transformer plugin callbacks)
- Some Perl modules (Check it with included webSSearch-check.cgi in installation step)
- Google API key ( Go and get key in http://www.google.com/apis/ )
- Technorati API key ( http://technorati.com/developers/api/ )
Prerequisite
I tried this plugin with non UTF-8 encoding, EUC-KR(Korean) and SHIFT_JIS(Japanese). Firefox, Opera have no problem, but IE6, 7 both broke response characters in that encoding. I don't know exact causes. So if you are using such encoding, this plugin doesn't guarantee you a cross-browsing.
And as you know, Technorati is a world wide meta site, so if you request some queries, the results might have various language's characters. If you are not using UTF-8 encoding, you may have higher possiblity with broken results. The web services are very very familiar with UTF-8 encoding.
Features
- Presently, Google, MT system, Technorati, Yahoo, Naver(for Korean) web service consumer CGI provided.
- Fully customizable per a blog, per a service, per a each service's target.
- Support dynamic publishing
- Adapted Rico's accrodion Javascript menu.
Installation
No thinking, just follow these steps.
0.
Prepate Google and Technorati API keys. No Yhaoo.
1.
Download this plugin and untar it under (mt home) with command like $ tar xvfz webSSearchy-3.3.01.tar.gz
2.
Add AltTemplate search_rss search_rss.tmpl line in (mt home)/mt-config.cgi file.
If you are not Korean, you can delete plugins/alogblog/webSSearchy-naver.cgi and plugins/alogblog/config/webSSearchy-config-system.korean.cgi file.
3.
Check (mt home)/plugins/alogblog/*.cgi's Execute permission, and check write permission of some folders: (mt home)/plugins/alogblog/config, (mt home)/plugins/alogblog/cache, and (mt home)/php/plugins/alogblog.
4.
Check http:/ / your_domain /(mt path) /plugins /alogblog /webSSearchy-check.cgi in your browser.
5.
Go to SYSTEM OVERVIEW's PLUGINS menu page, and then click webSSearchy action link. In there, just enter Google and Technorati API keys in proper fields, and then click "Save".
6.
In the upper breadcrumb menu of current page, select a blog you want to apply webSSearchy, then you can directly go to each blog's webSSearchy configuratin menu.
7.
Are you in a blog's config page? Then just click "Save".
8.
And you can see, in that page, "Create template modules for webSSearchy" button. Use no brain, just click it.
9.
In this step, you'll include 3 modules which was created in step 8 into proper templates.
Include webSSearchy::Javascript modue into Site Javascript Index template, ::CSS into Stylesheet Index template. Hey, what location of those templates? Doesn't care. So the bottom of it? Lastly,insert ::Links module into Individual Entry Archive template. Where? I included somewhere above trackback list. Avoid <div> tags overlapping.
If you finished module inclusion, you would know that you MUST rebuild each templates, site javascript/stylesheet/indiv. templates.
10.
Create a new entry, and enter search queries, one per a line. And apply service and proper target you want into that queries by selecting proper dropdown menu. For example, if you typed query as "AJAX", a final result can be "google.search.AJAX", or "yahoo.news.AJAX". Save the entry, and look your published result entry.
Congratulations!
I wish you succeed in just one time.
Explanation for each steps
Warning: Don't read the following, if you are novice user. You can use webSSearchy well, even if you never know these.
0.
You don't need Yahoo key, because Yahoo uses application key, not personal API key.
2.
This step is optional. If you don't want to make your MT blogs act as a simple Web Service provider, you can skip it. But why not? search_rss.tmpl template display MT's search result page with RSS format.
** So for example, if you want to control the length of Descrition field of search result, you can edit (mt home)/search_templates/search_rss.tmpl file, which is made from MT template tag, using trim_to="N" attribute.
3.
In plugins/alogblog/cache/ folder, each search results are stored. ** If you update some webSSearchy input and want to show its applied results at one, you should delete its cache file. Cache is created per a blog and per a each web service. One example is google_1.db file. This cache file is for a blog whose ID is 1, and its Google search results.
In plugins/alogblog/config/ folder, each blog's configuration settings are stored. This config file is plain text file, so you can open and edit if you can. Of course modifying by hands is a little dangerous. I choose this plain config text format because it's easy for other developers to add their own CGI.
In php/plugins/alogblog folder, a part of configuration settings is stored for a PHP's dynamic publishing.
You can simply give these folders a full permission 777, because these folder's file contents have no private data, and your API keys are protected by CGI mechanism.
4.
webSSearchy-check.cgi checks all required Perl modules and cache file write permission. The webSSearchy uses cache mechanism. Search results from web service are not frequently changed within one or two days, so it caches search results in a file DB for default 3 days(I call it cache-life).
If some modules are missing, consult your administrator or install that by yourself. I think all required Perl module are very basic.
5.
There are two levels of configuration in webSSearchy. These are system level and blog level. I provide default system level config file. But each blog's configs are not yet existing at first. When a blog has no saved config yet, if you open that blog's webSSearchy config page, all of its fields will be filled with system's corresponding settings. Once you save a blog's config, then system level config can't affect blog level in any way.
Whole purpose of system level config menu page is to provide default settings to existing blogs that have no saved config, and to newly created blogs.
So currently if you have muti blogs and use one API key, then it is easy for you to first enter that key into system level config menu.
6.
As I said above, setting config in system level is just for convenience of each blogs. The fact you have to remember is that you MUST save each blog's config file by using blog's webSSearchy config menu page. Each webSSearchy CGIs only read blog level config.
So go to a blog's webSSearchy config menu page. In normal environment, you can find its link in MT's blog page and template listing page. If you are already in webSSearchy config page, then you can jump to other blog's or system level config by selecting dropdown menu.
7.
Before these step, because we saved it in system level, current blog config's fields also have the same value. So just "Save". If you save a blog's config, a fiile called webSSearchy-config-blog_N.cgi is created in plugins/alogblog/config/ folder, where N is blog's ID.
8.
For webSSearchy to function rightly, you need some AJAX javascripts, CSS, MT template tags. If you click "Create template modules for webSSearchy" button, these required things are created in a form of MT template modules, named "webSSearchy::Javascript", "webSSearchy::CSS", "webSSearchy::Links".
9.
You can include those created MT modules into templates by using MT syntax like <$MTInclude module="webSSearchy::CSS"$>. Of course if you want, you can directly block-copy those module contents into corresponding target templates. If you use module form, don't forget to rebuild target templates.
10.
Dont confuse with webSSearchy search query and Tag or Keyword. Tag or Keyword are generally simple word. More complicate tag is, less possiblity to act as tag. But webSSearchy query is in contrast to tag. More simple query is, less possiblity to act as effective search.
The webSSearchy has its own terms.
"service" means each web service provider name, google, technorati, yahoo, mt. "mt" does not mean Six Aprat's web service. They don't provide such thing. It means your own MT system's search feature.
Each web service provides several sub-services. For example, Google web service provides "search", "cache", and "spelling" sub-services. Technorati provides "search", and "tag" sub-services and so on. Yahoo web service provides "image", "video", "news", and so on. These sub-services have generally different request protocol. So we call it as "target".
"google" service has a "search" target. "technorati" service has "search" and "tag" targets. "yahoo" service provides "image", "video", "news", "shopping", "search", "travel" targets.
Service and target names are lower case, and shouldn't be changed. Theses are for internal scripting. MT template tags <$MTWebSSearchyService$> and <$MTWebSSearchyTarget$>output these corresponding values. For example, you can use these tags for CSS id and class name.
Each service and target has its own alias name. You can set it in blog's config. These are solely for displaying purpose. If you are non English user, you can use it as localizing purpose.
In entry's webSSearchy query field, you can type a query term like servie_name(dot)target_name(dot)query by hands. But this process is routinary, so you can use service/target selection dropdown menu. First drag a query and select proper service/target menu. Then it automatically make you a proper query term. One query term per a line. Blank line is allowed.
Full customization
Warning again: You don't need to know the following right now.
1. Configuration fields
Cache life: For example, 60 and 60s means 60 seconds. 1m means 1 minute. 1h means 1 hour. 1d means 1 day. You can't mix its format like 1d15m. If some visitor click webSSearchy link, then its CGI first check its cache. If that query's result is in cache and its age is less than this "Cache life" value, then cached result is responded to visitors. If the age of cache data is older than cache life value, then CGI get new result from web service and reponse with that data. Default is "3d", 3 days. General search engine like Google and Yahoo's search result might not be changealbe in a short time, so we can increase this to 6d. However Technorati's tag search results may be different with Yahoo/Google. Then we can shorten it for Technorati.
Number of cache record: Each blog and each service has its own cache. For example, a blog whose ID is 3 can have google_3.db, yahoo_3.db, technorati_3.db cache files. Each cache file limits the record number. If cache record is larger than this value, old result record will be deleted.
Label / URL of help link: These are for entry's webSSearchy help link. Default URL is to each web service's reference page. If a web service provider has a localized help page, then you can change it.
No result message: When there is no search result, this will be shown to a webSSearchy link clicker.
Web Service side error message: When an error occurs between your blog server and web service provider, this message will be shown.
Service Alias: Service display name or localized name.
Service API Key: Some service provider need personal API key.
Blogs to exclude(specific to mt): It is comma seperated blog IDs. If your MT instance has multi blogs, you can control searchable blogs.
Blogs to include(specific to mt): 0 means that only current blog is to be searched. Of course you can explicitly current blog ID. One example is "1,2,5"
Max results per a blog(specific to mt): MT's native search CGI provides max search results per a blog. Default is 5.
Number of days to search(specific to mt): Default is 1 year, ie 365. Its a time limit.
2. webSSearchy template
Each blog can use serveral service. Each service has several its targets. Each target has its own webSSearchy template. Why do I call it as webSSearchy template instead of MT template? Because this template is proccessed by ONLY webSSearchy CGI, not by MT system. This means you CAN'T use MT template tags in here. Don't confuse with it.
In this template, you can use only HTML tags, text, <webSSearchy::*****> form tags. You can customize search results display for a each service's target using each template.
Each target provides different search result fields. For example, Google's search target provides URL, snippet, title, summary fields for each result records. So you can only only use that related webSSearchy tags like <webSSearchy::URL> in template.
Moreover, you can use <webSSearchy::EVENODD> tag. First result item gets 'webSSearchy_odd', second item gets 'webSSearchy_even'... You could use it in CSS selector.
In <webSSearchy::LOOP n="N"> loop tag, you can customize number of result N, max/default 10.
3. AJAX Javascript
When a visitor click a webSSearchy link, then it initiate AJAX Javascript routine. Javascript request to related webSSearchy CGI and wait a response from CGI for TIMEOUT period, temporarily displaying progress image to users. If search response is not arrived for that time, Javascript display "Timeout" message.
You can customize Timout mili seconds, Timeout message, progressing image, and so on in "webSSearchy::Javascript" module per a blog.
4. webSSearchy result displaying CSS
You can apply CSS rules in "webSSearchy::CSS" module. To be frank, my CSS skill is not good. So default CSS and target template id/class structure also are not best or optimized well.
CSS rules are dependent on document's DIV's class/id structure. Of course you can completely change it in each target's webSSearchy template and CSS.
5. webSSearchy MT template tags
Yes, one module is left. It's "webSSearchy::Links" module. If you want to use webSSearchy in individual entry archive page, anyway you will know you have to embed related MT template tags. Before, we just included it in a form of MT module. Of course you can customized it to your taste.
<MTWebSSearchys>
<MTWebSSearchyLink>
</MTWebSSearchys>
</div>
</div>
<MTIfWebSSearchy> is a conditional tag. Some entries have webSSearchy queries, but others will don't. So for such an entries, you should wrap core tags with <MTIfWebSSearchy> tag. If an entry has no query, then that entry will pass through that tag blocks.
Search results are generally a multiple of same form of fields. It's like with <MTEntries><MTEntryID>..</MTEntirs> structure. So you should use <MTWebSSearchys> loop tag.
<MTWebSSearchyLink> is a kind of utility tag. It is a combination of ohter basic tag. Only if you follow some rules to trigger AJAX request, you can customize webSSearchy link display.
By default, <MTWebSSearchyLink> is the exactly same with the following tag combination:
<a href='#' class='webSSearchy <$MTWebSSearchyService$> <$MTWebSSearchyTarget$> e<$MTEntryID$>'><$MTWebSSearchyTargetAlias$>::<$MTWebSSearchyQuery$></a>
You may know why I provided simple <MTWebSSearchyLink> utility tag. One example link output will be as follows: <a href='#' class='webSSearchy yahoo news e12'>News::AJAX</a>
Keep "class" attribute tags above, because AJAX JS routine uses it for some purpose. You may wonder why entry ID part is in there. General webSSearchy usage situation is in Individual entry archive. However, although I wouldn't explain it, you can embed webSSearchy links in other archives, ie. a situation where one HTML page contains many entries. For that, I implemented entry ID parts.
In <a ...>Link Label</a>, Link Label part always have to embed <MTWebSSearchyQuery>. Is it right? webSSearchy CGI gets a query from "Link Label" by using following method. CGI find "colon,colon(::)" in Link Label. So CGI considers the right side of it as query. If can't find (::), then Link Label itself is query.
As a result, following tag combination is valid for Link Label: (Using simple notation)
<a ...>Query</a>, <a ...>TargetAlias::Query</a>(It's a default), <a ...>ServiceAlias::TargetAlias::Query</a>, <a ...>Hey, do you want more?::Query</a>
Lastly, <div id="webSSearchyResult<$MTEntryID$>" class="webSSearchyResult"> part is a destination output area. After getting results from CGI, AJAX JS routine transfers results into DIV with ID is webSSearchyResult123.
So it is also possible for you to display search results to other postion, if you dont use default <MTWebSSearchyLink> and design MT tags properly. But I think it's not good.
6. Localization
Refer plugins/alogblog/lib/alogblog/webSSearchy/L10N/ko.pm file if you want to provide your own language file.
7. ETC
I included Six Apart's favicon in compressed file for a MT search display. You or your authors may confuse it with sixapart.com's web service, althought 6A doesn't provide such things. So I recommand you to replace mt-static/plugins/alogblog/images/mt.gif file with your site's favicon image. When doing that, your site's image name should alway be mt.gif. If your favicon is ICO format, convert it into GIF format. The webSSearchy only search exact mt.gif file for displaying.
License
Relased under the Creative Commons License.Version History
- 3.3.01: for MT 3.3 version, Technorati CGI supported.
- 3.2.01 : PHP version of Google API included.
- 3.2.01 : First release. Google, MT system, Yahoo, Naver(for Korean) supported
Comments
I cant make this mod work.
I want to have results similiar to your individual entry page. I followed 1-10 steps, such as:
Filled in API;
File webSSearchy-config-blog_ID.cgi has been saved;
Content of "webSSearchy::Javascript" has been manually put into mt-site.js;
Content of "webSSearchy::CSS" has been manually put into the orignial css file;
In Individual Entry Archive, what exactly code should I put into it? I tried the example code, but after rebuilding the entry, I can only see those image and links (such as: Yahoo...google...), but when I click the links, there is no search results.
Anything else am I missing? Should I do anything in blog level webSSearchy.cgi? such as editing "Target: Target Alias"?
Thanks!
I think errors might be from two sources. One is from missing customization, the other is from Perl's module problem. From your comment only, I have a difficulty in finding that source.
Did you look into blog/system Activity Log? It may have some erorr msgs. Or if you can see your web server's error log, it will very helpful.
When a visitor clicking webSSearchy link(eg. Google link), it triggers JS routine.(So if the page doesn'nt find that JS routine, anyrhing won''t happen, except some browser's JS error msg. . Plus it, browser could block your JS routine ?)
And then that JS routine calls related webSSearchy CGI. That CGI requests Google Web Service, and save its results into cache file.
So you can check plugins/alogblog/cache/ folder after clicking webSSearchy link.
To indiv. archive, you only have to insert that created webSSearchy::Links module.
Anyway, do you mind if I see your blog's published page? If you have some privacy problems, you can let me know it by email(alogblog dot com at gmail dot com)
Posted by: Chris | August 9, 2006 09:42 AM
Hello,
I think that I have seen this plugin in use on another site and I was very impressed. I have not however been able to get it to work on my site.
I am not able to save the config changes from the MT Admin. The error I receive is as follows:
File Manager gave the error: Opening local file 'D:\inetpub\tedoneill\cgi-bin\mt\php\plugins\alogblog\webSSearchy-config-blog_1.inc' failed: Permission denied . at D:\inetpub\tedoneill\cgi-bin\mt\plugins\alogblog\lib/webSSearchy.pm line 90.
Any help would be appreciated.
Thanks.
I think it is permission problem of directory/file creation (mt\php\plugins\**alogblog** folder).
First check the existence of php\plugins\alogblog folder.
If not exists, create alogblog folder and grant write permission.
If you save configs in MT admin's config menu, this plugin saves that config to php/plugins/alogblog/some_config_file.
Althoguh I'm not good at MS IIE web server, that folder should have 's writable permission.
If you have any problem, dont hesitate to ask help.
Good luck.
Posted by: Ted | September 10, 2006 11:50 AM