Monday, September 01, 2008

Web Service Discovery using Search engines

Few days back Kaushalye and I had a chat about features of WSO2 registry as well as supported standards. One of the reason behind the discussion was that he is doing a research on UDDI and nowadays registry like WSO2 registry. One of the issues with nowadays registry is that they do not follow an open standards , so that causes to number of issues when it come to federation and interoperability. If we just think about something like UDDI registry they are inteoperable and follow a common standards. Yes , I agree UDDI is somew hat complex and because of that not many people use UDDI in there applications. That is one of the main reason behind WSO2 registry , Mule Galaxy etc..

On the other hand if we take WSO2 registry for example , though it does not follow any particular standard for a SOA registry , which provides a way to interact with others. Which is ATOM or APP, anyone can build an application to interact with multiple WSO2 registries. So we can solve the problem of federation issues using APP. However that only among a set of WSO2 registry instances , then the problem is how can we federate and service discovery among registries from different vendors.

Let's forget about all those and focus on what I need to discuss here. The idea behind SOA registry is to

  • Register SOA artifacts like Web services
  • Mange artifacts
  • Discover them
  • Invoke them

If you look at UDDI registry , that exactly what it does.

When I read some of the articles and papers in the internet (Thank Kaushalye for the links) , I found most of the commonly used search engine (Google , Yahoo etc) can be used to Web service discovery. Those search engines knows how to talk to UDDI registries and get the data from there. I too agree that it is good if we can use search engines for Web service discovery purposes ,rather than building new applications for that .

If we look at most of the registries , they are isolated, meaning no connection with each other. So applications like Google can not find them. Then it is very difficult to do the service discovery. Therefore it is always good idea to have something like central registry. Then search engines can communicate with them and do the Web service discovery.

Therefore I think it is good if we can come up with open standard for SOA registry (of cource which should be which is much simpler than UDDI ) . And then build Crawler to talk to those registries and do the service discovery.


Tammo said...


crawling for WSDLs is one thing, discovery and selection is another, and a very tricky one. Sure, it depends on the lookup criteria, but I don't think that ratings, tags etc. are sufficient in a Web services world. You would need taxonomies to classify what services are about. You'd need means to define what data types (XSDs, messages) deal with (i.e. is that number a Purchase Order number or an ISBN) and to define (or at least classify) the operational semantics of operations. For instance hire and fire may take the same input parameters and produce the same output but do completely different things. How do you distinguish them? That's all knowledge that is not (yet) available in WSDLs, and even with the given set of information provided by WSDL, Google will not be able to understand what the WSDL is about.

Here are some links to enter the Semantic Web services world, I believe this might be helpful to improve today's WS registries (nevertheless I prefer the lightweight semantic approaches like SAWSDL, WSML-lite, etc., mostly because its very costly to add all these annotations...):

Lahiru Sandakith said...

Yeah Agreed.. , now we are down to the point of what will be the best fit of formal simple specification for a service/software component..

Are we going for maths ... Commons Notations .. XML .. Some body need to come up with one and Every Body need to agree ..