Services

Marieke Guy of JISC (UK) has released the long-anticipated "Good APIs" study, in two parts:

• Report 1: JISC Good APIs Management Report: "a background to API use in the UK HE [Higher Education] sector, the potential benefits of the provision of APIs and the challenges this provision can instigate. The report also reviews potential problems developers can face when using third-party APIs."
• Report 2: API Good Practice: "provides a number of good practice techniques for provision of and consuming APIs. The content of this report is based primarily on feedback provided from the developer community."

What came through strongly for me in this report was ease of use:

"make sure that there is a low barrier to access. The maximum entry requirements should be a login (username and password) which then emails out a link.

"If you need to use a Web API key make it straightforward to use. You should avoid the bottle neck of user authorisation, an overly complex or non-standard authentication process. One option is publish a key that anyone can use to make test API calls so that people can get started straight away. Another is to provide a copy of the service for developers to use that is separate from your production service. You could provide a developer account, developers will need to test your API so try to be amenable."

On the down side, I think the framing/title, "Good APIs", leads to a conceptual confusion between the notions of "API" vs. "Web Service". In some places, the JISC discussion is about the Service -- for example, finding out what users need first. In other places, the report recommendations are clearly about APIs specifically, for example when they talk about method naming conventions. I would argue that, to speak clearly about this domain, one needs to distinguish between these two concepts, and use the terms deliberately. So, a Web Service is a Web-based application oriented towards machine transactions. On the other hand, an API, or Application Programming Interface (not "Program" Interface), is a well-defined method for accessing a Service. (the machine-usability characteristic of APIs is mainly a consequence of this well-definedness). As an analogy, Web Services are like local or long-distance phone service, whereas the API is like a phone jack. A Service can have many APIs, like the various ways one might connect to long-distance phone service. Also, the mere existence of an API does not necessarily mean that the service behind it does something worthwhile, or does it with sufficient quality or reliability to meet your needs. A real, production-quality Service includes not only API(s) but documentation, a service-level agreement, performance benchmarks, and support. Finally, it's worth noting that a Web Service, in the general sense, might be delivered to users via means we wouldn't call APIs: for example, by depositing an updated Linked Data set to a public data repository. The problems with using "API" to mean Web Service include the following, in my opinion:

1. it confuses the wrapper for the package;
2. it tends to a view of Web Services as just simple data services, request-in and data-out -- which is only one type of WS, and often the least valuable.
3. It focuses attention on implementation detail, rather than the user- or business value of the actual service.
4. It hides the fact that a useful service might well, over time, have a number of different APIs implemented for it, as different usages, protocols, and environments emerge.
5. it tends to make people think of Web Services as just a matter of "exposing" some data or functions from an existing application.

Also, speaking of APIs separates us from the dominant technical and standards discussions, and rigorous term definitions, which center around the W3C and its Web Services program

I know, it might seem to be quibbling, to focus on terms. However, I've seen over and over that major issues are often decided in the choice of framing and vocabulary, rather than in the "actual" discussion that follows.

In any case, check out Marike's well-done and nicely consultative report, and learn more about how to make Web Services be, in Coleridge's famous phrase, "companionable forms."

[full version of article A Web Services Taxonomy (PDF 84k)] A Web Service, according to a standard definition, is "a software system designed to support interoperable machine-to-machine interaction over a network." 1 To put it another way, a Web Service is some useful service offered (usually) on the Internet, designed as a sort of building block you can use any way you want. So, for example, Google Maps, a free service that dynamically draws maps of any location and locates addresses, has been used by thousands of people to build new services such as crime-report maps and real-estate listing tools, Another way to wrap your mind around Web Services is to consider a range of well-known ones and what they do. That's what the chart below does, with services such as Paypal, Google, Twitter, and Sabre, the airline-reservations system. (click on chart to see full-size): This chart represents a taxonomy, or classification, of Web Services, constructed by characterizing all services according to two factors:

1. Data quality: from simple/commodity to complex/unique
2. Transaction level: from basic lookup to real-world transaction.

The full version of this article, A Web Services Taxonomy (PDF 84k), defines what is meant by those terms, and discuss representative examples of Services that exhibit varying degrees of these characteristics. Based on this, I suggest that the Services with the most usage, customer value, and/or revenues typically have more complex/unique data, and/or are more transactional. See also the above chart in full size, or the full article (PDF 84k).

xISBN bookmarklets now supports thousands more libraries by integration with Worldcat Registry. The previous xISBN bookmarket supports more than 300 libraries, however, the list was manually maintained and it's challenging to keep these links up-to-date, By ingesting good Registry OPAC information into xISBN bookmarklet, we are able to support thousands more libraries in a more sustainable way. xISBN bookmarklet automatically pulled new information from Worldcat Registry on a monthly basis, so if a library maintains up-to-date information in Registry, its data will be automatically reflected in xISBN bookmarklet. Furthermore, we have also developed mechanisms to validate and improve OPAC linking templates in Registry during the process. The previous xISBN bookmarklet is still maintained in http://xisbn.worldcat.org/liblook/

Written by colleague Joanna White, Product Manager of the WorldCat Registry. Users can use a new Registry Web Service to retrieve an institution record in XML using the OCLC symbol. For example: http://worldcat.org/webservices/registry/lookup/Institutions/oclcSymbol/OCL?serviceLabel=content Additional notes. Q: What happens when no records are found? When no WorldCat Registry record is found for a given OCLC Symbol , the web service returns "0" results in the XML response while the user interface presents options to adjust the search query. Q: Do I need to adjust my service for searches by special characters? Yes. Services that utilize these web services have to accommodate for special characters present in some OCLC symbols, for example symbols such as "A#2". Q: What is included in the returned content? If a user is not authorized, then the "public view" of the XML data is returned to the user. If the user is authorized, then the "authorized" version of the XML data is returned to the user. For example, "authorized" users will see IP Addresses for the record. More information on all the data fields is available with Data Fields Quick Reference at http://www.oclc.org/us/en/registry/support/Registryquickreference.pdf WorldCat Registry detailed search is also available as part of our more general Registry Search web service. This SRU service returns HTML to a web browser but XML to software agents (e.g., curl) Example of SRU search: http://www.worldcat.org/webservices/registry/search/Institutions?version=1.1&operation=searchRetrieve&recordSchema=info%3Arfa%2FrfaRegistry%2FschemaInfos%2FadminData&maximumRecords=10&startRecord=1&resultSetTTL=300&recordPacking=xml&query=local.oclcSymbol+exact+%22OCL%22+not+local.logicalDelete%3D%221%22&x-info-6-deletedRecord= Details about the three WorldCat Registry web services are listed here http://www.worldcat.org/wcpa/content/affiliate/default.jsp

ISBN number is structured and contains four parts of information: Group or country identifier, Publisher identifier, Title identifier and check digits. Because ISBN are allocated by blocks to publishers, all ISBNs in same block should have same publisher information. So it's possible to guess ISBN's publisher information from its structure, however this will only work well when there is a big database of ISBN-prefix and corresponding publisher name. Incidentally, there are over 19 million ISBNs in xISBN database, from this database, we can actually create a rather comprehensive ISBN-prefix database. Inspired by Publisher Name Server project in OCLC Research, in last data release we count 551,528 ISBN publisher-prefixes, and it is not very far away from ISBN agency's directory ( 880,000 prefixes) . From our ISBN prefix table, we add "guessing" service for xISBN, which tries to guess publisher information of any ISBN, even they are not covered by Worldcat, such as following:

http://xisbn.worldcat.org/webservices/xid/isbn/7806281622.js?method=hyphen

returns

{"stat":"ok",
 "list":[{
	"isbn":["7-80628-162-2"],
	"area":"China, People's Republic",
	"publisher":"San Qin chu ban she",
	"city":"Xi'an Shi"}]}

It will be interesting to see how this information can be used. On the other side, the "guessing" service is still approximate, we don't know all ISBN prefixes, and sometimes one ISBN prefix doesn't neatly identify one publisher, so use with caution.

Posted on behalf of my colleagues Joanna White and Xiaoming Liu: WorldCat Registry users now have an option to semi-automatically configure direct links to their catalog for over sixty ILS vendors. Authorized users can provide a sample catalog search link, click a button and receive suggested ISBN, ISSN and OCLC deep links. This guessing service is also available in a stand-alone user-friendly form and as a Web Service provided by xISBN LibLook. The WorldCat Registry provides an increasing number of links (library catalog links, OpenURL resolvers, etc.) to both WorldCat.org and the WorldCat Search API, as well as to external services like LibX. As more libraries use and maintain their WorldCat Registry information, the service can provide better and more accurate links for syndication. We are interested to learn about your experiences and how we can support more and better linking. For more information about the WorldCat Registry see Building the Grid and a short video tutorial on the Registry and how it can help your users connect to your services. Xiaoming Liu, xISBN and Joanna White, WorldCat Registry

xOCLCNUM has a less-used, but I think very useful feature of limiting search scope to a collection, we call it search in library feature. The goal is to limit FRBR expansion to a smaller scope, such as a library or special collection. Thanks for wonderful help from Jeremy York, we recently added hathitrust.org as a collection. This feature is implemented in following way: a request can put an additional parameter "library=hathi" in xOCLCNUM request, the service will only return records which marked as free access in hathitrust.org, when I gather hathitrust data last time, we collected 189,723 free access records with OCLCNUM. Similarly, we have 129,239 free access records from Open Content Alliance, and overall there are 322,629 records about ebooks (mostly free content) in xOCLCNUM service. Besides that, because of the FRBR expansion of xOCLCNUM, when a user requests an OCLCNUM, the service can lookup other OCLCNUM with free access content in same work group, for example, OCLCNUM: 51848364 is a book in copyright, by using the search in a library request, we can tell there are multiple versions of free ebooks about this book, including copies in archive.org and hathitrust.org. I just did a quick calculation, by using the FRBR expansion, 2,446,005 OCLCNUMs can link to a version of ebook. it is still a small percentage of the worldcat database, but I suspect it might be large enough for real world usage, so it would be nice to see this feature get better usage. We have a similar feature in xISBN service, but because most free access books don't have ISBNs, the xOCLCNUM's version might be better for real world usage.

xISBN service used to handle ISBN as a plain number, when a hyphenated ISBN is requested, we normalize it to a plain number internally, and present the response in plain number. However, hyphenated ISBNs carry structure information, and sometimes library OPAC system indexed hyphenated ISBN only. We just deployed a new version of xISBN service with better hyphen support, when requested ISBN is hyphenated, the response ISBNs will be hyphenated as well; we also added additional method of explicitly hyphenating ISBNs, such as: http://xisbn.worldcat.org/webservices/xid/isbn/9780596002817?method=hyphen It is implemented by downloading the ISBN range HTML file, turn it into a lookup table, and when an ISBN should be hyphenated, we use the lookup table to figure out where to add the hyphen. It is trivial to parse the ISBN range HTML file, but it would be great if ISBN agency has a machine-readable format of this file.

We added a few more features in this month's xID deployment, hopefully it could be useful in upcoming WorldCat Hackathon .

• support LCCN query such as: http://xisbn.worldcat.org/webservices/xid/lccn/2004273129?fl=isbn,lccn
• support deleted OCLCNUM (marc019 field) http://xisbn.worldcat.org/webservices/xid/oclcnum/47139964?method=getMetadata In this case OCLCNUM 47139964 was merged into 33100112, and we use a flag "presentOclcnum" to mark present OCLC numbers.
• xISSN project now supports tab-delimited and CSV dissemination http://xissn.worldcat.org/webservices/xid/issn/0036-8075?method=getEditions&format=csv&fl=issn,form,title http://xissn.worldcat.org/webservices/xid/issn/0036-8075?method=getEditions&format=txt&fl=issn,form,title
• start to support php dissemination format in all XID projects http://xisbn.worldcat.org/webservices/xid/isbn/0596002815?method=getEditions&fl=*&format=php

For more information, please check xISBN API, xISSN API, and xOCLCNUM API. Thanks Andrew Nagy and Jonathan Rochkind for valuable suggestions.

From the beginning, the promise of web services generally -- and OCLC Grid Services specifically -- has been that as more services are deployed, the more options there will be to weave them together in interesting, effective, and imaginative ways. Here at OCLC we're already beginning to benefit from that in our own services. For example, we've long known that a key piece of infrastructure for many of our web services would be a registry of all kinds of information related to institutions -- their catalog web address, the address of their OpenURL resolver, plus a pile of other things that would be needed by a variety of services we hoped to build down the road. We implemented this a while back as the WorldCat Registry. Now, with the release of the WorldCat Search API, we are starting to see how various Grid Services can be knitted together to enrich the whole. When the WorldCat Search API returns search results, depending on the response type requested, it can be sending back information pulled from the WorldCat Registry -- such as the link to a library's catalog. Therefore, keeping your institution's information accurate and up-to-date in the WorldCat Registry will populate that information out into a growing set of services in a pain-free, effective, and efficient way. Now isn't that what this should all be about?