OpenSRF is a message routing network that offers scalability and failover support for individual services and entire servers with minimal development and deployment overhead. You can use OpenSRF to build loosely-coupled applications that can be deployed on a single server or on clusters of geographically distributed servers using the same code and minimal configuration changes. Although copyright statements on some of the OpenSRF code date back to Mike Rylander’s original explorations in 2000, Evergreen was the first major application to be developed with, and to take full advantage of, the OpenSRF architecture starting in 2004. The first official release of OpenSRF was 0.1 in February 2005 (http://evergreen-ils.org/blog/?p=21), but OpenSRF’s development continues a steady pace of enhancement and refinement, with the release of 1.0.0 in October 2008 and the most recent release of 1.2.2 in February 2010.
OpenSRF is a distinct break from the architectural approach used by previous library systems and has more in common with modern Web applications. The traditional "scale-up" approach to serve more transactions is to purchase a server with more CPUs and more RAM, possibly splitting the load between a Web server, a database server, and a business logic server. Evergreen, however, is built on the Open Service Request Framework (OpenSRF) architecture, which firmly embraces the "scale-out" approach of spreading transaction load over cheap commodity servers. The initial GPLS PINES hardware cluster, while certainly impressive, may have offered the misleading impression that Evergreen is complex and requires a lot of hardware to run.
This article hopes to correct any such lingering impression by demonstrating that OpenSRF itself is an extremely simple architecture on which one can easily build applications of many kinds – not just library applications – and that you can use a number of different languages to call and implement OpenSRF methods with a minimal learning curve. With an application built on OpenSRF, when you identify a bottleneck in your application’s business logic layer, you can adjust the number of the processes serving that particular bottleneck on each of your servers; or if the problem is that your service is resource-hungry, you could add an inexpensive server to your cluster and dedicate it to running that resource-hungry service.
If you need to develop an entirely new OpenSRF service, you can choose from a number of different languages in which to implement that service. OpenSRF client language bindings have been written for C, Java, JavaScript, Perl, and Python, and server language bindings have been written for C, Perl, and Python. This article uses Perl examples as a lowest common denominator programming language. Writing an OpenSRF binding for another language is a relatively small task if that language offers libraries that support the core technologies on which OpenSRF depends:
Unfortunately, the OpenSRF reference documentation, although augmented by the OpenSRF glossary, blog posts like the description of OpenSRF and Jabber, and even this article, is not a sufficient substitute for a complete specification on which one could implement a language binding. The recommended option for would-be developers of another language binding is to use the Python implementation as the cleanest basis for a port to another language.
The XMPP messaging service underpins OpenSRF, requiring an XMPP server such
as ejabberd. When you start OpenSRF, the first XMPP
clients that connect to the XMPP server are the OpenSRF public and private
routers. OpenSRF routers maintain a list of available services and connect
clients to available services. When an OpenSRF service starts, it establishes a
connection to the XMPP server and registers itself with the private router. The
OpenSRF configuration contains a list of public OpenSRF services, each of which
must also register with the public router. Services and clients connect to the
XMPP server using a single set of XMPP client credentials (for example,
opensrf@private.localhost
), but use XMPP resource identifiers to
differentiate themselves in the Jabber ID (JID) for each connection. For
example, the JID for a copy of the opensrf.simple-text
service with process
ID 6285
that has connected to the private.localhost
domain using the
opensrf
XMPP client credentials could be
opensrf@private.localhost/opensrf.simple-text_drone_at_localhost_6285
.
Any OpenSRF service registered with the public router is accessible via the OpenSRF HTTP Translator. The OpenSRF HTTP Translator implements the OpenSRF-over-HTTP proposed specification as an Apache module that translates HTTP requests into OpenSRF requests and returns OpenSRF results as HTTP results to the initiating HTTP client.
Issuing an HTTP POST request to an OpenSRF method via the OpenSRF HTTP Translator.
# curl request broken up over multiple lines for legibility curl -H "X-OpenSRF-service: opensrf.simple-text" \ # --data 'osrf-msg=[ \ # {"__c":"osrfMessage","__p":{"threadTrace":0,"locale":"en-CA", \ # "type":"REQUEST","payload": {"__c":"osrfMethod","__p": \ {"method":"opensrf.simple-text.reverse","params":["foobar"]} \ }} \ }]' \ http://localhost/osrf-http-translator \ #
The X-OpenSRF-service header identifies the OpenSRF service of interest.
| |
The POST request consists of a single parameter, the osrf-msg value,
which contains a JSON array.
| |
The first object is an OpenSRF message (
| |
The URL on which the OpenSRF HTTP translator is listening,
/osrf-http-translator is the default location in the Apache example
configuration files shipped with the OpenSRF source, but this is configurable.
|
Results from an HTTP POST request to an OpenSRF method via the OpenSRF HTTP Translator.
# HTTP response broken up over multiple lines for legibility [{"__c":"osrfMessage","__p": \ # {"threadTrace":0, "payload": \ # {"__c":"osrfResult","__p": \ # {"status":"OK","content":"raboof","statusCode":200} \ # },"type":"RESULT","locale":"en-CA" \ # } }, {"__c":"osrfMessage","__p": \ # {"threadTrace":0,"payload": \ # {"__c":"osrfConnectStatus","__p": \ # {"status":"Request Complete","statusCode":205} \ # },"type":"STATUS","locale":"en-CA" \ # } }]
The OpenSRF HTTP Translator returns an array of JSON objects in its
response. Each object in the response is an OpenSRF message
("__c":"osrfMessage" ) with a collection of response parameters ("__p": ).
| |
The OpenSRF message identifier ("threadTrace":0 ) confirms that this
message is in response to the request matching the same identifier.
| |
The message includes a payload JSON object ("payload": ) with an OpenSRF
result for the request ("__c":"osrfResult" ).
| |
The result includes a status indicator string ("status":"OK" ), the content
of the result response - in this case, a single string "raboof"
("content":"raboof" ) - and an integer status code for the request
("statusCode":200 ).
| |
The message also includes the message type ("type":"RESULT" ) and the
message locale ("locale":"en-CA" ).
| |
The second message in the set of results from the response. | |
Again, the message identifier confirms that this message is in response to a particular request. | |
The payload of the message denotes that this message is an
OpenSRF connection status message ("__c":"osrfConnectStatus" ), with some
information about the particular OpenSRF connection that was used for this
request.
| |
The response parameters for an OpenSRF connection status message include a
verbose status ("status":"Request Complete" ) and an integer status code for
the connection status (`"statusCode":205).
| |
The message also includes the message type ("type":"RESULT" ) and the
message locale ("locale":"en-CA" ).
|
Before adding a new public OpenSRF service, ensure that it does
not introduce privilege escalation or unchecked access to data. For example,
the Evergreen open-ils.cstore
private service is an object-relational mapper
that provides read and write access to the entire Evergreen database, so it
would be catastrophic to expose that service publicly. In comparison, the
Evergreen open-ils.pcrud
public service offers the same functionality as
open-ils.cstore
to any connected HTTP client or OpenSRF client, but the
additional authentication and authorization layer in open-ils.pcrud
prevents
unchecked access to Evergreen’s data.
OpenSRF supports both stateless and stateful connections. When an OpenSRF
client issues a REQUEST
message in a stateless connection, the router
forwards the request to the next available service and the service returns the
result directly to the client.
REQUEST flow in a stateless connection.
When an OpenSRF client issues a CONNECT
message to create a stateful connection, the
router returns the Jabber ID of the next available service to the client so
that the client can issue one or more REQUEST
message directly to that
particular service and the service will return corresponding RESULT
messages
directly to the client. Until the client issues a DISCONNECT
message, that
particular service is only available to the requesting client. Stateful connections
are useful for clients that need to make many requests from a particular service,
as it avoids the intermediary step of contacting the router for each request, as
well as for operations that require a controlled sequence of commands, such as a
set of database INSERT, UPDATE, and DELETE statements within a transaction.
CONNECT, REQUEST, and DISCONNECT flow in a stateful connection.