Merge branch 'convert-to-org'

1 files changed, 232 insertions, 0 deletions

diff --git a/posts/2010-09-17-spore.org b/posts/2010-09-17-spore.org
new file mode 100644
index 0000000..390ca2e
--- /dev/null
+++ b/posts/2010-09-17-spore.org
@@ -0,0 +1,232 @@
+** Specification to a POrtable Rest Environment
+
+More and more web services offer
+[[http://en.wikipedia.org/wiki/Representational_State_Transfer][ReST
+API]]. Every time you want to access one of these services, you need to
+write a client for this API, or find an existing one. Sometimes, you
+will find the right library that will do exactly what you need.
+Sometimes you won't, and you end up writing your own.
+
+Some parts of an API client are always the same:
+
+-  build an uri
+-  make a request
+-  handle the response
+-  ...
+
+With SPORE, I propose a better solution for this. SPORE is composed of
+two parts:
+
+-  a specification that describes an API
+-  a "framework" for clients (think this as
+   [[http://www.python.org/dev/peps/pep-0333/][WSGI]],
+   [[http://plackperl.org/][Plack]],
+   [[http://rack.rubyforge.org/][Rack]],
+   [[http://jackjs.org/jsgi-spec.html][JSGI]], ...)
+
+** API Specifications
+
+I know, at this point, you're thinking "what ? isn't it just what
+[[http://en.wikipedia.org/wiki/Web_Services_Description_Language][WSDL]]
+does ?". Well, yes. But it's (in my opinion) simpler to write this than
+to write a WSDL. And when you say "WSDL" people think
+"[[http://en.wikipedia.org/wiki/SOAP_(protocol)][SOAP]]", and that's
+definitly not a good thing.
+
+The first part is the specification to API. A ReST request is mainly :
+
+-  a *HTTP method*
+-  a *path*
+-  some *parameters*
+
+This is *easy to describe*. For this example, I will use the
+[[http://dev.twitter.com/doc/get/statuses/public_timeline][twitter
+API]]. So, if i want to describe the user timeline method, we will get
+something like this:
+
+#+BEGIN_EXAMPLE
+    public_timeline:
+      method: GET
+      path: /statuses/public_timeline.:format
+      params:
+        - trim_user
+        - include_entities
+      required:
+        - format
+#+END_EXAMPLE
+
+Whatever your language of choice is, you'll always need this
+informations. The idea with the API description, is that it can be
+reused by every language. If the API provider publishes this file,
+everyone can easily use it. It's very similar to a documentation (for
+the twitter description, all I needed to do was to copy/paste the
+documentation, it's really that simple) but it can be used by a
+framework to generate a client.
+
+The specifications should be in JSON (I've written the example in YAML
+for the sake of readability). The complete description of the
+specifications are available
+[[https://github.com/SPORE/specifications][here]].
+
+There is many advantages to do it this way:
+
+-  if you have a client in Perl and Javascript, the names of the methods
+   are the same in both langages, and the names of parameters too
+-  if the API changes some endpoints, you don't have to change your
+   code, you only need to update the description file
+
+I've started to write some specifications for a few services
+([[https://github.com/SPORE/api-description/blob/master/services/twitter.json][twitter]],
+[[https://github.com/SPORE/api-description/blob/master/services/github.json][github]],
+[[https://github.com/franckcuny/spore/blob/master/services/backtype.json][backtype]],
+[[https://github.com/franckcuny/spore/blob/master/services/backtweet.json][backtweet]],
+...) and applications
+([[https://github.com/franckcuny/spore/blob/master/apps/couchdb.json][couchdb]],
+[[https://github.com/franckcuny/spore/blob/master/apps/presque.json][presque]]).
+They are not complete yet, so you're welcomed to
+[[https://github.com/franckcuny/spore][fork the repository]], add
+missings methods, and add your own specifications! :)
+
+** Client Specification
+
+Now that we have a simple description for the API, we want to have
+[[https://github.com/franckcuny/net-http-spore/blob/master/spec/spore_implementation.pod][an
+easy solution to use it]]. I will describe
+[[https://github.com/franckcuny/net-http-spore][the Perl
+implementation]], but there is also one for Ruby (will be published
+soon), and a early version for
+[[http://github.com/ngrunwald/clj-spore][Clojure]] and
+[[http://github.com/elishowk/pyspore][Python]].
+
+This kind of thing is really easy to implement in dynamic languages, and
+still doable in others.
+
+The client is composed of two parts: core and middlewares.
+
+The core will create the appropriate functions using the previous
+description. Thanks to metaprogramming, it's very easy to do it. If we
+use [[http://search.cpan.org/perldoc?Moose][Moose]], all I need to do,
+is to extend the
+[[http://search.cpan.org/perldoc?Moose::Meta::Method][Moose::Meta::Method]]
+and add new attributes like:
+
+-  path
+-  method
+-  params
+-  authentication
+-  ...
+
+For each method declared in the description, I build a new Moose method
+I will attach to my class. Basicaly, the code looks like this:
+
+#+BEGIN_SRC perl
+    foreach my $method_name ( keys %$methods_spec ) {
+        $class->meta->add_spore_method(
+            "user_timeline",
+            path     => '/statuses/public_timeline.:format',
+            required => [qw/format/],
+            params   => [qw/trim_user include_entities/]
+        );
+    }
+#+END_SRC
+
+The code of the =user_timelime= method will be generated via the
+=add_spore_method=.
+
+Middlewares are the nice part of it. By default, the core only creates a
+request, executes it, and gives you the result. Nothing more. By adding
+middlewares, you can handle the following stuff:
+
+-  headers manipulation
+-  authentication (basic, OAuth, )
+-  (de)serialization (JSON, XML, YAML, CSV, ...)
+-  caching
+-  proxying
+-  ...
+
+The most obvious middleware is the one that handles the format. When you
+load the middleware Format::JSON, it will set various headers on your
+request. In case of a GET method, the *Accept* header will be set to
+*application/json*. For a POST, the *Content-Type* will be also set.
+Before returning the result to the client, the content will be
+transformed from JSON to a Perl structure.
+
+For twitter, I can have a client with this few lines:
+
+#+BEGIN_SRC perl
+    my $client = Net::HTTP::Spore->new_from_spec('twitter.json');
+    $client->enable('Format::JSON');
+
+    my $timeline = $client->public_timeline( format => 'json', include_rts => 1 );
+    my $tweets = $timeline->body;
+    foreach my $tweet (@$tweets) {
+        say $tweet->{user}->{screen_name} . " says " . encode_utf8($tweet->{text});
+    }
+#+END_SRC
+
+Now, I want to use my friends timeline, which requires OAuth ? easy
+
+#+BEGIN_SRC perl
+    $client->enable(
+        'Auth::OAuth',
+        consumer_key    => $consumer_key,
+        consumer_secret => $consumer_secret,
+        token           => $token,
+        token_secret    => $token_secret,
+    );
+    my $friends_timeline = $client->friends_timeline(format => 'json', include_rts => 1);
+    my $tweets = $friends_timeline->body;
+    foreach my $tweet (@$tweets) {
+        print $tweet->{user}->{screen_name} . " says " . encode_utf8($tweet->{text}) . "\n";
+    }
+#+END_SRC
+
+Middlewares are easy to write. They should implement a /call/ method,
+which receive a request object as argument. The middleware can return:
+
+-  nothing: the next middleware will be executed
+-  a callback: it will be executed when the request is done, and will
+   receive a response object
+-  a response object: no more middlewares will be executed
+
+A simple middleware that add a runtime header to the response object
+will look like this:
+
+#+BEGIN_SRC perl
+    sub call {
+        my ( $self, $req ) = @_;
+
+        my $start_time = [Time::HiRes::gettimeofday];
+
+        $self->response_cb(
+            sub {
+                my $res      = shift;
+                my $req_time = sprintf '%.6f',
+                  Time::HiRes::tv_interval($start_time);
+                $res->header( 'X-Spore-Runtime' => $req_time );
+            }
+        );
+    }
+#+END_SRC
+
+I've tried to mimic as much as possible Plack's behavior. The result of
+a request is a Response object, but you can also use it as an arrayref,
+with the following values [http\_code, [http\_headers], body].
+
+** Conclusion
+
+The real target for this are not API developers (even if it's useful to
+have this when you write your own API, I will show some examples soon),
+neither client developers (even it's really easier to do with this), but
+people who want to play immediatly with an API to fetch data, without
+the coding skill or knowledge of what an HTTP request is, how to define
+headers, what is OAuth, ... As it was suggested to me, an implementation
+for [[http://en.wikipedia.org/wiki/R_(programming_language)][R]] would
+be really usefull to a lot of people.
+
+Right now, I'm looking for people interested by this idea/project, and
+to work on completing the specification. I'm pretty happy with the
+current status, as it works with most API I've encountered.
+
+I will present SPORE and its implementations
+[[http://act.osdc.fr/osdc2010fr/][during OSDC.fr]] next month.