diff options
| author | Franck Cuny <franckcuny@gmail.com> | 2016-07-02 20:06:31 -0700 |
|---|---|---|
| committer | Franck Cuny <franckcuny@gmail.com> | 2016-07-02 20:06:31 -0700 |
| commit | 4b8e43f75b394a4e6169884fbfb4c606865c6a22 (patch) | |
| tree | 48cae6b8e8f9b68cae29676d8a15cb3ddbfcccda /content/post/2010-04-14-presque-a-redis-tatsumaki-based-message-queue.md | |
| parent | Stop using Jekyll. (diff) | |
| download | lumberjaph-4b8e43f75b394a4e6169884fbfb4c606865c6a22.tar.gz | |
Import migration from Jekyll to Hugo.
All the posts were converted, and the layout is created. This looks like
it works just fine.
Diffstat (limited to '')
| -rw-r--r-- | content/post/2010-04-14-presque-a-redis-tatsumaki-based-message-queue.md | 87 |
1 files changed, 87 insertions, 0 deletions
diff --git a/content/post/2010-04-14-presque-a-redis-tatsumaki-based-message-queue.md b/content/post/2010-04-14-presque-a-redis-tatsumaki-based-message-queue.md new file mode 100644 index 0000000..ef3463c --- /dev/null +++ b/content/post/2010-04-14-presque-a-redis-tatsumaki-based-message-queue.md @@ -0,0 +1,87 @@ +--- +date: 2010-04-14T00:00:00Z +summary: In which I introduce presque +title: presque, a Redis / Tatsumaki based message queue +--- + +[presque](http://github.com/franckcuny/presque/tree/) is a small message queue service build on top of [redis](http://code.google.com/p/redis/) and [Tatsumaki](http://search.cpan.org/perldoc?Tatsumaki). It's heavily inspired by [RestMQ](http://github.com/gleicon/restmq) and [resque](http://github.com/defunkt/resque). + +* Communications are done in JSON over HTTP +* Queues and messages are organized as REST resources +* A worker can be writen in any language that make a HTTP request and read JSON +* Thanks to redis, the queues are persistent + +## Overview + +resque need a configuration file, writen in YAML that contains the host and port for the Redis server. + +```yaml +redis: + host: 127.0.0.1 + port: 6379 +``` + +Let's start the server: + +```bash +% plackup app.psgi --port 5000 +``` + +The applications provides some HTTP routes: + +* **/**: a basic HTML page with some information about the queues +* **/q/**: REST API to get and post job to a queue +* **/j/**: REST API to get some information about a queue +* **/control/**: REST API to control a queue (start or stop consumers) +* **/stats/**: REST API to fetch some stats (displayed on the index page) + +Queues are created on the fly, when a job for an unknown queue is inserted. When a new job is created, the JSON send in the POST will be stored "as is". There is no restriction on the schema or the content of the JSON. + +Creating a new job simply consist to : + +```bash +% curl -X POST "http://localhost:5000/q/foo" -d '{"foo":"bar", "foo2":"bar" }' +``` + +and fetching the job: + +```bash +% curl "http://localhost:5000/q/foo" +``` + +When a job is fetched, it's removed from the queue. + +## A basic worker + +I've also pushed [presque::worker](http://git.lumberjaph.net/p5-presque-worker.git/). It's based on [AnyEvent::HTTP](http://search.cpan.org/perldoc?AnyEvent::HTTP) and [Moose](http://search.cpan.org/perldoc?Moose). Let's write a basic worker using this class: + +```perl +use strict; +use warnings; +use 5.012; # w00t + +package simple::worker; +use Moose; +extends 'presque::worker'; + +sub work { + my ($self, $job) = @_; + say "job's done"; + ...; # yadda yadda! + return; +} + +package main; +use AnyEvent; + +my $worker = + simple::worker->new(base_uri => 'http://localhost:5000', queue => 'foo'); + +AnyEvent->condvar->recv; +``` + +A worker have to extends the presque::worker class, and implement the method *work*. When the object is created, the class check if this method is avalaible. You can also provide a `fail` method, which will be called when an error occur. + +## The future + +I plan to add support for [websocket](http://en.wikipedia.org/wiki/WebSocket), and probably [XMPP](http://en.wikipedia.org/wiki/Xmpp). More functionalities to the worker too: logging, forking, handling many queues, ... I would like to add priorities to queue also, and maybe scheluding job for a given date (not sure if it's feasable with Redis). |