-
Notifications
You must be signed in to change notification settings - Fork 14
/
Copy pathREADME
160 lines (121 loc) · 5.41 KB
/
README
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
NAME
Message::Passing - a simple way of doing messaging.
SYNOPSIS
message-pass --input STDIN --output STDOUT
{"foo": "bar"}
{"foo":"bar"}
DESCRIPTION
A library for building high performance, loosely coupled and
reliable/resilient applications, structured as small services which
communicate over the network by passing messages.
BASIC PREMISE
You have data for discrete events, represented by a hash (and serialized
as JSON).
This could be a text log line, an audit record of an API event, a metric
emitted from your application that you wish to aggregate and process -
anything that can be a simple hash really..
You want to be able to shove these events over the network easily, and
aggregate them / filter and rewrite them / split them into worker
queues.
This module is designed as a simple framework for writing components
that let you do all of these things, in a simple and easily extensible
manor.
For a practical example, You generate events from a source (e.g. ZeroMQ
output of logs and performance metrics from your Catalyst FCGI or
Starman workers) and run one script that will give you a central
application log file, or push the logs into Elasticsearch.
There are a growing set of components you can plug together to make your
solution.
Getting started is really easy - you can just use the "message-pass"
command installed by the distribution. If you have a common config that
you want to repeat, or you want to write your own server which does
something more flexible than the normal script allows, then see
Message::Passing::DSL.
To dive straight in, see the documentation for the command line utility
message-pass, and see the examples in
Message::Passing::Manual::Cookbook.
For more about how the system works, see
Message::Passing::Manual::Concepts.
COMPONENTS
Below is a non-exhaustive list of components available.
INPUTS
Inputs receive data from a source (usually a network protocol).
They are responsible for decoding the data into a hash before passing it
onto the next stage.
Inputs include:
Message::Passing::Input::STDIN
Message::Passing::Input::ZeroMQ
Message::Passing::Input::STOMP
Message::Passing::Input::AMQP
Message::Passing::Input::Syslog
Message::Passing::Input::Redis
Message::Passing::Input::Test
You can easily write your own input, just use AnyEvent, and consume
Message::Passing::Role::Input.
FILTER
Filters can transform a message in any way.
Examples include:
Message::Passing::Filter::Null - Returns the input unchanged.
Message::Passing::Filter::All - Stops any messages it receives from
being passed to the output. I.e. literally filters all input out.
Message::Passing::Filter::T - Splits the incoming message to multiple
outputs.
You can easily write your own filter, just consume
Message::Passing::Role::Filter.
Note that filters can be chained, and a filter can return undef to stop
a message being passed to the output.
OUTPUTS
Outputs send data to somewhere, i.e. they consume messages.
Message::Passing::Output::STDOUT
Message::Passing::Output::AMQP
Message::Passing::Output::STOMP
Message::Passing::Output::ZeroMQ
Message::Passing::Output::WebHooks
Message::Passing::Output::Search::Elasticsearch
Message::Passing::Output::Redis
Message::Passing::Output::Test
SEE ALSO
Message::Passing::Manual - The manual (contributions cherished).
<http://www.slideshare.net/bobtfish/messaging-interoperability-and-log-a
ggregation-a-new-framework> - Slide deck!
Log::Message::Structured - For creating your log messages.
Log::Dispatch::Message::Passing - use Message::Passing outputs from
Log::Dispatch.
THIS MODULE
This is a simple MooX::Options script, with one input, one filter and
one output. To build your own similar scripts, see:
Message::Passing::DSL - To declare your message chains.
Message::Passing::Role::CLIComponent - To provide "foo" and
"foo_options" attribute pairs.
Message::Passing::Role::Script - To provide daemonization features.
METHODS
build_chain
Builds and returns the configured chain of input => filter => output.
start
Class method to call the run_message_server function with the results of
having constructed an instance of this class, parsed command line
options and constructed a chain.
This is the entry point for the script.
AUTHOR
Tomas (t0m) Doran <[email protected]>
SUPPORT
Bugs
Please log bugs at rt.cpan.org. Each distribution has a bug tracker link
in its metacpan.org page.
Discussion
#message-passing on irc.perl.org.
Source code
Source code for all modules is available at <http://github.com/suretec>
and forks/patches are very welcome.
SPONSORSHIP
This module exists due to the wonderful people at Suretec Systems Ltd.
<http://www.suretecsystems.com/> who sponsored its development for its
VoIP division called SureVoIP <http://www.surevoip.co.uk/> for use with
the SureVoIP API -
<http://www.surevoip.co.uk/support/wiki/api_documentation>
COPYRIGHT
Copyright Suretec Systems Ltd. 2012.
Logstash (upon which many ideas for this project are based, but from
which we do not reuse any code) is copyright 2010 Jorden Sissel.
LICENSE
GNU Library General Public License, Version 2.1