-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy patharch.html
240 lines (187 loc) · 11.3 KB
/
arch.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<title>KiokuDB Architecture</title>
<link rel="stylesheet" type="text/css" href="css/main.css" />
</head>
<body>
<div id="main">
<a name="top"></a>
<div class="center">
<map name="GraffleExport">
<area shape=rect coords="470,194,542,251" href="#indexing">
<area shape=rect coords="370,194,442,251" href="#typemap">
<area shape=rect coords="257,194,341,251" href="#live_objects">
<area shape=rect coords="157,194,229,251" href="#collapser">
<area shape=rect coords="57,194,129,251" href="#linker">
<area shape=rect coords="221,298,375,372" href="#backends">
<area shape=rect coords="477,420,561,471" href="#simpledb">
<area shape=rect coords="151,420,223,472" href="#dbi">
<area shape=rect coords="246,420,352,472" href="#files">
<area shape=rect coords="375,420,454,472" href="#couchdb">
<area shape=rect coords="49,420,129,471" href="#bdb">
</map>
<img border=0 src="kiokudb_arch.png" usemap="#GraffleExport">
</div>
<hr/>
<h2><a name="high_level" href="#top">↑</a>High Level API</h2>
<p>The main API is provided by <tt><a
href="https://metacpan.org/module/KiokuDB">KiokuDB.pm</a></tt>.
This is the entry point into KiokuDB and is the API for users. It
brings together all of the other components.</p>
<p>When using KiokuDB you rarely need to delve deeper than this
interface, but knowing how the pieces fit together is helpful for
both knowing what to expect, and extending the system to better
suit your purposes.</p>
<hr/>
<h2><a name="processing" href="#top">↑</a>Processing Layer</h2>
<p>The processing layer contains a number of components that
integrate storage backed objects seamlessly into memory space.</p>
<p>The processing layer deals with objects in Perl's memory space
and is backend independent.</p>
<h3><a name="linker" href="#top">↑</a>The Linker</h3>
<p>The <a
href="https://metacpan.org/module/KiokuDB::Linker">linker</a>
manages the task of expanding <a
href="https://metacpan.org/module/KiokuDB::Entry">entries</a>
into objects and connecting them into the in memory graph of
objects.</p>
<p>Object expansion uses the <a href="#typemap">type map</a> to
locate the per-class expansion callbacks, and calls them while
maintaining all the links between objects consistent.</p>
<p>Inter object links are handled by recursively fetching the
necessary entries from the <a href="#backends">backend</a>, except
when <a
href="https://metacpan.org/module/KiokuDB::Meta::Attribute::Lazy">explicitly
deferred.</a>.</p>
<h3><a name="collapser" href="#top">↑</a>The Collapser</h3>
<p>The <a
href="https://metacpan.org/module/KiokuDB::Collapser">collapser</a>
is the storage-time counterpart of the <a
href="#linker">linker</a>. It walks the objects in memory and
<a href="#typemap">locates</a> the appropriate collapsing routine
for each class as it prepares a set of <a
href="https://metacpan.org/module/KiokuDB::Entry">entries</a>
to storage.</p>
<p>The collapser handles memory cycles, weak references, handling
of tied data structures, and various other nits and edge cases with
Perl data structures.</a>
<h3><a name="typemap" href="#top">↑</a>The Type Map</h3>
<p>The <a
href="https://metacpan.org/module/KiokuDB::TypeMap">type
map</a> is responsible for finding appropriate callbacks for
each class stored in the database. These callbacks assign IDs,
break down the object for the <a href="#collapser">collapser</a>,
and reassemble it during <a href="#linker">loading</a>.</p>
<p>The <a
href="https://metacpan.org/module/KiokuDB::TypeMap::Default">default
typemap</a> will handle all <a
href="http://moose.perl.org">Moose</a> based objects, with rich
support for metaclass based customization. This allows you to lazy
load certain attributes, skip cache attributes during
serialization, assign custom IDs, etc.</p>
<p>The default typemaps also contains entries for popular modules
such as <a
href="https://metacpan.org/module/DateTime">DateTime</a>.</p>
<p>Optional out of the box support for other popular class builders
is also provided.</p>
<p>Lastly, adding your own entries is trivial. You can <a
href="https://metacpan.org/module/KiokuDB::TypeMap::Entry::StorableHook">reuse
existing Storable hooks</a>, use <a
href="https://metacpan.org/module/KiokuDB::TypeMap::Entry::Callback">callbacks</a>,
or simply let the
collapser <a
href="https://metacpan.org/module/KiokuDB::TypeMap::Entry::Naive">break
down the object for you</a>.</p>
<h3><a name="live_objects" href="#top">↑</a>Live Objects</h3>
<p>Live objects are objects that KiokuDB has either loaded or
stored. The live object manager maps Perl reference addresses to
IDs and live entries allowing the <a href="#linker">linker</a> and
the <a href="#collapser">collapser</a> to maintain the integrity of
the object graph.</a></p>
<p>The <a
href="https://metacpan.org/module/KiokuDB::LiveObjects">live
object manager</a> additionally handles the various scoping
related tasks, such as keeping alive objects only pointed to by
weak references, and handling entry state during transactions.</p>
<h3><a name="indexing" href="#top">↑</a>Indexing</h3>
<p>Indexing is done before insertion to the database by extracting
the relevant search keys in Perl space. Although <a
href="https://metacpan.org/module/Search::GIN">Search::GIN</a>
is still in its infancy, it is capable of performing simple
searches on your database on arbitrary search keys.</a>
<p>An alternative to in memory indexing is relying on the backend
to provide searching facilities. The downsides are that this
technique breaks the encapsulation of your objects, and also limits
you to the data inside a single object, and does not let you index
relationship based data.</p>
<hr/>
<h2><a name="backends" href="#top">↑</a>Backends</h2>
<p>Almost any keyed data store can be used as a backend for
KiokuDB. BLOB or document oriented databases are especially
suitable.</p>
<p>Since backends are isolated from the complexity of the object
graph and all of Perl's quirks by the processing layer, the driver
code can remain clean and simple.</p>
<p>Implementing a backend is easy. The design lets you add features
incrementally instead of all at once, by using interface roles. The
core API contains just 4 methods.</p>
<p>Secondly, there are numerous
concrete roles to help backend authors reuse code, and a <a
href="https://metacpan.org/module/KiokuDB::Test">reusable
test suite</a> to check your code.</p>
<h3><a name="bdb" href="#top">↑</a>Berkeley DB</h3>
<p>The <a
href="https://metacpan.org/module/KiokuDB::Backend::BDB">Berkeley
DB backend</a> uses the <a
href="https://metacpan.org/module/BerkeleyDB">BerkeleyDB</a> module
to interface with <a
href="http://www.oracle.com/technology/products/berkeley-db/index.html">Oracle
Berkeley DB</a>. This is a high performance, full features
transactional store, supporting <a href="#index">GIN</a> based
queries.</p>
<h3><a name="dbi"href="#top">↑</a>DBI</h3>
<p>The <a
href="https://metacpan.org/module/KiokuDB::Backend::DBI">DBI
backend</a> utilizies Perl's <a
href="https://metacpan.org/module/DBI">DBI</a> module to
provide database independent transactional storage using simple,
portable SQL.</p>
<p>This backend supports <a href="#indexing">GIN indexing</a> as
well as simple SQL where clauses.</p>
<p>Tested databases include <a
href="http://www.sqlite.org/">SQLite</a>, <a
href="http://www.postgresql.org/">PostgreSQL</a> and <a
href="http://www.mysql.com/">MySQL</a>.</p>
<h3><a name="files" href="#top">↑</a>File Backend</h3>
<p><b>NOTE: This backend is currently broken. Patches are welcome to get it running again.</b></p>
<p>The <a
href="https://metacpan.org/module/KiokuDB::Backend::Files">files
backend</a> is the simplest storage provider. It uses <a
href="https://metacpan.org/module/Directory::Transactional">Directory::Transactional</a>
to provide transaction support to store each object in its own file
on disk.</p>
<p>This backend has no built in indexing mechanism.</p>
<h3><a name="couchdb" href="#top">↑</a>CouchDB</h3>
<p><a href="http://couchdb.apache.org">Apache CouchDB</a> can be
used as a backend using the <a
href="https://metacpan.org/module/KiokuDB::Backend::CouchDB">CouchDB
backend</a>. This utilizes <a
href="https://metacpan.org/module/AnyEvent::CouchDB">AnyEvent::CouchDB</a>
to communicate with CouchDB.</p>
<h3><a name="simpledb" href="#top">↑</a>Simple DB</h3>
<p>The <a
href="http://github.com/omega/kiokudb-backend-aws-sdb/">SimpleDB</a>
backend is still in development, and provides access to <a
href="http://aws.amazon.com/simpledb/">Amazon SimpleDB</a>
using the <a
href="https://metacpan.org/module/Amazon::SimpleDB">Amazon::SimpleDB</a>
module.</p>
</div>
<div id="footer">
© 2008, 2009 <a href="http://www.iinteractive.com/">Infinity Interactive</a> & <a href="http://nothingmuch.woobling.org/">Yuval Kogman</a>
</div>
</body>
</html>