00:06:16  <prettyrobots>https://github.com/bigeasy/r-tree
00:28:57  <prettyrobots>https://github.com/bigeasy/xy
00:28:58  * wolfeidaujoined
00:29:58  * paulfryzeljoined
00:34:05  * paulfryzelquit (Ping timeout: 246 seconds)
00:46:33  * ednapiranhaquit (Remote host closed the connection)
00:50:04  * mikealquit (Quit: Leaving.)
00:56:00  * jcrugzzjoined
01:06:40  * jxsonquit (Remote host closed the connection)
01:12:27  * paulfryzeljoined
01:15:55  * fallsemojoined
01:21:54  * thlorenz_quit (Remote host closed the connection)
01:36:59  * jxsonjoined
01:41:15  * thlorenzjoined
01:45:00  * jxsonquit (Ping timeout: 245 seconds)
01:45:12  * paulfryzelquit (Remote host closed the connection)
01:46:54  * tmcwjoined
01:56:42  * thlorenz_joined
01:56:42  * thlorenzquit (Read error: Connection reset by peer)
02:11:00  <levelbot>[npm] readme-getter@0.0.6 <http://npm.im/readme-getter>: A module for fetching README html from npm modules. (@danfinlay)
02:11:22  * mikealjoined
02:12:44  * copongcopongquit (Quit: Walk the walk, talk the talk.)
02:13:06  * tmcwquit (Remote host closed the connection)
02:16:53  * thlorenzjoined
02:21:45  * thlorenzquit (Ping timeout: 264 seconds)
02:42:02  * jxsonjoined
02:43:19  * paulfryzeljoined
02:46:15  * jxsonquit (Ping timeout: 245 seconds)
02:54:25  * julian_duquejoined
02:54:50  * julianduquequit (Disconnected by services)
02:54:55  * julian_duquechanged nick to julianduque
03:02:20  * thlorenz_changed nick to thlorenz
03:06:03  * paulfryzelquit (Remote host closed the connection)
03:15:09  * fallsemoquit (Ping timeout: 264 seconds)
03:17:34  * mikealquit (Quit: Leaving.)
03:17:39  * thlorenz_joined
03:22:04  * thlorenz_quit (Ping timeout: 264 seconds)
03:24:54  * fallsemojoined
03:42:32  * jxsonjoined
03:47:05  * jxsonquit (Ping timeout: 245 seconds)
03:54:20  * thlorenzquit (Remote host closed the connection)
04:16:42  * paulfryzeljoined
04:16:50  * jxsonjoined
04:18:05  * thlorenzjoined
04:19:03  * thlorenz_joined
04:19:03  * thlorenzquit (Read error: Connection reset by peer)
04:20:50  * paulfryzelquit (Ping timeout: 245 seconds)
04:26:57  * soldairquit (Ping timeout: 250 seconds)
04:29:57  * parshapjoined
04:34:42  * timoxleyquit (Ping timeout: 268 seconds)
04:38:41  * jxson_joined
04:39:02  * jxsonquit (Ping timeout: 240 seconds)
04:39:45  * mikealjoined
04:40:05  * mikealquit (Client Quit)
04:40:24  * mikealjoined
04:40:38  * timoxleyjoined
04:48:11  * parshapquit (Quit: WeeChat 0.3.7)
04:49:10  * ednapiranhajoined
04:49:30  * ednapiranhaquit (Remote host closed the connection)
05:15:46  * dguttmanquit (Quit: dguttman)
05:34:23  * mikealquit (Quit: Leaving.)
05:45:09  * ralphtheninjaquit (Ping timeout: 264 seconds)
05:45:40  * prettyrobotsquit (Quit: ZNC - http://znc.in)
05:51:01  * jxson_quit (Remote host closed the connection)
05:53:00  <levelbot>[npm] grunt-s3-sync@0.2.0 <http://npm.im/grunt-s3-sync>: A streaming interface for uploading multiple files to S3 (@calvein)
05:55:53  * mcollinajoined
06:04:34  * mikealjoined
06:05:24  * thlorenz_quit (Remote host closed the connection)
06:07:40  * eugenewarequit (Remote host closed the connection)
06:09:33  * eugenewarejoined
06:09:37  * eugenewarequit (Remote host closed the connection)
06:09:44  * eugenewarejoined
06:28:43  * julianduquequit (Ping timeout: 276 seconds)
06:34:03  * thlorenzjoined
06:38:20  * thlorenzquit (Ping timeout: 245 seconds)
06:40:49  * mcollinaquit (Remote host closed the connection)
06:41:15  * mcollinajoined
06:45:25  * mcollinaquit (Ping timeout: 245 seconds)
06:45:57  * wolfeidauquit (Remote host closed the connection)
06:47:50  * mcollinajoined
06:50:30  <levelbot>[npm] mosca@0.9.5 <http://npm.im/mosca>: The multi-transport MQTT broker for node.js. It supports AMQP, Redis, ZeroMQ, MongoDB or just MQTT. (@matteo.collina)
06:51:24  * mcollinaquit (Remote host closed the connection)
06:51:30  <levelbot>[npm] grunt-s3-sync@0.2.1 <http://npm.im/grunt-s3-sync>: A streaming interface for uploading multiple files to S3 (@calvein)
06:51:51  * mcollinajoined
06:56:15  * mcollinaquit (Ping timeout: 245 seconds)
07:04:03  * fallsemoquit (Quit: Leaving.)
07:06:23  * ramitosquit (Ping timeout: 245 seconds)
07:06:45  * ramitosjoined
07:10:43  * keslajoined
07:22:25  * mcollinajoined
07:27:30  * mcollinaquit (Ping timeout: 245 seconds)
07:34:17  * thlorenzjoined
07:38:54  * thlorenzquit (Ping timeout: 276 seconds)
07:53:04  * thlorenzjoined
07:57:30  * thlorenzquit (Ping timeout: 245 seconds)
07:57:58  * wolfeidaujoined
08:16:52  * mcollinajoined
08:21:50  * mcollinaquit (Ping timeout: 240 seconds)
08:22:07  * thlorenzjoined
08:27:00  * thlorenzquit (Ping timeout: 276 seconds)
08:40:04  * jcrugzzquit (Ping timeout: 264 seconds)
08:45:23  * thlorenzjoined
08:48:39  * mcollinajoined
08:49:26  * thlorenzquit (Ping timeout: 240 seconds)
08:50:40  * dominictarrjoined
08:52:55  * mcollinaquit (Ping timeout: 245 seconds)
09:29:38  * jcrugzzjoined
09:31:28  * thlorenzjoined
09:35:50  * thlorenzquit (Ping timeout: 240 seconds)
09:36:55  * jcrugzzquit (Ping timeout: 246 seconds)
10:13:25  * mcollinajoined
10:17:52  * mcollinaquit (Ping timeout: 264 seconds)
10:28:56  * thlorenzjoined
10:33:45  * thlorenzquit (Ping timeout: 264 seconds)
10:41:52  * keslaquit (Ping timeout: 264 seconds)
10:57:50  * thlorenzjoined
11:03:00  * thlorenzquit (Ping timeout: 276 seconds)
11:38:10  * mcollinajoined
11:43:04  * mcollinaquit (Ping timeout: 264 seconds)
11:49:12  * ralphtheninjajoined
11:51:01  * thlorenzjoined
11:55:39  * thlorenzquit (Ping timeout: 276 seconds)
12:17:12  * fallsemojoined
12:31:21  <dominictarr>brycebaril: just saw your through2-reduce module
12:33:46  <dominictarr>you can do interesting things with reducing streams
12:34:04  <dominictarr>I wrote this one, https://github.com/dominictarr/repred
12:34:16  <dominictarr>that uses a reduce function to replicate data
12:35:00  * fallsemoquit (Quit: Leaving.)
12:47:15  * thlorenzjoined
12:48:08  * tmcwjoined
12:51:18  * tmcwquit (Remote host closed the connection)
12:51:26  * thlorenzquit (Ping timeout: 240 seconds)
13:02:29  * mcollinajoined
13:07:04  * mcollinaquit (Ping timeout: 264 seconds)
13:44:47  * thlorenzjoined
13:56:39  * mcollinajoined
13:59:12  * tmcwjoined
14:01:21  * mcollinaquit (Ping timeout: 264 seconds)
14:04:17  * dguttmanjoined
14:09:55  * briancjoined
14:14:10  * dguttmanquit (Quit: dguttman)
14:18:28  * no9quit (Ping timeout: 264 seconds)
14:19:39  * dguttmanjoined
14:21:59  <dominictarr>we need this for level http://www.learndatalogtoday.org/
14:22:13  <dominictarr>(I mean: an datalog like module)
14:26:11  <brycebaril>I wrote a query language/parser with https://github.com/zaach/jison turned out pretty easy. You could probably make a good datalog-like parser out of it.
14:26:30  <brycebaril>The hard part might be settling on the interactions for different leveldb addons
14:27:05  * mcollinajoined
14:28:18  <brycebaril>re through2-reduce repred looks neat, to do what it does I think I'd need to make through2 do a partial reduce (which honestly might make it more generically useful)
14:31:57  * mcollinaquit (Ping timeout: 264 seconds)
14:32:56  * no9joined
14:34:04  <dominictarr>brycebaril: yeah, you can just emit the partial solution periodically or when it changes
14:34:49  * mcollinajoined
14:35:05  <dominictarr>although, there are lots of complications there, like, I make the user return the change so that I can just send that, to minimize bandwidth
14:35:05  * mcollinaquit (Read error: Connection reset by peer)
14:35:17  * mcollinajoined
14:35:34  <dominictarr>tbh I havn't actually used it for anything
14:36:13  <brycebaril>I'm working on a set of statistics libraries that will go on top of level-version that these sorts of things will be useful
14:36:47  <brycebaril>e.g. a partial reduction of db.versionStream(key).pipe( reduce ).pipe(...)
14:37:03  <brycebaril>where the reduction could be an average of a value by day
14:39:40  * paulfryzeljoined
14:48:40  * paulfryzelquit (Remote host closed the connection)
14:54:15  * ednapiranhajoined
14:56:51  * paulfryzeljoined
15:00:55  * paulfryzelquit (Remote host closed the connection)
15:01:30  * paulfryzeljoined
15:02:15  <dominictarr>brycebaril: right - that sort of thing would be useful for realtime gauges etc, I'm guessing.
15:02:30  * fallsemojoined
15:04:49  * paulfryz_joined
15:06:04  * jerrysv_joined
15:08:16  * paulfryzelquit (Ping timeout: 264 seconds)
15:09:09  * paulfryz_quit (Ping timeout: 264 seconds)
15:09:23  * timoxleyquit (Remote host closed the connection)
15:13:57  * mcollinaquit (Remote host closed the connection)
15:14:25  * mcollinajoined
15:16:34  * mcollinaquit (Read error: Connection reset by peer)
15:16:57  * mcollinajoined
15:18:48  * ednapiranhaquit (Remote host closed the connection)
15:19:01  * mcollinaquit (Read error: Connection reset by peer)
15:19:09  * mcollinajoined
15:22:18  * jerrysv_changed nick to jerrysv
15:29:48  * julianduquejoined
15:39:20  * paulfryzeljoined
15:43:26  * ednapiranhajoined
15:45:06  * julianduquequit (Ping timeout: 276 seconds)
15:51:00  * mikealquit (Read error: Connection reset by peer)
15:53:38  * ednapira_joined
15:53:42  * fallsemoquit (Quit: Leaving.)
15:54:51  * Guest65670joined
15:55:02  * mikealjoined
15:55:59  <Guest65670>https://github.com/bigeasy/paxos
15:56:00  * Guest65670changed nick to prettyrobots
15:56:37  <prettyrobots>https://github.com/bigeasy/paxos
15:57:27  * ednapiranhaquit (Ping timeout: 276 seconds)
15:59:45  * julianduquejoined
16:04:29  <levelbot>[npm] rss-emitter@0.0.1 <http://npm.im/rss-emitter>: Small library which import RSS feeds and emit upon new entries — Edit (@cerisier)
16:09:25  * mcollina_joined
16:09:27  * mcollinaquit (Read error: Connection reset by peer)
16:10:09  * mcollina_quit (Read error: Connection reset by peer)
16:10:14  * mcollinajoined
16:10:25  * mcollinaquit (Remote host closed the connection)
16:10:52  * mcollinajoined
16:12:07  * fallsemojoined
16:17:50  * fallsemoquit (Quit: Leaving.)
16:19:19  * mikealquit (Quit: Leaving.)
16:19:51  * mikealjoined
16:25:50  * julianduquequit (Ping timeout: 240 seconds)
16:30:32  * fallsemojoined
16:40:32  * mikealquit (Quit: Leaving.)
16:41:31  * mikealjoined
16:42:39  * mikealquit (Client Quit)
16:43:20  * mikealjoined
16:46:22  * tmcwquit (Remote host closed the connection)
16:48:48  * tmcwjoined
17:11:02  * timoxleyjoined
17:15:19  * timoxleyquit (Ping timeout: 256 seconds)
17:25:41  * jxsonjoined
17:29:52  * mikealquit (Quit: Leaving.)
17:36:41  * mikealjoined
17:49:10  * dominictarrquit (Quit: dominictarr)
18:15:16  * paulfryzelquit (Remote host closed the connection)
18:15:45  * mcollinaquit (Ping timeout: 264 seconds)
18:15:51  * paulfryzeljoined
18:19:18  * dominictarrjoined
18:24:11  * gwenbelljoined
18:27:31  * jcrugzzjoined
18:31:40  * dominictarrquit (Ping timeout: 264 seconds)
18:36:21  * gwenbellquit (Ping timeout: 256 seconds)
18:51:24  * paulfryzelquit (Remote host closed the connection)
18:52:00  * paulfryzeljoined
18:56:05  * paulfryzelquit (Ping timeout: 248 seconds)
19:05:29  <levelbot>[npm] level-gc@0.2.0 <http://npm.im/level-gc>: Garbage collection for levelup (@bryce)
19:11:59  <levelbot>[npm] level-version@0.0.3 <http://npm.im/level-version>: Versioned data for levelup. (@bryce)
19:12:00  * timoxleyjoined
19:16:32  * julianduquejoined
19:17:00  * timoxleyquit (Ping timeout: 276 seconds)
19:20:53  * Acconutjoined
19:21:26  * mcollinajoined
19:25:13  * jerrysvquit (Remote host closed the connection)
19:26:07  * mcollinaquit (Ping timeout: 260 seconds)
19:40:51  * Acconutquit (Quit: Acconut)
19:52:14  * dominictarrjoined
19:57:10  * mikealquit (Quit: Leaving.)
20:00:20  * mikealjoined
20:03:38  * fallsemoquit (Quit: Leaving.)
20:12:32  * timoxleyjoined
20:16:54  * timoxleyquit (Ping timeout: 264 seconds)
20:24:01  * mikealquit (Quit: Leaving.)
20:24:16  * mikealjoined
20:31:18  * paulfryzeljoined
20:53:46  * briancquit (Remote host closed the connection)
20:59:15  * timoxleyjoined
21:29:07  * mikealquit (Quit: Leaving.)
21:34:04  * jcrugzzquit (Ping timeout: 264 seconds)
21:37:11  * mikealjoined
21:38:41  * wolfeidauquit (Remote host closed the connection)
21:44:49  <rescrv>prettyrobots: You may find it useful to read http://www.cs.cornell.edu/courses/cs7412/2011sp/paxos.pdf
21:45:48  * tmcwquit (Remote host closed the connection)
21:48:10  * paulfryzelquit (Remote host closed the connection)
21:48:44  * paulfryzeljoined
21:53:09  * paulfryzelquit (Ping timeout: 248 seconds)
21:55:10  * tmcwjoined
21:58:14  * paulfryzeljoined
22:12:52  * fallsemojoined
22:13:19  * fallsemoquit (Read error: Connection reset by peer)
22:16:33  * jxsonquit (Remote host closed the connection)
22:18:43  * tmcwquit (Remote host closed the connection)
22:26:50  * mikealquit (Quit: Leaving.)
22:30:47  * jxsonjoined
22:34:19  <prettyrobots>rescrv: Thank you.
22:34:46  <prettyrobots>This paper is very accessible, too.
22:34:51  <prettyrobots>rescrv: http://www.inf.usi.ch/faculty/pedone/MScThesis/marco.pdf
22:36:03  * thlorenzquit (Remote host closed the connection)
22:41:21  <rescrv>prettyrobots: I'm not aware of that implementation. I'm not sure that using an "atomic broadcast" model is the best way to go with Paxos
22:42:19  <prettyrobots>Huh.
22:42:37  <prettyrobots>I was under the impression that atomic broadcast is an abstraction built on top of Paxos.
22:42:58  <prettyrobots>A queue with a garaunteed order.
22:43:09  <prettyrobots>That won't lose messages.
22:43:23  <rescrv>that's one way to go
22:43:45  * brianloveswordsquit (Excess Flood)
22:43:45  <rescrv>I'm not sure that it's the best because there is a more natural abstraction
22:44:26  <rescrv>when implementing the queue, who are the consumers of the queue? What does it mean to "pop"? Pushing is easy, but a "pop" operation can become tricky to program against
22:45:04  <rescrv>if "pop" means to remove and return the item, who is it returned to? Everybody? The one person requesting a "pop"
22:45:08  * mikealjoined
22:45:20  <rescrv>It's an API that's ripe for abuse.
22:45:33  <rescrv>My preferred design for this kind of thing is a replicated state machine
22:45:40  * brianloveswordsjoined
22:45:55  <rescrv>http://www.cs.cornell.edu/fbs/publications/smsurvey.pdf
22:46:24  <rescrv>with the state machine approach, you build a state machine
22:46:51  <rescrv>it starts in a well-defined starting state, and each transition takes it to another state (possibly a NOP)
22:47:33  <rescrv>You use paxos to agree upon and sequence the transitions in the state machine.
22:48:08  <rescrv>What you end up with is a program that is replicated and fault-tolerant, without the program having to know anything about the underlying mechanism for doing so.
22:48:33  <rescrv>You can see this in action with Replicant: https://github.com/rescrv/Replicant/blob/master/examples/echo.c
22:48:45  * paulfryzelquit (Remote host closed the connection)
22:49:22  * paulfryzeljoined
22:49:53  <rescrv>what I linked is a simple C program that simply echos the input as the output from the state transition.
22:50:24  <rescrv>You can build an RSM on top of atomic broadcast, and people have.
22:50:47  <prettyrobots>Do you have an application?
22:51:08  <rescrv>There are many examples of atomic broadcast that are strictly weaker than Paxos.
22:51:11  <rescrv>Application of?
22:51:14  <prettyrobots>I mean, is there a larger application in which Replicant is used.
22:51:31  <rescrv>Yes, HyperDex: https://github.com/rescrv/HyperDex/tree/master/coordinator
22:52:01  <rescrv>there's also a few internal projects here that use it for storing meta-state (as opposed to zookeeper)
22:54:06  * paulfryzelquit (Ping timeout: 276 seconds)
22:54:31  <rescrv>It'd be pretty neat to take Replicant and make objects for various scripting languages. Take a pure-javascript app, pass it off to an interpreter handled by Replicant, and have an automatically-fault-tolerant application.
22:54:31  <prettyrobots>rescrv: Thank you for the tour.
22:55:03  <rescrv>prettyrobots: Sure thing.
22:55:25  <rescrv>If you're looking for a more isolated version of Paxos than libpaxos, check out http://openreplica.org/
22:55:39  <rescrv>They've got an approachable paper and open source Python implementation
22:55:54  <prettyrobots>I was simply going to build an advisory read/write lock using Paxos.
22:56:16  <prettyrobots>Something remotely useful to understand the abstractions.
22:56:26  <rescrv>if it's advisory, why do you need paxos?
22:56:46  <prettyrobots>I don't know.
22:57:44  <prettyrobots>OpenReplica and Replicant suggest a better abstraction for an object state.
22:58:07  <prettyrobots>Paxos, atomicity.
22:58:29  <prettyrobots>Not sure how a networked lock could be anything but advisory.
22:58:34  <rescrv>Yeah. It'd be cool to have a Javascript object that you can just upload to a server, and then use a local proxy that just treats it like any old object.
22:59:04  <rescrv>That's basically the problem with the network lock abstraction. It's much easier to build a correct RSM than a correct program using advisory locks.
23:01:33  <rescrv>Paxos provides consensus which is strictly stronger than most forms of atomic broadcast. In consensus, at most one value is chosen, even if nodes fail. Many forms of atomic broadcast allow you to see more than one chosen value if others believe you to have failed. AB is strictly weaker than Paxos.
23:01:56  * thlorenzjoined
23:03:15  * mikealquit (Quit: Leaving.)
23:05:44  <prettyrobots>rescrv: That's a lot of detail that I won't be able to understand without building something.
23:06:13  * thlorenzquit (Ping timeout: 248 seconds)
23:08:29  <rescrv>prettyrobots: it's a lot of detail that must be understood to build something that's correct. Of course, there's no barrier between the two, but just building something won't give you the understanding for the difference.
23:08:47  <prettyrobots>rescrv: I disagree.
23:10:26  <eugeneware>my old faithful dell 24" monitor died yesterday. Anyone recommend a good 23" 16x9 which will be good for design, and not too glossy?
23:10:34  <eugeneware>I feel like a visit to scorptech is in order
23:10:58  <eugeneware>I saw this one: not sure if it's any good: http://www.scorptec.com.au/computer/48253-s2340l
23:11:21  * mikealjoined
23:11:30  <rescrv>I'm open to changing my mind. I've just seen way too many people attempt to implement Paxos and end up at something that is most certainly not Paxos. Usually, while building they decide to be clever and introduce broken optimizations.
23:11:54  <eugeneware>oops. sorry. wrong room! pardon me :-)
23:14:17  * eugenewareruns to #polyhack
23:14:48  * jcrugzzjoined
23:17:24  <rescrv>prettyrobots: http://www.cs.utexas.edu/users/lorenzo/corsi/cs380d/papers/paper2-1.pdf
23:18:54  <prettyrobots>I've collected these papers and many more, so I really appreciate your pointing out the ones you feel are important.
23:20:12  <eugeneware>rescrv: what are your thoughts about the raft protocol?
23:21:02  <eugeneware>how useful/scalable would that for building a distributed database architecture for guaranteeing consistency?
23:21:54  <rescrv>eugeneware: I haven't touched code with raft (I have with Paxos and Replicant), so I'm speaking just from what I've read in their paper.
23:23:54  <rescrv>The protocol is extremely similar to the one used by Zookeeper. You have a a primary-backup system that is reconfigured using a more expensive protocol. The more expensive protocol is where the consensus problem is addressed.
23:25:17  <eugeneware>it certainly seems quite understandable. https://github.com/coreos/etcd which is being used for coreos is using it. Made we want for a node.js version that I could play with levelup with.
23:25:32  <rescrv>The motivation for Raft is that it is simpler than Paxos, and more easy to understand. I think that this is a goal that sounds noble, but is extremely hard to quantify or define a success metric. I automatically exclude polls like they use in the paper because people with little-to-no understanding often overestimate their understanding.
23:25:52  <eugeneware>yeah.
23:25:57  <rescrv>In my experience TAing higher-level courses, and talking to people at conferences, very few people understand Paxos.
23:26:24  <eugeneware>I watched this video: http://www.youtube.com/watch?v=YbZ3zDzDnrw (even I could understand it). But understandability is good for implementation and use, but I wonder how it performs in real life.
23:28:00  <eugeneware>I'm still struggling to get my head around paxos that's for sure - and everyone talks about it with such mystique.
23:28:12  <rescrv>Understandability is good for implementation and use. I'd argue that what we mean by "understandability" is that the correct behavior of the program is easy to describe, enabling an implementation that meets that behavior.
23:28:32  * dominictarrquit (Quit: dominictarr)
23:29:37  <rescrv>Raft, Zookeeper, and Replicant all take an approach where the system configuration is managely separately (to varying degrees) from the actual replicated data.
23:29:40  <prettyrobots>Is there a log for this IRC.
23:30:03  <mbalho>i have one
23:30:36  <mbalho>https://www.dropbox.com/s/7gz3779sxg6ygkw/%23%23leveldb.log
23:31:30  <rescrv>There's nothing wrong with this approach (I wrote Replicant and chose this approach), but it does impose a higher burden on the programmer. For Raft/Zookeeper, the system elects a single master through which all state must pass. Replicant decides upon a chain of nodes.
23:32:40  * thlorenzjoined
23:33:24  <rescrv>When the configuration changes, you must move from the old configuration to the new one in a way that doesn't allow data to become "corrupt". For Raft/ZK, this means that the new leader must have the most up-to-date copy of all data *and* every host must throw away state not recognized by the new leader.
23:33:36  <eugeneware>rescrv: That's a distinction I didn't pick up on. But yeah, they don't make any guarantees about the data, but trust the participating systems when they say that are at a particulate state. I guess they could use some cyperhash or something to verify that the data had in fact been persisted correctly.
23:34:04  <prettyrobots>mbalho: Thank you.
23:34:23  <rescrv>This is as simple as saying that every node will do that. In the implementation, it's likely quite complex and requires very careful bookeeping. You need to make sure that command X under the old leader is not used instead of command X under the new leader.
23:34:23  <prettyrobots>mbalho: Does it update?
23:35:18  <rescrv>In Replicant, you have the chain, and you know that each successive node in the chain contains a prefix of what its predecessor has. It makes it very easy to reestablish state on configuration changes.
23:35:55  <rescrv>Paxos is a different beast. With Paxos, you just have "slots". Each slot is a round of consensus using a simplified Paxos protocol (http://www.cs.utexas.edu/users/lorenzo/corsi/cs380d/past/03F/notes/paxos-simple.pdf)
23:37:14  <rescrv>The "Synod" protocol as it's called decides a single slot. From an execution of the Synod protocol, a set of hosts can decide a single value
23:37:15  * jxsonquit (Remote host closed the connection)
23:38:02  <rescrv>Multi-Paxos makes Paxos more practical by introducing the notion of slots. Think of each slot as a positive integer. I run the synod protocol for slot 0, then slot 1, then slot 2, choosing a value for each slot in turn.
23:38:54  <rescrv>There is no leader. When people talk about electing leaders, they likely aren't using Paxos.
23:38:59  * wolfeidaujoined
23:40:00  <eugeneware>ok. they're just voting on the consensus value for each slot
23:40:36  <rescrv>eugeneware: It's not just voting. The prepare/promise/accept steps are set up to ensure that *at most* one value is chosen for a particular round.
23:40:38  * thlorenzquit (Ping timeout: 240 seconds)
23:42:15  <rescrv>Competing servers will go through prepare/promise repeatedly. Sharing of values in the promise step ensures that if two servers are trying to agree upon different values, they will cooperate and (likely) agree in short order
23:42:45  <rescrv>Note that there's no guarantee here. It's entirely possible (and indeed, necessary) fro the protocol to livelock in the prepare/promise phase
23:42:45  <mbalho>prettyrobots: i have an rsync thing i run from time to time that copies all my logs to dropbox
23:43:05  * timoxleyquit (Remote host closed the connection)
23:43:11  <prettyrobots>mbalho: Nice. Thank you.
23:46:35  <rescrv>The usual fix for that is to have each server introduce random delays (skewed based upon some unique identifier for the server), so that it's unlikely that they'll both immediately retry
23:49:32  <rescrv>Reconfiguring a Paxos cluster is a whole different beast. It requires that you limit the number of outstanding operations that have not yet been accepted, and introduce a special "reconfigure" command just beyond that limit. There's a lot of subtlety here.
23:50:16  <rescrv>The key idea to take away is that you're using Paxos to agree on a new configuration. The slot at which the configuration takes place is a clean barrier between the old and new views of the world.
23:51:13  <eugeneware>do you have to worry about rolling back state with paxos?
23:52:20  <rescrv>eugeneware: You have to deal with making sure you never accept state to roll back. It's why I trust it more than arbitrary Raft implementations.
23:52:49  <rescrv>It's a lot easier to always take transitions that preserve correctness than to try to adopt bad state and later un-adopt said state.
23:53:59  <eugeneware>yeah. rolling back badly mutated data could get complicated. Better to not even go there.
23:54:05  <rescrv>That's why Paxos is seen as more complex. Raft/ZK just bury the complexity in things that seem simple.
23:54:12  * wolfeidauquit (Remote host closed the connection)
23:54:38  <eugeneware>any more good paxos references you'd recommend?
23:54:57  <rescrv>I've linked most of them.
23:55:23  <eugeneware>cool. Thanks. Much reading ahead of me :-)
23:55:35  <rescrv>I think the best references are some of my peers, and some of the research scientists here. Unfortunately, I can't link them.
23:56:09  <rescrv>They'd probably be more than willing to chat about Paxos at a conference though
23:57:02  <rescrv>Especially one of the other people in my group (wrote Concoord herself). I have never seen anyone who understands Paxos better than her.
23:57:29  <rescrv>I've never met Lamport though ;-)