00:03:01  <ralphthe1inja>checked the references on that paper, all about networks and distributed stuff
00:03:24  <ralphthe1inja>we're on the right path alright :)
00:03:45  * shuaibquit (Quit: Textual IRC Client: http://www.textualapp.com/)
00:05:34  <ralphthe1inja>dominictarr: check out reference 27
00:05:41  <ralphthe1inja>R. C. Merkle. Secrecy, authentication, and public key
00:05:43  <ralphthe1inja>systems. PhD thesis, 1979.
00:05:51  <ralphthe1inja>rings a bell? :)
00:14:38  <Raynos>heaidng to substack house
00:14:50  <Raynos>what is the substack house address again? jjjjohnnny, dominictarr
00:17:45  * al__joined
00:18:03  * al__changed nick to Altreus
00:19:12  <ralphthe1inja>dominictarr: I checked the article on wikipedia on hash trees and from there I found the author of that article, apparently he is (or was) working on a project called Randpeer R. C. Merkle. Secrecy, authentication, and public key
00:19:34  <ralphthe1inja>aah god damn clipboard
00:20:02  <ralphthe1inja>http://www.randpeer.org/
00:22:35  <ralphthe1inja>seems like a dead project but the site has some interesting stuff
00:23:39  * st_lukejoined
00:25:00  * tphummeljoined
00:27:39  * Altreusquit (Quit: leaving)
00:27:50  * al__joined
00:27:55  * al__changed nick to Altreus
00:28:01  * Altreusquit (Changing host)
00:28:01  * Altreusjoined
00:28:25  * lyaunzbejoined
00:41:29  * nrknquit (Ping timeout: 252 seconds)
00:54:36  * tphummelquit (Quit: tphummel)
01:04:19  * defunctzombiejoined
01:04:19  * defunctzombiequit (Changing host)
01:04:19  * defunctzombiejoined
01:08:36  * defunctzombiequit (Ping timeout: 248 seconds)
01:21:51  * st_lukequit (Ping timeout: 244 seconds)
01:27:22  * fotoveritejoined
01:28:45  * st_lukejoined
01:28:48  * jibayquit (Quit: Leaving)
01:36:06  * fotoveritequit (Ping timeout: 264 seconds)
02:25:39  * lyaunzbequit (Ping timeout: 260 seconds)
02:39:52  <st_luke>bread machines
02:43:24  * AvianFlujoined
03:02:19  * ralphthe1injaquit (Ping timeout: 265 seconds)
03:05:14  * defunctzombiejoined
03:05:15  * defunctzombiequit (Changing host)
03:05:15  * defunctzombiejoined
03:09:59  * defunctzombiequit (Ping timeout: 252 seconds)
03:15:29  <st_luke>omfgdogs.com
03:30:18  <ik>!!!!
03:32:37  <ik>22:31:08 < shanse> requesting omfgturtles
04:24:34  * tphummeljoined
04:31:19  <st_luke>this cat is way too smart
04:33:52  * tphummelquit (Quit: tphummel)
05:06:47  * defunctzombiejoined
05:06:47  * defunctzombiequit (Changing host)
05:06:47  * defunctzombiejoined
05:11:30  * defunctzombiequit (Ping timeout: 264 seconds)
05:21:23  * ITproquit (Ping timeout: 276 seconds)
05:52:08  * st_lukequit (Remote host closed the connection)
06:07:06  * AvianFluquit (Remote host closed the connection)
06:47:26  * defunctzombiejoined
06:47:27  * defunctzombiequit (Changing host)
06:47:27  * defunctzombiejoined
06:55:35  * jden|awaychanged nick to jden|zzz
06:57:54  * st_lukejoined
07:07:51  * st_luke_joined
07:22:28  * defunctzombiequit (Remote host closed the connection)
07:27:33  * tphummeljoined
07:29:29  * tphummelquit (Client Quit)
07:50:14  * saijanai_joined
10:26:06  * zz_shykeschanged nick to shykes
10:30:54  * shykeschanged nick to zz_shykes
10:38:55  * st_luke_quit (Remote host closed the connection)
10:54:35  * fotoveritejoined
11:03:27  * jibayjoined
11:20:26  * ITprojoined
11:20:35  * st_luke_joined
11:25:07  * ITproquit (Ping timeout: 255 seconds)
11:25:51  * ITprojoined
11:52:34  * ralphtheninjajoined
11:52:34  * ralphtheninjaquit (Client Quit)
11:52:46  * ralphtheninjajoined
12:01:58  * AvianFlujoined
12:37:55  * ITproquit (Ping timeout: 260 seconds)
12:41:58  <ralphtheninja>twitter is a real time thief
12:45:54  <ralphtheninja>just got a little preview of the vids from node dublin, they look great
13:52:23  * ITprojoined
14:25:13  * defunctzombiejoined
14:25:14  * defunctzombiequit (Changing host)
14:25:14  * defunctzombiejoined
14:30:05  * defunctzombiequit (Ping timeout: 265 seconds)
14:49:06  <juliangruber>real time thief?
14:49:11  <juliangruber>ah, I read realtime :D
14:52:07  <AvianFlu>juliangruber: lol so did I XD
14:53:59  <ralphtheninja>hehe well I guess it's in realtime as well :)
15:03:15  * elliottcablechanged nick to pawsgordon
15:15:34  <juliangruber>rvagg dominictarr: started implementing streams for leveldb binding agnostic. so far only a put stream is implemented. https://npmjs.org/package/streamline-leveldb
15:18:46  * defunctzombiejoined
15:19:18  * lyaunzbejoined
15:20:19  * defunctzombiequit (Remote host closed the connection)
15:21:01  * defunctzombiejoined
15:28:33  * shuaibjoined
15:39:27  * jesusabdullahquit (Ping timeout: 248 seconds)
15:39:30  * jesusabd1llahjoined
15:59:47  * lyaunzbequit (Read error: Operation timed out)
16:04:44  * lyaunzbejoined
16:09:19  * lyaunzbequit (Ping timeout: 252 seconds)
16:19:19  * st_luke_quit (Remote host closed the connection)
16:28:25  * pawsgordonchanged nick to elliottcable
16:52:54  * tphummeljoined
17:09:41  * saijanai_quit (Quit: saijanai_)
17:10:09  * jiang-plusjoined
17:20:13  * tphummelquit (Quit: tphummel)
17:22:36  <ralphtheninja>if anyone of you use emacs I can really recommend dale harveys jshint-mode https://github.com/daleharvey/jshint-mode
17:30:51  * ralphtheninjaquit (Read error: Operation timed out)
17:32:30  * tphummeljoined
17:36:10  <defunctzombie>substack: any action on my part for required?
17:49:13  * ralphtheninjajoined
17:57:02  * CoverSlidequit (Quit: leaving)
18:08:40  * jiang-plusquit (Quit: Lost terminal)
18:28:54  * ircretaryjoined
18:39:20  * lyaunzbejoined
19:12:19  * zz_shykeschanged nick to shykes
19:40:21  * yorickjoined
20:44:01  * shykeschanged nick to zz_shykes
20:53:04  * dominictarrquit (Read error: Connection reset by peer)
20:53:46  * AvianFluquit (Remote host closed the connection)
21:02:30  * dominictarrjoined
21:19:06  <dominictarr>ralphtheninja, wow, just saw the randpeer site
21:19:14  <dominictarr>looks like interesting stuff
21:30:59  * tphummelquit (Quit: tphummel)
21:53:52  * zz_shykeschanged nick to shykes
21:57:10  <ralphtheninja>dominictarr: indeed, I think it has a lot of interesting stuff in there, but I also think that many of their arguments and problems might be non problems running through the browser
21:57:59  <dominictarr>I havn't read all their material yet, but it looks interesting
21:58:39  * shykeschanged nick to zz_shykes
22:11:50  * zz_shykeschanged nick to shykes
22:18:48  * thatguydanjoined
22:51:50  * st_lukequit
23:01:58  * tphummeljoined
23:09:05  <mbalho>guys i had a crazy idea
23:09:28  <mbalho>yehuda katz had this library that i forget the name of in ruby which was basically a high level KV store api
23:09:32  <mbalho>that worked with like every database ever
23:10:14  <mbalho>is it possible to write that for all the leveldb stuff we're doing so one api would work with all the bindings and also be pluggable into other databases like redis or even in memory? or am i crazy
23:10:51  <fotoverite>Not following
23:11:02  <fotoverite>Write an api in node basically?
23:11:31  * tphummelquit (Quit: tphummel)
23:11:38  <mbalho>well here is an example
23:11:42  <mbalho>isaac wrote a module called redsesh
23:11:50  <mbalho>which is for doing sessions in node and persisting them in redis
23:12:11  <mbalho>but i might not want to run redis, i might want to store sessions in memory or use scuttlebutt to replicate sessions to all my servers
23:12:39  <mbalho>but all isaac was really doing was KV store stuff into redis, so i started working on (didnt finish) a redis API clone module that stored things into memory instead
23:12:45  <mbalho>so you could hook it up to scuttlebutt
23:12:51  <mbalho>but now there are all these competing leveldb apis coming out
23:12:56  <rvagg>if you just want get/put/del then the 3 leveldb bindings are all pretty much the same
23:13:13  <mbalho>and i feel like we should consider the possibility of a 'standardized' api for common things
23:13:23  <mbalho>rvagg: yea but it would be etra nice if there was a test suite that proved compatibility or omsething
23:13:39  <rvagg>or y'all could just contribute to levelup to make it the standard binding !
23:14:03  <mbalho>good point, im in the brainstorm phase still
23:16:28  <rvagg>I think my8bird/leveldb isn't worth caring about at this stage unless someone comes along to revive it, it's CoffeeScript too so meh.. I'm hoping that we can integrate some of the performance lessons that leveled can teach into levelup too and I'm thinking that the batch-get might be worth integrating too for smaller batches
23:17:19  <mbalho>agreed
23:18:24  <rvagg>need to flesh out the benchmarks more and I'd also like to benchmark against a sqlite binding too to make sure we can be properly fast
23:18:55  <mbalho>you've seen this right http://www.youtube.com/watch?v=Kdwwvps4J9A&feature=plcp
23:18:57  <rvagg>but... time... the only time I have for it right now is the time created when I'm trying to avoid doing real work
23:19:05  <rvagg>ya
23:19:08  <mbalho>hahaha me too
23:19:17  <Raynos>o/
23:19:31  <mbalho>fucking jifasnif
23:19:37  <rvagg>heh
23:19:39  <Raynos>mbalho: I think jliangruber is writing that
23:20:12  <mbalho>Raynos: https://github.com/juliangruber/streamline-leveldb ?
23:20:19  <Raynos>yeah
23:20:26  <Raynos>not sure where he's going with it.
23:20:31  <Raynos>but he's writing it
23:20:38  <mbalho>juliangruber: you should make this non leveldb specific https://github.com/juliangruber/streamline-leveldb
23:20:38  <Raynos>I do feel that a KV binding is useful
23:20:41  <Raynos>but only for KV based backends
23:20:47  <mbalho>juliangruber: and more broadly useful for all databases
23:21:07  <Raynos>mbalho: have you seen model?
23:21:18  <mbalho>npm link?
23:21:26  <rvagg>mde/model?
23:21:32  <Raynos>https://npmjs.org/package/model
23:21:35  <Raynos>yeah.
23:21:46  <Raynos>when I say x I mean `npm docs x` :D
23:22:03  <mbalho>me too usually but that was a particularly general sounding term
23:22:12  <Raynos>true.
23:22:16  <Raynos>thats mde's fault
23:22:19  <Raynos>for writing a generic model
23:23:31  <mbalho>doesnt have streams :(
23:23:31  <rvagg>also 'downstairs' is another, he's hoping to make it k/v compatible too but it's mainly for sql models
23:26:02  <mbalho>model and downstairs both seem really big in scope compared to what i was thinking
23:26:35  <mbalho>like... npm install streaming-database
23:28:35  <mbalho>but i guess from a design perspective the streaming-database package would just be a test suite or something
23:28:58  <mbalho>and it wouldnt put multiple adapters in one repo like this https://github.com/mde/model/tree/master/lib/adapters
23:29:57  <Raynos>mbalho: well can we just write a test suite for the levelup interface?
23:30:07  <Raynos>oh wait someone already wrote that :D
23:30:41  <mbalho>well what im getting at is non leveldb use cases
23:31:07  <mbalho>but what i havent thought about as much is which databases would it make sense at all to use a leveldb style api on top of
23:31:27  <mbalho>given that leveldb exposes lexicographically sorted one dimensional indexes via single or batch atomic read/writes
23:31:39  <mbalho>you can do that in memory, you can do that in redis, you can do that in sql probably
23:31:54  <mbalho>couch is pretty much the same as leveldb at the api level
23:32:41  <Raynos>mbalho: but why use the leveldb api
23:32:45  <Raynos>on top of sql couch or redis
23:32:51  <Raynos>why not use it on top of level
23:32:58  <Raynos>an in memory clone of leveldb makes sense
23:33:03  <Raynos>and im going to write that
23:33:04  <mbalho>how would you run leveldb on nodejitsu
23:33:07  <rvagg>cause when you love hammers, everything looks like a nail?
23:33:10  <Raynos>well im going to write an in memory cache in front of leveldb
23:33:31  <Raynos>mbalho: you punch nodejitsu in the face until it lets you install arbitary shit on the smartos vm
23:33:43  <rvagg>levelup works on nodejitsu... minus persistence across drones of course... but that's why we need levelup-AAS !
23:33:56  <Raynos>levelup-AAS ?
23:34:01  <Raynos>as a service >_<
23:34:06  <mbalho>nodejitsu ensures disk persistence?? i did not realize
23:34:22  <mbalho>heroku doesnt
23:34:30  <rvagg>only on the current drone
23:35:06  <rvagg>mind you... I've never actually used levelup on nodejitsu for anything real, I just know it compiles and is usable there
23:35:52  <mbalho>anyway its not a hammer/nail thing, its just annoys me that modules like redsess are written in a database specific manner even though they could have just as easily been made generic but there is no pattern that people can use easily when authoring those types of modules
23:35:55  <Raynos>but I do agree
23:36:01  <Raynos>it feels that there is a more generic thing
23:36:13  <Raynos>that we should have
23:36:28  <Raynos>what does redis give you that leveldb doesnt have?
23:36:29  <mbalho>we have module.exports = function(){}, cb(err, data) and .pipe()
23:36:33  <Raynos>does redis have range queries?
23:36:46  <mbalho>Raynos: it has sets and set intersection stuff
23:36:50  <rvagg>Raynos: give us memory caching and bingo!
23:37:11  <Raynos>memory caching is interesting
23:37:14  <rvagg>with lru
23:37:18  <Raynos>because you can _optionally_ persist to the real thing
23:37:35  <Raynos>so ill write that
23:37:59  <rvagg>ya, we could make a redis clone backed by leveldb for persistence, that'd be neat
23:38:07  <rvagg>but again... avoiding real work
23:38:22  <mbalho>well now there are 5 people working on this stuff, roughly
23:38:31  <mbalho>but if we increased that number to 10 then more things would happen :D
23:38:43  <rvagg>perhaps nearForm wants to 'sponsor' more of us
23:38:44  <mbalho>and it turns out there are lotsa node programmers that use databases
23:40:33  <mbalho>anyway, on the other hand for things like redsess i could just fork it and write a new module based on it that is levelup specific and it would be less work than convincing a module author to switch their development approach
23:41:15  <Raynos>mbalho: what we really want
23:41:23  <Raynos>is `var session = RedSess(db, ...)`
23:41:31  <Raynos>and have db confirm to the "kv interface"
23:41:46  <mbalho>npm install lexicographically-sorted-one-dimensional-index
23:42:08  <Raynos>:D
23:43:09  <mbalho>Raynos: but redsess has red from redis in the name, might as well fork it to be generic. it isnt a large module
23:43:28  <mbalho>but i really want a session store that uses scuttlebutt to replicate between nodes, that would be super cool
23:44:26  <rvagg>mmmm
23:45:12  <mbalho>i wonder how many keys you can store in npm lru-cache before it hits the v8 memory limit
23:45:23  * paul_irish_changed nick to paul_irish
23:50:01  <dominictarr>mbalho, rvagg Raynos so juliangruber and I where talking about building a compatible API across levelup/leveled and using a separate test suite
23:51:04  <dominictarr>the problem with using a compatibility layer over a network accessable db is that those databases
23:51:35  <dominictarr>can be accessed by multiple processes, which it's assumed cannot be done via leveldb
23:51:57  <mbalho>its not just assumed it is confirmed :D
23:52:17  <dominictarr>yeah, so you'd want to run one db per client
23:52:25  <dominictarr>and then replicate between the clients
23:52:27  <mbalho>you can have 1 node process open many leveldbs
23:52:35  <dominictarr>yes
23:52:54  <dominictarr>although, you can't do atomic inserts across multiple db
23:53:32  <dominictarr>replication between your multiple processes is a better idea, nodejitsu compatible too
23:53:34  <mbalho>so redis, for instance, hosts an application server
23:53:38  <mbalho>that many node processes can talk to
23:53:53  <mbalho>if you were to use a compatibility layer to do redis on leveldb or something
23:54:01  <mbalho>node would be both the application server and the many clients
23:54:10  <dominictarr>yes
23:54:47  <dominictarr>most of the highlevel features that I am building depend on being in control of all the inputs
23:55:05  <mbalho>so what scenario were you referring to above when you mentioned lotsa dbs assume multiple process access
23:55:10  <dominictarr>so, you can intercept (or detect) when a user inserts something
23:55:20  <dominictarr>mysql, couch, etc
23:55:43  <dominictarr>databases which are not designed to be embedded in the user process.
23:55:59  <mbalho>so you make a local proxy compatibility server?
23:56:04  <dominictarr>pretty much all db except sqlite, leveldb, and tokyo cabnet
23:56:05  <mbalho>is that too much indirection
23:56:33  <dominictarr>well, you could have one process, and one db running on the same machine
23:56:41  <dominictarr>and stick to that
23:57:03  <dominictarr>you'd need a wacky reason not to just use leveldb
23:57:33  <mbalho>ok another crazy idea
23:57:47  <mbalho>npm install streaming-database; db = streamingDatabase(uri)
23:57:56  <mbalho>where uri is either 'idb://foobar' in browsers
23:58:03  <mbalho>'http://foobar' for http+json
23:58:08  <mbalho>'file://foobar' for leveldb
23:59:12  <dominictarr>well, we argeady have 2 idb->leveldb polyfills