00:14:32  <chrisdickinson>i keep getting "the data provided does not meet requirements"
01:02:15  * werlejoined
01:27:27  * dominictarrquit (Quit: dominictarr)
01:59:39  * werlequit (Quit: Leaving.)
03:22:23  * mreviljoined
03:41:53  * mrevil_joined
03:42:05  * mrevil_quit (Remote host closed the connection)
03:44:38  * mrevilquit (Ping timeout: 256 seconds)
04:18:05  * levelbotquit (Remote host closed the connection)
04:18:22  * levelbotjoined
04:18:30  * levelbotquit (Remote host closed the connection)
04:18:48  * levelbotjoined
04:19:05  <rvagg>made levelbot a bit cleverer, looks at deps now too
04:19:16  <rvagg>if anyone wants to hack: http://github.com/rvagg/levelbot
04:19:23  <rvagg>err: https://github.com/rvagg/level-bot
04:21:52  * levelbotquit (Remote host closed the connection)
04:22:10  * levelbotjoined
04:26:20  * levelbotquit (Remote host closed the connection)
04:26:48  * levelbotjoined
04:31:33  * levelbotquit (Remote host closed the connection)
04:31:53  * levelbotjoined
04:33:17  <levelbot>[npm] abstract-leveldown@0.3.0 <http://npm.im/abstract-leveldown>: An abstract prototype matching the LevelDOWN API (@rvagg)
04:33:26  * levelbotquit (Remote host closed the connection)
04:34:08  * levelbotjoined
04:34:09  <levelbot>[npm] abstract-leveldown@0.3.0 <http://npm.im/abstract-leveldown>: An abstract prototype matching the LevelDOWN API (@rvagg)
04:34:11  <rvagg>sorry for the channel spam, should be finished now
05:32:08  * ralphtheninjaquit (Ping timeout: 252 seconds)
08:45:19  * jez0990quit (Ping timeout: 260 seconds)
09:05:24  * dominictarrjoined
09:28:41  * dominictarrquit (Quit: dominictarr)
10:16:22  * dominictarrjoined
10:17:28  * levelbotquit (Ping timeout: 248 seconds)
10:17:57  * levelbotjoined
10:26:28  * mbalhoquit (*.net *.split)
10:26:28  * chrisdickinsonquit (*.net *.split)
10:26:28  * owen1quit (*.net *.split)
10:26:28  * wolfeidauquit (*.net *.split)
10:26:29  * juliangruberquit (*.net *.split)
10:28:06  * chrisdickinsonjoined
10:28:06  * owen1joined
10:28:06  * wolfeidaujoined
10:28:06  * mbalhojoined
10:28:06  * juliangruberjoined
10:34:21  * mbalhoquit (*.net *.split)
10:34:21  * chrisdickinsonquit (*.net *.split)
10:34:21  * owen1quit (*.net *.split)
10:34:21  * wolfeidauquit (*.net *.split)
10:34:22  * juliangruberquit (*.net *.split)
10:34:22  * Pwnnaquit (*.net *.split)
10:34:22  * chapelquit (*.net *.split)
10:34:22  * rvaggquit (*.net *.split)
10:34:23  * levelbotquit (*.net *.split)
10:34:23  * dominictarrquit (*.net *.split)
10:34:23  * Guest28778quit (*.net *.split)
10:34:23  * Raynosquit (*.net *.split)
10:34:24  * brianloveswordsquit (*.net *.split)
10:41:59  * juliangruberjoined
10:42:00  * mbalhojoined
10:42:00  * wolfeidaujoined
10:42:00  * owen1joined
10:42:00  * chrisdickinsonjoined
10:42:00  * levelbotjoined
10:42:00  * dominictarrjoined
10:42:00  * Pwnnajoined
10:42:00  * Guest28778joined
10:42:00  * chapeljoined
10:42:00  * rvaggjoined
10:42:00  * Raynosjoined
10:42:00  * brianloveswordsjoined
10:44:19  * dominictarrquit (Quit: dominictarr)
12:14:59  * dominictarrjoined
12:24:42  * werlejoined
12:41:39  * ralphtheninjajoined
12:42:54  * ralphtheninjaquit (Client Quit)
12:43:01  * ralphthe1injajoined
12:51:23  * dominictarrquit (Ping timeout: 245 seconds)
14:14:58  * dominictarrjoined
14:39:08  <juliangruber>dominictarr: the level-rpc benchmarks were wrong...multilevel does 11k ops/s and level-rpc does 25k ops/s
14:39:18  <juliangruber>and with msgpack ca. 22k ops/s
14:39:20  <juliangruber>hmm
14:39:25  <dominictarr>oh, oops
14:39:30  <juliangruber>that seems like the best we can get with node
14:40:03  <dominictarr>there is a c version of msgpack that we can use on the server maybe
14:40:28  <dominictarr>although, it would need to be extended to creationix msgpack to handle buffers.
14:40:36  <juliangruber>yeah
14:40:42  <juliangruber>hmmmmmm
14:40:44  <juliangruber>that sucks
14:41:13  <juliangruber>I don't think C msgpack will be faster
14:41:15  <dominictarr>is that parsing and stringifying ?
14:41:26  <juliangruber>since the orig. level-rpc is super lowlevel
14:41:30  <dominictarr>or pasing or stringifying
14:41:47  <juliangruber>it is db.{get,put} over a stream, without tcp
14:41:55  <juliangruber>so parse and stringify in both directions
14:42:16  <dominictarr>this includes going into the database?
14:42:39  <juliangruber>yes
14:42:51  <juliangruber>so it is about 50% of raw levelup performance
14:42:54  <dominictarr>oh, okay, that is pretty good then!
14:44:05  <juliangruber>cool
14:44:15  <juliangruber>I think I'll use msgpack-js and msgpack-js-browser
14:44:27  <juliangruber>sounds like the best option
14:44:48  <juliangruber>so, multilevel will do only strings and level-rpc only buffers
14:45:04  <juliangruber>for websockets that means that they should be in binary mode for level-rpc
14:45:16  <dominictarr>https://github.com/juliangruber/level-rpc/blob/master/parse.js#L83
14:45:20  <dominictarr>copy?
14:45:48  <dominictarr>we can use msg-pack with the currenty multilevel and mux-demux!
14:47:13  <juliangruber>why not copy?
14:47:50  <dominictarr>because then you have to copy memory
14:47:50  <juliangruber>ok, I'll make multilevel use msgpack-js and msgpack-js-browser
14:48:04  <dominictarr>if you can avoid copying memory it will be faster
14:48:44  <dominictarr>I think buffer has a thing to make a buffer out of other buffers - but just as a linked list
14:48:55  <juliangruber>oh right, buffer#slice
14:48:57  <juliangruber>will try that
14:48:58  <dominictarr>yeah
14:49:11  <juliangruber>cya, having bbq :)
14:49:14  <dominictarr>probably won't work in the browser but might work in node
14:49:22  <dominictarr>okay, catch you later!
15:41:17  <levelbot>[npm] level-hooks@4.1.9 <http://npm.im/level-hooks>: pre/post hooks for leveldb (@dominictarr)
15:41:20  <levelbot>[npm] level-hooks@4.1.9 <http://npm.im/level-hooks>: pre/post hooks for leveldb (@dominictarr)
15:41:32  <levelbot>[npm] level-hooks@4.1.9 <http://npm.im/level-hooks>: pre/post hooks for leveldb (@dominictarr)
15:41:34  <levelbot>[npm] level-hooks@4.1.9 <http://npm.im/level-hooks>: pre/post hooks for leveldb (@dominictarr)
15:41:39  <levelbot>[npm] level-hooks@4.1.9 <http://npm.im/level-hooks>: pre/post hooks for leveldb (@dominictarr)
15:41:39  <dominictarr>thats weird
16:42:11  <mbalho>levelbot: you're still drunk, go home
18:42:02  * _yxajoined
18:42:43  * _yxapart
18:54:51  <juliangruber>dominictarr: over a fake tcp socket, level-rpc does 9k ops/s (multilevel does 8k). with msgpack-js it is 7.5k ops/s. meh
18:55:09  <juliangruber>not copying the buffer anymore, btw
18:57:08  <dominictarr>fake tcp?
18:57:15  <dominictarr>you mean just a stream in memory?
18:57:24  <dominictarr>but still serializing
19:02:23  * werlequit (Quit: Leaving.)
19:05:05  <juliangruber>a local tcp socket that is
19:05:12  <juliangruber>a stream in mem does 20-25k ops/s
19:07:53  * werlejoined
19:11:14  <dominictarr>oh, cool.
19:11:18  <dominictarr>okay, that is good.
19:11:35  <dominictarr>hmm...
19:11:54  <dominictarr>what proportion is leveldb and what proportion is tcp?
19:12:02  <levelbot>[npm] multilevel@4.0.0 <http://npm.im/multilevel>: A levelUp instance exposed over a streaming interface (@juliangruber)
19:12:04  <levelbot>[npm] multilevel@4.0.0 <http://npm.im/multilevel>: A levelUp instance exposed over a streaming interface (@juliangruber)
19:12:09  <levelbot>[npm] multilevel@4.0.0 <http://npm.im/multilevel>: A levelUp instance exposed over a streaming interface (@juliangruber)
19:12:13  <dominictarr>maybe you could try the benchmark into levelmem?
19:12:18  <levelbot>[npm] multilevel@4.0.0 <http://npm.im/multilevel>: A levelUp instance exposed over a streaming interface (@juliangruber)
19:12:21  <levelbot>[npm] multilevel@4.0.0 <http://npm.im/multilevel>: A levelUp instance exposed over a streaming interface (@juliangruber)
19:12:28  <dominictarr>that would be a good comparison with redis
19:12:53  <juliangruber>mbalho dominictarr: https://github.com/juliangruber/multilevel/commit/f407d4bc02ef97ad3a1c5b3ad5fb011d35e1684f
19:13:09  <juliangruber>dominictarr: hm?
19:13:20  <dominictarr>looks good
19:13:28  <juliangruber>how do I track the time spend in tcp? dtrace?
19:13:32  <juliangruber>:)
19:13:41  <dominictarr>I guess so.
19:18:00  <dominictarr>hij1nx: so, what we need is a way to merge keys for lock maybe?
19:18:30  <dominictarr>prehaps the lock needs to be handled differently...
19:19:01  <juliangruber>dominictarr: going over tcp vs just the stream in memory makes it 50-80% slower
19:19:34  <juliangruber>node's net module should be crazy fast, right?
19:19:59  <dominictarr>right - but you cant compare actual io to memory
19:29:12  * werlequit (Quit: Leaving.)
19:29:21  <hij1nx>juliangruber: https://github.com/chrisa/node-dtrace-provider
19:29:54  <hij1nx>btw, i've been working on a dtrace branch of leveldown
19:32:10  <juliangruber>awesome!
19:33:02  * juliangruberis setting up a smartos virtualbox
19:51:12  <dominictarr>hij1nx: this fish thing is actually kinda similar to an idea I had about cad in some ways.
20:02:39  <hij1nx>dominictarr: worrydream does awesome work, he wrote all that software for that talk, to support his talking points. awesome.
20:03:18  <dominictarr>yeah, I really wanted to build something very similar
20:03:32  <dominictarr>but it was just gonna be for, uh, designing boats!
20:03:55  <hij1nx>dominictarr: ah! that would probably be a commercially viable application!
20:04:00  <dominictarr>but a similar idea about using geometry
20:04:19  <dominictarr>this is more original though
20:04:43  <hij1nx>both seasoned builders and amatures could understand the physical implications of their design decisions
20:04:57  <hij1nx>er, physics
20:05:21  <hij1nx>one for bikes would be cool too :)
20:05:28  <dominictarr>yeah, I wanted to make things in terms of parameters
20:05:39  <hij1nx>model a bike, see how it would work, ship it to the 3d printer
20:05:46  <dominictarr>so I could be like "I have this much wood" how long can the boat be
20:05:57  <hij1nx>you could 3d print your boat
20:06:03  <hij1nx>and then assemble it
20:06:11  <dominictarr>rather than how much wood to I need make a boat this long
20:06:28  <dominictarr>hij1nx: or, lazer cut it and then stitch it together.
20:06:47  <hij1nx>that would very cool too
20:07:18  <hij1nx>i wonder how soon it will be until people are 3d printing their houses?
20:07:48  <dominictarr>people are working on it.
20:08:06  <hij1nx>you can land on a plannet and 4d print a science station
20:08:08  <dominictarr>I am pretty excited about the 3d printing thing
20:08:29  <hij1nx>s/plannet/planet
20:08:48  <dominictarr>but we need better software to make it work as good as it can
20:09:00  <dominictarr>so, I'm no this at the moment!
20:09:16  <dominictarr>maybe custom clothing too
20:09:24  <dominictarr>people would be into that, I think.
20:09:58  <hij1nx>yeah, or quickly materialize a whatever you need with a small device, a bike, cloths, a ship, then when your done with it just decomposes
20:10:49  <hij1nx>the device just runs on dirt and garbage
20:11:00  <hij1nx>garbage is an awesome resource
20:11:13  <hij1nx>we should feed our robots garbage
20:11:37  <hij1nx>they will enjoy eating it and then make nice stuff out of it for us :)
20:11:48  <dominictarr>yes absolutely
20:12:20  <hij1nx>newer robots can eat older robots to make newer robots.
20:12:34  <hij1nx>lol. thats something to think about ;)
20:12:41  <dominictarr>and imagine what the world will be like when the sort of stuff that is happening to content industrys starts to happen to people who control physical manufacturing!
20:14:39  <dominictarr>feed a 3d printer some old 2d printers and it has a new 3d printer baby
20:14:42  <hij1nx>there will be new things to sell, like ships that bend space and time. good space transports will be too complex for the individual to manufacture
20:15:30  <dominictarr>yes, probably
20:15:41  <dominictarr>like my computer is pretty good,
20:15:46  <dominictarr>but it's not the whole internet.
21:12:33  <juliangruber>dominictarr: with several clients connected (seperate processes on my machine) one multilevel server handles ~15k ops/s
21:13:03  <dominictarr>SWEET!
21:13:14  <juliangruber>op=put "1234567890abcdef"
21:13:19  <dominictarr>so, it was maxing out one thread!
21:13:38  <juliangruber>I think so
21:13:40  <dominictarr>shit, so that is with regular multilevel?
21:14:12  <juliangruber>oh no...with one client it is the same results
21:14:27  <juliangruber>multilevel with msgpack-js
21:14:29  <dominictarr>but still separate client process
21:14:36  <juliangruber>yes
21:14:48  <juliangruber>so a server das max 15k ops/s
21:14:51  <dominictarr>awesome.
21:14:54  <juliangruber>does
21:15:10  <juliangruber>still using all the sweetness of mux-demux and rpc-stream :)
21:15:18  <dominictarr>so that was the same as mongo, if I recall your previous benchmarking!
21:17:28  <juliangruber>i think so
21:17:30  <juliangruber>oh btw
21:17:47  <juliangruber>multilevel over tcp with multiple clients is just as fast as multilevel over an in-memory stream
21:18:44  <dominictarr>wow
21:19:05  <dominictarr>oh, because of the event loop!
21:19:17  <juliangruber>that's just about the performance of mongodb and redis
21:19:19  <juliangruber>awsm
21:19:28  <juliangruber>yes
21:19:36  <dominictarr>except it's written in javascript!!!
21:19:46  <juliangruber>OMFG!!!
21:20:01  * jez0990joined
21:20:12  <dominictarr>that is the header on hacker news...
21:20:30  <dominictarr>"real written-in-C database shown up by javascript"
21:20:48  <dominictarr>… and die of shame...
21:22:02  <juliangruber>hmmm
21:22:08  <juliangruber>I just looked at the redis benchmarks
21:22:21  <juliangruber>they state >100k ops/s with 3 byte payload
21:22:30  <juliangruber>multilevel still does 15k ops/s with a 3 byte payload
21:22:39  <juliangruber>maybe mongo is comparable
21:23:01  <dominictarr>right - but you have to run these on the correct machine
21:23:05  <juliangruber>and redis cannot be beaten in performance doing just get and set
21:23:08  <juliangruber>mhm true
21:23:12  <dominictarr>I'm sure they don't run their benchmarks on a laptop
21:23:45  <dominictarr>redis isn't even going to disk so there won't be any way to beat that...
21:24:24  <juliangruber>true
21:34:48  * ralphthe1injachanged nick to ralphtheninja
21:36:45  * st_lukejoined
21:45:02  * st_lukequit (Remote host closed the connection)
21:54:57  * st_lukejoined
22:05:00  * st_lukequit (Remote host closed the connection)
22:26:01  * levelbotquit (Remote host closed the connection)
22:26:32  * levelbotjoined
23:06:22  <rvagg>morning ppl
23:07:43  <rvagg>sounds like you're making awesome progress juliangruber, good stuff
23:51:29  <rvagg>btw juliangruber, when you run your benchmarks, turn the cache size right up to act more like redis