00:01:36  * pfrazejoined
00:03:53  * eugenewarequit (Ping timeout: 245 seconds)
00:24:14  * ncthom91quit (Quit: My MacBook has gone to sleep. ZZZzzz…)
00:54:45  * nnnnathann_joined
00:54:53  * nnnnathannquit (Ping timeout: 272 seconds)
00:54:56  * nnnnathann_quit (Remote host closed the connection)
00:55:28  * nnnnathannjoined
01:24:57  <prettyrobots>ogd: Two things needed...
01:25:19  <prettyrobots>ogd: One some balance, merge and purge heuristics for Locket.
01:25:45  <prettyrobots>ogd: Two, an error reporting strategy for background operations; balancing, merging and purging.
01:26:00  <prettyrobots>Three, some real world data.
01:26:15  <prettyrobots>Three! Three things!
01:26:24  <prettyrobots>No body expects the Spanish Inquisition.
01:26:50  <prettyrobots>ogd, rescrv: So, I made a mistake.
01:27:01  <prettyrobots>Here's this:
01:27:02  <prettyrobots>https://github.com/bigeasy/locket/issues/105
01:27:08  <prettyrobots>That is an ogd idea of a test.
01:27:26  <prettyrobots>Here is my idea of a test:
01:27:27  <prettyrobots>https://github.com/bigeasy/locket/blob/master/t/locket/put.t.js
01:27:51  <prettyrobots>You can see I've tested Locket directly, but people are only going to use LevelUP, which adds things.
01:28:20  <prettyrobots>Please have a look here to help me de-dumbify my architectural understanding of LevelDB.
01:28:21  <prettyrobots>https://github.com/bigeasy/locket/issues/105
01:28:57  <prettyrobots>Note that Locket is rather stable.
01:28:58  <prettyrobots>https://www.prettyrobots.com/coverage/bigeasy/locket/119/119.1/lcov-report/locket/index.js.html
01:28:59  * blessYahujoined
01:29:23  <prettyrobots>And Strata:
01:29:23  <prettyrobots>https://www.prettyrobots.com/coverage/bigeasy/strata/546/546.1/lcov-report/strata/index.js.html
01:29:35  <prettyrobots>So, I'm only trying to unpack the last bits here.
01:30:58  <prettyrobots>How do you *know* you got a key not found?
01:31:45  * blessYahuquit (Excess Flood)
01:31:55  * blessYahujoined
01:33:09  <prettyrobots>404 and notFound
01:33:14  <prettyrobots>Fine. Got it.
01:39:30  <rvagg>and error instanceof NotFoundError
01:39:33  <rvagg>but that's levelup only
01:39:37  <rvagg>which you probably know
01:39:53  <rvagg>see the abstract-leveldown tests for how to actually tell when it comes from the underlying db
01:40:30  <rvagg>(/notfound/g).test(error.message) or something
01:40:38  <rvagg> /notfound/i sorry
01:41:57  <prettyrobots>rvagg: https://github.com/bigeasy/locket/issues/105
01:42:14  <prettyrobots>Or shorter: I'd like to test Locket through LevelUP so that my tests reflect actual API usage.
01:43:04  <prettyrobots>CC'd you on a GitHub message.
01:43:29  <prettyrobots>rvagg: I don't know much about LevelUP, but I know a bit about AbstractLevelDOWN now.
01:48:50  <ogd>prettyrobots: levelup is supposed to work 100% on top of any leveldown implementation, so I don't know why my test case fails because theoretically if locket passes all of the leveldown tests (it does) then levelup should work on top of it
01:50:15  <ogd>prettyrobots: levelup doesnt have an equivalent of abstract-leveldown (e.g. the tests in levelup arent abstract yet)
01:51:15  * prettyrobots_quit (Ping timeout: 245 seconds)
01:52:05  * prettyrobotsquit (Ping timeout: 245 seconds)
01:53:33  * brycebarilquit (Ping timeout: 252 seconds)
01:55:53  * prettyrobotsjoined
01:56:18  * prettyrobotschanged nick to Guest72605
01:56:27  * prettyrobots_joined
01:56:30  * prettyrobots_changed nick to Guest58729
01:56:35  * Guest72605changed nick to prettyrobots
02:00:50  * brycebariljoined
02:00:51  * nnnnathannquit (Ping timeout: 276 seconds)
02:01:19  * nnnnathannjoined
02:03:51  * Punnaquit (Ping timeout: 260 seconds)
02:05:58  * blessYahuquit (Ping timeout: 245 seconds)
02:06:07  * ednapiranhajoined
02:07:05  * Guest58729quit (Ping timeout: 245 seconds)
02:09:29  * prettyrobots__joined
02:33:15  * eugenewarejoined
02:33:16  * eugenewarequit (Remote host closed the connection)
02:33:43  * eugenewarejoined
03:05:08  * kenansulaymanchanged nick to kenan|afk
03:06:18  * kenan|afkchanged nick to kenansulayman
03:08:39  * thlorenzjoined
03:13:35  * nnnnathannquit (Ping timeout: 272 seconds)
03:59:38  * Sorellaquit (Quit: It is tiem!)
04:08:24  * nnnnathannjoined
04:12:54  * ednapiranhaquit (Quit: Leaving...)
04:25:33  * mafintoshquit (Quit: Leaving...)
04:40:21  * mikealjoined
04:41:01  * ncthom91joined
04:52:23  * thlorenzquit (Remote host closed the connection)
04:52:59  * thlorenzjoined
04:57:43  * thlorenzquit (Ping timeout: 272 seconds)
05:03:12  * nnnnathannquit (Remote host closed the connection)
05:09:46  * ncthom91quit (Quit: My MacBook has gone to sleep. ZZZzzz…)
05:12:03  * mikealquit (Quit: Leaving.)
05:17:27  * mikealjoined
05:23:55  * mikealquit (Quit: Leaving.)
05:28:44  * mikealjoined
05:30:22  * eugenewarequit (Remote host closed the connection)
05:41:12  * eugenewarejoined
05:44:53  * mikealquit (Quit: Leaving.)
05:47:04  * ncthom91joined
05:54:47  * guybrushquit (Excess Flood)
05:54:59  * guybrushjoined
06:24:27  * jcrugzzquit (Ping timeout: 265 seconds)
07:07:36  * DirtyColombianquit (Ping timeout: 251 seconds)
07:10:25  * eugenewarequit (Remote host closed the connection)
07:14:32  * eugenewarejoined
07:17:38  * DirtyColombianjoined
07:18:31  * mikealjoined
07:19:33  * eugenewarequit (Ping timeout: 265 seconds)
07:43:28  * eugenewarejoined
07:53:53  * eugenewarequit (Ping timeout: 245 seconds)
07:58:21  * eugenewarejoined
08:02:59  * eugenewarequit (Ping timeout: 240 seconds)
08:10:08  * ncthom91quit (Quit: My MacBook has gone to sleep. ZZZzzz…)
08:20:10  * eugenewarejoined
08:24:36  * eugenewarequit (Ping timeout: 252 seconds)
08:41:50  * eugenewarejoined
08:43:05  * book`quit (Read error: Operation timed out)
08:43:49  * book`joined
08:46:15  * eugenewarequit (Ping timeout: 245 seconds)
08:52:29  * pfrazequit (Ping timeout: 240 seconds)
09:03:34  * eugenewarejoined
09:08:09  * eugenewarequit (Ping timeout: 252 seconds)
09:09:17  * eugenewarejoined
09:14:13  * eugenewarequit (Ping timeout: 272 seconds)
09:25:18  * eugenewarejoined
09:30:03  * eugenewarequit (Ping timeout: 265 seconds)
09:42:33  * frankbuttjoined
09:42:35  * frankbuttpart
09:47:08  * eugenewarejoined
09:51:48  * eugenewarequit (Ping timeout: 245 seconds)
09:56:55  * mafintoshjoined
09:59:44  * jjmalinajoined
10:08:57  * eugenewarejoined
10:09:17  * dljcollettejoined
10:13:29  * eugenewarequit (Ping timeout: 240 seconds)
10:14:57  * dljcollettequit (Quit: Textual IRC Client: www.textualapp.com)
10:16:49  * dljcolle-joined
10:17:24  * dljcolle-changed nick to dljcollette
10:30:32  * eugenewarejoined
10:35:17  * eugenewarequit (Ping timeout: 272 seconds)
10:37:56  * werebuttjoined
10:37:57  * werebuttpart
10:52:17  * eugenewarejoined
10:53:45  * daviddiasjoined
10:56:37  * eugenewarequit (Ping timeout: 248 seconds)
11:15:58  * mafintoshquit (Quit: Leaving...)
11:54:09  * Sorellajoined
11:57:28  * eugenewarejoined
12:02:15  * eugenewarequit (Ping timeout: 252 seconds)
12:06:41  * cwmmajoined
12:40:56  * eugenewarejoined
12:45:48  * eugenewarequit (Ping timeout: 265 seconds)
13:00:17  * eugenewarejoined
13:22:30  * jjmalina1joined
13:23:17  * eugenewarequit (Remote host closed the connection)
13:23:28  * blessYahujoined
13:24:31  * eugenewarejoined
13:24:50  * blessYahuquit (Read error: Connection reset by peer)
13:24:57  * jjmalinaquit (Ping timeout: 265 seconds)
13:25:09  * blessYahujoined
13:29:23  * eugenewarequit (Ping timeout: 260 seconds)
13:38:31  * jjmalina1part
13:39:07  * jjmalinajoined
13:46:14  * eugenewarejoined
13:48:31  * brianloveswordsquit (Excess Flood)
13:49:05  * brianloveswordsjoined
13:51:03  * eugenewarequit (Ping timeout: 265 seconds)
14:00:18  * eugenewarejoined
14:08:00  * eugenewa_joined
14:08:00  * eugenewarequit (Read error: Connection reset by peer)
14:12:30  * eugenewa_quit (Ping timeout: 245 seconds)
14:29:44  * eugenewarejoined
14:34:10  * eugenewarequit (Ping timeout: 245 seconds)
14:41:28  * pfrazejoined
14:45:56  * jcrugzzjoined
15:08:35  * thlorenzjoined
15:28:40  * thlorenzquit (Remote host closed the connection)
15:34:54  * eugenewarejoined
15:39:50  * eugenewarequit (Ping timeout: 264 seconds)
15:45:26  * thlorenzjoined
15:51:54  * ednapiranhajoined
15:56:05  * tarrudajoined
15:56:42  * eugenewarejoined
15:59:45  * paulfryzeljoined
16:02:05  * eugenewarequit (Ping timeout: 272 seconds)
16:06:44  * jerrysvjoined
16:37:28  * thlorenzquit (Remote host closed the connection)
16:39:31  * thlorenzjoined
16:40:10  * eugenewarejoined
16:45:07  * kenansulaymanchanged nick to kenan|afk
16:45:14  * eugenewarequit (Ping timeout: 264 seconds)
16:58:18  * mikealquit (Quit: Leaving.)
16:58:58  * mikealjoined
17:01:55  * eugenewarejoined
17:06:29  * eugenewarequit (Ping timeout: 240 seconds)
17:06:58  * jerrysvquit (Read error: Connection reset by peer)
17:07:09  * jerrysvjoined
17:10:22  * mikealquit (Quit: Leaving.)
17:20:15  * jcrugzzquit (Ping timeout: 252 seconds)
17:39:32  * rcsolejoined
17:45:23  * eugenewarejoined
17:49:14  * rescrvquit (Quit: WeeChat 0.3.8)
17:50:15  * eugenewarequit (Ping timeout: 260 seconds)
18:07:09  * eugenewarejoined
18:07:45  * paulfryz_joined
18:08:08  * daviddiasquit (Remote host closed the connection)
18:08:37  * daviddiasjoined
18:10:58  * paulfryzelquit (Ping timeout: 245 seconds)
18:12:11  * eugenewarequit (Ping timeout: 272 seconds)
18:13:26  * daviddiasquit (Ping timeout: 264 seconds)
18:32:28  * rcsolequit (Quit: Page closed)
18:33:33  * pfrazequit (Read error: Connection reset by peer)
18:33:58  * pfrazejoined
18:40:25  * mikealjoined
19:07:46  * pfraze_joined
19:10:31  * pfrazequit (Ping timeout: 260 seconds)
19:12:38  * pfraze_changed nick to pfraze
19:15:19  <rvagg>prettyrobots: sorry, kind of snowed under atm, will try and get to it; I'm keen to help but it's just a matter of finding the time
19:16:03  * mikealquit (Quit: Leaving.)
19:16:53  * daviddiasjoined
19:21:23  * daviddiasquit (Ping timeout: 245 seconds)
19:23:17  * mikealjoined
19:32:51  * mikealquit (Quit: Leaving.)
19:33:44  * eugenewarejoined
19:37:59  * eugenewarequit (Ping timeout: 240 seconds)
19:40:15  * mikealjoined
20:08:25  * tarrudaquit (Quit: WeeChat 0.4.2)
20:11:36  * daviddiasjoined
20:12:16  * mikealquit (Quit: Leaving.)
20:15:58  * daviddiasquit (Ping timeout: 245 seconds)
20:17:34  * eugenewarejoined
20:21:37  * mikealjoined
20:21:52  <thlorenz>rvagg: have some weird issue with through2
20:22:09  <thlorenz>if I can log `src` right below that line: https://github.com/thlorenz/transfigurify/blob/async/index.js#L33
20:22:23  * eugenewarequit (Ping timeout: 260 seconds)
20:22:30  <thlorenz>shouldn't I get a callback here? https://github.com/thlorenz/transfigurify/blob/async/index.js#L78
20:22:35  <thlorenz>never hits though
20:25:31  * mikealquit (Client Quit)
20:26:35  <thlorenz>however if I push and cb here https://github.com/thlorenz/transfigurify/blob/async/index.js#L29 it does callback, seems like nested async functions are too much -- makes no sense though
20:33:55  <thlorenz>rvagg: nm I think I have an entire different problem going on
20:34:27  <rvagg>phew
20:39:16  * eugenewarejoined
20:42:11  * jcrugzzjoined
20:43:59  * eugenewarequit (Ping timeout: 240 seconds)
20:52:59  * jcrugzzquit (Read error: Connection reset by peer)
20:56:40  * mikealjoined
21:01:02  * eugenewarejoined
21:02:11  * blessYahu_joined
21:05:33  * eugenewarequit (Ping timeout: 245 seconds)
21:06:12  * jcrugzzjoined
21:06:20  * daviddiasjoined
21:06:33  * mikealquit (Quit: Leaving.)
21:09:44  * blessYahu_quit (Remote host closed the connection)
21:10:58  * daviddiasquit (Ping timeout: 245 seconds)
21:11:00  * mikealjoined
21:19:45  * rescrvjoined
21:20:53  * mikealquit (Quit: Leaving.)
21:22:37  * mikealjoined
21:22:44  * eugenewarejoined
21:26:00  * mikealquit (Client Quit)
21:26:08  * mikealjoined
21:26:59  * eugenewarequit (Ping timeout: 240 seconds)
21:33:11  * jerrysv_joined
21:33:44  * jerrysvquit (Read error: Connection reset by peer)
21:42:07  * mikealquit (Quit: Leaving.)
21:44:28  * eugenewarejoined
21:48:59  * eugenewarequit (Ping timeout: 252 seconds)
21:50:38  * mikealjoined
21:53:11  * mikealquit (Client Quit)
21:53:23  * dominictarrjoined
21:53:32  * mikealjoined
21:56:10  * mikealquit (Client Quit)
21:56:20  * mikealjoined
21:56:57  <dominictarr>I made a new base64 encoding that sorts correctly!
21:56:58  <dominictarr>https://github.com/dominictarr/d64
21:57:38  * mikealquit (Client Quit)
22:01:07  * daviddiasjoined
22:02:56  <pfraze>dominictarr cool idea
22:05:33  * daviddiasquit (Ping timeout: 245 seconds)
22:06:12  * eugenewarejoined
22:11:07  * eugenewarequit (Ping timeout: 260 seconds)
22:14:33  * dominictarrquit (Ping timeout: 252 seconds)
22:15:14  * cwmmaquit (Quit: cwmma)
22:23:47  * paulfryz_changed nick to paulfryzel
22:24:12  * mikealjoined
22:27:56  * eugenewarejoined
22:28:28  * jjmalinaquit (Quit: Leaving.)
22:30:01  * mikealquit (Quit: Leaving.)
22:32:51  * eugenewarequit (Ping timeout: 272 seconds)
22:33:36  <thlorenz>rvagg: my lesson for today: "don't cache transform streams, instead cache the functions to make them"
22:33:44  <thlorenz>at least my problem is solved :)
22:34:19  <rvagg>thlorenz: ah, you were reusing the stream?
22:34:34  <thlorenz>yep without much thinking :(
22:35:05  <rvagg>thlorenz: you can do it with through2.ctor()
22:35:12  <rvagg>thlorenz: i.e. make a reusable form
22:35:24  <rvagg>thlorenz: https://github.com/rvagg/through2/blob/master/through2.js#L48-L66
22:35:47  <thlorenz>rvagg: it's ok, I just needed to change this line: https://github.com/thlorenz/transfigurify/blob/async/resolve-transforms.js#L44
22:36:12  <rvagg>righto
22:36:13  <thlorenz>to not call (file), but just cache the function and call it on each use, but good to know through2 has that feature
22:36:31  <thlorenz>rvagg: I also saw through2.obj() +1
22:36:34  * daviddiasjoined
22:36:41  <thlorenz>better than passing that verbose opts thing
22:37:00  <rvagg>thlorenz: yeah, totally, I'm more often using object streams than binary streams so it's a biggie for me
22:37:37  <rvagg>thlorenz: there's also a PR to add through2.term() to make a terminator stream (i.e. a /dev/null stream), haven't merged it yet, still pondering
22:38:03  <thlorenz>ah, interesting you could just use: https://github.com/thlorenz/dev-null
22:38:09  <thlorenz>and pipe through into that
22:38:11  * kenan|afkchanged nick to kenansulayman
22:38:44  <thlorenz>I use that for tests all the time, where I have to drain the response to be able to close the test server
22:39:21  <thlorenz>rvagg: I'd argue though that the dev/null stuff is not through's responsibility
22:39:32  <thlorenz>just use the module for it that I posted
22:39:46  <rvagg>thlorenz: yeah, brycebaril has a devnull too, which is why I'm wavering, if there are others in npm then perhaps it doesn't belong in through2
22:40:02  <thlorenz>agreed, I'll comment on the PR
22:40:33  <rvagg>thlorenz: https://github.com/thlorenz/dev-null/blob/master/index.js#L21 why setImmediate?
22:40:51  <thlorenz>to have consistent behavior
22:41:21  <thlorenz>that can be important if you depend on something happening on the next loop
22:42:19  <thlorenz>rvagg: may actually not apply in this case, but rather be on the safe side
22:42:58  <thlorenz>rvagg: I can't find brycebaril's dev null
22:43:37  <rvagg>it's in terminus
22:43:52  <rvagg>https://github.com/brycebaril/node-terminus#terminusdevnulloptions
22:44:17  <thlorenz>ah, ok, thanks I'll link that too
22:47:21  <ggreer>in general, you probably want to use setImmediate instead of process.nextTick
22:48:48  <ggreer>setImmediate sticks the function after any IO in the event queue. nextTick sticks the function first in the event queue. that means recursive calls to nextTick can cause IO starvation
22:53:24  <kenansulayman>Just finished my leveldb talk, really a success! 3 companies in berlin are interested in me introducing them to level for production workloads; overall, a really enthusiastic audience
22:53:48  <rvagg>kenansulayman: fantastic
22:54:14  <rvagg>ggreer: my point is-why defer at all, just call the callback and be done with it
22:54:20  <ggreer>because you could blow the stack
22:54:35  <rvagg>kenansulayman: are you still using lmdb? I heard of a possible *very significant* new user of it so it may get some more love soon
22:54:58  <rvagg>ggreer: not with the stream, it's already taken care of in ReadableStream
22:55:32  <ggreer>ah, you're doing this after async stuff has happened, so there's no chance of a huge stack?
22:57:23  <kenansulayman>rvagg yes, but the issue with the memory allocation is *really* an issue. if it was able to scale over the amount of available memory (like hyperlevel afaik) we'd love to continue with it
22:57:58  <kenansulayman>Right now, we translated every lmdb database to hyperlevel because there was an unexpected peak of users which wasn't doable with our memory anymore
23:02:25  <ggreer>https://github.com/rvagg/node-leveldown/pull/88 <-- rvagg: do I need to do anything for this? it seemed like the ralphtheninja came to the same conclusion as me
23:04:28  <rescrv>kenansulayman: any chance you guys have written about your experiences with hyperleveldb anywhere?
23:06:33  <kenansulayman>rescrv No, sorry. Nothing explicitly on it, yet. I'll ping you should be publish anything alike soon
23:06:57  <rescrv>kenansulayman: please do! It'll help us secure more resources to keep improving it
23:10:54  <kenansulayman>rescrv Sure :) I'd love to see it improve. We have some issues with scaling, though, already. As you know, I'd love to see HyperDex and it's inception in Node!
23:11:27  * eugenewarejoined
23:11:39  <rvagg>hyperleveldb as a gateway drug for hyperdex in node
23:12:02  <rvagg>ggreer: I think it's good to go, just need patience now, I'm swamped atm
23:12:08  <ggreer>no worries
23:12:13  <ggreer>I just wanted to make sure it wasn't blocked on me
23:13:57  <rescrv>kenansulayman: I'd love to too. Our current work-selection function is to find something people need and implement it. We only have so many resources, so we prioritize by the likelihood of it going into someone's application. Can you give me an estimate of how serious about it you are? If you have sensitive info, you can send it to support@hyperdex.org rather than putting it out in the channel.
23:16:11  * eugenewarequit (Ping timeout: 272 seconds)
23:17:38  * daviddiasquit (Ping timeout: 245 seconds)
23:20:19  <kenansulayman>rescrv The topic actually also captures financing; we're currently working towards a A financing round and will see infrastructure construction with the investment; However, we will definitely use it, when it becomes available. Our current solution of scaling is qualitatively limited — (strange self constructed sharding, where we direct a user to a specific server in order to be able to handle the load). And since I am aware of
23:20:19  <kenansulayman> hyperlevel and it's the very base of every data storage we do (even for avatar caching), it's not a real topic on how serious we're going to use it; more of when we'll be able to use it
23:20:21  * thlorenzquit (Remote host closed the connection)
23:23:32  * eugenewarejoined
23:27:16  <rescrv>kenansulayman: can you send that in an email to support@hyperdex.org? Even if you just format it in paragraphs, that's good enough. We can then use it to justify node.js. Our previous attempt saw little use, so I have to convince the neysayers that round two will be worth it.
23:31:44  <kenansulayman>rescrv Sure! I'll right do it tomorrow. It's 12:30 am already; I'll do my best to explain as best as possible on how we're requiring it
23:32:11  <rescrv>kenansulayman: sure thing! Thanks!
23:32:25  <rescrv>btw, I've pushed changes to HyperLevelDB that make it compatible with *.ldb files.
23:32:40  <rescrv>I'm going to look into sending rvagg a pull request for level-hyper
23:33:04  <kenansulayman>Nice! Thanks for that
23:34:10  <rvagg>rescrv: https://github.com/rvagg/node-leveldown/issues/89 got a release coming up, will sync up to latest hyperleveldb when that's done
23:34:25  <rvagg>i.e. the forks get a simultaneous release when leveldown is released
23:35:56  <rescrv>rvagg: I've not cut a release of hyperleveldb. I was just going to throw in the code from git. What should I do to best mesh with your workflow?
23:36:59  <rvagg>rescrv: I think I'm still using a git submodule for hyperleveldb but I really hate submodules so perhaps I'll just use a plain directory
23:37:24  <rescrv>so should I just tag a version and make that tag known to you?
23:37:44  <rvagg>rescrv: yeah, do that and put the tag in that leveldown issue and I'll use that
23:38:04  <rvagg>for reference, node-leveldown repo, hyper-leveldb is the branch that releases come from