01:13:52  * mreviljoined
01:14:19  * timoxleyjoined
01:18:20  * mrevilquit (Ping timeout: 246 seconds)
01:25:57  * ramitosquit (Ping timeout: 252 seconds)
01:29:08  * no9quit (Ping timeout: 256 seconds)
02:06:38  * werlejoined
02:11:55  * Pwnnajoined
02:21:02  * levelbotquit (Remote host closed the connection)
02:21:27  * levelbotjoined
03:18:23  * ralphthe1injaquit (Ping timeout: 246 seconds)
03:47:00  * mreviljoined
03:53:53  <rvagg>new version of second article, https://gist.github.com/rvagg/b1dd8e11d478009cbfda will pass it on soon but would still appreciate feedback
03:56:32  <owen1>rvagg: wow. pretty long!
03:57:09  <owen1>it will be nice to have it in the wiki
03:57:47  <rvagg>owen1: yeah, I'm worried that it's too long
04:04:20  <owen1>but insightful so who cares
04:04:52  <owen1>cutting it to smaller pieces might be annyoing
04:05:15  <rvagg>owen1: yeah, I wish I had have committed to a 4 part series now, but I'm locked in to 3 so this'll have to do!
04:05:52  <owen1>"if you really can't stomach it" love this
04:37:28  * timoxleyquit (Quit: Computer has gone to sleep.)
04:56:25  * mrevilquit (Remote host closed the connection)
04:57:05  * mreviljoined
05:01:32  * mrevilquit (Ping timeout: 252 seconds)
05:35:14  * Pwnnaquit (Ping timeout: 246 seconds)
06:00:02  <mbalho>leveldown question: does 'db.iterator()' (no args) create a full scan iterator?
06:00:24  <chilts>yes if you don't give it a start and end I think
06:00:31  <chilts>if you do, then it'll be for just that range
06:00:42  <mbalho>k
06:00:54  * chiltsis about 80% sure
06:01:00  <chilts>rvagg: ^^^ ?
06:01:03  <mbalho>that makes sense, just checkin
06:01:10  <mbalho>i need to figure out how to do the same in indexeddb now
06:01:20  <mbalho>e.g. tell it to start from the beginning
06:03:03  <mbalho>oh i think i just pass it nothing
06:03:05  <mbalho>ahha
06:03:15  <mbalho>less is more
06:06:48  <rvagg>chilts, mbalho correct
06:11:00  * chapelquit (Ping timeout: 264 seconds)
06:12:15  * keslajoined
06:12:34  * chapeljoined
06:20:27  * keslaquit (Quit: kesla)
06:35:20  <mbalho>rvagg: running this https://github.com/rvagg/node-abstract-leveldown/blob/master/abstract/iterator-test.js#L105-L113 causes a DataError: https://developer.mozilla.org/en-US/docs/IndexedDB/IDBObjectStore#Exceptions
06:35:46  <mbalho>rvagg: but changing them to {type: 'put', value: {'hello': 'world'}} works
06:36:13  <mbalho>rvagg: do you have any ideas on how to interpret those DataError definitions and how they might be incompatible with the leveldown API?
06:48:50  <mbalho>ahh http://www.w3.org/TR/IndexedDB/#dfn-in-line-keys
06:49:20  <mbalho>wtfz
07:00:49  * mreviljoined
07:05:36  * mrevilquit (Ping timeout: 276 seconds)
07:18:38  * werlequit (Quit: Leaving.)
07:31:49  <mbalho>opened issue here https://github.com/jensarps/IDBWrapper/issues/27
07:31:53  <mbalho>because idb is crazy
07:57:17  * no9joined
08:03:38  * levelbotquit (Ping timeout: 246 seconds)
08:43:46  <rvagg>heh, that sounds interesting, will have a look when I have a moment
09:01:45  * mreviljoined
09:03:53  * dominictarrjoined
09:06:13  * mrevilquit (Ping timeout: 248 seconds)
09:06:34  <rvagg>the latest details here are interesting: http://code.google.com/p/leveldb/issues/detail?id=164
09:06:54  <rvagg>I think LevelDB could do with a bunch of optimisation around its use of thread locking
09:07:12  <rvagg>I wonder how low-hanging that fruit is..
09:10:36  * jez0990_quit (*.net *.split)
09:32:36  <rvagg>mbalho: are you using indexeddb transactions for batches?
10:02:08  * mreviljoined
10:06:52  * mrevilquit (Ping timeout: 268 seconds)
10:19:08  * ralphtheninjajoined
10:34:41  * jez0990joined
10:40:46  * no9quit (Ping timeout: 256 seconds)
10:53:44  * no9joined
10:59:28  * no9quit (Quit: Leaving)
11:24:31  <dominictarr>rvagg: when asm.js arrives it a pure js leveldb might actually be pretty good
11:34:22  * dominictarrquit (Quit: dominictarr)
11:52:00  * Pwnnajoined
12:13:17  * dominictarrjoined
12:42:37  <rvagg>dominictarr: can you have a quick look at those last 2 sections of the article for me? then I can fire it off to alex for publication
12:43:47  <dominictarr>I posted a comment just before
12:43:57  <rvagg>oh, right, ta
12:43:59  <dominictarr>you have updated since - what was the link?
12:44:36  <rvagg>no I haven't updated since, I just didn't get a notification of a comment
12:45:27  <rvagg>ok, so if I remove sections about deferred open then it'll shorten it a bit
12:46:09  <dominictarr>then it will be a good length.
12:46:30  <rvagg>and btw, I use close instead of end cause it's much safer, technically it works properly now but it didn't used to, db.close() on readStream 'end' was likely to involve a crash
12:46:40  <rvagg>but I'll change them, cause it should be fine now
12:50:02  <dominictarr>right - do you need to do db.close() ?
12:50:06  <dominictarr>cos I never do that
12:51:21  <rvagg>I tend to, but I think (without looking at the code) a close should happen when the GC collects your instance or the process closes
12:52:00  <dominictarr>generally, I only want to close when the process exists
12:52:13  <dominictarr>and since it's append only, just crashing should work fine
12:52:37  <dominictarr>something deletes the lock file, which is okay.
12:53:16  <rvagg>ok, 'close' all replaced with 'end'
12:53:30  <rvagg>removed reference to deferred open and just showed the one way to open a db
12:54:52  <rvagg>wolfeidau reckoned my north korea data set was silly and I should replace it with something like geo data
12:54:55  <rvagg>http://leafletjs.com/examples/geojson.html
12:57:39  <rvagg>I think I'll leave it cause multi-dimensional objects are just going to complicate the comparison of multiple entries vs a json blob of the same data
12:58:56  <dominictarr>yeah. that would be a really good example for a more advanced post though
13:01:28  * werlejoined
13:09:10  * dubbanjoined
13:42:47  <werle>morning guys
13:42:55  <werle>rvagg: had lots of fun playing with Blorg
13:49:12  * ralphtheninjaquit (Ping timeout: 256 seconds)
13:53:02  * dubbanquit (Remote host closed the connection)
13:53:09  <rvagg>werle: cool, wolfeidau and I have been discussing the next iteration with easier config
13:54:04  <rvagg>feel free to open an issue if you want to discuss anything about it, it'd be great to have additional dev help to refine it a bit more
13:54:31  <rvagg>werle: and here's your perfect companion for editing markdown content: https://github.com/rvagg/morkdown
13:59:16  <werle>rvagg: my hero
13:59:26  <werle>rvagg: I'm going to spend a little more time with it
13:59:36  <werle>get to know it and maybe break it ;)
13:59:43  <rvagg>excellent
13:59:48  <rvagg>I'm going to spend some time with my bed
13:59:50  <rvagg>g'night all
13:59:58  <werle>then I'll open some issues hopefully by this weekend
14:00:00  <werle>cheers
14:01:23  <juliangruber>rvagg: dominictarr's new browser editor is called bed!
14:07:32  * timoxleyjoined
14:08:44  <timoxley>rvagg re delimiters: perhaps "select" a delimiter that is "appeasing" and simply transform it into \xff
14:12:17  <rvagg>timoxley: yeah, except then you'll need to control for at least 2 characters to disallow in your keys
14:12:32  <rvagg>I think that whole area is a little confusing so I might not fluff it up too much more
14:12:58  <timoxley>rvagg sure but don't you have that that issue anyway, just informally?
14:14:00  <rvagg>timoxley: yeah, but why bother, jut to make them look nice?
14:14:24  <rvagg>you should try using map-reduce and look at the contents after it's got some data in there, ugly as, but works really well
14:15:05  * werlequit (Quit: Leaving.)
14:17:31  <rvagg>anyway, I'm not supposed to be here, ttfn
14:17:56  <timoxley>rvagg bbfn
14:38:44  * werlejoined
15:02:23  * mreviljoined
15:31:35  * ralphtheninjajoined
15:50:15  * werlequit (Quit: Leaving.)
16:01:56  * levelbotjoined
16:01:56  <levelbot>[npm] level-rpc@0.3.1 <http://npm.im/level-rpc>: Super fast rpc mechanism for LevelUp
16:18:21  * werlejoined
17:18:17  * jluisjoined
17:18:41  * jluischanged nick to Guest28778
17:20:12  * weetabeexquit (Ping timeout: 252 seconds)
17:43:43  * mrevilquit (Remote host closed the connection)
17:48:49  <levelbot>[npm] level-autotable@0.0.5 <http://npm.im/level-autotable>: Auto increment key table for node-levelup/leveldb
18:26:03  <mbalho>rvagg: im using idb-wrapper which uses a transaction for batch writes, is that not the way it is supposed to work?
18:28:00  <dominictarr>that is exactly what you should do.
18:28:37  <mbalho>dominictarr: im trying to figure out https://github.com/jensarps/IDBWrapper/issues/27
18:29:46  <dominictarr>hmm.
18:30:09  <dominictarr>Raynos has also wrestled with this stuff.
18:30:14  <Raynos>hi
18:30:27  <dominictarr>^
18:30:42  <dominictarr>whats the deal with indexedDb transactions?
18:31:06  <Raynos>they are weird
18:31:46  <dominictarr>whats that autoIncrement thing?
18:32:46  <Raynos>auto incrementing ids
18:32:56  <Raynos>every "object" in the object store
18:32:58  <Raynos>needs an id
18:33:04  <Raynos>it auto generates them for you if you tell it to
18:33:24  <Raynos>I dont know what the batch issue is
18:33:42  <Raynos>his batch impl looks like mine ( https://github.com/Raynos/levelidb/blob/master/index.js#L195 ) and that worked for chrome 22 6 months ago
18:35:07  <owen1>rvagg: it looks similar to https://github.com/ypocat/gfms
18:35:22  <owen1>rvagg: i am talking about your https://github.com/rvagg/morkdown
18:36:11  <owen1>the nice thing about fgms is you don't need to refresh the browser
18:36:21  <owen1>i meant gfms
18:36:24  <dominictarr>Raynos: what if you don't want ids?
18:37:56  <Raynos>dominictarr: you disable it
18:38:07  <Raynos>you can have the "key" for a document be either external or embedded on the value
18:38:48  <Raynos>if you look at the examples ( http://www.w3.org/TR/IndexedDB/#key-generator-concept )
18:39:00  <Raynos>you see that put(value/* NO KEY */) auto generates a key
18:42:10  <Raynos>Oh!
18:42:13  <Raynos>This is the leveldb room
18:47:55  * mreviljoined
18:52:13  * mrevilquit (Ping timeout: 245 seconds)
18:53:36  * timoxleyquit (Ping timeout: 264 seconds)
18:53:45  <hij1nx>dominictarr: re: hooks. a pre hook is only done when add is called, even if i dont pass any values to it? like, `add() === done()`
18:56:38  <dominictarr>don't call add() empty
18:56:48  <dominictarr>just return, it's not allowed to be async anyway.
18:57:03  <dominictarr>yeah add !== done
18:58:30  <mbalho>Raynos: my issue is that autoIncrement is set to true on transaction.objectStore https://github.com/jensarps/IDBWrapper/issues/27
18:58:35  <mbalho>Raynos: and i cant figure out how to turn that off
18:58:56  <Raynos>mbalho: dont know
18:59:06  <Raynos>I may try to write a good indexeddb thing this weekend
18:59:18  <Raynos>mainly because epic rageface
18:59:27  <mbalho>yea something with testling tests would be nice
18:59:37  <mbalho>idb-wrapper just has an examples folder that doesnt even test the whole api
18:59:47  <Raynos>well
18:59:54  <Raynos>a good indexeddb wrapper has a leveldown api
18:59:59  <Raynos>remind me of this
19:00:19  <mbalho>im working on a leveldown api implementation on top of idb-wrapper https://github.com/maxogden/level.js
19:00:30  <mbalho>that uses the leveldown test suite
19:03:25  <hij1nx>dominictarr: awesome
19:09:09  <levelbot>[npm] level-static@1.0.0 <http://npm.im/level-static>: use leveldb as a static file store.
19:10:29  <levelbot>[npm] level-static@1.0.1 <http://npm.im/level-static>: use leveldb as a static file server
19:16:00  <levelbot>[npm] level-hooks@4.1.7 <http://npm.im/level-hooks>: pre/post hooks for leveldb
19:22:03  * dominictarrquit (Quit: dominictarr)
19:40:46  <mbalho>omgz indexeddb is crazy
19:43:35  <Raynos>mbalho: its the worst
19:43:36  <Raynos>thing ever
19:47:26  * Pwnnaquit (Ping timeout: 255 seconds)
19:48:53  <mbalho>Raynos: you can only create an object store from within a db version change event
19:49:36  <mbalho>Raynos: but you can get a default object store by making a transaction that you never use and just grabbing the .objectStore off of it
19:49:51  <mbalho>Raynos: but theres no way to pass in config to that object store, but its easier so idb-wrapper was using it
19:52:24  <Raynos>blargh
19:52:52  <Raynos>this problem should not exist :D
19:54:06  <juliangruber>mbalho: https://github.com/juliangruber/level-rpc does 50k ops/s now! just need to add type hints to the protocol and you already have db{get,put} over a stream with binary support
19:55:42  <mbalho>AHHA I FIGURED IT OUT
19:55:51  <mbalho>juliangruber: awesome!
20:04:50  <Raynos>mbalho: what do you mean with "Don't abstract away features of IndexedDB" ?
20:14:26  <mbalho>Raynos: leveldown is a smaller API than indexeddb
20:14:55  <mbalho>Raynos: but i wanna make sure that all the indexeddb features are still possible to get to, like storing types other than strings and buffers
20:15:07  <mbalho>Raynos: though now that i dive into it i might not care about certain indexeddb features
20:15:33  <mbalho>Raynos: cause theyre a pain in the butt
20:16:26  <Raynos>xd
20:16:30  <Raynos>so basically you mean
20:16:36  <Raynos>expose nice features like storing ArrayBuffer's
20:16:37  <Raynos>and shit
20:17:15  <mbalho>yea
20:18:55  * st_lukejoined
20:20:32  <hij1nx>juliangruber: hell yeah. btw, will you deprecate multilevel?
20:21:06  <juliangruber>hmmmmm
20:21:27  <juliangruber>multilevel could be easier to port since it doesn't do buffer stuff
20:21:45  <juliangruber>dunno yet
20:22:16  <Raynos>mbalho: are you close to finishing leveljs?
20:28:00  <mbalho>Raynos: yep i have like 10 tests left before all of the leveldown tests pass
20:28:14  <mbalho>Raynos: i finally figured out the iterator stuff that i needed to figure out
20:32:18  <Raynos>Cool!
20:42:05  * hij1nxquit (Ping timeout: 246 seconds)
20:43:44  * hij1nxjoined
20:45:31  * mreviljoined
21:02:40  <juliangruber>sweet!
21:05:49  <levelbot>[npm] levelnet@0.1.1 <http://npm.im/levelnet>: remote API for levelUP
21:08:27  * dominictarrjoined
21:11:00  * dominictarrquit (Client Quit)
21:21:49  <levelbot>[npm] levelnet@0.2.0 <http://npm.im/levelnet>: remote API for levelUP
21:26:40  * st_lukequit (Remote host closed the connection)
21:29:01  * nebenjoined
21:31:29  <neben>I published and refactored a module levelnet that provides a remote API for levelUP. I wrote it not knowing about Julian's multilevel, which is fine, I am new at all of this and it was fun. Anyway, I refactored it a little so it exposes a stream interface, and indeed now just copies Multilevel's interface so you can drop it in without changing any code.
21:32:10  <neben>Rand some bench's on it appears to be slower. Which I think means that Dnode is slower than rpc-stream, since that is the main difference between the levelnet and multilevel
21:33:47  <neben>Though I might be able to optimize it some. I have marked on the readme that Multilevel exists and have encouraged users to use that one instead. I am interested in the work going on Level-rpc
21:41:06  * dominictarrjoined
22:11:43  <mbalho>neben: i think this week julian has been working on binary support also, cause i think both rpc-stream and dnode are ascii only
22:17:16  <rvagg>owen1: re gfms vs morkdown, the syntax highlighting is the important bit to me, I'm using pygments like GitHub does so it comes out near perfect, gfms doesn't seem to even do syntax highlighting at all
22:22:33  <rvagg>article is published, thanks for the help ppl, http://dailyjs.com/2013/05/03/leveldb-and-node-2/
22:22:37  <mbalho>wewt
22:22:53  <rvagg>(url at the top is in the process of being fixed btw, jekyll takes an age to build dailyjs these days apparently)
22:24:21  * Pwnnajoined
22:25:44  <owen1>rvagg: i'll give morkdown a try. can it be used as part of a blog engine or is the purpose is just for yourself to review md files before pushing to git?
22:26:23  <rvagg>owen1: oh sorry, I meant 'brucedown', that's my github 2 html component
22:26:27  <rvagg>owen1: it's used in morkdown
22:26:37  <rvagg>owen1: and it's used in blorg, my new blog generator
22:26:44  <rvagg>and a bunch of other places!
22:36:11  <hij1nx>mbalho: nice work! level.js looking great!
22:37:49  <hij1nx>mbalho: it would be awesome if when levelweb went offline you could still manage data and then it would do a sync when you got reconnected. it seems like this may allow for such a thing
22:44:47  <mbalho>hij1nx: yea exactly
22:45:14  <mbalho>hij1nx: level.js will just provide a leveldown interface in the browser so any sync modules can layer on top of it
22:53:44  <mbalho>just gotta implement limit and itll be done
22:53:56  * werlequit (Quit: Leaving.)
22:54:06  * st_lukejoined
23:01:07  <mbalho>hmm anyone know how to tell an indexeddb objectstore cursor to abort gracefully?
23:08:10  <juliangruber>dominictarr: there seems to be a problem with level-hooks and multilevel
23:08:18  <juliangruber>hij1nx stumbled upon it
23:08:38  <juliangruber>as soon as multilevel.server(db) the hooks on the db aren't called anymore
23:08:47  <juliangruber>can you think of a reason?
23:09:36  <dominictarr>on the server?
23:09:49  <juliangruber>yes
23:10:08  <juliangruber>hij1nx is filling a bug/pr on level-hooks, seems to have found the problem already
23:10:20  <dominictarr>that is strange
23:11:51  * st_lukequit (Remote host closed the connection)
23:12:46  <mbalho># tests 282
23:12:47  <mbalho># pass 282
23:12:53  <mbalho>UNF
23:16:15  <juliangruber>parteey
23:19:06  <hij1nx>dominictarr: trying to just fix it.
23:26:26  <hij1nx>dominictarr: fixed. https://github.com/dominictarr/level-hooks/pull/3
23:26:47  <hij1nx>oh, i can write a test case for this if its not obvious enough.
23:27:02  <dominictarr>yes please!
23:27:07  <hij1nx>k
23:27:13  <dominictarr>how does it happen?
23:27:51  <dominictarr>what calls add?
23:28:08  <hij1nx>hook something, use add() inside the hook without any params, it never gets put.
23:29:02  <dominictarr>oh, so your patch is a safer behaviour
23:29:03  <hij1nx>`add()` is like (or should be like) `done()`, because inside the hook you may have an async operation that results in nothing to add to the batch
23:29:24  <dominictarr>no, you can't have an async add.
23:29:34  <dominictarr>if you do it throws.
23:30:01  <hij1nx>you cant do a get inside a hook???
23:30:44  <hij1nx>there are very strong use cases for needing to say, look up another value and then decide what to do next
23:31:01  <dominictarr>hmm, okay, I thought I had implemented that.
23:31:22  <hij1nx>implemented that?
23:31:39  <dominictarr>hij1nx:sorry, thought I had it fail fast.
23:31:53  <hij1nx>dominictarr: not sure I understand
23:31:54  <dominictarr>I agree there is a strong case for async hooks
23:32:15  <dominictarr>but I havn't implemented something in hooks that I think will be reliable.
23:32:19  <hij1nx>why would you not want to allow an async operation to occur inside a hook?
23:32:36  <dominictarr>well, you do, but then you'd need locks
23:32:41  <hij1nx>there are many compelling arguments to do so
23:32:52  <dominictarr>I'm not against it
23:32:52  <hij1nx>dominictarr: only for some cases
23:33:12  <hij1nx>in any case, my patch still makes sense
23:33:13  <dominictarr>it's just that if you use the current hooks I think you will get problems.
23:33:26  <hij1nx>possibly
23:33:29  <dominictarr>yes, I will merge - and we should discuss your usecase
23:33:36  <hij1nx>but in my particular cases i wont
23:33:46  <dominictarr>what is your case?
23:34:05  <hij1nx>but you are correct that doing async stuff inside a hook could cause issues
23:34:28  <mbalho>rvagg: ok im donw messing with abstract-leveldown
23:34:30  <dominictarr>I have experimented with this idea, in level-update
23:34:59  <dominictarr>but I havn't needed it for anything yet, but now that you are actually using it for something
23:35:14  <dominictarr>then it's the right time to address the problem!
23:35:32  <hij1nx>im hooking a put from multilevel, checking its value, adding to its value and then letting the operation continue as usual
23:35:57  <dominictarr>checking the current value?
23:36:03  <dominictarr>for that key?
23:36:07  <hij1nx>change.value
23:36:25  <dominictarr>that isn't async
23:36:36  <hij1nx>but a get is
23:36:53  <dominictarr>yes, but you arn't doing a get - are you?
23:37:00  <hij1nx>sometimes
23:37:04  <mbalho>rvagg: oh wait i broke leveldown tests, lemme fix that
23:37:39  <dominictarr>what sort of data is it?
23:37:43  <hij1nx>checking its value may mean doing a get using the change.key
23:37:52  <hij1nx>logs
23:38:02  <hij1nx>i decided to work on my logger project again
23:38:10  <dominictarr>right - merging with the current value for that key
23:38:18  <dominictarr>or checking whether it's an overwrite?
23:38:34  <hij1nx>logger.incr
23:38:40  <dominictarr>right
23:38:56  <juliangruber>incr is a bitch
23:39:00  <hij1nx>;)
23:39:08  <dominictarr>so, there is another way you can handle this with multilevel
23:39:09  <dominictarr>hmm
23:39:36  <hij1nx>id like to use hooks, i think its a very maintainable abstraction
23:39:43  <dominictarr>agree
23:39:49  <juliangruber>agree
23:39:57  <hij1nx>agree
23:39:59  <hij1nx>;)
23:40:05  <juliangruber>incr always needs locks, right?
23:40:06  <dominictarr>but for the pursoses of discussion
23:40:29  <hij1nx>server should lock/buffer
23:40:29  <dominictarr>you could add a method to the db and expose that in the multilevel manifest
23:40:30  <mbalho>ok FINALLY the same test suite passes all tests in leveldown, abstract-leveldown and level.js
23:40:42  <dominictarr>mbalho: +10000000000000
23:40:45  <hij1nx>mbalho: GOOD WORK NICE STUFF YAY. CATS
23:40:50  <juliangruber>dominictarr: we don't have documentation for that yet :O
23:41:35  <dominictarr>right - the format of the manifest is not described
23:41:46  <mbalho>rvagg: you can make an abstract-leveldown relese whenevs
23:41:52  <juliangruber>anyways, need to catch some zzzzs
23:42:10  <dominictarr>okay, so you can add a method, but for incr you'd still need a lock.
23:42:36  <dominictarr>because you don't want this to happen:
23:42:43  <hij1nx>dominictarr: yep, i've got that taken care of, im doing some fancy buffering
23:43:00  <dominictarr>client1.incr (get, +1, put)
23:43:33  <juliangruber>idea: incr as a live map reduce
23:43:35  <dominictarr>but then cilent2 does a get at the same time (before client1 has put, but then puts after client1 does.
23:43:55  <dominictarr>juliangruber: exactly - that is the other option
23:44:00  <hij1nx>dominictarr: client2 cant get
23:44:09  <hij1nx>(in my case)
23:44:18  <juliangruber>storing incrs sucks, because of the gets
23:44:26  <dominictarr>hij1nx: how does it work?
23:44:30  <juliangruber>hij1nx: would a live map reduce for each key be feasible?
23:44:32  <hij1nx>logs go in, thats it
23:45:11  <dominictarr>you could do a simpler thing than map-reduce
23:45:20  <dominictarr>but still keeping the same basic idea.
23:45:25  <juliangruber>yes
23:45:35  <dominictarr>just have a simple reduce
23:46:02  <dominictarr>put -> seperate area
23:46:21  <dominictarr>and then asyncly reduce what you have in that area, and write it to the main key
23:46:36  <dominictarr>(and delete what you combined, also)
23:46:41  <juliangruber>true, no need to block the put
23:46:57  <hij1nx>you can just do c1.incr(key) -> get, +1, put; c2.incr(key) -> buffer, get, +1, put
23:46:58  <juliangruber>I guess strict consistency isn't required
23:47:01  <dominictarr>we should explore both methods
23:47:20  <dominictarr>hij1nx: that is a lock.
23:47:35  <hij1nx>dominictarr: thats ok, for what im doing.
23:47:48  <hij1nx>dominictarr: im ok with a lock if its very short
23:47:55  <dominictarr>sure - if that is what you need, then do that
23:48:00  <hij1nx>dominictarr: mostly because its unlikely as hell
23:48:04  <dominictarr>I'm not anti-locks
23:48:15  <dominictarr>yeah, that too
23:48:23  <rvagg>remember that leveldb's cache is pretty good so storing a simple incrementable and fetching it each time you need it isn't going to be a huge cost if you have a direct key
23:48:23  <dominictarr>but data is sacred
23:48:46  <dominictarr>and if it's not predictably consistent (with it self) it's not so useful.
23:49:06  <dominictarr>what sort of stuff are you counting, by the way?
23:51:34  <dominictarr>hij1nx: https://github.com/hij1nx/level-hooks/blob/b41e93f26b283a19e568c2da8c64740b09a1c7cb/index.js#L131
23:51:47  <dominictarr>if you call add asyncly, nothing will happen
23:52:01  <dominictarr>because it passes it to the db syncly.
23:52:18  * levelbotquit (Remote host closed the connection)
23:52:47  * levelbotjoined
23:53:18  <rvagg>mbalho: `result = String.fromCharCode.apply(null, new Uint16Array(value))`, what are you expecting levelup to do with these non-buffers if you have to do that kind of crazy in the tests?
23:53:57  <rvagg>mbalho: seems that we're going to have to add some special code to levelup too to deal with this cruft
23:54:11  <levelbot>[npm] level-js@1.0.0 <http://npm.im/level-js>: leveldown/leveldb library for browsers
23:55:22  <mbalho>rvagg: that will only ever run if you start storing ArrayBuffers in leveldown
23:56:10  <mbalho>rvagg: other than the two places where i had to do that i feel good about the changes i had to make
23:56:16  <rvagg>mbalho: yeah, so does that mean ArrayBuffers aren't supported via levelup in the browser until we deal with it explicitly there too?
23:57:02  <mbalho>rvagg: probably
23:57:18  <mbalho>rvagg: i mean, they are a basic JS type, it makes sense to support them
23:57:33  <mbalho>rvagg: you just cant call .toString() on them to get their contents, you have to do that
23:57:36  <rvagg>ok, but maybe later
23:57:44  <mbalho>rvagg: yea i am sure we can make it less ugly