00:04:50  <mbalho>dominictarr: http://macwright.org/2012/11/14/indexing-searching-big-static-data.html
00:04:53  <mbalho>dominictarr: http://blog.notdot.net/2009/11/Damn-Cool-Algorithms-Spatial-indexing-with-Quadtrees-and-Hilbert-Curves
00:08:05  <mbalho>dominictarr: also fork this and make it streaming http://square.github.com/crossfilter/
00:09:53  <mbalho>dominictarr: http://www.cs.umd.edu/~hjs/pubs/NelsSIGG86.pdf
00:10:20  <chrisdickinson>mbalho: streaming graphs are actually really difficult
00:10:42  <mbalho>i was kind of saying it tongue-in-cheek hehe
00:16:25  <chrisdickinson>ah, haha
00:16:44  <chrisdickinson>yeah, i was big into the idea for a while, but what i eventually landed on was a sort of slippymap for graphing data
00:17:02  <chrisdickinson>(loading data in "tiles" and then transitioning when a new range was found.)
00:20:41  * ITproquit (Ping timeout: 245 seconds)
00:20:53  * ITprojoined
00:28:06  <substack>chrisdickinson: http://github.com/substack/graph-stream
00:28:32  <chrisdickinson>interesting
00:28:45  <chrisdickinson>does it automatically handle new maxima?
00:29:57  <substack>yes
00:33:17  <chrisdickinson>awesome!
00:36:58  <chrisdickinson>Raynos: do you still need (or did you ever need?) a streaming CSS parser?
00:37:45  <substack>dominictarr: http://paulgraham.com/startupideas.html
00:37:52  <substack>"Live in the future and build what seems interesting."
00:38:13  <dominictarr>I saw someone tweeted that.
00:38:37  <dominictarr>but didn't click it, because I knew it would say that.
00:39:06  <dominictarr>"few others realize are worth doing"
00:39:15  <dominictarr>that is the most important thing for a good idea.
00:39:44  <dominictarr>too many people are building systems that are new ways to order takeout.
00:39:51  <dominictarr>or find a place to eat.
00:39:58  <ik>no substack
00:39:59  <ik>19:39:47 <@iank> GIR startup idea
00:39:59  <ik>19:39:49 <+GIR> STARTUP IDEA: GIRAFFE TOWER
00:40:01  <ik>you use a robot
00:40:03  <ik>suck it, paul graham
00:40:18  <substack>giraffe tower, of course
00:40:24  <substack>seems so obvious in retrospect
00:40:41  <dominictarr>a giraffe on a tower? or a tower of giraffes
00:40:48  <substack>yes
00:41:45  <dominictarr>drone recharging stations on the heads of giraffes.
00:42:32  <dominictarr>then the drones could surveil even while they are recharging
00:43:08  <dominictarr>here is a startup: markov chains of paul graham.
00:43:21  <substack>drones could drop an inductive grappling hook
00:43:24  <dominictarr>predict pg's new essays before he writes them
00:44:00  <dominictarr>markov hacker news
00:44:09  <substack>dominictarr: you should figure out what needs to exist to build a passive money machine with crdt
00:44:14  <dominictarr>it shows what will probably be posted to hacker news tomorrow.
00:45:04  <dominictarr>yes, I should think about that.
00:45:46  <substack>maybe you could spin up hosted peers to replicate with that save their data to disk
00:46:13  <ralphtheninja>this really IS the mad science channel :)
00:46:16  <dominictarr>we are working on that for nearForm
00:47:02  <dominictarr>ralphtheninja, you are on the inside now.
00:47:30  <substack>and/or replicated hash ring peers
00:47:43  <dominictarr>that is on the TODO list too.
00:47:47  <substack>where you just mash the "more peers" button
00:47:51  <substack>and you get more cloud
00:48:33  <dominictarr>what about you pass a function to a function...
00:49:34  <dominictarr>spinup(function (stream) { var doc = new (require('crdt').Doc); stream.pipe(doc.createStream()).pipe(stream) })
00:49:58  <dominictarr>and it starts the server on a node somewhere and you can stream to it.
00:50:09  <substack>stream serialization as a service?
00:50:29  <substack>stream bucket
00:50:35  <dominictarr>this would be a tiny PaaS
00:50:44  <dominictarr>a module platform
00:50:55  <substack>if only there were a distribution platform for tiny services...
00:51:27  <dominictarr>and maybe it could pick servers that are nearby the client requesting the service...
00:52:03  <substack>distributed computing as a service
00:52:11  <substack>but not a single service, lots of tiny services
00:52:25  <ralphtheninja>aye, I've thought about that as well
00:52:27  <substack>services that perform their specialized tasks
00:52:45  <ralphtheninja>ONE thing and doing it awesomely
00:52:51  <substack>yes
00:53:00  <ralphtheninja>like a distributed unix
00:53:43  <ralphtheninja>and you can configure your own pipelines .. pick stream-nodes and pipe them together with patterns/arguments in between
00:53:48  <dominictarr>currently, when you want some compute, you have to get a WHOLE VM that is like a linux instance.
00:54:37  <dominictarr>but that is way too big.
00:55:12  <jjjjohnnny>functions as a service (it deos)
00:55:30  <dominictarr>you could also allow outside vendors to contribute compute nodes,
00:55:34  <jjjjohnnny>i dont think this buinsess idea has legs
00:55:50  <dominictarr>so you'd just pay commodity rates
00:55:57  <jjjjohnnny>there is gobs of excess cloud computing lying around
00:56:00  <ralphtheninja>you don't need a business idea if you're taking over the world :p
00:56:29  <dominictarr>^^ exactly
00:56:33  <jjjjohnnny>http://www.nytimes.com/2012/09/23/technology/data-centers-waste-vast-amounts-of-energy-belying-industry-image.html?pagewanted=all&_r=0
00:56:35  <ralphtheninja>you could charge companies bitcoins to route the traffic faster or something like that
00:57:26  <jjjjohnnny>what you do is buy up the spare computational power of these data centers and resell it
00:57:57  <dominictarr>COMPUTE FUTURES
00:58:12  <dominictarr>jjjjohnnny, exactly.
00:58:25  <dominictarr>this could just run when they are off peak
00:58:49  * _sorensenquit (Ping timeout: 246 seconds)
00:59:09  <jjjjohnnny>with a few of them then you would have distributed stability up to a certain threshold
00:59:16  <rowbit>Hourly usage stats: [developer: 9, free: 14]
00:59:25  <jjjjohnnny>where are the fucking biz hackers at
00:59:32  <jjjjohnnny>i need ten
01:00:20  <jjjjohnnny>in other news, oakland is setting up a ideo bot net to fight crime
01:00:25  <jjjjohnnny>ideo
01:00:27  <jjjjohnnny>video
01:01:00  <dominictarr>oh, that isn't so good
01:01:24  <jjjjohnnny>ist people powered
01:01:35  <dominictarr>I read a thing the other day that said london's CCTV hadn't reduced crime rates at all
01:01:36  <jjjjohnnny>independent community watches
01:01:50  <dominictarr>huh?
01:01:59  <substack>jjjjohnnny: the videos should go straight to youtube
01:02:14  <jjjjohnnny>substack: I think they will be available online at all times
01:02:26  <dominictarr>oh, now that is REALLY QUITE INTERESTING
01:02:33  <jjjjohnnny>yeah
01:02:48  <jjjjohnnny>i want to go to th emeeting tonight, but dont have a ride
01:02:49  <dominictarr>there is a lot of cool stuff you could build off that.
01:02:55  <ralphtheninja>stream from youtube and do face recognition
01:03:01  <ralphtheninja>with node of course
01:03:09  <dominictarr>+ read number plates
01:03:35  <jjjjohnnny>dominictarr: the guy pushing this gets every plate that enters his hood
01:03:38  <dominictarr>or just have a live feed of Where The Party At
01:03:54  <dominictarr>dominictarr, already?
01:03:56  <jjjjohnnny>http://neighborhoodguard.org/
01:04:00  <dominictarr>jjjjohnnny, already?
01:04:06  <jjjjohnnny>yeah
01:04:29  <jjjjohnnny>the guard group exists to help eighborhoods get the cameras and set up the servers and stuff
01:05:00  * _sorensenjoined
01:05:18  <jjjjohnnny>one neat thing is there are many open volunteer positions available on the committee of this guard, such as software development and web master
01:05:57  <dominictarr>"4) Any person who hides or turns they face away from you when they notice you are observing them."
01:06:12  <dominictarr>"2) Any car that drives slower than usual, occupants that more interested in the houses or drive ways than the road in front of them. (sometimes making notes)"
01:06:24  <jjjjohnnny>yeah thats from the police
01:06:33  <dominictarr>"6) Any person that is dressed or acting in a way that is unusual (peeking into windows), standing on a corner talking into a cellphone (lookout) etc.etc."
01:07:11  <st_luke>juliangruber: jilla is nice
01:07:17  <st_luke>juliangruber ++
01:07:36  <dominictarr>really, I don't think that street crime is a huge threat to society
01:08:03  <dominictarr>it's white collar corruption that is the threat to the future of humanity.
01:08:20  <ralphtheninja>white collar corruption?
01:08:32  <st_luke>dominictarr: street crime is a threat to communities
01:08:34  <dominictarr>yeah, government & corporations.
01:09:08  <ralphtheninja>agree
01:09:17  <dominictarr>yeah, but people don't get into street crime because it's fun, but because they are hungry.
01:09:36  <dominictarr>I'm not saying it's not a problem.
01:09:48  <dominictarr>I'm saying that it's a small problem in the scheme of things
01:09:52  <st_luke>dominictarr: or because of drug addiction, or boredom
01:10:01  <Raynos>jjjjohnnny: it would be stupid if it didnt work on both node & browsers
01:10:07  * _sorensenquit (Ping timeout: 244 seconds)
01:10:33  <dominictarr>and that it's a problem that is exaggerated for political reasons (such as to distract from white collar corruption)
01:10:43  <jjjjohnnny>Raynos: so it streams webcam and mix to the server peer then?
01:11:00  <Raynos>jjjjohnnny: it doesnt emulate mediastreams. Only peer connection & data channel
01:11:24  <jjjjohnnny>dominictarr: all levels of crime can be fought through positive interventions like education and opportunity-making
01:11:25  <dominictarr>st_luke, that is another problem. there is a suplus of creative talent and energy, and it's really hard to find someplace to direct that.
01:11:30  <Raynos>chrisdickinson: Hell yes! Preferably one in the browser as well
01:11:58  <dominictarr>jjjjohnnny, agree
01:12:00  <st_luke>A couple of years ago a couple dumb kids pulled a knife on me in upstate NY and jacked my cell phone and sold it for < $100, anyway it got recovered and the kid got sentenced to 4-6 years for armed robbery. I don't think he did it due to hunger or drug addiction, most likely boredom.
01:12:36  <jjjjohnnny>i would put all my money on educating people so we had 10 millions coders, 10 million microbioloists, 10 million every scientist
01:12:36  <substack>we need to get these kids building robots
01:12:42  <substack>exactly
01:12:46  <dominictarr>yeah agree
01:13:03  <substack>I mean if you are going to rob people at least do it with a robot
01:13:06  <dominictarr>st_luke, that is a really really expensive way for the state to deal with that problem
01:13:12  <st_luke>I could relate, because I committed crimes as a teenager due to boredom also. boredom is the same reason I never finished high school and college.
01:13:40  <ralphtheninja>st_luke: same here
01:14:02  <jjjjohnnny>st_luke: idleness happens in absence
01:14:09  <st_luke>dominictarr: I agree, especially when by convicting a kid of a felony the state is sentencing him for life, even if he only spends a year in jail.
01:14:12  <dominictarr>like it probably cost hundreds of thousands to send that guy to prison for 4 years
01:14:38  <dominictarr>would be cheaper to send him to college
01:15:08  <jjjjohnnny>yeah saving people from crimes and punishments would effectively redistribute wealth
01:15:20  <ralphtheninja>and politicians respond by increasing the penalties instead of trying to find the root cause
01:15:49  <st_luke>I'm not personally for wealth distribution, I'm for wealth redefinition if anything.
01:16:13  <dominictarr>in most western countries, they basically pay people not to steal, to some degree.
01:16:18  <dominictarr>(except US obv.)
01:16:19  <jjjjohnnny>rephrase to wealth balancing
01:16:39  <substack>dominictarr: the US still has welfare and many indirect forms of benefits
01:17:04  <dominictarr>yes. It just doens't seem as comprehensive.
01:17:54  <dominictarr>jjjjohnnny, when you say it like that is sounds like you are gonna take wealth from some people and give it to other people
01:17:56  <st_luke>US culture, finance, justice and education are based on classism derived from the tribal nature of humans
01:17:58  <ralphtheninja>that's why we need to work on developing more p2p stuff, it's essential for many things, p2p = freedom
01:19:00  <jjjjohnnny>dominictarr: SIR I SAID NO SUCH THING /RAEGAN
01:19:20  <jjjjohnnny>i demand a takeback
01:19:29  <dominictarr>but what I'm wanting really, is to run the govt more efficently, making better decisions, so that you achive better results for less.
01:19:50  <dominictarr>jjjjohnnny, I didn't say you said anything, I just said it could be interpereted like that.
01:20:51  <substack>if governments had a continuous taxation function that went negative under a threshold that would be much simpler and cheaper to administer than means-tested social programs
01:20:55  <ralphtheninja>dominictarr: like making the government more and more lean, strip of rotten and useless flesh
01:21:12  <substack>https://npmjs.org/package/buffer-browserify
01:21:15  <substack>wups
01:21:15  <dominictarr>ralphtheninja, fighting the system by improving it
01:21:16  <substack>https://en.wikipedia.org/wiki/Negative_income_tax
01:21:19  <substack>silly clipboard
01:22:29  <dominictarr>jjjjohnnny, okay, I take it back.
01:22:50  <ralphtheninja>dominictarr: totally with you there, might be hard to get things through thought, lots of bureaucracy
01:23:05  <ralphtheninja>s/thought/though
01:23:40  <substack>ralphtheninja: start at the state level
01:23:49  <substack>many US states have income taxes
01:23:52  <substack>good place to start
01:23:58  <substack>and US states administer welfare programs
01:24:01  <jjjjohnnny>dominictarr: say whats wrong with your government anyhow
01:24:08  <substack>and it's much easier to experiment at the state level
01:24:19  <dominictarr>ralphtheninja, read "innovator's delemma"
01:24:34  <substack>and california has a budget shortfall
01:24:51  <substack>if such a plan could be rephrased in terms of saving welfare programs while cutting the budget
01:24:55  <substack>then you win
01:25:00  <substack>by eliminating waste
01:25:21  <st_luke>substack: do you have any modules for parsing incoming email
01:25:31  <jjjjohnnny>determined, dominictarr needs an American bride
01:25:35  <substack>st_luke: the text?
01:25:40  <substack>it's just MIME
01:25:52  <substack>http-style headers
01:25:57  <ralphtheninja>sure, you still need to convince bureaucrats that your idea is sound and that they should use your idea for improvement
01:26:03  <substack>there are some modules on npm for that, pretty sure
01:26:20  <st_luke>substack: yeah, I just need something quick to pipe to from /etc/aliases on a server
01:26:21  <ralphtheninja>and there will be people against your idea, because it will put them out of work
01:26:28  <substack>ralphtheninja: california has ballot initiatives
01:26:28  <st_luke>I can actually probably write up something quickly
01:27:15  <dominictarr>jjjjohnnny, what is wrong with the NZ govt is that it's lobbied and influenced by the same things as the US govt, and by the US govt.
01:27:22  <jjjjohnnny>"Pure truth cannot be assimilated by the crowd; it must be communicated by contagion."
01:27:47  <ik>hornet spray
01:28:07  <dominictarr>part of the problem IS bureaucrates
01:28:31  <dominictarr>they are the slow, bad, expensive way to make decisions.
01:29:15  <substack>lobbyists lobby the US to lobby NZ
01:29:23  <substack>so many lobbies
01:29:27  <substack>all that wasted floor space
01:30:00  <substack>ik: so I have a lot of japanese electronics parts now I should probably build robots or whatever
01:30:33  <substack>also electronics components are huge here
01:30:51  <ralphtheninja>dominictarr: aye, that's what I meant, they will put up a fight if they feel their jobs are being threatened
01:31:41  <dominictarr>so, we gotta find them better jobs
01:31:54  <ralphtheninja>sure :)
01:32:03  <dominictarr>or just obliterate their jobs in a way they can't compete with
01:32:31  <dominictarr>innovator's delemma describes that sort of thing.
01:32:48  <ralphtheninja>hmm there must be a shitload of common problems to be solved, like govermental anti patterns
01:32:50  <dominictarr>how little new things come to be and destroy old crappy things
01:33:13  <ralphtheninja>sounds like my previous job :)
01:33:21  <substack>bureaucrats are some of the easiest things to automate away
01:33:24  <ralphtheninja>everything is focused on what the customer wants
01:33:26  <dominictarr>and how it usually catches the old things in the blind spot.
01:35:10  <dominictarr>READ "INOVATOR'S DILEMMA"
01:35:25  <ralphtheninja>searching for it right now
01:35:28  <dominictarr>^there, put it into LOUDBOT's databanks
01:35:47  <mbalho>this talk is good www.youtube.com/watch?v=3grzYoJ2oPQ
01:35:59  <mbalho>robot industrialization
01:39:52  * mikealjoined
01:41:39  * captain_morganjoined
01:42:10  * mikealquit (Client Quit)
01:42:51  * fentjoined
01:43:30  * jibayquit (Remote host closed the connection)
01:59:16  <rowbit>Hourly usage stats: [developer: 0, free: 45]
02:01:01  * thatguydan_joined
02:03:53  * thatguydanquit (Ping timeout: 265 seconds)
02:03:53  * thatguydan_changed nick to thatguydan
02:04:03  * dominictarrquit (Ping timeout: 276 seconds)
02:09:27  * zz_shykeschanged nick to shykes
02:20:48  * thatguydanquit (Remote host closed the connection)
02:21:08  * thatguydanjoined
02:21:31  * ITproquit (Ping timeout: 245 seconds)
02:21:33  * shykeschanged nick to zz_shykes
02:26:11  * dominictarrjoined
02:37:15  * dominictarrquit (Quit: Leaving)
02:37:33  * dominictarrjoined
02:38:35  * rvaggjoined
02:39:06  <dominictarr>rvagg, this is where the cool people hang out.
02:39:14  <rvagg>dominictarr, Raynos: btw, have you looked at writeStream() ? it converts writes into batch operations where they are within the same tick.. probably not exactly what you're after tho I spoze
02:39:24  <rvagg>this is the substack cult eh?
02:39:29  <Raynos>yes
02:39:36  <Raynos>write stream wont do what we need for hooks
02:39:43  * rvagglooks for the streaming-altar
02:39:43  <Raynos>I think hooks and before / after events should go into levelup
02:39:49  <dominictarr>well, it just happens to be substack's channel.
02:40:01  <dominictarr>that is incidental
02:40:43  <dominictarr>so, rvagg Raynos and I have been working on a map-reduce extension for levelup
02:41:19  <rvagg>yes, I've had a look at your code but it's too dense to take in in 5 mins
02:41:28  <dominictarr>yeah, it's ugly
02:41:32  <dominictarr>I'm tidying it up
02:41:37  <dominictarr>and refactoring it now
02:41:42  <dominictarr>spliting the parts, etc.
02:41:51  * rvaggwill wait for a readme
02:42:00  <dominictarr>haha. good decision.
02:42:05  <Raynos>rvagg: I think keyStream and valueStream are too much
02:42:08  <rvagg>so what do you need?
02:42:08  <Raynos>it bloats the API
02:42:24  <Raynos>the same way I think createReadStream and createWriteStream are too much in mux-demux
02:42:37  * ralphtheninjaquit (Read error: Operation timed out)
02:42:41  <rvagg>Raynos: the reason they are in there is so that you can avoid passing too much from the C layer to V8, if you just abstract around a complete readStream() then you pass too much
02:42:46  <dominictarr>they are just aliases, who cares.
02:43:21  <dominictarr>I guess you could have an readstream option {keys: true}
02:43:32  <dominictarr>if that is what Raynos is thinking
02:43:46  <rvagg>https://github.com/rvagg/node-levelup/blob/master/src/iterator.cc#L52-L55
02:43:53  <dominictarr>anyway Raynos, this isn't important.
02:43:54  <rvagg>yes, those options are there, that's how valueStream and keyStream work
02:44:10  <Raynos>rvagg: keyStream and valueStream just call readStream({ keys: })
02:44:16  <Raynos>oh your right
02:44:19  <Raynos>its not important
02:44:22  <Raynos>its pure bikeshedding :D
02:44:28  <dominictarr>okay, so what IS important
02:44:29  <Raynos>hooks!
02:44:32  <Raynos>hooks are important
02:44:33  * rvagglikes the bikeshed
02:44:38  <dominictarr>sush! I'm getting to that!
02:44:56  <dominictarr>so, the thing that is soooooo great about level db
02:45:05  <dominictarr>is that it's only the HARD part of a data base
02:45:05  <Raynos>dominictarr: https://github.com/rvagg/node-levelup/issues/29
02:45:31  <dominictarr>& we get to build all the soft parts in js.
02:45:52  <dominictarr>this is the javascript way: polyfills all the way down.
02:46:22  <dominictarr>we just get given a crappy, but basically there API from the browser, or whatever
02:46:29  <dominictarr>and then we polyfill it in JS
02:46:49  <dominictarr>when you think about it, node.js is really just a polyfill.
02:47:25  <dominictarr>rvagg, so basically, we are building the other features that a database normally has
02:47:42  <rvagg>re hooks, can't you just mess with __proto__ ?
02:47:42  <dominictarr>but since we are doing it in JS we can make it really flexible and modular.
02:47:55  <dominictarr>rvagg, so, you COULD
02:48:31  <dominictarr>but we'd probably get better compatibility if we figured out a more disiplined way to do it.
02:48:50  <dominictarr>so, one thing you need is a queue system
02:49:15  <dominictarr>I'm using this already to make map-reduce reliable
02:49:37  <dominictarr>so, like, if the process crashes, we need to restart unfinished map-reduce jobs
02:50:10  <dominictarr>so, we need a reliable job queue for that.
02:50:18  <dominictarr>but if we expose that feature,
02:50:26  <dominictarr>then we get a DB with a job queue too!
02:50:53  <dominictarr>and in some applications, maybe you don't care about M-R
02:51:02  <dominictarr>but you want job queues
02:51:32  <dominictarr>with levelup, you can have that.
02:52:26  <Raynos>Im thinking that before and after events might be the same as hooks
02:53:04  <Raynos>whats the performance penalty between put and batch with a single value?
02:53:08  <dominictarr>Raynos, well, for the queue to work right, we need to turn a PUT into a BATCH
02:53:22  <rvagg>Raynos: we can test that now!
02:53:32  <Raynos>i mean you can implement before and after as hooks not the other way around
02:53:38  <dominictarr>you could also implement a scuttlebutt style replication with that method.
02:53:48  <dominictarr>Raynos, yes.
02:54:05  <dominictarr>even SQL has triggers.
02:54:15  <dominictarr>it's the same thing, but more flexible
02:54:43  <substack>https://github.com/substack/shell-stream/blob/master/server.js
02:54:48  <substack>ssh over http, pretty much
02:55:06  <substack>now I just need to figure out how to make the stream seem like a real tty
02:55:12  <jjjjohnnny>can you set outgoing port on a net.connect?
02:55:17  <dominictarr>substack, you should call "secure-peer" "secure-stream"
02:55:24  <substack>and https://github.com/substack/shell-stream/blob/master/client.js
02:55:37  <dominictarr>because that is like "ssh" is "secure shell"
02:55:52  <dominictarr>or maybe "sst"?
02:56:47  <dominictarr>substack, now we just need the jQuery of the fs
02:57:20  <substack>the cool thing about this ssh is that you don't need root to run it
02:57:23  <dominictarr>and then we can do everything from a node repl instead of ever using sh.
02:57:24  <substack>and you can drop it into any application
02:57:30  <substack>tcp, http, whatevs
02:57:54  <dominictarr>ABSOLUTELY
02:58:16  <substack>running this on browserling before I head out so I can ssh on wifi that blocks ssh
02:58:42  <Raynos>substack: on xhr-write-stream mention the order feature
02:58:50  <Raynos>it's important you mention how it preserves order
02:58:55  <Raynos>thats the only reason to use it
02:59:16  <rowbit>Hourly usage stats: [developer: 0, free: 37]
02:59:36  * mikealjoined
03:01:11  <rvagg>Raynos: benchmarks for single-put batch() in master now.. it's kind of expensive but I don't know how much of that is because of the JS overhead, there's a fair bit of preprocessing going on before it gets passed to leveldb
03:01:30  <Raynos>#winning
03:01:39  <substack>Raynos: it also gives you a single stream instead of having to care that the xhr requests are all separate
03:01:51  <Raynos>oh I see
03:02:04  <Raynos>it sends multiple writes as single xhr's
03:02:04  <substack>and it does timeouts if it hasn't heard any data from a request id in a while
03:02:08  <substack>yes
03:02:19  <substack>since xhr can't keep requests open
03:02:25  <Raynos>it can :p
03:02:29  <Raynos>but it cant send more data
03:02:30  <substack>for writing?
03:02:32  <substack>yes
03:02:38  <substack>you can only .send() once
03:05:24  * dominictarrquit (Ping timeout: 268 seconds)
03:06:52  <jjjjohnnny>how am i gonna bind to a socket on c-76-102-195-134.hsd1.ca.comcast.net
03:07:13  <Raynos>who knows :D
03:07:47  <jjjjohnnny>its my public address
03:08:02  <Raynos>rvagg: should put / del / batch events fire even if they error?
03:08:03  <jjjjohnnny>but its removed from my machine several nets
03:08:27  <ik>jjjjohnnny: you don't
03:08:46  <jjjjohnnny>ik: but
03:08:50  <rvagg>Raynos: good question, I suspect not
03:09:01  <ik>jjjjohnnny: you forward traffic from there to you
03:09:02  <Raynos>I think it might be useful
03:09:08  <ik>jjjjohnnny: and you bind to a socket on your local network interface
03:09:12  <Raynos>maybe we should have before & after events
03:09:18  <Raynos>or maybe before and after hooks
03:09:41  <rvagg>give it a go and see what you come up with!
03:09:47  <rvagg>I'm easy
03:09:53  <Raynos>cool
03:12:44  <substack>sweeeet https://en.wikipedia.org/wiki/Pseudo_terminal#Unix98_PTYs
03:19:04  <Raynos>rvagg: your whitespace style ._.
03:19:22  <rvagg>is gangnam style
03:19:27  <jjjjohnnny>ik: is this impossible http://en.wikipedia.org/wiki/TCP_hole_punching
03:19:52  <rvagg>Raynos: comma-first but I don't like my commas squishing up against the previous indent level so they get indented
03:20:10  <Raynos>rvagg: I also have comma first. it works way better with 4 spaces
03:20:14  <rvagg>like isaacs's style but extra indent for nested commas
03:20:49  <rvagg>Raynos: I like my var names to line up so 2 char indenting works perfectly with the width of 'var'
03:21:14  <Raynos>ok I'm writing in 4 spaces because my text editor can't auto indent to your style
03:21:19  <rvagg>I'm no format-nazi tho., but I appreciate consistency
03:21:22  <Raynos>:D
03:21:40  <rvagg>bah, whatever, just do what you like and if it bothers me enough I'll reformat
03:23:01  <Raynos>its ok
03:23:06  <Raynos>ive fixed my text editor
03:24:44  <substack>isaacs: node's tty should expose ptsname()
03:24:49  <substack>tty.open()
03:25:11  <substack>or maybe not but I have no idea how windows does that
03:27:01  <jjjjohnnny>can a net.server create connections through its listen port?
03:28:43  <Raynos>rvagg: what is good about buster?
03:30:11  <substack>haha node 2.4 is still on nexus
03:32:20  <rvagg>Raynos: beautifully integrated Sinon for mocks/stubs/spies, excellent assertion framework, none of the pollution that Mocha does (leaves your env completely alone), plus it's great for browser-side testing too.
03:32:35  <Raynos>I see
03:32:42  <Raynos>well i hate its assertion framework already
03:32:42  <rvagg>Raynos: unfortunately they haven't released a buster-node yet so you get a ton of browser deps even when you want it just for node testing
03:32:46  <Raynos>its not backwards compatible with node assert
03:33:47  <rvagg>http://docs.busterjs.org/en/latest/modules/buster-assertions/#buster-assertions
03:34:07  <fent>hello node.js stream folks :)
03:34:09  <Raynos>yeah I found it, just frustrating
03:34:11  <Raynos>too complex!
03:34:41  <rvagg>k, well I prefer it to node's assert module
03:34:45  * defunctzombiejoined
03:34:48  * mikealquit (Quit: Leaving.)
03:34:52  <rvagg>custom assertions are neat too, roll up a complex condition into a single assert
03:35:22  <rvagg>https://github.com/rvagg/traversty/blob/master/test/setup.js#L14-L27
03:39:22  * defunctzombiequit (Ping timeout: 252 seconds)
03:41:27  <isaacs>substack: it used to, i think
03:41:33  <isaacs>substack: talk to bert and nate about it
03:41:56  <substack>it's so hard to open a pts with ptmx >_<
03:43:41  * substackheads out to explore tokyo
03:51:04  <jjjjohnnny>https://developers.google.com/talk/libjingle/
03:51:55  * thatguydanquit (Read error: Connection reset by peer)
03:52:04  * thatguydan_joined
03:55:01  <Raynos>rvagg: see hooks implementation!
03:55:35  <jjjjohnnny>libjingle is a collection of open-source C++ code and sample applications that enables you to build a peer-to-peer application
03:56:56  * dominictarrjoined
03:57:31  <dominictarr>back
03:57:52  <dominictarr>rvagg, anyway, so I just wanted to tell you what I was up to.
03:58:20  <rvagg>dominictarr: that's cool, I'd like to try out your examples when you've cleaned up a bit and perhaps made a basic readme, sounds fun
03:58:27  <dominictarr>cool!
03:58:30  <dominictarr>ALSO
03:59:03  <dominictarr>I'm gonna add replication. that is what I need the map-reduce for - to calculate a merkle tree.
03:59:16  <rowbit>Hourly usage stats: [developer: 0, free: 33]
04:00:28  <rvagg>oh, that's cool
04:01:05  <rvagg>my original plan with levelup was to build a larger db ontop of it... if I ever have time... but I see people doing such cool stuff in that area already now
04:01:39  <rvagg>dominictarr: did you see the foojs entry to nko3? https://github.com/nko3/foojs/ awesome work in there, just needs persistent storage (via levelup of course!)
04:02:28  <rvagg>Raynos: I'll have a look at the code a little later, I have to pop out for a while
04:02:44  <dominictarr>rvagg, yeah I saw that.
04:03:00  <Raynos>dominictarr: https://github.com/rvagg/node-levelup/pull/31
04:03:02  <dominictarr>I've gotta meet up with that guy, while I am here
04:03:22  <Raynos>dominictarr: he was a nko. He uses a centralized cordination system :(
04:03:48  <rvagg>oh really? I thought he had a gossip-style thing going on?
04:04:06  <rvagg>that's a shame
04:04:25  <rvagg>oh well, he has vectorclocks and other neat stuff in there that should make it to reusable modules
04:05:13  <Raynos>yeah
04:05:18  <Raynos>he already made vectorclocks re-usable
04:05:25  * st_lukequit
04:05:51  <dominictarr>yeah, I'm not sure that the part he's made consistent really needs to work like that.
04:06:09  <dominictarr>since the rest of the design works deals with that anyway.
04:06:28  <dominictarr>Raynos, rvagg it's not actually centralized. I think
04:06:48  <Raynos>Hmm
04:06:53  <Raynos>I dont think its centralized actually
04:06:56  <Raynos>its just consistent
04:06:56  <dominictarr>It uses the same approach as zokeeper
04:06:58  <Raynos>thats what it was
04:07:00  <dominictarr>zookeeper
04:07:41  <Raynos>i cant remember
04:07:45  <Raynos>the problem was that its not seaport :P
04:08:26  <dominictarr>haha, I'm gonna organise a distributed systems js meeting
04:08:44  <dominictarr>between the people who are really into this stuff in the bay area while I'm here.
04:08:49  <dominictarr>gonna invite him
04:09:03  <dominictarr>substack, when do you get back?
04:25:06  * tphummelquit (Quit: tphummel)
04:27:40  * AvianFluquit (Remote host closed the connection)
04:38:10  * fentquit (Ping timeout: 244 seconds)
04:48:37  * st_lukejoined
04:51:20  * thatguydanjoined
04:51:22  * thatguydanquit (Read error: Connection reset by peer)
04:51:29  * thatguydan_quit (Read error: Connection reset by peer)
04:52:09  * thatguydanjoined
04:52:19  <rvagg>dominictarr: need an irc channel for that, sad I'm not in SF!
04:52:56  <dominictarr>yeah, this is the benefit of travelling and conferences.
04:53:04  <dominictarr>did you go to jsconf.au?
04:54:35  <rvagg>dominictarr: yes! it was so good to meet all these people I only know online
04:55:02  <rvagg>and isaacs, mikeal & brendan too of course
04:55:42  <rvagg>you missed out, but we have some steam building for a node.au event sometime in the future, I'll give you enough warning to be on this side of the world for that
04:56:17  <dominictarr>haha, well I've been at several conferences this side of the globe on this trip!
04:57:00  <dominictarr>It's looking like a possibility I might be back in the antipodes for a bit this summer!
04:57:10  <dominictarr>will have to drop by!
04:58:39  * defunctzombiejoined
04:59:16  <rowbit>Hourly usage stats: [developer: 0, free: 30]
05:00:45  <ik>DAYCHANGE!
05:04:00  * fentjoined
05:19:22  <rvagg>so dominictarr, you're doing some map/reduce work for nearForm, how do you fit in Raynos, do you work for nearForm?
05:19:38  <Raynos>I WANT TO WRITE ALL THE CODE
05:20:04  <rvagg>ya, fair enough
05:20:29  <rvagg>and this fellow? https://twitter.com/ramitos/status/270541142842540032
05:20:53  <rvagg>or he's just an observer perhaps..
05:21:17  <Raynos>dont know him
05:28:57  <Raynos>juliangruber: ping
05:29:56  * fentquit (Ping timeout: 248 seconds)
05:38:28  <rvagg>Raynos: derp - https://travis-ci.org/rvagg/node-levelup
05:38:50  <Raynos>o_o
05:38:52  <Raynos>sec
05:39:07  <Raynos>of course >_<
05:39:24  <rvagg>feel free to put yourself on the travis email list if you want to be complained at
05:41:53  <Raynos>ah of course!
05:41:59  <Raynos>herps
05:42:52  <Raynos>fixed.
05:42:53  * defunctzombiequit (Remote host closed the connection)
05:55:48  * st_lukequit (Ping timeout: 255 seconds)
05:59:16  <rowbit>Hourly usage stats: [developer: 1, free: 51]
06:07:42  * st_lukejoined
06:08:56  <st_luke>I need to move somewhere where a cappella groups of hipsters don't gather outside my window to do dumb rap concerts
06:09:41  <fotoverite>Wait what?
06:09:45  <fotoverite>THat never happens to me.
06:09:57  <fotoverite>Luke you doing anything for thanksgiving?
06:12:32  <st_luke>nothing different than any other day of the week
06:23:29  <ik>moonlight!
06:45:08  * st_lukequit (Remote host closed the connection)
06:48:17  <Raynos>writing a distributed peer list
06:48:25  <Raynos>anyone have any api suggestions
06:48:31  <Raynos>or want to point me to a module that already does this
06:59:16  <rowbit>Hourly usage stats: [developer: 0, free: 42]
07:06:20  <dominictarr>rvagg, Raynos and I have just been collaborating on stuff. Raynos is crazy, he'll post loads and loads of issues on your projects.
07:06:31  <Raynos>>:(
07:06:34  <Raynos>i am crazy
07:06:49  <Raynos>ive been doing it to airportyh >_<
07:06:49  <dominictarr>that was a complement
07:06:50  <Raynos>I feel bad
07:06:58  <dominictarr>no it's good
07:07:03  <Raynos>https://github.com/airportyh/testem/issues?state=open
07:07:04  <Raynos>9 !
07:07:10  <Raynos>thats like more then 50% of his open issues
07:07:24  <Raynos>btw im writing peer nodes!
07:07:42  <dominictarr>you are making up for all the people who never even tell you that they are using something.
07:08:06  <dominictarr>I gotta relocate. BRB.
07:08:24  <jjjjohnnny>Raynos: write node-p2p
07:08:44  <Raynos>jjjjohnnny: Pretty much am
07:08:46  <Raynos>seen my gists?
07:09:07  <jjjjohnnny>no
07:09:10  <Raynos>https://gist.github.com/4116515
07:09:51  <jjjjohnnny>is this real p2p with NAT traversal or is it nodes in the cloud p2p
07:10:35  <Raynos>now with comments!
07:11:17  <Raynos>its p2p using webrtc
07:11:27  <Raynos>but because i havnt done the hard work of implementing webrtc it uses a relay server
07:12:42  * dominictarrquit (Ping timeout: 264 seconds)
07:12:55  <jjjjohnnny>the decntralized crypto anonymous protocol requires real p2p, real nat punching p2p
07:14:12  <Raynos>webrtc does nat punching
07:14:27  <mbalho>read that as cat punching and got really angry for a second
07:18:51  <jjjjohnnny>cat 5s for everybody
07:19:14  <jjjjohnnny>I SAID CAT FIVES FOR EVERYBODY
07:20:17  <Raynos>Ok
07:20:21  <Raynos>how do I do reliable Date.now()
07:20:28  <Raynos>I need a reliable heartbeat
07:20:35  <Raynos>but any timestamp the client gives me is full of shit
07:21:00  <jjjjohnnny>thats a sad poem
07:21:55  <jjjjohnnny>*tries online dating*
07:22:19  <Raynos>I guess I can use vector clocks
07:32:49  * tphummeljoined
07:32:51  * thatguydan_joined
07:36:51  * thatguydanquit (Ping timeout: 276 seconds)
07:37:11  * thatguydan_quit (Ping timeout: 252 seconds)
07:59:16  <rowbit>Hourly usage stats: [developer: 0, free: 31]
08:00:16  * dominictarrjoined
08:14:56  <Raynos>FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF------------------------
08:15:08  <Raynos>INTERNETS Y U NO SIMPLER
08:19:28  * ITprojoined
08:23:46  * tphummelquit (Quit: tphummel)
08:24:08  * ITproquit (Ping timeout: 276 seconds)
08:25:32  <Raynos>dominictarr: vector clocks! Should I put them in scuttlebutt instead of timestamps?
08:25:36  <Raynos>or do they go somewhere else
08:26:01  * niftylettucequit (Max SendQ exceeded)
08:26:02  <dominictarr>Raynos, no
08:26:15  <dominictarr>scuttlebutt is a vector clock already
08:28:05  <dominictarr>if you want to do something useful, we need a synchronized auto adjusting clock
08:28:54  * niftylettucejoined
08:28:58  * niftylettucequit (Max SendQ exceeded)
08:29:44  <Raynos>how is scuttlebutt a vector clock?
08:31:41  <Raynos>i want a vector clock myself because I cant trust clients the have a correct timestamp
08:31:59  <Raynos>Does the synchronized auto adjusting clock handle the "some client has his local PC time 3 weeks in the future" problem?
08:37:17  * niftylettucejoined
08:39:08  <dominictarr>well, what you would do, it just tell him to change his clock.
08:39:32  <dominictarr>or rather, use a "virtual clock" that is set to a reasonable time.
08:39:47  <Raynos>how do you create a virtual clock
08:39:56  <Raynos>without asking a central service for what the current time is
08:40:10  <dominictarr>the problem with normal vector clocks in SB is that currently it's using a clock to decide which writer wins.
08:40:55  <dominictarr>a vector clock tells you what update is concurrent, but it doesn't help you decide who should win.
08:41:23  <dominictarr>Raynos, I think you could do a distributed p2p synchronization
08:41:33  <Raynos>maybe
08:41:41  <Raynos>the problem is its a pain in the fuckign ass to bootstrap
08:41:58  <Raynos>because I need it for peer nodes which I use to bootstrap the list of nodes to create the network topology
08:41:58  <dominictarr>you'd just ping back and forth between a number of clients, and average that out.
08:42:08  <Raynos>i see
08:42:48  <dominictarr>thats okay, it doesn't matter if the clocks are correct, just that they show approximately the same value on their face.
08:43:35  * captain_morganquit (Ping timeout: 252 seconds)
08:43:39  <dominictarr>if a clock joins that is too far ahead, just bump the other clocks forward.
08:43:49  <dominictarr>or set it's clock back, so that it's even.
08:43:50  <Raynos>I see
08:44:04  <dominictarr>or, maybe you could note that it's jumped
08:44:14  * defunctzombiejoined
08:44:25  <dominictarr>and store that information somewhere so that when you are comparing times
08:44:56  <dominictarr>you can check that Raynos has been 20 min ahead since saturaday
08:45:01  * captain_morganjoined
08:46:34  <dominictarr>once, they discovered that the date in the outer reaches of the brittish empire was 17 wrong compared to England.
08:46:59  * rannmannquit (Ping timeout: 260 seconds)
08:47:10  <dominictarr>so, they just told everyone, "okay, don't count 17 days!" and then everyone was back in sync.
08:47:59  <Raynos>ehm
08:48:40  * defunctzombiequit (Ping timeout: 252 seconds)
08:50:29  <Raynos>FUCK IT
08:51:13  <dominictarr>Raynos, ah maybe it was a different thing;
08:51:14  <dominictarr>http://en.wikipedia.org/wiki/Gregorian_calendar
08:51:51  <dominictarr>the pope realized that the roman calendar had slightly the wrong length of the year,
08:52:03  <dominictarr>and easter was being celebrated 10 days off,
08:52:13  * captain_morganquit (Ping timeout: 245 seconds)
08:52:30  <dominictarr>so they removed 3 leap years in 100
08:53:34  <dominictarr>year % 100 are not leap years unless year % 400
08:54:07  <dominictarr>by the time russia and greece adopted the julian calendar (in the 20th CENTURY!)
08:54:18  <dominictarr>sorry, gregorian calendar
08:54:26  <dominictarr>they where 13! days out
08:54:31  <Raynos>https://github.com/Raynos/ready-signal/commit/747edf6433cc2348bc775b01e07530db7a688eb2
08:54:33  <Raynos>BEST REFACTOR EVER
08:54:51  <Raynos>Inlined everything that was relevant AND removed 2 dependencies
08:56:46  <Raynos>https://gist.github.com/4116665
08:56:48  <Raynos>forever iterating!
08:57:39  <Raynos>is there a scuttlebutt implementation yet for a memory constrained doc?
08:58:06  <Raynos>What would be a good name for model except it purges anything that is too old
08:58:17  <Raynos>like a model with expiry
08:59:16  <rowbit>Hourly usage stats: [developer: 0, free: 49]
09:02:26  * captain_morganjoined
09:04:44  <dominictarr>model only keeps the latest update for a key
09:05:01  <dominictarr>so the document only gets as big as the number of keys.
09:05:24  <dominictarr>scuttlebutt/events wants expiry though.
09:06:00  * substackonline briefly
09:06:07  <substack>some unsecured android hotspot
09:06:17  <substack>wifi is tricky to find around these parts
09:06:51  <dominictarr>hey, so how is the japanese node scene?
09:06:57  <dominictarr>is there mad science?
09:07:20  <substack>loads of it
09:07:35  <substack>lots of hardware hackers especially
09:07:37  <substack>so much hardware
09:08:29  <substack>dominictarr: if you ever visit japan just go to akihabara
09:08:58  <dominictarr>I feel like I've heard of that place
09:09:08  <dominictarr>is that the place with all the electronics?
09:09:10  <substack>electric city https://en.wikipedia.org/wiki/Akihabara
09:09:11  <substack>yes
09:09:58  <Raynos>dominictarr: Oh events wants expiry
09:10:00  <dominictarr>it's the (extra) nerdy part of japan.
09:10:03  <Raynos>I'm writing expiry for Model
09:10:11  <Raynos>because I have a lot of keys
09:10:49  <Raynos>dominictarr: Would you like a PR for an expired model or is that userland?
09:11:27  <dominictarr>why do you have so many keys?
09:11:58  <Raynos>its the peer nodes set
09:12:02  <dominictarr>the problem with expiry is it's no longer consistent.
09:12:04  <Raynos>its the set of all peers connected to the network
09:12:15  <Raynos>thats ok
09:12:22  <Raynos>the peers auto update themself
09:12:26  <Raynos>every 5 seconds as a heartbeat
09:12:33  <Raynos>if the value expires it means the peer is no longer in the network
09:12:36  <dominictarr>right, expiry is okay for keys that are deleted.
09:12:53  <Raynos>expiry is also ok for keys that auto update at high frequency
09:13:19  <dominictarr>well, here I think you don't even want scuttlebutt.
09:13:30  <Raynos>hmm
09:13:32  <Raynos>maybe true.
09:13:42  <Raynos>how would I do it without scuttlebutt
09:13:42  <dominictarr>there is no point doing the hand shake, because you are gonna update all the keys anyway
09:13:46  <dominictarr>use repred
09:14:04  <dominictarr>it already has a example for peer to peer.
09:14:12  <dominictarr>with heart beats.
09:14:38  <Raynos>oh I see
09:17:08  <Raynos>repred is too confusing >_<
09:18:08  <dominictarr>what is confusing?
09:19:18  <Raynos>i dont understand what that map function does
09:21:13  <Raynos>ill optimize peers later
09:21:57  * jden|zzzchanged nick to jden
09:26:20  <dominictarr>map turns the repred.update(X) into the TypeX that reduce uses.
09:27:32  <Raynos>and update(X) is called internally based on the changes object returned from reduce?
09:40:59  <dominictarr>no, you are meant to mutate the state object inside the reduce function.
09:41:29  <dominictarr>it's stateful, but the user is expected to use self disipline
09:44:46  <dominictarr>Raynos, much updated queue and map-reduce
09:44:53  <Raynos>cool
09:45:01  <dominictarr>next is supporting multiple map-reduce views.
09:45:06  <Raynos>cool
09:45:16  <dominictarr>but now, it's my bed time.
09:45:19  <dominictarr>night all.
09:49:43  * dominictarrquit (Ping timeout: 244 seconds)
09:56:07  * thatguydanjoined
09:56:07  * captain_morganquit (Read error: Connection reset by peer)
09:57:20  * captain_morganjoined
09:57:44  * thatguydanpart
09:59:16  <rowbit>Hourly usage stats: [developer: 0, free: 90]
10:04:21  <Raynos>juliangruber: https://github.com/Raynos/peer-nodes#example
10:04:24  <Raynos>too late man! too late
10:12:22  * thatguydanjoined
10:12:22  * thatguydanquit (Client Quit)
10:35:48  * ITprojoined
10:44:51  * defunctzombiejoined
10:49:11  * defunctzombiequit (Ping timeout: 245 seconds)
10:59:16  <rowbit>Hourly usage stats: [developer: 1, free: 92]
11:31:53  * fotoverite_joined
11:34:18  * fotoveritequit (Ping timeout: 264 seconds)
11:34:18  * fotoverite_changed nick to fotoverite
11:36:32  * purrchanged nick to plurr
11:37:18  * captain_morganquit (Ping timeout: 264 seconds)
11:49:35  * captain_morganjoined
11:51:11  * ralphtheninjajoined
11:59:16  <rowbit>Hourly usage stats: [developer: 2, free: 136]
12:07:18  * devaholicquit (Ping timeout: 256 seconds)
12:14:41  * fentjoined
12:32:14  <substack>jjjjohnnny: so it turns out that computer vision algorithms use a lot of the same algorithmic primitives as audio filters
12:32:27  <substack>like band-pass filters
12:34:03  <substack>oh sweet the parrot ar drone is in the US now
12:34:09  <substack>not delivered yet though
12:59:16  <rowbit>Hourly usage stats: [developer: 0, free: 149]
12:59:25  * rannmannjoined
13:08:05  * LOUDBOTquit (Remote host closed the connection)
13:08:39  * LOUDBOTjoined
13:27:29  * yorickjoined
13:27:30  * yorickquit (Changing host)
13:27:30  * yorickjoined
13:31:55  * AvianFlujoined
13:46:14  * defunctzombiejoined
13:50:59  * defunctzombiequit (Ping timeout: 265 seconds)
13:59:16  <rowbit>Hourly usage stats: [developer: 0, free: 103]
14:37:09  * defunctzombiejoined
14:55:03  * mikealjoined
14:59:16  <rowbit>Hourly usage stats: [developer: 5, free: 104]
15:29:43  * chadskidmorequit (Ping timeout: 260 seconds)
15:31:36  * chadskidmorejoined
15:39:14  * st_lukejoined
15:59:16  <rowbit>Hourly usage stats: [developer: 1, free: 44]
16:02:37  * yorickquit (Remote host closed the connection)
16:09:51  * _sorensenjoined
16:14:27  * yorickjoined
16:14:28  * yorickquit (Changing host)
16:14:28  * yorickjoined
16:58:58  * dominictarrjoined
16:59:16  <rowbit>Daily usage stats: [developer: 43, free: 1646]
16:59:17  <rowbit>Hourly usage stats: [developer: 1, free: 51]
17:27:55  * plurrchanged nick to purr
17:30:12  * captain_morganquit (Remote host closed the connection)
17:32:56  * fotoverite_joined
17:34:51  * fotoveritequit (Ping timeout: 276 seconds)
17:34:51  * fotoverite_changed nick to fotoverite
17:35:08  * _sorensenquit (Ping timeout: 245 seconds)
17:36:35  * jdenchanged nick to jden|away
17:39:32  * zz_shykeschanged nick to shykes
17:39:57  * _ritchjoined
17:55:53  * paul_irish_changed nick to paul_irish
17:59:16  <rowbit>Hourly usage stats: [developer: 7, free: 74]
18:10:00  * blakmatrixjoined
18:10:20  * blakmatrixpart
18:14:35  * tphummeljoined
18:14:58  * tphummelquit (Client Quit)
18:15:55  * tphummeljoined
18:19:14  * shykeschanged nick to zz_shykes
18:20:44  * dominictarrquit (Ping timeout: 256 seconds)
18:25:12  * zz_shykeschanged nick to shykes
18:40:57  * wiwilliajoined
18:42:32  * _sorensenjoined
18:45:12  * paul_irishchanged nick to paul_irish_
18:50:55  <mbalho>guize
18:50:58  <mbalho>we gotta read this http://dvcs.w3.org/hg/streams-api/raw-file/tip/Overview.htm
18:51:08  <mbalho>it has a thing for high water mark streams
18:59:16  <rowbit>Hourly usage stats: [developer: 13, free: 51]
19:03:16  * dominictarrjoined
19:07:36  * defunctzombiequit (Remote host closed the connection)
19:08:30  * shykeschanged nick to zz_shykes
19:10:29  * tphummelquit (Ping timeout: 268 seconds)
19:10:50  * captain_morganjoined
19:11:07  * tphummeljoined
19:17:52  * captain_morganquit (Read error: Connection reset by peer)
19:18:04  * captain_morganjoined
19:18:10  * fotoveritequit (Remote host closed the connection)
19:24:54  * dominictarrquit (Ping timeout: 240 seconds)
19:26:33  <jjjjohnnny>mbalho: are you coming for thxgvg?
19:29:32  * captain_morganquit (Read error: Connection reset by peer)
19:33:41  * captain_morganjoined
19:38:07  * defunctzombiejoined
19:46:42  * defunctzombiequit (Ping timeout: 265 seconds)
19:47:42  * captain_morganquit (Remote host closed the connection)
19:54:03  * tphummelquit (Quit: tphummel)
19:57:13  * yorickquit (Ping timeout: 246 seconds)
19:59:16  <rowbit>Hourly usage stats: [developer: 6, free: 61]
19:59:22  * tphummeljoined
20:06:07  * zz_shykeschanged nick to shykes
20:11:39  * yorickjoined
20:11:39  * yorickquit (Changing host)
20:11:39  * yorickjoined
20:14:36  * AvianFluquit (Remote host closed the connection)
20:19:21  * tilgovijoined
20:29:57  <st_luke>mikeal: finally got anarcho-syndicalism delivered
20:30:00  <st_luke>I am excited to read through it
20:36:11  <jjjjohnnny>Alameda County App Challenge http://code.acgov.org/
20:41:03  * defunctzombiejoined
20:41:13  * shykeschanged nick to zz_shykes
20:41:59  * zz_shykeschanged nick to shykes
20:59:16  <rowbit>Hourly usage stats: [developer: 1, free: 48]
21:00:26  * shykeschanged nick to zz_shykes
21:07:47  <ralphtheninja>anarcho syndicalism is the shit :)
21:11:55  * dominictarrjoined
21:14:29  * fotoveritejoined
21:23:44  <mbalho>jjjjohnnny: SORRY yes i am!
21:24:04  <mbalho>jjjjohnnny: im having a hardware hack thing at my place tomorrow at 6pm with some rad peeps who are in town
21:24:10  <mbalho>jjjjohnnny: including dominictarr
21:25:45  <fotoverite>Damn it I want my transporter already
21:28:09  <rowbit>/!\ ATTENTION: (default-local) systems+browserling@... successfully signed up for developer browserling plan ($20). Cash money! /!\
21:28:09  <rowbit>/!\ ATTENTION: (default-local) paid account successfully upgraded /!\
21:35:09  * zz_shykeschanged nick to shykes
21:36:04  <Raynos>mbalho: what kind of hardware?
21:36:09  <Raynos>wood or computers
21:36:12  <mbalho>yes
21:36:15  * tilgoviquit (Quit: No Ping reply in 180 seconds.)
21:36:21  <mbalho>kites, coffee roasters
21:36:41  * tilgovijoined
21:45:09  <Raynos>I see
21:45:36  <Raynos>not sure how to hack those but might be fun to join you
21:59:16  <rowbit>Hourly usage stats: [developer: 1, free: 41]
21:59:48  <mbalho>Raynos: ya please do!
22:07:04  <mbalho>https://gather.at/e/5e2f10ff <-- hardware.js hack night
22:08:42  <ralphtheninja>sounds really fun, too bad I'm on the other side of the world, perhaps I should go to Oakland and check it out
22:09:21  <ralphtheninja>.. to check out Oakland that is, not necessarily jump on a plane right now for the hack night ;)
22:09:26  <mbalho>haha
22:09:32  <mbalho>build a nodecopter and fly yourself
22:09:43  <mbalho>we calculated that you would need 300 nodecopters to lift a human
22:09:47  <ralphtheninja>haha
22:10:05  <ralphtheninja>an african or european nodecopter? :p
22:10:23  <mbalho>haha
22:10:38  <dominictarr>ralphtheninja, on my todo list is to build a telepresense bot
22:10:41  <ralphtheninja>"I dunno .. aaaaah"
22:11:00  <dominictarr>so that I can still hangout in the bay area where ever I happen to physically be.
22:11:31  <ralphtheninja>dominictarr: you have some serious projects on your todo list :)
22:11:41  <dominictarr><3 that everyone just calls them nodecopters now
22:12:08  <ralphtheninja>felix did an awesome job there
22:12:23  <dominictarr>ralphtheninja, yeah, I try to talk about them so that there is a chance someone else just builds it.
22:13:16  <ralphtheninja>dominictarr: lol, trying to plant evil plans in my brains eh? good! :)
22:13:22  <ralphtheninja>brain*
22:13:58  <dominictarr>indeed
22:14:16  * mike-dquit (*.net *.split)
22:14:16  * Raynosquit (*.net *.split)
22:15:43  * fent_joined
22:16:03  * fent_quit (Client Quit)
22:22:48  * shykeschanged nick to zz_shykes
22:24:16  <dominictarr>here is another idea: skype puppets.
22:24:30  <fotoverite>YES!
22:24:34  * fentquit (*.net *.split)
22:24:44  <dominictarr>you'd have a camera, that recognised your facial expression
22:24:53  <dominictarr>but instead of sending your image
22:24:54  * rvaggchanged nick to polyhackbot
22:25:08  <dominictarr>just sent a cartoon of you
22:25:24  <dominictarr>so it would be semi anonymous, but still expressive
22:26:23  * polyhackbotchanged nick to rvagg
22:26:30  * tilgoviquit (Ping timeout: 264 seconds)
22:28:24  * Raynosjoined
22:28:37  <dominictarr>also, it would be way lower bandwidth than video
22:29:01  <dominictarr>and would also obscure distracting features, like the persons race.
22:30:41  * yorickquit (Remote host closed the connection)
22:44:54  * tilgovijoined
22:47:52  <Raynos>juliangruber: feel free to patch / fix / recommend changes to peer-nodes. Thanks for the help
22:48:20  <Raynos>Added you as collab!
22:48:22  * wiwilliaquit
22:59:16  <rowbit>Hourly usage stats: [developer: 6, free: 55]
23:00:52  <dominictarr>Raynos, actually, should have this conversation here
23:01:02  <Raynos>#winnign
23:01:20  <dominictarr>investigating ways to set up keys to get ranges
23:01:26  <dominictarr>this will be useful for map reduce
23:01:31  <dominictarr>like you can go
23:02:02  <dominictarr>~BUCKET~2,"KEY1","KEY2" - ~BUCKET~2,"KEY1","KEY2"~
23:02:23  <dominictarr>to get everything in [key1, key2] group
23:02:28  <dominictarr>but also,
23:02:46  <dominictarr>the 2 at the start specifies the depth you want
23:02:52  <dominictarr>but you can ALSO do it like this
23:03:07  <dominictarr>~BUCKET~2,"KEY1" - ~BUCKET~2,"KEY1"~
23:03:24  <dominictarr>and you get every sub group under key1
23:03:48  <dominictarr>this will be super awesome for spatial queries
23:04:02  <Raynos>yes!
23:04:37  <dominictarr>because you can just go, give me the line level detail of everything in tenderloin
23:04:40  <dominictarr>or whatever
23:05:09  <dominictarr>but also, you could do: give me the summary in the tenderloin
23:05:13  <dominictarr>the first is
23:05:29  <dominictarr>~SF~2,"tenderloin" ...
23:05:34  <dominictarr>but the second is
23:05:44  <dominictarr>~SF~1~"tenderloin"
23:05:58  <dominictarr>the N specifies the depth you want to go to.
23:07:50  <dominictarr>can you do this is couchdb?
23:09:51  <dominictarr>I think you might, not sure.
23:10:14  <dominictarr>I'm writing some stuff to make these ranges easy to specify
23:10:17  <dominictarr>like,
23:10:43  * AvianFlujoined
23:10:50  <dominictarr>b = bucket('bucname') ; b.range([big, true])
23:11:40  <dominictarr>returns a range that will return everything inside big, at level 2
23:11:48  <dominictarr>b.range([big])
23:12:28  <dominictarr>will just give the summary for the [big] group.
23:13:00  <dominictarr>you'll also be able to select through mutiple groups b.range([bigA, true], [bigB, true])
23:14:26  <ralphtheninja>dominictarr: what are you working on?
23:14:57  <dominictarr>map-reduce for levelup.
23:17:26  <ralphtheninja>how come you started on that? any special plans for it?
23:18:05  * thatguydanjoined
23:20:00  <dominictarr>need map reduce to build queries over the distributed real time stuff
23:20:34  <dominictarr>but ALSO, to calculate merkle trees
23:20:45  <dominictarr>which is a clever sort of replication.
23:20:55  * zz_shykeschanged nick to shykes
23:26:05  <dominictarr>AvianFlu, no, Ralph Merkle trees.
23:30:08  * dominictarrpart ("Leaving")
23:30:08  <Raynos>dominictarr: we need version numbers in the leveldb replicating thing
23:30:14  * dominictarrjoined
23:30:15  <Raynos>and a rejected handshake if it fails
23:30:36  <Raynos>So that if you replicate from leveldb a to b they should have the same version
23:30:43  <Raynos>to stop old data from being replicated into your app
23:30:47  <dominictarr>Raynos, can you repeat? I accidentiall quit for a sec
23:30:51  <Raynos>STALE DATA MUST DIE
23:31:03  <dominictarr>I got "and a rejected handshake..."
23:31:04  <Raynos>if we have a leveldb replicate function it should use semver
23:31:14  <Raynos>to close sockets that try to replicate old / stale data into my db
23:31:23  <Raynos>although actually
23:31:28  <Raynos>that can just be done at the network level
23:31:30  <Raynos>with header stream
23:31:31  <Raynos>never mind
23:31:43  <dominictarr>maybe we want a migration plugin
23:32:06  <dominictarr>so you specify a function to migrate 0.0.1 -> 0.0.2
23:32:20  <dominictarr>and a function to tell you what version a data is
23:32:23  * Krinklejoined
23:32:40  <dominictarr>and then whenever you read, that migration happens, if it needs to.
23:32:51  <Raynos>level.fn.migrate = function () { /* do it! */ }
23:33:30  <dominictarr>I think you'd need to specify a bit more detail
23:33:38  <dominictarr>like what ranges should be migrated
23:34:02  <Raynos>well
23:34:05  <dominictarr>if this is data that is replicated, you'd need to be able to distribute the migration too
23:34:07  <Raynos>migration is just a map reduce!
23:34:15  <Raynos>if someone writes data in the old format
23:34:17  * mikealquit (Quit: Leaving.)
23:34:21  <Raynos>map reduce it to the new format
23:34:24  <Raynos>PROBLEMS SOLVED
23:34:31  <Raynos>your just forced to change key names
23:34:35  <Raynos>each time you make a db change
23:35:06  <dominictarr>not sure if key changes are what you want.
23:36:04  <Raynos>maybe
23:36:05  <dominictarr>also, it would be a particular type of MR, probably just a map
23:36:06  <Raynos>but its simple
23:36:14  <Raynos>probably
23:36:21  <Raynos>destructive map
23:36:22  <dominictarr>the implications arn't
23:36:25  <Raynos>one that cleans up after itself
23:36:36  <dominictarr>yeah, cos you are writing back into source
23:36:36  <Raynos>well if you dont want any db versioning hell
23:36:47  <Raynos>ensure you never use the same key to mean two different things
23:36:59  <Raynos>the only problem your left with is a shit ton of denormalization
23:37:04  <Raynos>but thats ok MR it
23:37:16  <dominictarr>you could do it that way, but then you'd have to build awareness of versions thoughout your whole application.
23:38:16  <dominictarr>so here is a crazy idea
23:38:31  <dominictarr>build a offline personalized npm index
23:38:37  <dominictarr>using level db
23:39:04  <Raynos>what would it index on
23:39:11  <rvagg>! DO IT !
23:39:17  <Raynos>would it just map reduce npm to valuable stuff
23:39:18  <ralphtheninja>is leveldb a mvcc db?
23:39:23  <dominictarr>rank everything like page rank,
23:39:27  <dominictarr>but personalized
23:39:38  <dominictarr>starting from the modules that you use
23:39:54  <Raynos>npmRank(function rank(module) { if (module.name === 'hoarders') { return 9999 } else { return 0 } })
23:40:07  <dominictarr>flowing into the modules that the authors of the modules you use use, and so on.
23:40:19  <Raynos>ralphtheninja: nope. but you can make it a mvcc db!
23:40:22  <Raynos>implement it in js
23:40:51  <dominictarr>what is MVCC?
23:41:07  <ralphtheninja>dominictarr: multi version concurrency control
23:41:16  <ralphtheninja>save everything, don't update in place
23:41:17  <dominictarr>ralphtheninja, ah
23:41:22  <Raynos>did substack write an npm module to go back in time?
23:41:52  <dominictarr>no. it's just put, del, get, range, and batch (deletes + puts in a single trx)
23:41:58  <dominictarr>but you can build anything with that.
23:41:59  <ralphtheninja>I'm looking for a db that's similar to Datomic (Rich Hickey the clojure dude)
23:42:04  <dominictarr>you could build MVCC
23:43:18  <dominictarr>rvagg, would it be possible to add a feature like a range query, but tell it to only return N results starting or ending from a particular value?
23:43:28  <dominictarr>like the first value after "A"?
23:43:46  <Raynos>ralphtheninja: were you at the datomic thing a few weeks ago?
23:43:46  <ralphtheninja>imagine it like git, you can have snap shots of the db .. and when you have bugs you can save the snapshot id .. makes it super easy to reproduce bugs etc
23:43:55  <ralphtheninja>Raynos: nope
23:43:57  <rvagg>you could just write a wrapper around a readStream() cause that's what it'd be doing anyway--an iterator ending prematurely
23:44:30  <dominictarr>rvagg, you just destroy the stream after the first data?
23:45:07  <rvagg>ya, destroy() and it'll clean up for you
23:45:13  <dominictarr>cool!
23:45:24  <rvagg>internally it'll make 1 more fetch than you need but that's done async anyway
23:45:25  <dominictarr>ralphtheninja, ^ you could use that to implement MVCC
23:46:06  <dominictarr>rvagg, you could make an option that was {start: x, n: number} though, right?
23:46:29  <dominictarr>(as long as it's possible, then can defer that :)
23:46:32  <Raynos>dominictarr: just use take
23:46:36  <Raynos>take(stream, n)
23:46:45  <Raynos>composition!
23:46:56  <rvagg>dominictarr: yes, a number/count option would probably be neat
23:46:58  <dominictarr>Raynos, this is a low level performance issue
23:46:59  <rvagg>make an issue for it
23:47:10  <dominictarr>oh, this reminds me
23:47:11  <Raynos>true
23:47:35  <dominictarr>do the levelup streams respect .pause()?
23:47:45  <rvagg>yes
23:48:01  <dominictarr>SWEETNESS
23:48:15  <rvagg>dominictarr: but because it prefetches 1 entry ahead on the iterators then it'll buffer and discard if you don't read it
23:48:41  <Raynos>doesnt look like it
23:48:42  <dominictarr>that is reasonable behavior for pause.
23:49:12  <Raynos>oh wait
23:49:14  <Raynos>I see.
23:49:56  <Raynos>rvagg: https://github.com/rvagg/node-levelup/blob/master/lib/read-stream.js#L123
23:50:06  <Raynos>you seem to just _read() without checking for whether your paused
23:50:07  <rvagg>actually.... it may not be buffering...
23:50:30  <Raynos>:D
23:50:34  <Raynos>thats ok
23:50:37  <Raynos>just wrap it into streams2
23:50:41  <Raynos>or in pause stream
23:50:56  <rvagg>needs more tests on stream behaviour I suppose
23:51:30  <rvagg>actually, maybe I'm wrong!
23:51:40  <dominictarr>to be fast we need it to stop reading the database when you pause.
23:51:42  <rvagg>Raynos: check out the _read implementaiton, it only goes ahead if you're "ready"
23:52:04  <Raynos>oh true
23:52:09  <Raynos>and you change status to ready+paused
23:52:11  <Raynos>so its cool
23:52:19  <Raynos>yeah it respects pause
23:52:22  <rvagg>I have a slow-stream test in there to make sure it follows the correct stream backpressue semantics, it should be all good
23:52:48  <rvagg>I put a lot of sweat into that darn stream implementation... made me hate implementing streams even more
23:52:58  <Raynos>use an abstraction
23:53:15  <Raynos>https://github.com/Raynos/levelidb/blob/master/index.js#L109
23:53:20  <rvagg>use streams2
23:53:30  <dominictarr>rvagg, stream2 arn't stable yet
23:53:35  <rvagg>ya, I know
23:53:37  <rvagg>getting there..
23:54:06  <dominictarr>we can test the streams with stream-spec
23:54:49  <Raynos>yeah I use streams2 through read-stream
23:55:06  <Raynos>but thats probably suboptimum
23:55:22  <Raynos>best to jsut wait for 0.10
23:58:03  <ralphtheninja>is there any quick way to go to a corresponding github-repo for a npm module?
23:58:45  <dominictarr>ralphtheninja, there should be a link from npm.im/the_module
23:59:16  <rowbit>Hourly usage stats: [developer: 10, free: 45]