00:00:01  * ircretaryquit (Remote host closed the connection)
00:00:08  * ircretaryjoined
00:05:35  * Domenic__joined
00:05:39  <Raynos>substack: what's a good way to tell the dart core team "your streams without backpressure is a disaster waiting to happen"
00:05:58  <Raynos>isaacs: ^, actually is "stream without backpressure" really a disaster?
00:06:47  <substack>aren't they pull-streams?
00:07:11  <isaacs>Raynos: they do backpressure, i thought.
00:07:24  <Raynos>they are push streams
00:07:28  <isaacs>Raynos: the ones that don't do backpressure are buffered seekable streams
00:07:54  <isaacs>but i mean, whatver, that's buffering all your shit up anyway
00:07:59  <isaacs>at least, that's how i thought it worked
00:08:41  <substack>I have a use-case to mess with the new streams stuff finally now
00:08:48  <substack>first I'm going to publish it as classic streams
00:09:07  <substack>but using streams2 will prevent unnecessary buffering
00:09:12  <Raynos>oh ok they do pause
00:09:14  <Raynos>false alarm.
00:09:30  <Raynos>its just hard to find in their docs -.-
00:10:18  * Domenic__quit (Ping timeout: 264 seconds)
00:12:24  <Raynos>isaacs: idea, make all streams support `.close()`
00:13:31  <isaacs>Raynos: what's close() do if it isn't a thing that closes?
00:13:35  <Raynos>isaacs: are there any Readable streams where it would not make sense for them to have a close()
00:13:48  <Raynos>Oh wait duplex transforms >_<
00:13:58  <isaacs>Raynos: :)
00:14:10  <Raynos>yeah I live in lazy land >_<
00:14:20  <isaacs>Raynos: zlib streams have a .close() method
00:14:23  <Raynos>where there is just Readable.pipe(Writable) and no duplexes
00:14:30  <isaacs>Raynos: but it should only be called when both sides are done.
00:14:55  <Raynos>Ok what about this
00:14:58  <Raynos>Duplex.close() means
00:15:01  <isaacs>fs streams have a destroy(), which closes the fd
00:15:04  <isaacs>same for sockets
00:15:06  <Raynos>if there is any data left in the buffer just kill it
00:15:12  <isaacs>that's destroy()
00:15:19  <isaacs>what you probably want is destroySoon()
00:15:22  <Raynos>does fs.createReadStream().close() clean up the fd ?
00:15:29  <isaacs>s/close/destroy/ yeah
00:15:33  <Raynos>I just want an ABORT thing
00:15:36  <Raynos>which is like pause
00:15:37  <isaacs>naming things is hard.
00:15:45  <Raynos>but it actually means deallocate all resources and die
00:15:48  <isaacs>you want destroy()
00:16:02  <Raynos>so whats the difference between close() and destroy()
00:16:07  <Raynos>other then "oops naming is hard and back compat"
00:17:43  <isaacs>so, you have destroy() and destroySoon()
00:17:50  <isaacs>destroySoon() should really be renamed to close()
00:18:07  <isaacs>what it means is, write the remainder of what you have, and then destroy() once it's flushed out
00:19:04  <rvagg>so destroy() should just cancel anything in process? discard buffers etc.?
00:20:19  <Raynos>isaacs: I'm talking about Readable's
00:21:03  <isaacs>sure, but you can't have method called "close" on readables, that does a different thing on writables, because duplexes.
00:21:24  <sorensen>random question
00:21:28  <sorensen>wtf is the order of npm installs
00:21:39  <sorensen>seems ranom
00:21:41  <sorensen>random*
00:21:42  <rowbit>Hourly usage stats: [developer: 4, free: 21]
00:21:56  <Raynos>isaacs: oh ok
00:22:08  <isaacs>sorensen: it's kinda random
00:22:10  <Raynos>so there is no close, thats just a bad name, there is only destroy & destroySoon
00:22:13  <Raynos>and they work on Readable's ?
00:22:15  <sorensen>so glad lol
00:22:16  <isaacs>sorensen: more approrpiately, it's parallel
00:22:18  <sorensen>im not crazy
00:22:21  <sorensen>ah
00:22:25  <sorensen>explains it
00:22:28  <isaacs>sorensen: so, depends on the vagaries of disks and https
00:22:36  <Raynos>for a Readable if you call destroySoon do you have to deplete the internal buffer by calling read() until it returns null before it closes sockets & fds ?
00:22:44  <sorensen>parallel is all i needed ;) thanks
00:22:52  <isaacs>there is no concept of destroySoon for a Readable, relaly
00:22:56  <isaacs>"soon", what?
00:23:00  <isaacs>what is "soon"?
00:23:16  <sorensen>hot dam i'm excited for nodeconf
00:23:21  <isaacs>Raynos: most of the time, we just close when we get the EOF chunk
00:23:26  <isaacs>sorensen: :D
00:23:34  * chrisdickinsonwill be there too!
00:23:38  <sorensen>first time goin, couldnt make it last year
00:23:40  <Raynos>isaacs: what if I want to readFiveLines(fd) ?
00:23:55  <sorensen>woo! beer is on isaac!
00:23:57  <sorensen>:D
00:24:00  <Raynos>because thats the header of the file and dependending on the header I either read the content or abort
00:24:07  <isaacs>please don't get beer on me...
00:24:15  <sorensen>oh its happening
00:24:25  <sorensen>"yes sir, tab under the name isaac"
00:24:35  <sorensen>"shots for everyone, yes i'm sure
00:24:37  <isaacs>sorensen: nodeconf won't have "tabs" exactly.
00:24:38  <sorensen>;)
00:24:45  <isaacs>sorensen: have you ever been to summer camp?
00:24:49  <sorensen>no
00:24:53  <sorensen>just messing around
00:24:57  <sorensen>but no
00:24:58  <isaacs>it'll be more like coolers with bottles that you just open
00:25:05  <isaacs>in the woods, around a campfire
00:25:05  <sorensen>aww
00:25:09  <isaacs>for realsies
00:25:11  <sorensen>i do know about that
00:25:15  <isaacs>coyotes and shit
00:25:17  <sorensen>grew up in nebraska after all
00:25:18  <isaacs>deer
00:25:26  <isaacs>see, you know the deal :)
00:25:27  <sorensen>it'll be like home
00:25:37  <isaacs>but with trees and hills, so not so much like nebraska
00:25:38  <isaacs>but yeah
00:25:46  <sorensen>well, very much like nebraska ;)
00:26:05  <isaacs>where does nebraska have hills?
00:26:10  <sorensen>eas
00:26:11  <sorensen>east*
00:26:15  <isaacs>ohhhh...
00:26:20  <chrisdickinson>sort of like east kansas / missouri, huh?
00:26:20  <sorensen>like... omaha
00:26:22  <isaacs>i had relatives in lincoln a while ago
00:26:24  <sorensen>yeap
00:26:28  <sorensen>lincoln a bit flat
00:26:30  <isaacs>was pretty flat.
00:26:33  * chrisdickinsonis from east kansas
00:26:41  <sorensen>eastern nebraska gets towards the bluffs of iowa
00:26:44  <chrisdickinson>i think i've accidentally wound up in nebraska before
00:26:47  <sorensen>chrisdickinson: midwest!
00:26:58  <chrisdickinson>yeah, lawrence ks :D
00:26:59  <isaacs>it was just slightly flatter than 2d euclidian space
00:27:06  <sorensen>lol
00:27:15  <sorensen>it all varies
00:27:23  <isaacs>and not really trees.
00:27:27  <sorensen>for whatever reason i remember omaha being on a list of hilliest cities
00:27:39  <sorensen>lincoln... *sigh*
00:27:43  * isaacslives in Oakland, near SF
00:27:44  <chrisdickinson>isaacs: the only thing of note i saw in west kansas on the drive to oregon was a single ditch, full of broken down 1950's pickup trucks.
00:27:53  <sorensen>i like oakland
00:27:56  <isaacs>y'all ain't got nothin on hills.
00:28:00  <isaacs>:)
00:28:02  <sorensen>shiiiiit
00:28:17  <sorensen>could drive a stick in SF due to my omaha training
00:28:19  <sorensen>in the dam snow
00:28:26  <sorensen>yall dont know bout hills
00:28:36  <sorensen>:P
00:28:44  <Raynos>isaacs: http://nodejs.org/api/fs.html#fs_class_fs_readstream readstream does not have a close / destroy or destroySoon documented
00:29:04  <sorensen>what i love most about oakland is that the city of oakland is nicer than the city of SF
00:29:08  <isaacs>Raynos: yeah, it has a destroy, but not a destroySoon
00:29:12  <sorensen>crazy nice building
00:29:13  <isaacs>Raynos: not sure if it's documented
00:29:29  <Raynos>isaacs: so to clarify again in your mind does "destroySoon" and "close" mean the same thing?
00:29:30  <isaacs>Raynos: it's in the javascript low-level API docs.
00:29:39  <isaacs>Raynos: yeah, i guess so
00:29:45  <isaacs>Raynos: but both are app-specific
00:29:47  <Raynos>where is the low-level API docs?
00:30:07  <isaacs>Raynos: https://github.com/joyent/node/tree/master/lib/
00:30:16  <Raynos>oh.
00:30:17  <isaacs>Raynos: the docs that we write in javascript
00:30:19  <Raynos>so its in the code :P
00:30:19  <isaacs>;P
00:30:22  <mbalho>the sf hobart building is like the crappier version of the oakland cathedral building
00:30:32  <isaacs>sf has more stuff
00:30:55  <isaacs>like a crazy cat person, their carpets reeking of urine, and old photographs and rotten garbage from last month.
00:31:31  <sorensen>i do love sf
00:31:40  <sorensen>probably since i grew up in the midwest
00:31:43  <Raynos>oh interesting
00:31:49  <Raynos>you can't close() a stream, that's for servers
00:31:50  <sorensen>have an uncle that does city planning in SF
00:49:07  <jez0990>nodeconf
00:49:23  <jez0990>hmm
00:50:00  <mbalho>go!
00:53:14  <Raynos>always go
00:53:26  <Raynos>nodeconf is a must
00:57:49  <isaacs>Raynos: right, servers have close()
01:05:56  * Guest55310quit
01:09:03  * fallsemo1quit (Quit: Leaving.)
01:21:42  <rowbit>Hourly usage stats: [developer: 0, free: 42]
01:22:32  * jxsonquit (Ping timeout: 256 seconds)
01:24:52  * ralphtheninjaquit (Ping timeout: 260 seconds)
01:34:16  * devaholicjoined
01:42:42  * thl0joined
01:46:25  * Domenic__joined
02:05:26  * ins0mniajoined
02:06:36  <thl0>so substack, could we use deep-is in difflet since in that case seems to make more sense?
02:07:05  <thl0>i.e. you wouldn't want to see a diff indicator when you do 'diff(NaN, NaN)'
02:08:21  <Domenic__>thl0 enable issues on deep-is
02:08:36  <Domenic__>also make it work so that deepIs(-0, +0) === false
02:09:28  <mikolalysenko>why are all the red black tree libraries in npm useless?
02:09:42  <mikolalysenko>they all give you find, but none of them has a working lower_bound
02:09:50  <mikolalysenko>which is the whole point of using a tree in the first place?
02:09:56  <thl0>Domenic__: done
02:10:05  <thl0>Domenic__: why?
02:10:10  <mikolalysenko>if I didn't care about successor/predecessor queries I'd use a hash (end rant)
02:10:16  <thl0>is that bc Objec.is works that way?
02:12:08  * ins0mniaquit (Remote host closed the connection)
02:16:54  * timoxleyjoined
02:18:39  * stlsaintquit (Quit: leaving)
02:21:42  <rowbit>Daily usage stats: [developer: 33, free: 1307]
02:21:43  <rowbit>Hourly usage stats: [developer: 1, free: 27]
02:42:12  * thl0quit (Remote host closed the connection)
02:42:44  <niftylettuce>isaacs: can you help with this? https://github.com/nodejitsu/node-http-proxy/issues/399
02:42:58  <niftylettuce>isaacs: i noticed that crypto/tls now requires a ".pfx" file apparently in 0.10.x
02:43:16  <niftylettuce>isaacs: so all my my existing ssl setup is flawed without this pfx stuff, not certain how to create
02:43:54  * thl0joined
02:44:46  * thl0quit (Remote host closed the connection)
02:46:24  * Benviequit (Ping timeout: 264 seconds)
02:54:06  * Benviejoined
02:57:10  * mmckeggpart
02:59:28  <Raynos>miko: write a good one!
02:59:51  <Raynos>also how do you "know" that red black trees are good at lower bounds
02:59:59  <Raynos>I dont even know what lower bound means wrt trees
03:21:42  <rowbit>Hourly usage stats: [developer: 0, free: 24]
03:25:52  <mikolalysenko>Raynos: basically balanced binary trees are a way to store dynamic ordered sequences
03:26:19  <mikolalysenko>and the main thing you get with an oredered sequence is that you can binary search it in O(log n) to find lower bounds
03:26:49  <mikolalysenko>so if you want the items that come before and immediately after some query, you can figure that out
03:27:03  <mikolalysenko>it is slower and more expensive than a hash table, but you can answer stronger queries
03:27:37  <mikolalysenko>however, when I look in npm none of the balanced binary search tree libraries implement bounds queries!
03:27:46  <mikolalysenko>which means that all they end up being is just a crappy version of a hash table
03:28:11  <mikolalysenko>if all you can do is insert/delete/find, then you are strictly better off using a hash: why would you ever bother with a tree?
03:30:54  <Domenic__>thl0: yes, and of course the deeper reason for that is because that's how the ES spec works.
03:33:16  <mikolalysenko>actually, I just found one that looks good but it had a weird name: https://npmjs.org/package/leafy
03:34:36  <mikolalysenko>wait, never mind. It is still terrible
03:34:50  <mikolalysenko>the findLessThan/findGreaterThan methods return arrays of elements
03:35:00  <mikolalysenko>which means if you use them in a loop, you will do O(n^2) work
03:35:04  <mikolalysenko>retarded...
03:53:21  <Raynos>write one ;)
03:58:44  * dominictarrjoined
03:59:49  * dominictarrquit (Client Quit)
04:13:18  <niftylettuce>isaacs: https://github.com/nodejitsu/node-http-proxy/issues/399#issuecomment-16216048 it looks like there are some recent changes to the crypto module that might have caused the break
04:15:00  <niftylettuce>not sure which one though https://github.com/joyent/node/commits/v0.10.2-release/lib/crypto.js
04:15:08  <niftylettuce>im on 0.10.2
04:15:27  * jxsonjoined
04:18:17  * kenperkinsquit (Quit: Computer has gone to sleep.)
04:19:11  * kenperkinsjoined
04:21:42  <rowbit>Hourly usage stats: [developer: 0, free: 26]
04:32:09  * shamaquit (Remote host closed the connection)
04:33:19  * jcrugzzquit (Ping timeout: 245 seconds)
04:35:13  * AvianFluquit (Remote host closed the connection)
04:45:49  * fallsemojoined
04:57:08  * dguttmanquit (Quit: dguttman)
05:05:11  * timoxleyquit (Quit: Computer has gone to sleep.)
05:21:42  <rowbit>Hourly usage stats: [developer: 13, free: 31]
05:38:47  * jcrugzzjoined
05:49:33  * thatguydanjoined
05:58:12  * kenperkinsquit (Quit: Computer has gone to sleep.)
06:02:16  * mikolalysenkoquit (Ping timeout: 246 seconds)
06:05:27  * defunctzombiechanged nick to defunctzombie_zz
06:08:40  * jcrugzzquit (Ping timeout: 256 seconds)
06:21:42  <rowbit>Hourly usage stats: [developer: 0, free: 22]
06:39:53  * fallsemoquit (Quit: Leaving.)
07:06:07  * timoxleyjoined
07:08:57  * mikolalysenkojoined
07:13:20  * mikolalysenkoquit (Ping timeout: 245 seconds)
07:21:42  <rowbit>Hourly usage stats: [developer: 10, free: 28]
07:41:33  * jxsonquit (Ping timeout: 240 seconds)
08:01:07  * defunctzombie_zzchanged nick to defunctzombie
08:08:04  <substack>pow https://github.com/substack/slice-file
08:08:18  <substack>now implementing .follow()
08:08:19  <substack>like tail -f
08:21:42  <rowbit>Hourly usage stats: [developer: 0, free: 60]
08:22:09  * timoxleyquit (Quit: Computer has gone to sleep.)
08:23:17  * timoxleyjoined
08:37:01  * jxsonjoined
08:46:30  * ralphtheninjajoined
08:49:08  * dominictarrjoined
08:49:25  * no9joined
09:08:10  * devaholicquit (Remote host closed the connection)
09:10:02  * ralphtheninjaquit (Quit: leaving)
09:21:42  <rowbit>Hourly usage stats: [developer: 0, free: 28]
09:22:52  * dominictarrquit (Quit: dominictarr)
09:33:00  * dominictarrjoined
10:08:07  * dominictarrquit (Quit: dominictarr)
10:21:42  <rowbit>Hourly usage stats: [developer: 0, free: 59]
10:36:33  * no9quit (Ping timeout: 240 seconds)
10:37:49  * stagasjoined
10:40:31  <stagas>if anyone can help get the facts straight in this I would be eternally grateful :) https://github.com/stagas/browserify-vs-component/wiki/Browserify-vs-Component
11:07:15  * thl0joined
11:08:52  * jibayjoined
11:13:48  <defunctzombie>substack: I can has cool drawing?
11:18:22  <substack>oh right
11:21:42  <rowbit>Hourly usage stats: [developer: 0, free: 57]
11:21:42  * no9joined
11:38:13  * no9quit (Ping timeout: 248 seconds)
11:42:22  * saijanai_quit (Quit: saijanai_)
11:43:24  * spionjoined
11:47:11  * mikealjoined
12:08:20  * thatguydanquit (Ping timeout: 252 seconds)
12:09:33  * thatguydanjoined
12:15:07  * Guest55310joined
12:15:08  * Guest55310changed nick to ITpro
12:15:37  * ITprochanged nick to Guest33047
12:21:42  <rowbit>Hourly usage stats: [developer: 0, free: 42]
12:22:40  * st_lukejoined
12:34:36  * yorickjoined
12:49:12  * AvianFlujoined
12:52:27  * mikealquit (Read error: Connection reset by peer)
12:52:36  * mikeal1joined
12:53:52  * mikeal1quit (Client Quit)
12:59:08  * st_lukequit (Remote host closed the connection)
13:01:30  * thl0quit (Remote host closed the connection)
13:06:50  * st_lukejoined
13:09:25  * jibayquit (Quit: Leaving)
13:21:42  <rowbit>Hourly usage stats: [developer: 0, free: 32]
13:36:42  * thl0joined
13:40:46  * fallsemojoined
13:46:15  * mikolalysenkojoined
13:49:02  * tmcwjoined
14:00:27  * fallsemoquit (Quit: Leaving.)
14:02:11  * Domeni___joined
14:06:53  * gozalaquit (Ping timeout: 240 seconds)
14:07:10  * gozalajoined
14:14:18  * stagasquit (Read error: Connection reset by peer)
14:17:01  * Domeni___quit (*.net *.split)
14:17:01  * wolfeidauquit (*.net *.split)
14:17:02  * rook2pawnquit (*.net *.split)
14:17:02  * jesusabdullahquit (*.net *.split)
14:19:00  * Domeni___joined
14:19:00  * wolfeidaujoined
14:19:00  * rook2pawnjoined
14:19:00  * jesusabdullahjoined
14:21:42  <rowbit>Hourly usage stats: [developer: 0, free: 76]
14:26:27  * kenperkinsjoined
14:36:46  * fallsemojoined
14:38:55  * ralphtheninjajoined
14:46:28  * Benviequit (Quit: Benvie)
14:46:43  * Benviejoined
14:55:23  * Benviequit (Ping timeout: 255 seconds)
14:59:41  * Benviejoined
15:01:16  * Benviequit (Client Quit)
15:04:11  * Benviejoined
15:11:58  * jiang_plusjoined
15:13:47  * dguttmanjoined
15:18:12  <jiang_plus>is there any good website/tool for cooperation?
15:18:46  <tmcw>github.com
15:21:42  <rowbit>Hourly usage stats: [developer: 0, free: 52]
15:28:10  * jibayjoined
15:32:17  <st_luke>defunctzombie: are you still running that bouncer
15:37:42  * tmcwquit (Remote host closed the connection)
15:40:53  * jiang_plusquit (Ping timeout: 248 seconds)
15:41:29  * jiang_plusjoined
15:42:50  * jibayquit (Remote host closed the connection)
15:46:23  * dominictarrjoined
15:48:15  <defunctzombie>st_luke: yea
15:50:03  * jcrugzzjoined
15:51:44  <st_luke>where do I sign up? :)
15:59:23  * Guest33047quit (Ping timeout: 260 seconds)
16:02:16  * Domeni___quit (Remote host closed the connection)
16:04:04  * Domeni___joined
16:05:04  * Guest33047joined
16:05:04  * Guest33047changed nick to ITpro
16:05:34  * ITprochanged nick to Guest95467
16:07:31  * st_lukequit (Remote host closed the connection)
16:21:42  <rowbit>Hourly usage stats: [developer: 4, free: 36]
16:36:12  * stagasjoined
16:37:02  <pkrumins>50 repositories out of 270 on testling-ci have a badge on github: https://github.com/pkrumins/testling-ci-badge-checker/blob/master/readme.txt#L7
16:38:28  <Domeni___>pkrumins: i think putting the badges on GitHub will be a lot more popular now that there are not spurious failures making you look bad
16:38:52  * dominictarrquit (Quit: dominictarr)
16:39:09  <chapel>pkrumins: is testling-ci free for open source repos?
16:39:15  <pkrumins>yes it's free
16:39:27  * fotoveritejoined
16:39:32  <chapel>okay, will have to check it out finally
16:39:43  <pkrumins>chapel: here http://www.catonmat.net/blog/howto-write-testling-ci-tests/
16:39:52  <pkrumins>Domeni___: exactly!
16:42:10  <chapel>thanks pkrumins
16:42:19  <pkrumins>you're welcome :)
16:45:50  * timoxleyquit (Quit: Computer has gone to sleep.)
16:54:13  * thl0quit (Remote host closed the connection)
16:55:57  * dominictarrjoined
16:56:16  * crankquit (Ping timeout: 256 seconds)
16:58:04  * crankjoined
16:58:37  * Guest95467quit (Remote host closed the connection)
16:59:00  * Guest95467joined
17:01:27  <pkrumins>I just wrote a new blog post called "I published another 10 of my projects to GitHub": http://www.catonmat.net/blog/i-published-another-10-of-my-projects-to-github/
17:03:47  * fotoveritequit (Quit: fotoverite)
17:04:50  * fotoveritejoined
17:05:28  * dguttmanquit (Quit: dguttman)
17:08:50  * fotoveritequit (Client Quit)
17:20:42  * shamajoined
17:21:24  * tmcwjoined
17:21:42  <rowbit>Hourly usage stats: [developer: 0, free: 40]
17:23:04  * jcrugzz_joined
17:23:26  * jxsonquit (Remote host closed the connection)
17:23:49  * tmcwquit (Remote host closed the connection)
17:24:06  * jxsonjoined
17:24:36  * thl0joined
17:24:41  * tmcwjoined
17:26:00  * jcrugzzquit (Ping timeout: 258 seconds)
17:32:55  * thl0quit (Ping timeout: 245 seconds)
17:33:12  * jcrugzz_changed nick to jcrugzz
17:35:58  * thatguydanquit (Quit: thatguydan)
17:36:44  * dguttmanjoined
17:38:34  * thl0joined
17:47:52  * spionquit (Ping timeout: 246 seconds)
18:05:15  * jiang_plusquit (Ping timeout: 256 seconds)
18:05:23  * fallsemoquit (Quit: Leaving.)
18:05:58  * fallsemojoined
18:13:30  <isaacs>anyone in here use Readable.unshift() ever?
18:13:43  <isaacs>Raynos, dominictarr, etc? ^
18:15:03  <dominictarr>isaacs: that would be useful for writing a parser -- actually I'm working on one like that now -- but it's my opinion that that is not readable stream job
18:19:47  <isaacs>dominictarr: the reason i agreed to put it there was to use it for a better websockets API
18:19:57  <isaacs>so the "bodyPart" bullshit can go away.
18:20:02  <isaacs>just like, stuff it back up in there :)
18:20:23  <dominictarr>oh, right - I'm totally for that then
18:20:40  <isaacs>ok... so.. here's the weird thing..
18:21:07  <isaacs>three parties in a streams interaction: The Stream class (S) (ie, node)
18:21:19  <isaacs>the Author/implementor (ie, dominic) (A)
18:21:30  <isaacs>and the Consumer/user of the stream (ie, someone installing your module) (C)
18:21:42  <rowbit>Hourly usage stats: [developer: 8, free: 45]
18:21:53  <dominictarr>aha
18:21:56  <isaacs>C calls read(), which kicks off som emachinery in S and triggers A's _read() method
18:22:06  <dominictarr>yes
18:22:08  <isaacs>in A's _read(), A calls 'this.push(chunk)' at some point
18:22:28  <isaacs>so S says, "Great, i can unset that `reading` flag, and have you start getting some more!"
18:22:46  <isaacs>however, `unshift(chunk)` *also* sets reading=false
18:22:59  * st_lukejoined
18:23:15  <isaacs>so when C calls stream.unshift(bitThatDidntGetParsedYet)
18:23:22  <isaacs>A might be in the middle of a _read()
18:23:34  <isaacs>but S says, "Great! I see you're done reading, DO ANOTHER _READ!"
18:23:46  <isaacs>which A doesn't guard against, because S promised it would never ever do that!
18:23:50  <dominictarr>oh, right - hmm, well the consumer should use pipe
18:24:10  <dominictarr>and consume by passing to a writable stream
18:24:23  <isaacs>sure, pipe() to a transform stream which does this.push(output)
18:24:35  <dominictarr>yeah
18:24:37  <isaacs>if it's a parser for instance.
18:25:17  <isaacs>i hate our ws api
18:25:19  <isaacs>it's such garbage.
18:25:33  <isaacs>why couldn't we just make it keep using req/res.
18:25:34  <dominictarr>so, maybe there is a better way to remove bodyPart from ws
18:25:43  <isaacs>and you can then write to res and read from req, just like you do now
18:27:36  <dominictarr>I don't like a parser stream using unshift, because that is roping a different object into doing your job
18:27:43  <dominictarr>the parser should have it's own buffer
18:28:16  <dominictarr>unshift should be private, and you should only call your own unshift
18:30:22  <isaacs>hm.
18:30:40  <isaacs>as it is now, it's public, and the border between C and A are pretty flimsy
18:32:34  <dominictarr>with streams, I recommend people to treat write, end, on('data'…) as private and only use pipe.
18:32:51  <dominictarr>('end', 'close', 'error' are okay)
18:35:56  * st_lukequit (Remote host closed the connection)
18:36:26  <isaacs>hm
18:36:34  <isaacs>ok, well... that's not how everyone does it :)
18:36:51  * spionjoined
18:36:52  <isaacs>people are using unshift() with fs streams and breaking them
18:36:55  <isaacs>and then getting upset.
18:37:06  <Raynos>isaacs: nope
18:38:58  <dominictarr>it shouldn't matter what type of stream it is
18:39:26  <isaacs>well... it does. because some streams are tolerant of being interrupted during a _read()
18:39:29  <isaacs>others are not
18:39:33  <isaacs>fs streams are, for a bit
18:39:34  <Raynos>isaacs: wait what, why would C do unshift ?
18:39:38  <isaacs>then they suddenly aren't :)
18:39:52  <dominictarr>Raynos: it's trying to be a parser
18:40:00  <isaacs>Raynos: because C is pulling data off a socket, and then realizing it got too much, and putting some back
18:40:13  <dominictarr>but using the source's buffer as a work bench
18:40:15  <Raynos>why cant it just put the chunk in an array
18:40:20  <Raynos>nistead of callign unshift ?
18:40:22  <dominictarr>exactly
18:40:25  <isaacs>Raynos: becuase then it passes that same socket off to someone else.
18:40:39  <Raynos>isaacs: complex :(
18:40:45  <isaacs>it says, "Good news! It's a websocket! here's the req and res, and ALSO the socket!"
18:41:22  <isaacs>you know what...
18:41:31  <isaacs>i SHOULD just deprecate unshift()
18:41:41  <dominictarr>isaacs: +1
18:41:41  <isaacs>and we figure out a better way to do this.
18:42:05  <isaacs>but, without rewriting the http parser, i don't see how to do that
18:42:31  <isaacs>you won't know how much to pull off of the socket to get to the body of a req that's upgraded.
18:42:45  <dominictarr>what about you create a stream and initialize the buffer with some initial data?
18:43:12  <chrisdickinson>i've been just keeping a pool around
18:43:12  <Raynos>isaacs: cant the parser be a transform stream?
18:43:22  <chrisdickinson>but i'm also still in classic streams world on top of through
18:43:46  <isaacs>Raynos: moving http_parser.c to a transform stream is a great idea. but it's also a 6 month project.
18:43:56  <dominictarr>worth it
18:43:57  <chrisdickinson>this problem came up quite a bit while doing git-list-pack
18:44:01  <isaacs>and any effect can only be negative, so it'd be very difficult to do properly
18:44:09  <Raynos>sounds more like a one weekend userland module :D
18:44:14  <isaacs>Raynos: a slow one.
18:44:21  <Raynos>oh :(
18:44:26  <isaacs>Raynos: good luck with that :)
18:44:30  <isaacs>http is hard to get right.
18:44:31  <chrisdickinson>since i have to pool bytes while inflating because node's zlib doesn't give an indication of where the deflated data ends and the next object header begins
18:44:32  <isaacs>surprisingly so.
18:44:41  <Raynos>is it specced well?
18:44:49  <isaacs>Raynos: not really, no.
18:44:54  <isaacs>it's completely fucking bonkers.
18:45:10  <isaacs>if you think you can do it in a week, it's because you don't understand it that well. dig in deep. the rabbit hole goes all teh way down.
18:45:57  <isaacs>what i could do is create a Passthrough, push() the first chunk into it, and hten pipe() the socket into it
18:46:06  <isaacs>but now you don't have the "real" socket, you have this fakey thing
18:46:46  <dominictarr>whats flakey about that?
18:47:45  <dominictarr>just reemit error and close on it
18:49:01  <isaacs>well, the handle object isn't there, you can't setKeepAlive() on it, etc.
18:49:41  <isaacs>no, you need the real socket.
18:49:54  <dominictarr>in userland we all do stuff like this occasionally
18:50:01  <isaacs>yeah, but this is core :)
18:50:02  <isaacs>it's different
18:50:55  <dominictarr>so, just rename unshift to _unshift ?
18:51:17  <isaacs>the problem is that C *should* be the one to unshift!
18:51:20  <isaacs>and A *shouldn't*
18:51:32  * Guest95467changed nick to ITpro
18:51:33  <isaacs>so, i'm thinking, maybe unshift() shouldn't unset the reading flat
18:52:14  <dominictarr>not if it's it's already reading
18:52:43  <isaacs>right
18:52:49  <isaacs>push() unsets the "i am currently reading" flag
18:52:54  <isaacs>so _read can be called again
18:57:08  * st_lukejoined
19:19:13  <Raynos>isaacs: why is this so hard?
19:19:29  <Raynos>isaacs: would `peek()` solve this better then `unshift()` ?
19:19:45  * st_lukequit (Remote host closed the connection)
19:20:17  <isaacs>Raynos: not really
19:20:21  <isaacs>i mean, a little
19:20:40  <isaacs>but you want to be able to be kind of optimistic in what you pull off the wire
19:20:49  <isaacs>read() first, ask questions later, that kind of thing
19:20:53  <isaacs>since most of the time, that'll be faster.
19:21:42  <rowbit>Hourly usage stats: [developer: 4, free: 38]
19:25:52  * spionquit (Ping timeout: 246 seconds)
19:28:13  * stagasquit (Read error: Connection reset by peer)
19:38:55  * st_lukejoined
19:44:17  * jcrugzzquit (Ping timeout: 252 seconds)
19:49:11  * jcrugzzjoined
19:58:05  <defunctzombie>st_luke: find me once things have settled down with the bitcoin shits and I can hook you up
19:58:18  * jcrugzz_joined
19:58:33  <st_luke>defunctzombie: oh i gotta fund my bitfloor account also
19:58:54  <defunctzombie>right now it is very chaotic
20:02:00  * jcrugzzquit (Ping timeout: 264 seconds)
20:03:34  * jcrugzz_changed nick to jcrugzz
20:05:16  <Domeni___>isaacs: why does npm init insert "gitHead" into package.json?
20:05:33  <isaacs>Domeni___: probably just because readPackageJson does that
20:05:43  <isaacs>Domeni___: and init doesn't un-do it.
20:05:52  <isaacs>Domeni___: gitHead is important for other reasons.
20:06:22  <Domeni___>hmm
20:06:42  <isaacs>but init should probably remove it.
20:07:29  <Raynos>Domeni___: you need an irc bot thing
20:07:43  * ITproquit
20:07:45  <Domeni___>Raynos: to fix my name problems?
20:07:51  <Raynos>yes
20:08:18  * ITprojoined
20:09:23  <Domeni___>yeah
20:13:38  <Raynos>isaacs: should the new http parser be written in JS ?
20:17:54  * st_lukequit (Remote host closed the connection)
20:20:40  <Raynos>Domeni___: Is it bad to have a `then` property of a javascript object when its not a promise
20:21:00  <Raynos>has the JavaScript promise community come far enough to be able to say "fuck you. the word then belongs to us. your being a dick"
20:21:42  <rowbit>Hourly usage stats: [developer: 4, free: 40]
20:22:57  <Domeni___>Raynos: only if you want to use that object as the return value inside a promise callback.
20:23:20  <Raynos>blargh this is a similar problem
20:23:22  <Domeni___>if your system does not have promises, or does not use the other thenables inside a promise context, you're fine
20:23:23  <Raynos>that streams2 has
20:23:36  <Raynos>"your json stream cant contain null & undefined. Those are special symboles meaning END OF STREAM"
20:25:29  <Domeni___>isn't that only applicable to binary streams?
20:25:34  <Domeni___>i.e. object streams can contain them?
20:25:42  <Raynos>no
20:25:49  <Raynos>you cant put null & undefined in object streams
20:25:50  <Domeni___>oh, wow
20:26:02  <Domeni___>that's amazingly horrible
20:26:08  <Raynos>just like
20:26:12  <Raynos>not having an object with a then property
20:26:14  <Raynos>in a promise
20:26:15  <Domeni___>but then again object streams are a strange hack anyway
20:26:17  <Raynos>is amazingly horrible
20:26:33  <Raynos>its a trade off
20:26:46  <Raynos>for streams its "why would you want null as a chunk in a stream. push(null) is convenient"
20:27:11  <Raynos>for promises its "Having a thenable being unpacked properly is useful for chaining. Why would you want a value with a then property"
20:27:27  * ins0mniajoined
20:27:41  <Domeni___>in both cases, the answer is you need a wrapper if you want arbitrary data.
20:28:00  <Raynos>correct
20:28:12  <isaacs>Raynos, Domeni___: how would one signal the end, then?
20:28:17  <Raynos>however I actually think when building new abstractions this problem should die
20:28:39  <Domeni___>isaacs: .signalEnd() doesn't work?
20:28:42  <Raynos>isaacs: `read(function (err, value) { })`
20:28:53  <isaacs>Raynos: that doesn't work for the many reasons i've discussed with you in the past.
20:29:02  <Raynos>if (err === true) { /* END */ } else if (err) { /* OOPS */ } else { /* CHUNK VALUE */ }
20:29:14  <isaacs>Raynos: that works only for files, and some synthetic stream.s
20:29:29  <isaacs>Raynos: it doens't work for crypto, tcp, tls, tty, zlib, or basically any other stream ever.
20:29:47  <Raynos>why not
20:29:53  <Raynos>just do cb(null, "")
20:30:03  <isaacs>because you're not guaranteed to have a single piece of data from a single attempted read.
20:30:04  <Raynos>https://gist.github.com/Raynos/6b3b2b99aa0972e58b71#file-stream-thing-js
20:30:35  <isaacs>Raynos: what happens when you get chunks at arbitrary times, sometimes 2 from a single _read() call?
20:30:46  <isaacs>now it's on the implementor to juggle a callback, and make sure they call it exactly once.
20:30:46  <Raynos>buffer them
20:30:51  <isaacs>yeah, no.
20:31:01  <isaacs>that's why streams1 doesn't work.
20:31:15  <Raynos>anyway this is a completely disjoint problem
20:31:18  <isaacs>saying "buffer them" = saying "they're your bugs"
20:31:20  <isaacs>right
20:31:24  <Raynos>the answer to how do you signal end without null
20:31:25  <isaacs>read(cb) = not gonna happen, sorry.
20:31:28  <isaacs>right
20:31:30  <Raynos>is read returns a tuple [token, chunk]
20:31:43  <isaacs>we could add a setEndSigil(blerg)
20:31:44  <Domeni___>that's actually how ES6 iterators work now
20:31:44  <Raynos>thats the only way to do arbitary chunks in a stream
20:31:46  <Domeni___>no more StopIteration
20:31:59  <isaacs>Domeni___: oh, really? throw SI is dead?
20:32:04  <isaacs>Domeni___: that's GREAT news.
20:32:07  <Domeni___>isaacs: yeah :)
20:32:19  <Raynos>isaacs: no, saying buffer them is everyone uses buffer(stream) -> Stream function
20:33:01  <isaacs>every time i heard someone talk about how it's not a big deal it sounded like, "Look, guns are only a problem if you point them at people. we need to turn off lights from across the room, so you use the LightSwitchGun, and point it directly at the switch, and shoot very gently, and it Just Works. I don't see what the problem is."
20:33:28  <Raynos>anyway
20:33:37  <Domeni___>i <3 analogies ^_^
20:33:39  <Raynos>I like the idea mine & dominictarr's pull-streams
20:33:54  <isaacs>ok, Raynos, yes, of course, you can hack around any interface in userland, because it's software.
20:33:56  <Raynos>the only thing left to do is rewrite node core to use them and proof there are no problems
20:34:09  <Raynos>then the only remaining problem is back compat
20:34:25  <isaacs>pretend for a moment that the node core stream interface has to be the lowest common denominator, and also back compat, and also efficient and fast, and also work for all the streams in node-core.
20:34:33  <isaacs>because that's the constraints here.
20:34:41  <Raynos>yes
20:34:49  <isaacs>read(cb) = no, sorry.
20:34:53  <isaacs>not a realistic solution.
20:34:58  <Raynos>ignoring back compat I want to believe read(cb) can do it
20:35:23  <isaacs>again, what happens when you have a stream with assymmetric data?
20:35:33  <rowbit>/!\ ATTENTION: (default-local) thelevybreaks@... successfully signed up for developer browserling plan ($20). Cash money! /!\
20:35:33  <rowbit>/!\ ATTENTION: (default-local) paid account successfully upgraded /!\
20:35:35  <Raynos>what happens with read() right now ?
20:35:43  <Domeni___>i kind of remember reading somewhere, but why does .read() even exist? it seems the only sensible way to consume a readable stream is to build a writable stream.
20:35:58  <isaacs>Domeni___: it's perfectly sensible to use 'data' events, or call read() directly.
20:36:03  <isaacs>Domeni___: i do that all the time. it's fine.
20:36:17  <Domeni___>isaacs: but calling .read() gives you an unpredictable amount of data. when is that useful?
20:36:32  <isaacs>Domeni___: read(10) gives you 10 bytes, or the last <=10 bytes.
20:36:36  <isaacs>it's not that unpredictable.
20:36:42  <Raynos>consider https://gist.github.com/Raynos/3c8812c0262b5ee0493b
20:37:08  <Domeni___>that's pretty unpredictable. anywhere from 0-10 bytes? what kind of app logic can use that? i'm genuinely curious.
20:37:10  <isaacs>Raynos: ee leak
20:37:12  <CoverSlide>read(Infinity)
20:37:22  <Raynos>isaacs: fixed.
20:37:24  <isaacs>Raynos: s.onCE('readable', read)
20:37:25  <isaacs>yep
20:37:40  <Raynos>i actually want to understand the problem with that function
20:37:44  <Raynos>other then performance & back compat
20:37:59  <Raynos>performance penalty is a few extra function calls
20:38:00  <isaacs>Raynos: other than ${the two things i care most about}
20:38:09  <isaacs>Raynos: the perf penalty is EE.once()
20:38:22  <isaacs>Raynos: that's why we juggle so many flags in streams2
20:38:31  <isaacs>because once is prohibitive for main-flow code.
20:38:43  <Raynos>isaacs: the claim is "we can remove read() & "readable" event in favor of read(cb) and increase performance"
20:38:55  <Raynos>but that's an empty claim.
20:39:07  <isaacs>it is
20:39:20  <isaacs>Raynos: also, how do you read not-all-of-it
20:39:24  <Raynos>what I'm really asking is a callback based pull stream a bad abstraction for crypto, tls & zllib ?
20:39:33  <isaacs>Raynos: var ninteenBytes = read(19)
20:39:43  <Raynos>isaacs: out of scope
20:39:51  <Raynos>the parser use case is way too small imho
20:39:51  <isaacs>Raynos: haha, ok
20:40:04  <Domeni___>parsers can't use .read(19) can they?
20:40:08  <Domeni___>because that's 0-19 bytes
20:40:12  <Raynos>var parser = ParseReader(stream); parser.read(19)
20:40:12  <Domeni___>i'm still confused about this
20:40:20  <CoverSlide>have your parser check the length afterwards
20:40:27  <Domeni___>sure then what?
20:40:32  <Raynos>Domeni___: Its 19 bytes unless its n bytes followed by EOS
20:40:40  <CoverSlide>do nothing until the next chunk comes in
20:40:44  <isaacs>Domeni___: if the stream is ended, you get whatever's left.
20:40:57  <isaacs>it's the same as how read(2) works if you give it a length requremnet
20:41:00  <Domeni___>Raynos: isaacs: oh. i thought it was like 5 bytes if only 5 bytes have arrived on the client so far.
20:41:14  <Domeni___>(e.g. for an incoming http response stream)
20:41:36  <isaacs>Raynos: so, with this read(cb) model, we just wait around for there to be more data, before calling the callback
20:41:44  <pkrumins>guys, what's the difference between 'close' and 'exit' for require('child_process').spawn'ed processes? (talking about node 0.8 as that's what we use in production). I'm seeing problems that sometimes 'exit' wont fire and 'close' will, but sometimes it's the other way around, 'exit' will fire, but 'close' won't. has anyone seen anything like this?
20:42:03  <pkrumins>so what i'm trying right now is: ps.on('exit', f); ps.on('close', f); var psFired = 0; function f () { if (!psFired) { ...; psFired = 1 } }
20:42:07  <isaacs>pkrumins: 'exit' = we have an exit code. 'close' = we have an exit code, and the stdio fd's are closed.
20:42:27  <isaacs>pkrumins: if you only listen on 'exit', you might miss some stdout data
20:42:28  <pkrumins>isaacs: does that mean 'exit' should always happen?
20:42:32  <isaacs>pkrumins: yes.
20:42:37  <Raynos>isaacs: yes. so read(cb) should be trivial to implement for pull sources like file descriptors
20:42:44  <pkrumins>so weird because i'm not seeing 'exit'! but i'm seeing 'close'
20:42:48  <pkrumins>(or the other way around)
20:42:53  <isaacs>'close' will generally follow it closely, *unless* the stdio is shared.
20:42:54  <CoverSlide>bug report!
20:42:54  <Raynos>its harder to implement for push sources like sockets but that's already hard given that streams2 is pull
20:43:08  <isaacs>Raynos: it's actualy really easy to use the current interfaces for sockets.
20:43:14  <isaacs>and other pushy things
20:43:24  <isaacs>though, rather than "pushy", i should say, "assymmetric"
20:43:32  <pkrumins>isaacs: do you think the workaround that i'm trying is reasonable?
20:43:39  <isaacs>ie, the availability of data is not strictly tied to the calls to _read()
20:43:39  <pkrumins>until i figure out why i never see 'exit'
20:43:48  <isaacs>pkrumins: i dunno, seems fine, i guess.
20:43:50  <Raynos>isaacs: thats because you have a 700 line abstraction that makes the problem dissappear
20:44:03  <pkrumins>isaacs: yes, thanks. trying that. :)
20:44:11  <Raynos>isaacs: can you explain more with what you mean with assymmetric
20:44:35  <isaacs>pkrumins: you can also do var f = once(fucntion() { do stuff }); ps.on('close', f).on('end', f)
20:44:44  <isaacs>pkrumins: npm install once
20:44:53  <isaacs>Raynos: i mean, data may come at arbitrary times.
20:45:11  <pkrumins>isaacs: oh cool!
20:45:19  <isaacs>Raynos: so, i end up needing the same exact machinery as i have now, with push() that returns a boolean, and a _read() function to open teh gates again.
20:45:56  <isaacs>Raynos: so, basically, read(cb) is strictly an increase in complexity
20:46:19  <isaacs>Raynos: also, i still have to make sure i don't have parallel _read()s happening, because threads and files.
20:46:47  <isaacs>Raynos: and, i still dont' see what this has to do with passing null as the EOF chunk.
20:46:59  <Raynos>well
20:47:06  <Raynos>either read(function (token, chunk) { })
20:47:10  <Raynos>or [token, chunk] = read()
20:47:15  <Raynos>solves the EOF chunk problem
20:47:17  <isaacs>Raynos: the simpler thing is to add an eofSigil:<value> option to the ReadableState, and have it default to null, and push(thatThing) is EOF
20:47:24  <isaacs>Raynos: then you can set it to your own object or whatever.
20:47:32  <Raynos>that solves the problem too :P
20:47:38  <isaacs>and tha'ts WAY simpler.
20:47:40  <isaacs>than rad(cb)
20:47:44  <dominictarr>or have another function this.pushEnd()
20:47:45  <isaacs>or read() that returns not-data
20:48:04  <dominictarr>isaacs: Raynos use NaN for end!
20:48:09  <Raynos>:D
20:48:18  <isaacs>dominictarr: that's kind of a dick move ;)
20:48:20  <Raynos>if (chunk === NaN) { /* end */ }
20:48:26  <isaacs>Raynos: SURPRISE!
20:48:29  <Raynos>:D
20:48:36  <isaacs>.. NaN === NaN
20:48:39  <isaacs>purr: ?
20:48:44  <Raynos>issue: "WHY MY STREAM NEVER END" ?!?!?!
20:48:47  <dominictarr>trololololol
20:49:06  <dominictarr>Schrodingers Stream
20:49:42  <isaacs>dominictarr: we contemplated a stream.eof() that would basically push(null)
20:49:54  <isaacs>maybe that's a better way to do it, but at least for non-objectMode streams, push(null) must be kept forever.
20:50:27  <jesusabdullah>tanepiper: Tell me about 15 stone babbies
20:50:29  <thl0>funny had that issue yesterday
20:50:31  <thl0>NaN
20:50:36  <jesusabdullah>tanepiper: a stone is like 15, 20 lbs no?
20:50:45  <dominictarr>isaacs: 0.10 has only just come out, is it really a legacy problem?
20:51:07  <dominictarr>Raynos: just box your nulls
20:51:17  <dominictarr>push(new Null())
20:51:17  <jesusabdullah>I'm gonna make my OWN stream api! With blackjack!
20:51:26  <isaacs>dominictarr: yeah, i mean, you can easily abstract around it.
20:51:28  <jesusabdullah>and then bitch that nobody will use my stream api
20:51:37  <dominictarr>juliangruber: welcome to the club!
20:51:59  <Raynos>this isnt a problem i actualy have :D
20:52:11  <isaacs>ok, coffee is wearing off. food tiems.
20:52:28  <Raynos>isaacs: is there a good documentation somewhere for a "raw pushy stream" API ?
20:52:54  <isaacs>Raynos: I think the docs for Readable.push() has an example or something
20:52:58  <dominictarr>pushy? like classic-streams?
20:52:58  <Raynos>like if I wanted to make read(cb) work with an assymettric stream what would such a thing look like
20:53:11  <Raynos>i mean more what's the raw socket look like
20:53:19  <Raynos>readStart() and .onValue or something
20:53:44  <Raynos>i guess the source will do
20:53:53  <isaacs>yeah, read the JS docs.
20:54:06  <isaacs>they're written in JavaScript, the universal language of JavaScript
20:55:26  <jesusabdullah>lol
20:57:42  <Raynos>:D
20:57:53  * ins0mniaquit (Remote host closed the connection)
20:58:11  <jesusabdullah>I wonder if the ghettomart has AA batteries.....
21:00:18  <AvianFlu>YOU'RE a AA battery!
21:03:25  <Raynos>dominictarr: how do pull streams deal with errors. Buffering errors feels wrong
21:04:07  <dominictarr>you get the error on the next read
21:04:29  <dominictarr>but… a sink would call read(err) immediately
21:04:32  <Raynos>that makes me sad :(
21:04:49  <Raynos>what about take(5)
21:05:02  <dominictarr>what about it?
21:05:04  <Raynos>i take 5 items and then theres a buffered error sitting around
21:05:14  <Raynos>because the error happened between item 5 & 6
21:05:51  <dominictarr>you are talking in abstract. errors are concrete.
21:06:09  <Raynos>i have a tcp socket
21:06:11  <dominictarr>platonic streams never error
21:06:14  <Raynos>and wrap it in a pull stream abstraction
21:06:24  <Raynos>now im a slow and lazy reading
21:06:29  <Raynos>so i just pull a chunk out once per second
21:06:32  <Raynos>or maybe
21:06:37  <Raynos>once per whenever I feel like it
21:06:59  <Raynos>the push stream emits an error like it would when im not reading from it
21:07:06  <Raynos>The basic problem is
21:07:12  <dominictarr>if you get an error, respond with that in the next read…
21:07:29  <Raynos>dominictarr: but the next read can happy in an arbitrary time in the future
21:07:38  <dominictarr>you won't have much in the buffer anyway, because you will only pull it in when someone calls read
21:07:56  <Raynos>the problem is I dont like a pull mechanism of reading errors
21:08:05  <Raynos>because it can lead to boxed errors that are never read or looked at
21:08:24  <dominictarr>well… give your stream a special onError handler
21:08:54  <dominictarr>net.connect(port, host, onError) -> PullSourceStream
21:09:42  <dominictarr>anything else is horribly complex, or extends the api in an ugly way
21:10:39  <Raynos>basically waht I want
21:10:43  <Raynos>is for paused things to never error
21:11:58  * dguttmanquit (Quit: dguttman)
21:11:59  * st_lukejoined
21:13:25  * dguttmanjoined
21:14:15  <dominictarr>what? but that is the opposite of what you just said
21:14:51  <dominictarr>getting the error in the next read is the same as paused things to not error
21:16:15  <Raynos>ok
21:16:20  <Raynos>what I want is to be able to pause a thing
21:16:23  <Raynos>and never resume it
21:16:26  * st_lukequit (Remote host closed the connection)
21:16:26  <Raynos>and not miss an error
21:16:52  * nicholasfjoined
21:21:33  <dominictarr>so, pass in an extra error handler to the thing that errors
21:21:42  <rowbit>Hourly usage stats: [developer: 5, free: 15]
21:22:28  <defunctzombie>well, now I know what it is like to get DDoS'd
21:32:41  <dominictarr>how did node handle it?
22:02:15  * ITproquit (Read error: Connection reset by peer)
22:03:04  * ITprojoined
22:05:18  * ralphthe1injajoined
22:05:33  * ralphthe1injaquit (Client Quit)
22:05:45  * ralphthe1injajoined
22:06:23  * ralphthe1injaquit (Client Quit)
22:06:31  * ralphthe1injajoined
22:06:41  * ralphtheninjaquit (Ping timeout: 245 seconds)
22:07:12  * ralphthe1injachanged nick to ralphtheninja
22:19:15  * thl0quit (Remote host closed the connection)
22:21:42  <rowbit>Hourly usage stats: [developer: 0, free: 31]
22:24:32  <isaacs>defunctzombie: orly?
22:24:37  <isaacs>how'd your app handle it?
22:27:30  <Domeni___>hmm streams piped to each other don't propagate errors, interesting
22:27:40  <isaacs>Domeni___: correct.
22:27:45  <isaacs>stream.pipe(stream)
22:27:58  <isaacs>^ where to propagate?
22:28:04  <Domeni___>heh
22:28:19  <Domeni___>i mean it doesn't need to propagate anywhere then, stream just emits the error
22:28:25  <Domeni___>i was thinking... tentatively... more like:
22:28:43  <Domeni___>readable.pipe(writable): if readable emits 'error' then writable probably should too?
22:28:44  <isaacs>if you say, "only propagate to a stream that it's not already been to" then that means that you still need a handler on all streams in a cycle, but now the one that catches the error ISNT the one that raised it!
22:29:05  <isaacs>Domeni___: but there are streams in this world that are readable and also writable nad pipe to themselves
22:29:22  <Domeni___>yeah piping to yourself, or cycles generally, confuses me.
22:29:22  <isaacs>Domeni___: pipe != then ;)
22:29:34  <isaacs>Domeni___: how else would you write an echo server?
22:29:42  <isaacs>or a "gzip as a service" server?
22:29:46  <Domeni___>fair
22:30:26  <isaacs>not propagating errors is kind of annoying. but *sometimes* propagating errors is like a hundred times worse, and propagating errors in infinite loops is a million times worse.
22:30:44  <isaacs>we do stop the pipe if either one has an error, thpugh
22:30:52  <Domeni___>right yeah, saw that
22:32:17  * thatguydanjoined
22:32:17  <defunctzombie>isaacs: not well
22:32:34  <defunctzombie>isaacs: too much network traffic
22:32:39  <isaacs>ah, i see
22:34:45  <isaacs>dominictarr, Raynos: regarding our conversation earlier about unshift(): https://github.com/joyent/node/pull/5280
22:36:01  * nicholasfquit (Remote host closed the connection)
22:36:35  * nicholasfjoined
22:37:52  <dominictarr>Domeni___: in my pull-stream experiment error is propagated the same way that close is
22:38:08  <dominictarr>(and stream.pipe(stream) is forbidden)
22:38:14  <Domeni___>dominictarr: interesting....
22:38:28  <isaacs>dominictarr: what about stream.pipe(transformer).pipe(stream)?
22:38:37  <dominictarr>also, close "abort" is propagated back to the source
22:38:47  <dominictarr>isaacs: that is handled differently
22:38:59  <isaacs>dominictarr: orly? but errors get propagated you said...
22:39:09  <dominictarr>a duplex stream like that is handled as a source and a sink
22:39:25  <isaacs>dominictarr: so, you do s.pipe(t).pipe(s)
22:39:26  * Dr_Kquit (Quit: Leaving.)
22:39:36  <isaacs>s.emit('error')
22:39:38  <isaacs>propagated to t?
22:39:43  <dominictarr>like: stdin.pipe(transform).pipe(stdout)
22:39:52  <isaacs>no, that's different, because stdin !== stdout
22:40:01  <isaacs>(usually ;)
22:40:09  <dominictarr>well, what about req, and res?
22:40:16  <isaacs>what about sock and sock?
22:40:35  <isaacs>net.createServer(function(sock) { sock.pipe(gzip.Gzip()).pipe(sock) })
22:40:45  <isaacs>now sock.emit('error')
22:41:10  <dominictarr>okay, so - it's impossible to do s.pipe(s) because s is not an event emitter
22:41:15  * nicholasfquit (Ping timeout: 276 seconds)
22:41:22  <isaacs>?
22:41:27  <isaacs>can you pipe anything into s?
22:41:31  <dominictarr>instead pull-streams are functions that take a read function and return a read function
22:41:34  <isaacs>i'm so confused about what youer' doing.
22:41:55  <isaacs>so they're like foreachables?
22:42:14  <dominictarr>function (read) { return function (abort, cb) { read(abort, cb) }} // that is the most basic transform pull-stream
22:43:17  <isaacs>and abort and cb are both functions?
22:43:24  <dominictarr>you have a reader function, which calls a read function repeatedly, until it calls back(true) or (err)
22:43:31  <dominictarr>abort is just a value
22:43:41  <isaacs>oh, what do you do with abort?
22:44:07  <dominictarr>if (!!abort) then the source drops everything and cleans up
22:44:47  <dominictarr>and callbacks(abort || err)
22:44:49  <isaacs>oh, ok
22:45:00  <isaacs>it's an argument
22:45:04  <dominictarr>yes
22:45:17  <isaacs>"i am teling you to abort now" not "use this to tell me when you have aborted"
22:45:18  <isaacs>got it
22:45:26  <dominictarr>yes
22:45:43  <isaacs>tht's great for streams that can be stateless.
22:45:59  <isaacs>you're still going to have an object in that closure a lot of the time, though
22:46:13  <dominictarr>yeah, this is for object streams
22:46:51  <dominictarr>you think it would be more performant with a prototype?
22:47:07  <Raynos>everything is more performant with a prototype
22:47:08  * thatguydanquit (Quit: thatguydan)
22:47:22  <Domeni___>^ sad but true
22:47:26  <isaacs>dominictarr: JS shines when you don't make it something other than JS
22:47:39  <isaacs>use new. use prototypes. use constructors. pretend it's java
22:47:59  <dominictarr>isaacs: how much difference does it make?
22:48:12  <isaacs>dominictarr: depends how much of your program's time is spent in JS
22:48:23  <isaacs>dominictarr: but like, up to 80% of whatever that amount of time is :)
22:48:30  <dominictarr>vs v8?
22:48:41  <isaacs>dominictarr: vs waiting on IO
22:48:49  <dominictarr>oh, right
22:48:59  <isaacs>it's a significant difference.
22:49:21  <isaacs>making the streams code more optimization friendly bought back a lot of perf losses in streams2
22:49:38  <isaacs>trevnorris's changes to the nextTick and makecallback stuff was even more beneficial
22:52:54  <niftylettuce>https://getprove.com
22:53:01  <niftylettuce>and if you guys want special promo https://getprove.com/hn
22:53:04  <niftylettuce>going on HN tomorrow
22:54:38  <niftylettuce> /cc Nexxy jesusabdullah
22:55:29  * nicholasfjoined
22:55:45  <Nexxy>niftylettuce, sweet deal
22:55:50  <Nexxy>niftylettuce, twilio?
22:56:34  <isaacs>niftylettuce: i just signed up for this thing! i don't know what it is!
22:56:43  <niftylettuce>isaacs: it's awesome sauce
22:56:53  <isaacs>ooh... neat.
22:56:58  <dominictarr>surely, a scope could be optimized the same way as objects
22:57:00  <niftylettuce>isaacs: try a call out, a friendly computer lady wants to give you a pin code
22:57:19  <isaacs>dominictarr: functions can be inlined in many cases, which is sort of what you're suggesting.
22:57:28  <isaacs>dominictarr: but it's not "the same way", in any relevant sense.
22:57:45  <isaacs>dominictarr: functions that nest a bunch of closures and other fancy crap generally are harder to optimize.
22:57:48  <dominictarr>no, I mean that a scope stack seems very similar to a prototype chain
22:58:07  <dominictarr>why couldn't you have a hidden class for a scope, too?
22:58:46  <isaacs>dominictarr: take it up with V8
22:59:07  <isaacs>dominictarr: i'm just telling you what i've learned from banging my head against this wall a lot with node core :)
22:59:20  <isaacs>dominictarr: "could" is academic, i'm talking about what it actually does.
22:59:29  <dominictarr>true
22:59:36  <dominictarr>I'll try some benchmarks
22:59:41  <niftylettuce>isaacs: phone verifiction api btw is what it is
22:59:48  <isaacs>niftylettuce: ah, neato cheato
22:59:51  <dominictarr>I don't want to be a compiler guy
23:01:07  <isaacs>dominictarr: me neither :)
23:01:22  <isaacs>niftylettuce: this way you can make sure that your users all have cell phones ;P
23:01:29  <niftylettuce>isaacs: yea
23:01:48  <niftylettuce>isaacs: it was a fun 24 hour hack
23:02:01  <isaacs>seems like it :)
23:03:25  <niftylettuce>isaacs: here's node module https://github.com/getprove/prove-node#prove-node-001
23:03:32  <niftylettuce>2 api calls and your done
23:03:41  <isaacs>that's nice
23:08:06  * nicholasfquit (Read error: Connection reset by peer)
23:08:30  * nicholasfjoined
23:08:46  * nicholasfquit (Read error: Connection reset by peer)
23:09:09  * nicholasfjoined
23:18:06  * thatguydanjoined
23:19:52  * nicholasfquit (Read error: Connection reset by peer)
23:20:18  * nicholasfjoined
23:20:48  <jez0990>mikolalysenko: did you decide to start on that Red Black tree module? would AA trees work for your use case?
23:21:42  <rowbit>Hourly usage stats: [developer: 0, free: 23]
23:23:19  * Domeni___quit (Remote host closed the connection)
23:24:13  * sveisveiquit (Ping timeout: 246 seconds)
23:24:13  * ehdquit (Ping timeout: 246 seconds)
23:24:34  * Domenic_quit (Ping timeout: 246 seconds)
23:24:35  * ryanseddonquit (Ping timeout: 246 seconds)
23:26:17  * stlsaintjoined
23:29:34  * ITproquit (Ping timeout: 245 seconds)
23:29:41  * thl0joined
23:30:16  * ITprojoined
23:30:37  <mikolalysenko>jez0990: I am working toward it
23:30:57  <mikolalysenko>but pretty much any balanced binary search tree would work, doesn't have to be red-black
23:31:08  <mikolalysenko>I am thinking about implementing weight balanced trees since they are a bit simpler
23:33:57  * thl0quit (Ping timeout: 248 seconds)
23:35:22  * yorickquit (Remote host closed the connection)
23:52:13  * dominictarrquit (Quit: dominictarr)
23:53:04  * tmcwquit (Remote host closed the connection)
23:56:53  * Domenic__quit (Ping timeout: 240 seconds)