00:00:01  * ircretaryquit (Remote host closed the connection)
00:00:08  * ircretaryjoined
00:01:32  * AvianFluquit (Remote host closed the connection)
00:15:02  * loladirojoined
00:19:06  <trevnorris>isaacs: i'll be able to get ReadableStream back up to par.
00:19:24  * piscisaureus_quit (Quit: ~ Trillian Astra - www.trillian.im ~)
00:19:39  * paddybyersquit (Ping timeout: 240 seconds)
00:30:16  * qmxchanged nick to qmx|away
00:31:45  * c4miloquit (Remote host closed the connection)
00:45:09  * EhevuTovquit (Quit: This computer has gone to sleep)
01:03:29  * EhevuTovjoined
01:17:45  * c4milojoined
01:22:26  * trevnorrisquit (Quit: Leaving)
01:37:30  * c4miloquit (Remote host closed the connection)
01:38:19  * c4milojoined
01:40:06  * c4miloquit (Remote host closed the connection)
01:43:21  * abraxasjoined
01:50:26  * sblomquit (Ping timeout: 276 seconds)
02:03:40  * dapquit (Quit: Leaving.)
02:03:41  * pooyaquit (Quit: pooya)
02:05:04  * EhevuTovquit (Quit: This computer has gone to sleep)
02:08:53  * googoljoined
02:25:24  * lohkeyquit (Ping timeout: 244 seconds)
02:26:04  * aittjoined
02:26:17  <aitt>hi, anyone around?
02:27:40  * qmx|awaychanged nick to qmx
02:32:42  * pooyajoined
02:33:26  * karupaneruraquit (Ping timeout: 276 seconds)
02:37:30  * EhevuTovjoined
02:39:28  * karupanerurajoined
02:49:44  * sgallaghjoined
02:54:13  * sgallaghquit (Client Quit)
02:56:02  * sgallaghjoined
02:58:44  * jmar777quit (Remote host closed the connection)
03:01:51  * jmar777joined
03:03:26  * wolfeidaujoined
03:07:08  * sgallaghquit (Ping timeout: 256 seconds)
03:14:05  * TooTallNatequit (Quit: ["Textual IRC Client: www.textualapp.com"])
03:15:15  * wolfeidauquit (Remote host closed the connection)
03:26:33  * pooyaquit (Quit: pooya)
03:33:39  <txdv>aitt: es
03:33:45  <txdv>yes
04:04:49  * pooyajoined
04:08:47  * pooyaquit (Client Quit)
04:09:02  <aitt>sorry, i forgot about this window, anyway, i was going to ask a lame question and figured it out myself, thanks anyway =)
04:11:47  * qmxchanged nick to qmx|away
04:16:49  * loladiroquit (Quit: loladiro)
04:27:23  * AvianFlujoined
04:28:10  * brsonquit (Quit: leaving)
04:38:21  * c4milojoined
05:05:01  * wolfeidaujoined
05:05:50  * kazuponjoined
05:06:43  * kazuponquit (Remote host closed the connection)
05:06:53  * kazuponjoined
05:08:13  * c4miloquit (Remote host closed the connection)
05:09:44  * brsonjoined
05:11:51  <aitt>i was wondering, am i doing anything wrong if the only way i can get my program to compile is having the uv.h include before windows.h ?
05:12:02  <aitt>i see a lot of redefinition errors if i don't do it that way
05:12:10  <aitt>i'm using msvc btw
05:12:13  <aitt>2010 IIRC
05:23:13  * stagas_joined
05:25:45  * stagasquit (Ping timeout: 276 seconds)
05:25:52  * stagas_changed nick to stagas
05:29:39  * CoverSlidequit (Ping timeout: 240 seconds)
05:31:26  * CoverSlidejoined
05:46:32  * jmar777quit (Remote host closed the connection)
05:50:31  * wolfeidauquit (Remote host closed the connection)
06:03:26  * pooyajoined
06:36:23  * paddybyersjoined
07:20:39  * pooyaquit (Quit: pooya)
07:26:46  * mikealjoined
07:28:33  * kazuponquit (Remote host closed the connection)
07:33:45  * kazuponjoined
07:37:37  * `3rdEdenjoined
07:51:30  * aittquit (Quit: Page closed)
07:52:17  * rendarjoined
07:53:25  * brsonquit (Read error: Connection reset by peer)
08:20:31  * EhevuTovquit (Remote host closed the connection)
08:53:11  * hzjoined
08:54:41  * hzquit (Client Quit)
08:56:31  * hzjoined
09:03:06  * paddybyers_joined
09:06:11  * EhevuTovjoined
09:06:26  * hzquit (Disconnected by services)
09:06:28  * paddybyersquit (Ping timeout: 248 seconds)
09:06:28  * paddybyers_changed nick to paddybyers
09:06:30  * hzjoined
09:09:57  * hzquit (Disconnected by services)
09:10:01  * hzjoined
09:10:31  <indutny>morning
09:10:54  <rendar>hi
09:11:15  <indutny>isaacs: yt?
09:11:17  * hzquit (Read error: Connection reset by peer)
09:18:52  * hzjoined
09:23:44  * bnoordhuisjoined
09:32:40  * AvianFluquit (Remote host closed the connection)
09:34:11  * rje`macquit (Ping timeout: 245 seconds)
09:35:47  * rphillipsquit (Ping timeout: 260 seconds)
09:36:00  <indutny>bnoordhuis: morning
09:36:21  * rphillipsjoined
09:36:39  * rje`macjoined
09:38:23  <bnoordhuis>indutny: morning
09:41:11  <indutny>so, I'm seriously thinking about rewriting some parts of tls.js
09:41:23  <indutny>especially this big .cycle() thing
09:41:38  <indutny>it looks like it isn't needed anymore after streams2 came
09:41:59  <indutny>since all interactions are on-demand, and should be done explicitely by either user of encrypted side
09:42:03  <indutny>or user of clear side
09:42:18  <indutny>s/explicitely/explicitly/
09:42:25  <indutny>bnoordhuis: what do you think?
09:42:59  <bnoordhuis>do i need to have an opinion?
09:43:14  <indutny>not sure, but it would be good to hear one
09:43:20  <indutny>otherwise its my thoughts only
09:44:14  <bnoordhuis>it sounds reasonable
09:45:16  <indutny>haha
09:45:17  <indutny>thanks :)
09:46:39  * googolquit (Ping timeout: 240 seconds)
09:49:01  * EhevuTovquit (Quit: This computer has gone to sleep)
09:54:38  <indutny>bnoordhuis: btw, what are you working on?
09:56:22  <bnoordhuis>indutny: profiling tooling
09:56:34  <bnoordhuis>also some strongloop issues
09:58:01  <indutny>understood
09:59:47  * hzquit
10:10:18  * kazuponquit (Remote host closed the connection)
10:45:53  <bnoordhuis>you know that feeling when you write reasonably complex code and it works on the first try?
10:46:07  <indutny>yes
10:46:17  <indutny>that feeling when you think something is not right here
10:46:24  <bnoordhuis>exactly :)
10:46:26  <indutny>but I can't see what exactly
10:46:27  <indutny>:)
10:50:24  <indutny>As the wise contemporary philosophers the Wu-Tang Clan once said:
10:50:24  <indutny>"Cache Rules Everything Around Me ( C.R.E.A.M ) "
10:50:24  <indutny>Many people believe this is just typical rappers exhibiting the materialistic nature of modern life. However, they were in fact visionaries describing the importance of memory hierarchy utilisation on processor performance.
11:24:05  * mralephquit (Read error: Connection reset by peer)
11:24:11  * mraleph1joined
11:38:27  * felixgejoined
11:38:27  * felixgequit (Changing host)
11:38:27  * felixgejoined
11:42:53  <indutny>shit, streams2 are broken by definition
11:43:04  <indutny>:)
11:43:19  <indutny>many things are too complex for 3rd-party developers
11:43:32  <mmalecki>indutny: streams2 are too complex IMO
11:43:51  <indutny>isaacs: where art thou
11:43:57  <indutny>mmalecki: not really
11:43:58  <mmalecki>not only for 3rd-party, the implementation is complex
11:44:06  <indutny>idea is good
11:44:09  <indutny>implementation is not that good
11:44:20  <indutny>I think it should be the same as with BSD sockets
11:44:32  <indutny>and APIs seems to follow this
11:44:37  <indutny>but they don't in details
11:51:40  <indutny>so basically, I think both ._read and ._write should be able to return partial data
11:51:52  <indutny>return/or write
11:52:04  <indutny>otherwise it only complicates things
11:52:29  <indutny>for example, right now in ._read method I can't call cb(null, new Buffer(0)) because it'll be treated as EOF
11:53:00  <indutny>well, it's the same as read(2)
11:53:08  <indutny>hm... I need to think about it
12:22:44  * hzjoined
12:46:51  * jmar777joined
13:00:28  * jmar777quit (Read error: Connection reset by peer)
13:01:00  * jmar777joined
13:07:41  * qmx|awaychanged nick to qmx
13:21:02  * abraxasquit (Remote host closed the connection)
13:30:52  * sgallaghjoined
14:07:22  * c4milojoined
14:28:35  <MI6>joyent/libuv: Ben Noordhuis master * 7f3c783 : unix: don't clobber errno in signal handler - http://git.io/rCROmA
14:31:45  <indutny>isaacs: morning?
15:09:55  * bradleymeckjoined
15:38:57  * piscisaureus_joined
15:40:23  <isaacs>Good morning
15:40:27  <isaacs>call in 0:20
15:41:20  <isaacs>indutny: what would be the value of calling cb(null, new Buffer(0))?
15:51:57  <indutny>isaacs: I'll discuss it soon with you :)
15:52:07  <indutny>basically, I've a strong feel that streams2 are not ready for prime time
15:53:09  <indutny>isaacs: for example, http is expecting underlying socket to be push-stream
15:53:26  <indutny>while, by default, every stream is pull-stream
15:53:36  <indutny>and it doesn't play nicely together
15:53:53  <indutny>I just can't handle it the way I want in tls.js
15:54:12  <indutny>because http requires me to push data, but I want it to pull data
15:54:34  <indutny>isaacs: em... yt?
16:00:24  * TooTallNatejoined
16:02:09  <bnoordhuis>call?
16:02:12  * sblomjoined
16:02:45  <indutny>call?
16:02:57  <TooTallNate> /cc isaacs
16:05:44  <piscisaureus_>all?
16:05:52  <piscisaureus_>ISAAAAAAAAAAACS
16:05:53  <isaacs>call
16:05:54  <isaacs>one sec
16:05:55  <isaacs>sorry
16:06:06  <bnoordhuis>first time isaacs is late
16:06:12  <indutny>ohaha
16:07:17  * bradleymeckquit (Quit: bradleymeck)
16:10:32  * bradleymeckjoined
16:12:40  * `3rdEdenquit (Remote host closed the connection)
16:17:41  * qmxchanged nick to qmx|lunch
16:20:00  * dapjoined
16:21:47  <indutny>isaacs: so returning to our burritas
16:21:59  <indutny>isaacs: net.Socket and http should be really streams2
16:22:03  <indutny>not just emulation
16:22:20  <indutny>well former one looks like 100% compatible
16:22:28  <indutny>but http.Connection is a bit problematic
16:22:47  <indutny>and the thing is that you might think its used internally and wtf, why anyone needs it
16:22:59  <indutny>but I need it for at least three things: tls.js, spdy and tlsnappy:)
16:23:08  <indutny>the all heavily rely on how it works
16:23:16  <indutny>and before 0.9.x it was pretty stable
16:28:50  <indutny>isaacs: yt?
16:28:51  <indutny>:)
16:29:01  <piscisaureus_>sblom: hey
16:29:08  <piscisaureus_>sblom: ComponentGroupRef Id="Product.Generated" <-- do you know what that does?
16:32:35  <TooTallNate>indutny: i'd like to understand the problem as well
16:32:54  <TooTallNate>indutny: i *think* i'm still seeing some weirdness with net.Socket and streams2
16:33:03  <TooTallNate>but i don't have a definitive test case yet
16:33:07  <indutny>TooTallNate: interesting
16:33:16  <indutny>the problem is that http module is relying on .ondata property
16:33:29  <indutny>which don't really fit streams2 ideology
16:33:37  <indutny>because I want my stream to be pulled
16:33:43  <indutny>but there're noone to do this
16:33:46  * sblomquit (Quit: leaving)
16:34:42  <TooTallNate>indutny: are you depending on the http "createConnection" option?
16:34:59  <indutny>no, not really
16:35:18  <indutny>I'm porting tls.js CryptoStream on streams2
16:35:25  <piscisaureus_>sbom: nvm. However what bugs me is the ungodly mess with components and componentgroups in the wix project
16:35:44  <brucem>he left.
16:39:49  * AvianFlujoined
16:48:05  <piscisaureus_>:-(
16:48:06  <piscisaureus_>boo
16:48:33  <indutny>piscisaureus_: you <3 msoft? :)
16:53:10  <isaacs>indutny: hey
16:53:18  <indutny>ho
16:53:29  <indutny>isaacs: so like I said
16:53:37  <indutny>I think we should fix http before 0.10
16:53:43  <isaacs>right, so, http should not be flipping connection into a different mode.
16:53:45  <isaacs>agreed.
16:53:46  <isaacs>let's do it
16:53:50  <indutny>haha
16:53:53  <isaacs>http should be calling read() like a gentleman
16:53:56  <indutny>yes
16:54:09  <isaacs>indutny: the tricky thing there is the parser.
16:54:09  <indutny>if underlying stream doesn't support ondata
16:54:12  <indutny>or something like this
16:54:14  <isaacs>right
16:54:35  <indutny>surely, parser is pretty tricky
16:54:35  <isaacs>but we can have http IncomingMessage streams be something like a transform.
16:54:44  <isaacs>that was actually my plan for refactoring http in 0.12
16:54:45  <indutny>yes, that's how I imagine it
16:54:58  <indutny>nice.
16:55:04  <isaacs>so you actually can do socket.pipe(message)
16:55:18  <isaacs>then the message's _transform function writes it to the parser, and gets the body bits out
16:55:54  <indutny>isaacs: that's how I see tls.js http://twitpic.com/bzr5fx
16:56:02  * c4milo_joined
16:56:31  * AvianFlu_joined
16:56:50  * stagas_joined
16:57:14  <indutny>isaacs: and it fits streams2 model pretty well
16:57:15  <isaacs>indutny: so on the left is the socket, and on the right is http?
16:57:16  * AvianFluquit (Disconnected by services)
16:57:18  * AvianFlu_changed nick to AvianFlu
16:57:33  * c4miloquit (Read error: Connection reset by peer)
16:57:33  * stagasquit (Read error: Connection reset by peer)
16:57:35  <indutny>yes
16:57:45  * stagas_changed nick to stagas
16:57:49  <indutny>and right now it doesn't work properly
16:57:54  <indutny>not just because of http
16:58:01  <indutny>I'm still trying to figure out best way of doing it
16:58:22  <indutny>also, what should implementor do if it has received ._read() call, but it can't read any data right now
16:58:38  <indutny>i.e. it needs some information to come before it.
16:58:55  <indutny>I was trying to call cb(null, new Buffer(0)), but this emits EOF
16:58:58  <isaacs>indutny: so, you can think of _read as either "go get some data, and then call this callback"
16:59:06  <isaacs>indutny: or just a signal tat more data is requested.
16:59:13  <isaacs>indutny: and then call Stream.push(chunk) when you have it
16:59:19  <indutny>ok
16:59:20  <isaacs>indutny: ie, if _read is called, and you have no data, then just do nothing.
16:59:29  <isaacs>indutny: and call Stream.push(chunk) whenever you DO have some data
16:59:31  <indutny>so probably I made a right decision by implementing .read in tls.js
16:59:47  <isaacs>you should implement _read, not read
16:59:54  <indutny>isaacs: well, with CryptoStream I will not know when to call .push()
17:00:00  <indutny>because I want someone to pull data from me
17:00:08  <indutny>at least, in my implementation
17:00:22  <indutny>thus, I can emit 'readable' and expect someone to try reading from me
17:00:28  <indutny>even if I've 0 bytes
17:00:38  <isaacs>hmm..
17:00:46  <indutny>is this incorrect?
17:00:49  <isaacs>why don't you know when to call push()?
17:01:05  <indutny>because I've removed .cycle() function
17:01:10  <isaacs>indutny: when someone calls _read(), you check if there's data. if not, then do nothing. if so, then call push(chunk)
17:01:15  <indutny>and doing SSL_read/SSL_write explicitely
17:01:24  <isaacs>ok
17:01:40  <indutny>isaacs: I should invoke cb(), otherwise .read() won't call ._read()
17:01:42  <indutny>next time
17:01:50  <isaacs>the callback is optional, actually
17:02:00  <indutny>that's pretty complex
17:02:09  <isaacs>because we expect users to implement _read(), and we know that users suck at getting streams right, it's pretty flexible.
17:02:24  <isaacs>now, if you NEVER call the cb, and you NEVER call push(), then ok, you're not going to do much.
17:02:43  <indutny>well, hm...
17:02:43  <isaacs>but Readable.push() just calls the cb that it would call when data is available.
17:02:46  <isaacs>it's the same function, actually
17:03:00  <indutny>I believe something is good in this approach
17:03:04  <indutny>err
17:03:06  <indutny>s/good/bad/
17:03:08  <indutny>sorry :)
17:03:23  <isaacs>hah
17:03:25  <isaacs>> console.error(stream.Readable.prototype.push.toString())
17:03:26  <isaacs>function (chunk) {
17:03:26  <isaacs> var rs = this._readableState;
17:03:26  <isaacs> rs.onread(null, chunk);
17:03:45  <indutny>yes, but why .onread should be the same as ._read()'s cb?
17:03:54  <indutny>it might change somewhere between?
17:04:05  <indutny>not in node's core, but by some users
17:04:08  <isaacs>because when we call _read() in read(), we do this._read(size, state.onread)
17:04:20  <isaacs>we always pass the exact same callback to _read
17:04:22  <indutny>yeah, I know, but why can't anyone change state.onread?
17:04:32  <isaacs>well, you can, but GAH yo ushould not do that.
17:04:32  <indutny>isaacs: better not pass callback at all then
17:04:57  <indutny>this is inconsistent
17:05:11  <isaacs>indutny: in practice, in every stream we do, we always pass the same callback to our read functions. it's fine.
17:05:20  <isaacs>why would we create a new function every time that just does the same thing?
17:05:28  <isaacs>all it's doing is putting the data into the readable queue
17:05:43  <indutny>well, not calling callback is something wrong to me
17:06:00  <isaacs>stream.push() IS calling the cb
17:06:09  <indutny>it's still wrong
17:06:14  <indutny>because .push() doesn't accept cb as argument
17:06:20  <isaacs>push IS the callback
17:06:24  <indutny>oh...
17:06:27  <isaacs>(almost)
17:06:29  <indutny>why do we need cb at all?
17:06:30  <indutny>:)
17:06:40  <isaacs>because it's an API that people are familiar with? i don't know.
17:06:52  <isaacs>it makes it a bit nicer when you do fs.read(this.fd, cb)?
17:07:25  <isaacs>rather than fs.read(this.fd, function(er, data) { if (er) return this.emit('error', er); this.push(data) })
17:09:15  <indutny>ok
17:09:20  <indutny>let me wrap it up in my head
17:09:29  <indutny>I guess I misunderstanding something
17:09:45  <indutny>s/guess I/guess I am/
17:10:26  <isaacs>indutny: the idea is that you can use _read/push as a way to interact with push-style streams under the hood
17:10:47  <isaacs>indutny: _read->src.resume(); src.on('data') -> if (!push(data)) src.pause()
17:10:47  <indutny>ok, but there're still other type of streams
17:10:52  <isaacs>sure
17:11:04  <indutny>and lets suppose this streams can't push :)
17:11:11  <isaacs>right
17:11:13  <isaacs>so youer' pulling out of openssl
17:11:16  <indutny>I see streams2 API as sort of BSD sockets API
17:11:39  <indutny>tls' work is to notify user when he can pull data
17:12:07  <indutny>and user tries to read data
17:12:27  <indutny>look at read(2) call in man, it can return -1 and set errno to, lets say, EINTR
17:12:39  <indutny>err
17:12:42  <indutny>I mean EAGAIN
17:13:12  <indutny>my opinion, is that it'd be good to have same thing in streams2
17:13:29  <isaacs>yes
17:13:37  <isaacs>read() -> null, wait for 'readable;'
17:13:40  <isaacs>that's the exposed interface.
17:13:43  <indutny>yes, something like this
17:13:45  <isaacs>push/_read is just the machinery
17:13:49  <isaacs>that's how it already works
17:13:52  <indutny>huh?
17:14:04  <indutny>well, that's how it works for external users
17:14:09  <indutny>but what about stream internals
17:14:21  <isaacs>if you use push(), and you impelemtn _read(), then what you get *exposed* is that read() returns a buffer or null, and if it returns null, then you get a readable event
17:14:47  <isaacs>you can certainly implemnet stream.read instead of stream._read
17:14:58  <isaacs>but it's more work, and less dry
17:15:28  <indutny>well, I can ignore watermarks in it ;)
17:15:35  <isaacs>yes.
17:15:44  <indutny>ok, good
17:15:52  <isaacs>but ignoring water marks is not so nice
17:16:03  <indutny>so it seems for me that overriding .read() is the only way to make pull-stream
17:16:15  <isaacs>i don't see why
17:16:55  <indutny>because otherwise I would net to call ._read() myself and push data that was read
17:17:25  <isaacs>stream._read = function(n,cb) { var d = pullSomeData(); if (d) return cb(null, d); }
17:17:39  <indutny>that doesn't work
17:17:44  <indutny>suppose if there're no data
17:17:44  <isaacs>why not?
17:17:47  <isaacs>ok
17:17:49  <indutny>i.e. callback isn't called
17:17:52  <isaacs>how will you know when there is data?
17:17:53  <indutny>stream.reading = true
17:17:57  <indutny>s/stream/state/
17:18:02  <indutny>I will not know
17:18:09  <indutny>but I can emit 'readable'
17:18:10  <isaacs>how would you impleemnt read()?
17:18:22  <isaacs>i mean, how do you know when to emit 'readable'?
17:18:23  <indutny>see my image :)
17:18:33  <indutny>I'll emit it everytime I receive any data
17:18:35  <indutny>on both ends
17:19:39  <indutny>wait a minute, I'll show you a WIP
17:20:02  <isaacs>how do you receive data?
17:20:14  <isaacs>BIO_read is async? or synchronous?
17:21:03  <indutny>synchronous
17:21:05  <indutny>same is SSL_read
17:21:07  <indutny>and SSL_write
17:21:22  <indutny>isaacs: https://github.com/indutny/node/compare/feature-crypto-tls-streams2
17:21:57  <indutny>so right now I'm overriding .read =
17:22:52  <isaacs>ok, so, you just call BIO_read/SSL_read on read() call
17:23:00  <indutny>yes
17:23:09  <indutny>and call BIO_write/SSL_write on ._write() call
17:23:15  <isaacs>and then how do you know to emit 'readable'?
17:23:18  <indutny>but I'd like to call BIO_read/SSL_read in _read
17:23:31  <isaacs>so, you're emitting 'readable' when you might not actually be readable
17:23:45  <isaacs>that'll work, but it's not "correct"
17:23:46  <indutny>well, I'm always readable, but not always have data to read ;)
17:23:50  <indutny>why not?
17:23:50  <isaacs>right
17:23:59  <isaacs>'readable' should only be emitted when you have data to read
17:24:16  <indutny>hm...
17:24:18  <indutny>I see your point
17:24:24  <isaacs>you should be calling BIO_read in your _maybeReadable function
17:24:25  <indutny>so probably I need to cycle data manually
17:24:32  <isaacs>and then if there's data, self.push(chunk)
17:24:35  <indutny>yeah
17:24:43  <isaacs>so, your _maybeReadable effectivelyIS the cycle function
17:24:44  <indutny>but how would I stop?
17:25:03  <indutny>i.e. it looks like watermarks wouldn't really work well with it
17:25:07  <isaacs>well... actually... you call your _read instead of maybeReadable
17:25:20  <indutny>well, suppose I receive a lot of data
17:25:24  <indutny>from net.Socket
17:25:27  <isaacs>in yoru _read function, you check if youre below the highWaterMark
17:25:32  <isaacs>if you're below it, pull from ssl
17:25:45  <indutny>oh, so I should do it myself? :)
17:25:48  <isaacs>if you're not, then just don't do anything,a nd let hte data stay wherever it is, so that it exerts backpressure on whoever's putting it there
17:25:53  <isaacs>one second, gonna gist something
17:25:55  <indutny>I mean, I can do it
17:26:01  <indutny>and I understand internal logic
17:26:10  <indutny>I just want to do it like it's 3rd party module
17:26:22  <indutny>and avoid using internal stuff
17:26:28  <indutny>as much as possible
17:28:25  * qmx|lunchchanged nick to qmx
17:29:35  * mikealquit (Quit: Leaving.)
17:32:58  <isaacs>indutny: https://gist.github.com/4684623
17:33:16  <isaacs>indutny: yeah, i want that as well.
17:33:32  <indutny>isaacs: I see
17:33:35  <indutny>and it's understood
17:33:38  <isaacs>here's how i look at it: if you have to override the public read() or write() methods, and manage backpressure etc yourself, then the API is lacking.
17:33:44  <indutny>I'm just afraid that CryptoStream's case might be very common
17:33:52  <isaacs>of course, the API might be lacking, and maybe we can't finish it very easily, ok, whatever.
17:33:52  <indutny>isaacs: agreed
17:34:04  <isaacs>but we should be using the documented methods to build our stuff.
17:34:08  <isaacs>so, one thing i dont' like in my gist:
17:34:32  <isaacs>there should be a clear way to know whether or not you *should* get data, other than doing _read
17:34:42  <isaacs>so maybe the _write functions should just do read(0) to trigger a _read conditionally.
17:34:47  <isaacs>yeah, actually, that's the right answer.
17:34:49  <isaacs>updating gist.
17:35:20  <indutny>yes, that's what I'm doing now
17:35:26  <indutny>sort of
17:35:41  <isaacs>ok, updated gist: https://gist.github.com/4684623
17:36:08  <isaacs>we DO need a way to say, "I'm done reading, but I don't have data right now. ask again later."
17:36:13  <isaacs>maybe Buffer(0) could be that.
17:36:19  <indutny>yeees!
17:36:20  <isaacs>i don't know.
17:36:23  <indutny>that's what I'm talking about
17:36:25  <isaacs>yes.
17:36:26  <isaacs>i see this now.
17:36:27  <isaacs>thanks
17:36:31  <indutny>I'm not sure about way of doing it
17:36:34  <isaacs>right
17:36:36  <indutny>is Buffer(0) good fit for it
17:36:37  <mmalecki>creating an empty Buffer might be a significant overhead
17:36:39  <isaacs>without delving into private bs
17:36:44  <indutny>mmalecki: not really, it's cached now
17:36:48  <isaacs>mmalecki: an empty buffer is effectively free.
17:36:51  <mmalecki>oh. good.
17:36:54  <isaacs>mmalecki: it's up there with new Array(0)
17:37:10  <isaacs>indutny: i don't know.
17:37:22  <isaacs>indutny: there are cases like fs where "no data" is effectively treated like eof
17:37:24  <indutny>btw, I'm going to talk about streams2 at Yandex soon
17:37:28  <isaacs>indutny: awesome :)
17:37:31  <indutny>yeah
17:37:37  <indutny>so I need to get better understanding of it :)
17:37:44  <isaacs>indutny: do you use keynote? you can feel free to steal from my slides.
17:37:55  <indutny>isaacs: yep, I'll use keynote
17:38:14  <indutny>do you have your slides somewhere online?
17:41:38  <isaacs>yeah, lemme grab you the url...
17:41:59  <isaacs>indutny: http://dl.dropbox.com/u/3685/presentations/streams2/streams2-ko.key
17:42:06  * sblomjoined
17:42:15  <isaacs>indutny: that's the one from korea, which is the third one i gave in that trip, so it's the most polished.
17:42:20  * pooyajoined
17:42:40  <indutny>:)
17:42:41  <indutny>great
17:42:45  <indutny>thank you
17:43:45  <isaacs>some stuff has changed since then, of course.
17:43:55  <isaacs>but the basics are still accurate
17:44:06  <sblom>piscisaureus_: What did I break in the installer? I meant to mention that there's a change that works with WiX 3.6 and 3.7, but probably not with 3.5 (which I think is what we currently use).
17:44:33  <piscisaureus_>sblom: I get an error that a feature/@level tag is missing
17:44:37  <piscisaureus_>sblom: it's trivial to fix though.
17:44:45  <sblom>oh, interesting.
17:44:46  <isaacs>sblom: i should probably pull your changes onto my build vm and make sure it'll work when i do an actual release.
17:44:55  <sblom>isaacs: I'd love it if you would.
17:45:00  <indutny>isaacs: so what do you think about stream.dontHaveAnythingPleaseComeAgainLater() ? :)
17:45:16  <sblom>This is one of those things that's pretty much impossible to write auto-tests for. :(
17:45:41  <sblom>I've been doing upgrades and uninstalls and reinstalls and non-standard directory installs, but I've still possibly missed some things.
17:46:01  <piscisaureus_>sblom: maybe it's an issue with _my_ wix version
17:46:17  <sblom>piscisaureus_: It sounds like a legitimate complaint from WiX to me.
17:46:56  <piscisaureus_>sblom: it's this line:
17:46:56  <piscisaureus_><Feature Id="nodejs.shortcuts" Title="node.js shortcuts" Description="$(var.ProductDescription) Shortcuts">
17:47:02  <piscisaureus_>It lacks a Feature="1"
17:47:11  <piscisaureus_>er, Level="1"
17:47:23  <sblom>piscisaureus_: I'll double-check that it works with WiX 3.7.
17:47:24  <isaacs>indutny: yeah, need to figure out some nice API for that.
17:47:32  <sblom>It probably does.
17:47:36  <isaacs>indutny: in the meantime, just set this._readableState.reading = false if you're not going to call the cb
17:47:53  <indutny>isaacs: this sounds like a plan
17:48:33  * mikealjoined
17:49:14  <sblom>piscisaureus_: Still builds for me, so it definitely should be there.
17:49:24  <sblom>i.e. it's compatible across the gamut
17:51:46  <sblom>piscisaureus_: While you're thinking about the <Feature> stuff in WiX, jimschubert's change uses feature.application.shortcuts and feature.internet.shortcuts for Feature/@Id. I was thinking it should be nodejs.shortcuts.application and nodejs.shortcuts.internet just to keep the hierarchy alive.
17:52:18  <sblom>jimschubert raised a mild defense of starting them with "feature" in his pull request.
17:52:22  <sblom>What are your thoughts?
17:52:50  <sblom>I actually feel fairly strongly that we should be consistent here, and I think that means I should rename them.
17:52:59  <sblom>In fact, let me upgrade: I'm renaming them unless you object.
17:55:39  <sblom>piscisaureus_: done (and included your Level="1" fix)
18:04:22  * brsonjoined
18:05:48  * MI6quit (Remote host closed the connection)
18:05:59  * MI6joined
18:06:16  <isaacs>sblom: whoohoo! destination folder!!
18:06:36  <isaacs>sblom: woohoo! custom shortcut optional setups!!
18:08:37  <indutny>isaacs: surprisingly it works :)
18:08:44  <indutny>isaacs: with .ondata quirk and reading = false
18:08:55  <indutny>I need to figure out some details like ending streams and etc
18:08:59  <indutny>and make it stable :)
18:09:08  <indutny>and after that we can replace old retarded tls.js with new shining one
18:09:14  * loladirojoined
18:09:31  <isaacs>indutny: !!!! \o/
18:09:33  <isaacs>indutny: you are a hero
18:12:08  <indutny>meh, not really
18:12:10  <isaacs>indutny: i think we can do Buffer(0) as a "i have no more, but not eof yet". and null as eof
18:12:13  <indutny>it's still looks awkward
18:12:15  <isaacs>indutny: that might work.
18:12:24  <indutny>isaacs: yes, this lgtm
18:12:32  <indutny>brb
18:13:46  * rendarquit (Ping timeout: 245 seconds)
18:19:30  * AvianFluquit (Remote host closed the connection)
18:21:21  * rendarjoined
18:26:31  <sblom>isaacs: Did it build and run for you? Or was that excitement just from reading the pull request?
18:26:38  <isaacs>sblom: yeah, it ran
18:26:40  <isaacs>it installed
18:26:40  <isaacs>it works
18:26:41  <sblom>Great.
18:26:52  <isaacs>oddly, i now have a nodejs in C:\program files (x86) as well as one in c:\program files
18:27:02  <isaacs>maybe some older install? an x64 one?
18:27:17  <bnoordhuis>template <class Type, class ReturnType>
18:27:17  <bnoordhuis>struct Sorter<std::less<ReturnType>, Type, ReturnType>
18:27:17  <bnoordhuis>Sort(ReturnType (Type::*fn)() const)
18:27:17  <bnoordhuis>{ return Sorter<std::less<ReturnType>, Type, ReturnType>(fn);
18:27:17  <bnoordhuis>}
18:27:21  <bnoordhuis>^ c++ is so awesome :/
18:27:33  <bnoordhuis>that'd be a oneliner in any other language
18:27:56  <isaacs>piscisaureus_, bnoordhuis: Hey, so... odd question: why do we not do feof, and instead just assume that read() returning 0 means eof?
18:28:01  <sblom>isaacs: I'm sure that isn't supposed to happen. Lemme try to reproduce it.
18:28:16  <bnoordhuis>isaacs: feof? what/where?
18:28:17  <isaacs>sblom: the one that's not in x86 is 0.9.8
18:28:29  <isaacs>bnoordhuis: when we read files in a fs.ReadStream
18:28:36  <indutny>bnoordhuis: ++
18:28:38  <isaacs>bnoordhuis: we just go until we get no bytes.
18:28:43  <isaacs>bnoordhuis: which is fine, i guess
18:28:57  <indutny>isaacs: because it's ok :)
18:29:05  <sblom>isaacs: and the one in x86 is earlier?
18:29:14  <isaacs>sblom: no, the one in x86 is the one i just installed
18:29:22  <isaacs>sblom: the one in non-x86 is older
18:29:22  <sblom>ahh, I get it.
18:29:41  <sblom>Okay--I'll go play with this for a bit.
18:29:50  <isaacs>sblom:
18:29:50  <isaacs>c:\node>node
18:29:50  <isaacs>> process.execPath
18:29:50  <isaacs>'C:\\Program Files\\nodejs\\node.exe'
18:29:50  <isaacs>> process.version
18:29:52  <isaacs>'v0.9.8'
18:30:06  <isaacs>sblom: i assume that menas that i installed the x64 msi
18:30:07  <bnoordhuis>isaacs: that's how it works. feof() is a libc abstraction thing, the actual syscall just returns 0 once eof is hit
18:30:09  <isaacs>sblom: which i might have done
18:30:19  <isaacs>bnoordhuis: oh, ok. good answer :)
18:30:29  <sblom>yeah--I believe that part. Thing is, they both have the same product GUID, so installing one should remove the other.
18:30:35  <sblom>Or at least that's my expectation.
18:30:37  <isaacs>sblom: oh, ok
18:30:45  <sblom>We actually had someone request the ability to install both.
18:30:47  <isaacs>sblom: so it might be an actual bug with the new one
18:30:53  * mikealquit (Quit: Leaving.)
18:30:55  <isaacs>sblom: or maybe a new feature!
18:30:55  <sblom>But I think that's a complication we don't want to have to figure out just yet.
18:31:00  <sblom>Heh--maybe.
18:31:09  <sblom>I'm betting more bug.
18:31:13  <sblom>:s
18:31:50  * trevnorrisjoined
18:31:55  * mikealjoined
18:32:25  <isaacs>bnoordhuis: while i've got you: https://github.com/joyent/node/issues/1776
18:32:26  * lohkeyjoined
18:32:45  <isaacs>bnoordhuis: i think we need to expose RSTs somehow to net cliens
18:32:47  <isaacs>*clients
18:33:00  <bnoordhuis>yeah, i'm not sure why we do it like that
18:33:10  <isaacs>bnoordhuis: yes, it is a thing that you need to handle.. but how can you handle it if we don't show you it?
18:33:29  <indutny>RST?
18:33:32  <isaacs>resets
18:33:41  <indutny>yes, but how would we handle it
18:33:46  <indutny>it's not really exposed AFAIK
18:33:49  <isaacs>i dunno
18:33:57  <isaacs>self.emit('reset'); self.destroy()?
18:33:57  <indutny>bnoordhuis: am I right?
18:33:59  * `3rdEdenjoined
18:34:02  <isaacs>something
18:34:05  <indutny>well...
18:34:08  <indutny>not exactly this
18:34:14  <indutny>I mean how to handle this in C-land
18:34:38  <bnoordhuis>isaacs: apparently it's because of this -> https://github.com/joyent/node/issues/1571
18:35:06  <isaacs>ugh, that guy again?
18:35:10  <isaacs>who gave him commit access??
18:35:13  <bnoordhuis>yeah. good thing we let him go
18:35:26  <indutny>aaah
18:35:31  <indutny>ECONNRESET
18:35:36  <bnoordhuis>indutny: ECONNRESET just bubbles up from libuv/binding layer
18:35:40  <indutny>yes
18:35:42  <indutny>I remember now
18:35:49  <bnoordhuis>but lib/net.js squelches it
18:35:59  <indutny>squelches?
18:36:05  <indutny>yeew
18:36:13  <bnoordhuis>suppresses :)
18:36:21  <indutny>ook
18:37:01  <isaacs>what's weird is that the referenced test *doesn't* fail with econnreset now
18:37:03  <isaacs>when i take out that bit
18:37:22  <indutny>isaacs: oh, btw
18:37:29  <indutny>I remember another problem with streams2
18:37:30  <indutny>partial writes
18:37:32  <isaacs>k
18:37:39  <bnoordhuis>isaacs: indeed. (just checked it too)
18:37:41  <isaacs>partial writes?
18:37:43  <indutny>why ._write() can't succeed with partial write?
18:37:44  <indutny>yeah
18:37:56  <indutny>suppose ._write() was called with 16kb buffer
18:37:59  <isaacs>ohh, like, you gave me a buffer, and i was able to write some of it, but not all of it?
18:38:04  <indutny>but only 8kb was written
18:38:06  <indutny>yes
18:38:24  * bnoordhuisruns tests
18:38:25  <indutny>I can buffer it
18:38:28  <isaacs>well, the way it actually works in streams1 land, is that you wouldn't take a new pending chunk until you're done with hte old pending chunk
18:38:34  <indutny>but my internal buffering won't affect watermarks
18:38:39  <isaacs>indutny: that's how it works in streams2, also
18:38:46  <indutny>hm...
18:38:48  <isaacs>indutny: _write(chunk,cb) don't call cb until you're done writing the whole thing
18:38:52  <indutny>should I just stop calling callback
18:38:55  <isaacs>if you have to call your underlying write function a bunch of times, that's fine
18:38:58  <indutny>that's what I was afraid of :)
18:39:00  <isaacs>but don't call cb till your'e done
18:39:05  <isaacs>and only call that one once :)
18:39:05  <indutny>ok
18:39:15  <indutny>you see, I learned a lot today :)
18:39:28  <isaacs>the writable side doesnt' actually change much from streams1 to streams2
18:39:39  <isaacs>except that you have to do less work to handle pending stuff
18:40:03  <indutny>yes
18:41:02  <isaacs>bnoordhuis: so, yo udo get a TON of econnresets in other tests when you remove that bit.
18:41:16  <isaacs>(ton = 5. they're 400lbs each, apparently.)
18:41:22  <bnoordhuis>:)
18:41:29  <bnoordhuis>yeah, i'm seeing that as well
18:41:36  <bnoordhuis>also AssertionError: "ECONNRESET" == "EPIPE"
18:41:54  <bnoordhuis>in pummel/test-tls-ci-reneg-attack
18:42:21  <bnoordhuis>[03:49|% 100|+ 558|- 3]: Done <- all failures in the pummel tests interestingly enough
18:43:13  <isaacs>simple/test-http-multi-line-headers is failing with an econnreset
18:43:19  <trevnorris>sup everyone? isaacs if you push to a Readable stream, but there's nothing reading it out, what should happen?
18:43:19  <isaacs>which is kind of expected, looking at the code.
18:43:23  <isaacs>i mean, the server destroys the connection
18:43:32  <isaacs>trevnorris: it should eventually have push() return false
18:43:39  <bnoordhuis>isaacs: passes for me
18:43:42  <trevnorris>isaacs: for me the process just dies, out of memory
18:43:53  <isaacs>trevnorris: well, you should stop push()ing when it returns false
18:44:01  <isaacs>trevnorris: wait for another _read call
18:44:32  <isaacs>bnoordhuis: weird.
18:44:39  <bnoordhuis>the pummel/test-tls-ci-reneg-attack failure is more or less expected, i think
18:44:49  <isaacs>ok, so... we can't just yank that code out. we have to do *something* with ECONNRESETs, i guess.
18:44:56  <bnoordhuis>it gets ECONNRESET but that's squelched, and the next write errors with EPIPE
18:44:57  <isaacs>they're not so abnormal, after all.
18:45:05  <trevnorris>isaacs: figured. just wanted to know if there was a failsafe to dump unread data if it was filling up.
18:45:15  <isaacs>bnoordhuis: right, sounds like the test could just be updated to expect ECONNRESET instead of EPIPE
18:45:19  <bnoordhuis>yep
18:45:23  <isaacs>trevnorris: nope.
18:45:28  <isaacs>trevnorris: it's up to you to respect backpressure.
18:45:38  <isaacs>trevnorris: it's like calling write(chunk) forever and not respecting the return value
18:45:58  <trevnorris>isaacs: got it. yeah. I just put a .push() if a for loop and tried to kick the crap out of it.
18:46:05  <isaacs>trevnorris: hahah
18:46:16  <isaacs>trevnorris: should be a while(push()); loop
18:46:32  <trevnorris>ah, that makes sense.
18:49:40  * jmar777quit (Read error: Connection reset by peer)
18:50:02  * jmar777joined
18:51:41  <trevnorris>isaacs: so if I am just pushing constantly, I should only see continued increase of memory consumption, right?
18:51:55  <isaacs>yes.
18:52:02  <isaacs>that's what i would expect
18:53:07  <trevnorris>hm. don't think that's what I'm seeing. I'll run massif on it and post the results.
18:55:57  <indutny>again this word
18:56:02  <indutny>bnoordhuis: wash your mouth
18:56:12  <bnoordhuis>indutny: wut?
18:57:40  <MI6>joyent/node: bnoordhuis created branch issue1776 - http://git.io/bJlL_A
18:57:43  <pfox__>howdy. so.
18:57:57  <bnoordhuis>isaacs: ^ if you can look into why that other test is failing for you
18:58:04  <pfox__>looks like joyent/libuv master is broken in mingw32. is that a high priority for you guys?
18:58:06  <bnoordhuis>then you're free to steal my commit :)
18:58:12  <pfox__>just trying to figure out if i need to patch/fix on my own or what.
18:58:13  <bnoordhuis>pfox__: no, but we'll take patches
18:58:17  * pfox__nods
18:58:20  <bnoordhuis>what is broken?
18:58:25  <pfox__>lemme get a paste
18:59:11  <isaacs>bnoordhuis: i'm seeing 4 simple tests failing with that same diff
18:59:18  <isaacs>bnoordhuis: testing another thing atm, i'll be on that in a few minutes
18:59:23  <bnoordhuis>okay, no rush
18:59:30  <isaacs>bnoordhuis: (the fifth was @#$!@#!ing debugger tests)
18:59:30  <pfox__>bnoordhuis: http://pastebin.mozilla.org/2101375
18:59:35  <bnoordhuis>it's not like it's terribly bothering me :)
18:59:52  <bnoordhuis>ah, the debugger tests... they make me sad :9
19:00:21  <trevnorris>pfox__: you work out of MTV?
19:00:32  <bnoordhuis>pfox__: ah, it's trying to compile unix code
19:00:52  <isaacs>indutny: so.. with returning Buffer(0)
19:01:10  <isaacs>indutny: let's say, you call read(). that triggers _read(). i call teh cb(null,Buffer(0))
19:01:28  <isaacs>indutny: now, we're in a state where, you don't know that you'll need to read() again, and you're waiting for a readable event, and i'm not doing anything.
19:01:32  <bnoordhuis>pfox__: right, i know what that is (and i'm afraid i kind of broke it)
19:01:40  <isaacs>indutny: i think for anything other than crypto streams, this is very dangerous.
19:01:45  <isaacs>indutny: we should not tell people about this :)
19:01:46  <bnoordhuis>pfox__: if you compile with `make OS=mingw`, it should work
19:01:47  <pfox__>trevnorris: no. im a rust contributor, so i use their pastebin out of habit
19:02:06  <pfox__>bnoordhuis: ok, i'll try that..
19:02:07  <isaacs>indutny: it'll only work if some *other* thing is going to trigger another read() at some point in the future (which crypto streams will)
19:02:10  <trevnorris>piscisaureus_: was it you that fixed a Buffer(0) bug since the class points to NULL or something?
19:02:25  <trevnorris>pfox__: ah, cool.
19:02:40  <pfox__>wouldn't i love to work out of their, though :)
19:02:44  <pfox__>someday, perhaps.
19:02:48  <pfox__>there, even
19:02:53  <bnoordhuis>pfox__: it's because of this -> OS ?= $(shell sh -c 'uname -s | tr "[A-Z]" "[a-z]"')
19:04:11  <pfox__>bnoordhuis: OS=mingw cleared it right up. thanks.
19:04:32  <bnoordhuis>i guess i didn't break it after all, it just never worked. before, you had to set MSVC=1
19:04:53  <bnoordhuis>anyway...
19:06:11  <pfox__>hey, no biggie
19:06:22  <pfox__>i'm trying to land the libuv update and mingw32 is my last checkbox
19:06:49  * jmar777quit (Read error: Connection reset by peer)
19:07:19  * jmar777joined
19:12:48  <indutny>isaacs: sorry, was afk
19:12:57  <isaacs>np
19:13:11  <indutny>isaacs: why is it dangerous?
19:13:16  <indutny>it should be documented
19:13:18  <indutny>and that's it
19:13:31  <indutny>like if you call cb(null, new Buffer(0)) we'll wait for 'readable' event
19:14:02  <MI6>joyent/libuv: bnoordhuis created branch fix-mingw - http://git.io/80G3ZQ
19:14:13  <bnoordhuis>pfox__: can you try that patch? ^
19:14:22  <indutny>brb
19:14:30  * c4milo_quit (Ping timeout: 264 seconds)
19:14:54  * bnoordhuisis off to dinner
19:16:12  <pfox__>bnoordhuis: ok
19:21:04  * mikealquit (Quit: Leaving.)
19:22:38  <MI6>joyent/node: isaacs created branch stream-nodata-noteof - http://git.io/u1WEtQ
19:22:39  <isaacs>indutny: here's the patch ^
19:22:47  <isaacs>indutny: well, that's the thing. there won't BE a readable event.
19:23:10  <isaacs>indutny: because we wn't even ask for more data until you read() again, but there wont' be more data until you read(), so how do you know when to read() if you don't know when there'll be more data?
19:23:30  <isaacs>indutny: you need something to have at least some guess as to when there MIGHT be data, and that thing call read(0)
19:23:54  <isaacs>indutny: really, if you're not writing crypto pair streams, you probably should just not be returning empty buffers
19:24:14  * cucucujoined
19:24:24  * c4milojoined
19:24:35  <isaacs>indutny: here's another possibility...
19:24:59  <isaacs>indutny: instead of having the pair be a "cleartext" side and a "encrypted" side
19:25:06  <isaacs>indutny: what if it was a pair of transform streams.
19:25:22  <isaacs>indutny: so one stream is ClearToEncrypted, and the other is EncryptedToClear
19:27:05  * wolfeidaujoined
19:27:36  <cucucu>hi, i was wondering, is there a way to install libuv on linux using the default makefile? the compile step is fine but i wasn't able to install and i didn't see the point on using gyp since i already have it compiled, am i missing something?
19:28:15  <pfox__>bnoordhuis: actually that patch breaks everything, even w/ OS=mingw
19:28:17  <pfox__>let's just leave it be
19:31:01  <trevnorris>isaacs: here: https://gist.github.com/4685660
19:31:02  * loladiroquit (Quit: loladiro)
19:31:12  * mikealjoined
19:31:47  <TooTallNate>isaacs: Transform streams makes sense to me for crypto
19:32:23  <TooTallNate>isaacs: plus, are we using any in core yet? it would be a good thing to do so
19:32:43  <TooTallNate>oh, zlib probably
19:32:52  <TooTallNate>yup
19:33:02  <isaacs>yeah, we're using them for crypto hash decipher, etc.
19:33:56  <TooTallNate>oh ok
19:34:08  <TooTallNate>isaacs: i was saying makes sense in the context of what you and indutny were talking about
19:34:10  <TooTallNate>"tls" i gues
19:34:12  <TooTallNate>guess
19:34:17  <isaacs>yah
19:34:21  <isaacs>Cryptostream
19:34:57  <TooTallNate>that's one of the ones internal to tls.Socket right?
19:35:34  <isaacs>yeah
19:35:36  * EhevuTovjoined
19:35:53  <isaacs>actually, there is no tls.Socket
19:36:03  <TooTallNate>ya, i was just noticing that
19:36:07  <TooTallNate>it's just tls.connect()
19:36:10  <isaacs>https uses tls.CleartextStream for stuff
19:36:17  <isaacs>that's the thing you actually see in tls.connect
19:36:25  <isaacs>the actual socket is connected to the encrypted side of the pair
19:38:06  <indutny>isaacs: interesting
19:38:38  <indutny>though its not really transforming
19:38:42  <indutny>and things a bit more interesting
19:38:46  <isaacs>yeah.
19:38:47  <isaacs>i know
19:38:48  <indutny>since for example
19:39:02  <indutny>TCP data might be written to encrypted side
19:39:07  <indutny>and immediately read from it
19:39:13  <indutny>without producing any cleartext output
19:39:16  <isaacs>right
19:39:18  <indutny>or getting any input
19:39:19  <isaacs>ok, scratch that.
19:39:21  <indutny>so it won't apply here
19:39:22  <indutny>yes
19:39:27  <indutny>just wanted to make my point clear
19:39:30  <isaacs>it's actually two separate duplexes that are just conencted at the hip
19:39:34  <indutny>yes
19:39:38  * mikealquit (Quit: Leaving.)
19:39:38  <trevnorris>isaacs: from the table you'll see that it'll be impossible to get the js streams api as fast as v0.8 since the raw transfer is only just as fast.
19:39:45  <indutny>my diagram is the most correct model of it
19:39:52  <isaacs>trevnorris: yes.
19:39:53  <indutny>big mess inside called OpenSSL
19:40:08  <isaacs>trevnorris: we have to figure out why the raw bindings are slower, first.
19:40:13  <isaacs>trevnorris: then we can dig into the s
19:40:14  <isaacs>*js
19:40:47  <trevnorris>isaacs: ok. i'll leave that one up to ya'll and keep working on _streams_readable
19:40:58  <isaacs>trevnorris: what are you doing in _streams_readable, exactly?
19:41:05  <isaacs>trevnorris: just wondering because i was going to dig into pipe() today
19:41:55  <trevnorris>isaacs: first thing is checking where it's deopting, switching contexts, etc.
19:42:00  <isaacs>oh, ok
19:42:01  <isaacs>kewl.
19:42:14  <isaacs>that's good stuff
19:42:40  <TooTallNate>i like trev's low-level optimizations :)
19:43:10  <trevnorris>TooTallNate: heh, thanks. for some reason I can do this for hours.
19:43:28  <TooTallNate>well i welcome it cause it's the kind of crap i don't want to deal with :p
19:43:35  <indutny>isaacs: https://github.com/joyent/node/commit/8916a6cf63c0 <- lgtm
19:54:38  * AvianFlujoined
19:59:19  <isaacs>indutny: ok, i'm gonna pull that in, and we can go from there.
19:59:26  <isaacs>indutny: if it gets ugly or something, we can work something else out
19:59:47  <MI6>joyent/node: isaacs master * 7e1cf84 : stream: Don't signal EOF on '' or Buffer(0) Those values, if passed to t - http://git.io/HxkdWA
20:02:09  <trevnorris>isaacs: I swear I remember piscisaureus_ talking about an issue returning Buffer(0) because something like data_ would just point to null.
20:02:41  <trevnorris>and v8 would throw a fit about it when doing a specific operation.
20:03:17  <isaacs>trevnorris: nah, it's fine
20:03:18  <isaacs>> b = new Buffer(0); [b.length, b[0], b.toString(), Array.prototype.slice.call(b)]
20:03:21  <isaacs>[ 0, undefined, '', [] ]
20:03:28  <isaacs>trevnorris: maybe SlowBuffer(0)?
20:03:35  <trevnorris>isaacs: yeah. maybe that was it.
20:04:18  <isaacs>> b = new buffer.SlowBuffer(0); [b.length, b[0], b.toString(), Array.prototype.slice.call(b)]
20:04:21  <isaacs>[ 0, undefined, '', [] ]
20:04:23  <isaacs>nah, that's fine, too
20:06:24  <trevnorris>isaacs: ah, it was bnoordhuis (http://logs.nodejs.org/libuv/2013-01-15)
20:06:30  <trevnorris>search for 'Buffer(0)'
20:06:55  * cucucuquit (Quit: Page closed)
20:08:10  <isaacs>ah. well that's weird.
20:08:41  <isaacs>if new Buffer(0) is a problem, then it's a problem.
20:08:51  <isaacs>ie, a problem we need to fix, not just avoid by not using it
20:08:57  <isaacs>people creaet empty buffers sometimes
20:09:08  <isaacs>indutny: fwiw, "" is treated teh same way as Buffer(0)
20:09:48  <indutny>huh?
20:10:07  * mikealjoined
20:10:11  <isaacs>indutny: if returning Buffer(0) in the crypto stream makes bad happen, you can use an empty string, it's the same.
20:10:27  <isaacs>indutny: (see my conv with trevnorris right above this)
20:10:44  <TooTallNate>i don't get what Buffer(0) has to do with crypto
20:10:47  * isaacsaway for the insertion of food into mouth
20:11:01  <isaacs>TooTallNate: the CryptoStream classes will use that for "no data right now, but maybe try again later"
20:11:10  <isaacs>oh, shit, there's a problem with that patch.
20:11:12  <trevnorris>TooTallNate: just saw a mention of returning a zero buffer, and remembered a conversation bnoordhuis and indutny had a while back
20:11:20  <isaacs>you could have a decoder return a 0-length string...
20:11:24  <isaacs>but i guess that was a problem before, also.
20:11:40  <isaacs>oh well, lunch time
20:12:13  <TooTallNate>isaacs: wouldn't you just not call the _read's callback function until you had more data?
20:12:28  <isaacs>TooTallNate: scroll up. you don't know when to ask for more data.
20:12:36  <isaacs>TooTallNate: so you're actually not reading right now
20:12:53  <isaacs>and you need a future read() call to trigger a _read
20:13:10  <isaacs>and the actual _read is sync
20:13:15  <isaacs>ok, srsly, gotta run
20:13:17  * isaacs&
20:13:34  * TooTallNate!
20:20:52  * mikeal1joined
20:20:52  * mikealquit (Read error: Connection reset by peer)
20:22:58  <trevnorris>think there's something funky with assert.throws. the test throws a RangeError, but the test only checks for Error.
20:23:00  <trevnorris>shouldn't that fail?
20:24:04  <trevnorris>but if I throw an Error and expect a RangeError then it fails.
20:29:39  * `3rdEdenquit (Remote host closed the connection)
20:29:44  * mikeal1quit (Ping timeout: 255 seconds)
20:31:29  * `3rdEdenjoined
20:32:22  * `3rdEdenquit (Remote host closed the connection)
20:33:05  <bnoordhuis>trevnorris: that Buffer(0) bug is already fixed
20:33:20  <bnoordhuis>i landed a workaround in deps/v8 and the v8 people have fixed it upstream
20:33:31  * CoverSlide?
20:33:35  <trevnorris>bnoordhuis: cool. that's the part I couldn't remember. just remembered there was discussion on it, and wanted to make sure.
20:38:24  * EhevuTovquit (Quit: This computer has gone to sleep)
20:43:39  * googoljoined
20:43:52  * mikealjoined
20:44:23  <pfox__>bnoordhuis: don't know if you saw my comment above.. that patch broke everything on mingw32. i've already worked around in our build scripting using OS=mingw. thanks for the help.
20:44:42  <bnoordhuis>pfox__: yeah, i saw. already killed the branch :)
20:44:49  <bnoordhuis>thanks for trying it
20:44:58  <pfox__>np
20:48:51  * stagas_joined
20:50:02  * stagasquit (Ping timeout: 244 seconds)
20:50:16  * stagas_changed nick to stagas
20:51:50  * EhevuTovjoined
21:02:31  * stagas_joined
21:05:30  * stagasquit (Ping timeout: 264 seconds)
21:05:41  * stagas_changed nick to stagas
21:11:05  * sgallaghquit (Remote host closed the connection)
21:12:45  * c4miloquit (Remote host closed the connection)
21:13:15  * c4milojoined
21:14:36  * bradleymeckquit (Ping timeout: 245 seconds)
21:15:45  * hzquit (Disconnected by services)
21:15:48  * hzjoined
21:17:23  * c4miloquit (Ping timeout: 248 seconds)
21:18:16  * hzquit (Disconnected by services)
21:18:19  * hzjoined
21:20:31  <isaacs>trevnorris: RangeError inherits from Error
21:20:47  <isaacs>trevnorris: > new RangeError instanceof Error
21:20:47  <isaacs>true
21:21:01  <trevnorris>mother effing. forgot to use `new`
21:21:11  <trevnorris>>> RangeError instanceof Error
21:21:25  <trevnorris>fasle
21:21:34  <trevnorris>isaacs: anyways. thanks
21:22:19  <trevnorris>isaacs: oh and bnoordhuis reminded me that he made a fix to v8, and filed a bug. is now fixed in upstream.
21:22:28  <trevnorris>(for the SlowBuffer(0) thing)
21:24:55  * loladirojoined
21:28:46  * bnoordhuisquit (Ping timeout: 245 seconds)
21:31:18  * hzquit
21:34:10  <isaacs>trevnorris: yeah, RangeError is a function ;)
21:34:16  * jmar777quit (Remote host closed the connection)
21:34:21  <isaacs>kewl
21:34:28  <isaacs>can someone review this? https://github.com/isaacs/node/commit/master
21:35:14  <isaacs>basically, the zero-length-ness of a _read result must be *before* being written to the decoder, since base64 returns zero-length strings routinely
21:35:16  * indexzerojoined
21:35:52  * hzjoined
21:38:07  * EhevuTovquit (Quit: This computer has gone to sleep)
21:38:40  <trevnorris>isaacs: that part makes sense. and crazy amount of additional testing for it. =)
21:39:29  * stagasquit (Read error: Connection timed out)
21:40:05  * stagasjoined
21:40:36  * CoverSlidequit (Remote host closed the connection)
21:42:09  <isaacs>trevnorris: it's more of moving the code around
21:42:15  <isaacs>than actually adding that much more stuff
21:42:27  <isaacs>i just moved the existing test into a function so i could call them both
21:42:35  <isaacs>landing it.
21:42:45  <MI6>joyent/node: isaacs master * a6c1847 : stream: Don't stop reading on zero-length decoded output Fixes regressio - http://git.io/pLKwcw
21:48:20  * EhevuTovjoined
21:54:33  * wolfeidauquit (Remote host closed the connection)
22:00:49  * c4milojoined
22:03:52  <trevnorris>isaacs: how is Readable push failing with a JS out of memory error? does that mean the incoming bits are being stored in the v8 heap?
22:08:38  * rendarquit
22:10:35  <trevnorris>and wtf. valgrind is telling me that the process memory usage doesn't grow beyond 1.8 MB. where's bnoordhuis when you need him?
22:17:38  * wolfeidaujoined
22:19:44  <isaacs>trevnorris: Buffers aren't in the V8 heap. it's dying with a standard OOM error.
22:19:55  <isaacs>trevnorris: the OS is shooting your process in the head, because it's using a buttload of memory
22:20:11  <isaacs>ok, this push->pipe refactor is not doing anything.
22:20:15  <isaacs>has no effect.
22:20:24  <isaacs>trevnorris: what exact IS a deopt?
22:21:00  <trevnorris>isaacs: i'm confused. after running it until it dies, v8 tells me it's gc'd 7GB memory. does that include external memory?
22:21:35  <isaacs>trevnorris: i'm not su
22:21:36  <isaacs>re
22:22:21  <trevnorris>and the error msg is "FATAL ERROR: JS Allocation failed - process out of memory". I've only ever gotten that when the v8 heap overfills. anyways, about deopt:
22:23:17  <trevnorris>this explains it way better than I can: http://floitsch.blogspot.com/2012/03/optimizing-for-v8-inlining.html
22:32:09  <isaacs>oh, hahhahaha
22:32:11  <isaacs>so..
22:32:23  <isaacs>i've been trying to track down why net-pipe is deopting Socket.write() so often
22:32:27  <isaacs>turns out... i'm doing a console.log
22:32:32  <isaacs>BADERP, that writes a string.
22:33:33  <trevnorris>heh, awesome.
22:34:34  <isaacs>i could probably make this benchmark faster by having console.log convert to a buffer before doing process.stdout.write
22:35:24  <trevnorris>(oh, and if you click on "Sidebar" in top left, the entire thing is awesome)
22:35:58  <trevnorris>that sounds good. writing a benchmark that really hits the exact problem can be a real pain when you're checking the tiny bits.
22:36:56  <trevnorris>he has an article on polymorphic calls in assembly. talk about micro optimizations.
22:37:53  <trevnorris>isaacs: think you'd also be surprised how many type checks can cause a DEOPT.
22:38:19  <trevnorris>especially if you're expecting types of arguments in specific places, for optional arguments.
22:38:40  <trevnorris>that's why I created: https://gist.github.com/4670377
22:38:53  <trevnorris>(but haven't had a lot of time to work on it)
22:40:51  <isaacs>well, that gets rid of the write deopts.
22:40:59  <isaacs>but the benchmark doesn't improve in the slightest.
22:41:18  <isaacs>i just had it push all the output lines into an array, and then dump at the end
22:41:40  <trevnorris>isaacs: yeah. next hardest thing about doing this is identifying which deopts are screwing you.
22:41:50  <trevnorris>how often were you running console.log?
22:43:18  <trevnorris>isaacs: and awesome idea about the buffer/stdout thing. i'm going to convert my tests in my PR to do that.
22:46:43  <isaacs>trevnorris: every second
22:46:53  <isaacs>trevnorris: so, it looked like a lot
22:47:06  <isaacs>most of the readable side is where the slowness is, thoguh, and that's not getting deopted
22:47:09  <isaacs>it's just slow :)
22:47:28  <trevnorris>heh
22:47:32  <trevnorris>i'm looking at that now.
22:51:13  <trevnorris>isaacs: i'm getting a fair amount of deopts from "process._makeCallback", but i'll save that for another time.
22:51:18  <trevnorris>you working on _stream_readable?
22:51:47  <trevnorris>also, what cli options are you using to trace?
22:52:08  <isaacs>trevnorris: --trace_inlining and --trace_deopts
22:52:27  <isaacs>yeah, _makeCallback gets deopted a lot
22:52:29  <tjfontaine>isn't a bailout worse than a deopt?
22:52:31  <isaacs>the args array is a lot of different types.
22:52:34  <isaacs>so that kind of makes sense.
22:52:39  <isaacs>tjfontaine: i'm not sure
22:52:44  <trevnorris>isaacs: also try add --trace-opt and --code-comments
22:53:34  <trevnorris>code comments is awesome. it shows the assembly trace to why it was deopt'd
22:54:21  <tjfontaine>lets be serious, v8 is just an awesome vm :)
22:54:31  * loladiroquit (Quit: loladiro)
22:54:51  <trevnorris>tjfontaine: yeah. I can let my micro optimization spirit fun free.
22:55:22  <trevnorris>isaacs: and just to make sure it wasn't a typo. you using --trace_deopts or --trace_deopt?
22:55:31  <isaacs>--trace_deopt
22:55:37  <trevnorris>ah, cool
22:55:47  <isaacs>i'm gonna relocate. bbiab
22:57:13  <trevnorris>tjfontaine: oh, and the tick-parser for --prof makes my little heart leap for joy.
22:58:06  <tjfontaine>trevnorris: then you add dtrace+ustack resulting flamegraphs and there's little reason to have slow code
22:58:39  <trevnorris>still have to learn how to use that.
22:58:53  <trevnorris>bnoordhuis just taught me how to use valgrind w/ massif
23:01:11  <trevnorris>tjfontaine: i've seen the flame graphs. they look cool, but I have a hard time reading them.
23:01:27  <trevnorris>can you like click on them and get additional info?
23:01:44  <tjfontaine>trevnorris: well you hover you see the function name at the top of the graph
23:01:52  <trevnorris>ahhhh. ok
23:02:10  <tjfontaine>trevnorris: what you're seeing are the call stacks that are seen the most often
23:02:40  <trevnorris>interesting.
23:02:46  <trevnorris>tjfontaine: you know much about valgrind?
23:03:04  <trevnorris>getting results that I don't understand.
23:03:14  <tjfontaine>trevnorris: I've used it, not regularly, but I'll try and help
23:04:33  <trevnorris>tjfontaine: created a for loop that Readable.push continuously. doesn't wait for drain. tracing the gc, says it has cleaned up 7GB before the process crashed.
23:04:48  <trevnorris>but valgrind only reports the process using a few MB of memory.
23:06:13  * wolfeidauquit (Read error: Connection reset by peer)
23:06:28  * wolfeidaujoined
23:08:49  * indutnyquit (Ping timeout: 248 seconds)
23:09:00  * mikealquit (Quit: Leaving.)
23:10:05  <tjfontaine>trevnorris: have you pastebin'd the valgrind output?
23:11:23  * loladirojoined
23:11:27  * brsonquit (Remote host closed the connection)
23:11:37  * indutnyjoined
23:13:01  <trevnorris>tjfontaine: https://gist.github.com/4687602
23:13:20  <trevnorris>wait... what?
23:14:02  <trevnorris>now it's saying 3.2GB (i had just run this test with a bunch of different options, but failed to check it)
23:14:29  * brsonjoined
23:15:47  <trevnorris>tjfontaine: well, thanks for helping me realize my complete lack of attention to that detail. ;-)
23:16:22  <tjfontaine>:)
23:17:08  * mikealjoined
23:17:30  <trevnorris>indutny: strange thing. if you push continuously to a Readable stream, the process will fail b/c the gc can't keep up.
23:20:12  * stagas_joined
23:21:22  <trevnorris>indutny: or something like it. my test shows 2.5GB of buffers fill with back pressure, but the process then fails, reaching 1GB usage.
23:21:56  * stagasquit (Ping timeout: 252 seconds)
23:22:07  * stagas_changed nick to stagas
23:22:09  <trevnorris>tjfontaine: so guess I accidentally passed the right option. just ran it again as a sanity check and said max usage was 1.8MB
23:23:32  * paddybyersquit (Read error: Connection reset by peer)
23:23:39  * googolquit (Ping timeout: 260 seconds)
23:23:53  * paddybyersjoined
23:28:51  * indexzeroquit (Quit: indexzero)
23:38:51  * jmar777joined
23:38:51  * loladiroquit (Quit: loladiro)
23:38:52  * hzquit (Disconnected by services)
23:38:54  * hzjoined
23:39:44  <trevnorris>TooTallNate: how do you build w/ node-gyp using a custom build of node?
23:41:56  <tjfontaine>--nodedir=/path/to/node
23:42:12  <tjfontaine>don't use relative though, in my experience it isn't always happy with it
23:42:59  <TooTallNate>trevnorris: --nodedir=path/to/node
23:43:13  <TooTallNate>where "node" is the node source code dir
23:43:17  <TooTallNate>and/or git checkout
23:43:27  <TooTallNate>oh sorry tjfontaine
23:43:30  <TooTallNate>i'm retarded hahah
23:43:36  <tjfontaine>sok, I'm happy to be right :)
23:44:00  <TooTallNate>i wasn't aware of the relative problem though
23:44:07  <TooTallNate>i always do --nodedir=~/node
23:44:12  <TooTallNate>which expands to absolute path
23:44:28  <tjfontaine>ya, but I think ../ isn't as nice
23:45:39  <trevnorris>so like: node-gyp build --nodedir=...?
23:45:59  <TooTallNate>trevnorris: ya
23:47:15  <trevnorris>TooTallNate: tells me I have to run configure first, and when I run the same w/ configure still tells me "gyp info using node@0.8.17"
23:50:16  <TooTallNate>trevnorris: oh, well nodedir will build a binary *for* the version in the dir you specify
23:50:32  <TooTallNate>trevnorris: but if you want to run node-gyp using a different node executable
23:50:51  <TooTallNate>then you can do like "./node `which node-gyp` rebuild"
23:50:53  <TooTallNate>something like that
23:50:58  <trevnorris>oh I don't care. just thought that line meant what version of node it was building for.
23:50:59  * TooTallNatequit (Quit: Computer has gone to sleep.)
23:52:04  <trevnorris>whoot! it worked. awesome.
23:52:33  <trevnorris>thanks. was just misreading the config output.
23:55:47  * paddybyersquit (Ping timeout: 248 seconds)
23:59:40  * felixgequit (Quit: felixge)