00:00:01  * ircretaryquit (Remote host closed the connection)
00:00:09  * ircretaryjoined
00:02:49  * indexzerojoined
00:03:56  * ericktquit (Quit: erickt)
00:30:03  * loladiroquit (Quit: loladiro)
00:36:34  * loladirojoined
00:36:43  * loladiroquit (Client Quit)
00:36:57  * ericktjoined
00:37:17  * loladirojoined
00:43:34  * ericktquit (Quit: erickt)
00:45:16  * ericktjoined
00:55:22  * wolfeidaujoined
01:01:17  * loladiroquit (Quit: loladiro)
01:07:32  * ericktquit (Quit: erickt)
01:08:05  <indutny>isaacs: no
01:08:45  * ericktjoined
01:09:35  * loladirojoined
01:11:09  * ericktquit (Client Quit)
01:18:06  * abraxasjoined
01:41:21  * bradleymeckquit (Quit: bradleymeck)
01:44:52  * bradleymeckjoined
01:48:11  * indexzeroquit (Quit: indexzero)
01:48:42  * indexzerojoined
02:37:35  * brsonjoined
02:50:57  * jmar777joined
02:58:49  * jmar777quit (Remote host closed the connection)
02:59:22  * jmar777joined
03:04:20  * jmar777quit (Ping timeout: 272 seconds)
03:17:53  * jmar777joined
03:28:45  * loladiroquit (Quit: loladiro)
03:49:36  * jmar777quit (Remote host closed the connection)
04:06:47  * loladirojoined
04:28:16  * bradleymeckquit (Quit: bradleymeck)
04:39:34  * loladiroquit (Quit: loladiro)
04:42:50  * bradleymeckjoined
04:45:29  * loladirojoined
04:48:22  * joshthecoderjoined
05:00:10  * AvianFluquit (Read error: Connection reset by peer)
05:00:33  * AvianFlujoined
05:16:20  <isaacs>
05:16:58  <mmalecki>I can agree with that
05:17:30  <mmalecki>isaacs: never got to know wtf was up with that parse error, can't repro
05:27:22  * brsonquit (Quit: leaving)
05:30:00  * AvianFluquit (Ping timeout: 272 seconds)
05:39:52  * AvianFlujoined
05:48:52  * AvianFlu_joined
05:49:36  * AvianFluquit (Read error: Connection reset by peer)
05:53:42  * benoitcquit (Excess Flood)
05:54:38  * benoitcjoined
05:55:01  * AvianFlu_changed nick to AvianFlu
06:19:42  <indutny>heya
06:19:46  <indutny>how are you doing, guys?
06:26:11  * wolfeidauquit (Remote host closed the connection)
06:27:38  * benoitcquit (Excess Flood)
06:36:08  * benoitcjoined
06:40:00  * AvianFluquit (Remote host closed the connection)
07:17:39  * mraleph1quit (Read error: Connection reset by peer)
07:17:45  * mralephjoined
07:19:24  * indexzeroquit (Quit: indexzero)
07:22:54  * stagasjoined
07:38:49  * `3rdEdenjoined
08:18:42  * mjr_quit (Quit: mjr_)
08:25:57  * felixgejoined
08:40:15  * joshthecoderquit (Quit: Linkinus - http://linkinus.com)
08:41:10  * rendarjoined
09:10:19  * bradleymeckquit (Quit: bradleymeck)
09:37:41  * felixgequit (Quit: felixge)
10:00:20  * bnoordhuisjoined
10:30:35  * abraxasquit (Remote host closed the connection)
10:30:52  <MI6>joyent/node: Ben Noordhuis master * ddb1560 : buffer: use MAP_ANON, fix OS X build - http://git.io/GsrFSw
10:36:36  <indutny>bnoordhuis: hey man
10:36:42  <bnoordhuis>indutny: ho man
10:36:52  <indutny>bnoordhuis: so I've a problem with 0.9.4 on osx
10:36:55  <indutny>probably not only osx
10:37:01  <bnoordhuis>what problem?
10:37:02  <indutny>some openssl symbols are not exported
10:37:09  <indutny>well, they're in libopenssl.a
10:37:13  <indutny>but not in node's binary
10:37:21  <bnoordhuis>but not visible to add-ons?
10:37:25  <indutny>yes
10:37:33  <indutny>I thought you've already dealt with it
10:37:35  <bnoordhuis>not in node binary == stripped by the linker?
10:37:45  <indutny>I suppose so
10:37:50  <bnoordhuis>no, that was bert - and it hasn't landed yet
10:38:08  <indutny>aaah
10:38:12  <indutny>and where is bert?
10:38:18  <indutny>drinking at monday?
10:38:19  <bnoordhuis>who knows? i don't
10:38:22  <bnoordhuis>probably :)
10:38:42  <indutny>haha
10:38:43  <indutny>ok
10:55:25  <indutny>ircretary: tell piscisareus_ to tell me about stripping symbols, openssl and osx
10:55:25  <ircretary>indutny: I'll be sure to tell piscisareus_
10:55:34  <indutny>oops
10:55:46  <indutny>tell piscisaureus_ to tell me about stripping symbols, openssl and osx
10:56:39  * `3rdEdenchanged nick to `3rdEden|BRB
10:59:38  <bnoordhuis>so about streams2... https://github.com/joyent/node/commit/945f877
10:59:52  <bnoordhuis>i'm not really pleased with the fact that it requires changes to existing code
11:00:04  <bnoordhuis>api stability and all that
11:00:59  <bnoordhuis>indutny: you reviewed isaacs' code, right? did you discuss that?
11:01:11  <indutny>yes
11:01:16  <indutny>with him?
11:01:24  <bnoordhuis>yes?
11:01:27  <indutny>yes
11:01:35  <bnoordhuis>what was the conclusion?
11:01:57  <indutny>the conclusion is that landing streams is ok
11:02:04  <indutny>but probably not for http thing
11:02:08  <indutny>because it's not ready
11:02:23  <indutny>I mean landing in 0.9
11:02:24  <bnoordhuis>but it landed in master though, didn't it?
11:02:32  <indutny>http?
11:02:39  <indutny>oh
11:02:40  <indutny>I see now
11:02:43  <indutny>well...
11:02:46  <indutny>I dunno :)
11:02:48  <indutny>ask isaacs
11:02:57  <bnoordhuis>yeah, i guess i'll do that
11:03:35  <indutny>but code base looks pretty solid
11:03:48  <indutny>though resuming streams...
11:03:49  <indutny>idk
11:03:59  <indutny>it was working without it :)
11:04:14  <indutny>isaacs: we need some compatibility mode
11:04:20  <indutny>like for 'data' event
11:05:57  * felixgejoined
11:05:58  * felixgequit (Changing host)
11:05:58  * felixgejoined
11:06:16  <MI6>joyent/node: Ben Noordhuis v0.8 * 53b826e : install: fix openbsd man page location Man pages go into $PREFIX/man on - http://git.io/y6MOxw
11:14:05  * peterrsjoined
11:15:47  * piscisaureus_joined
11:16:13  <rendar>can libuv does synchronous i/o?
11:17:05  <piscisaureus_>morning
11:17:54  <piscisaureus_>rendar: only synchronous file io
11:18:00  <indutny>piscisaureus_: heya
11:18:14  <piscisaureus_>indutny: mornign
11:18:19  <piscisaureus_>indutny: how is it going?
11:19:22  <indutny>piscisaureus_: good good
11:19:30  <indutny>piscisaureus_: I wanted to ask you about exporting openssl symbols on osx
11:19:45  <indutny>piscisaureus_: what is the status of it?
11:19:46  <indutny>:)
11:20:02  <piscisaureus_>indutny: it already works on os x, because the os x linker doesn't strip
11:20:10  <indutny>hm...
11:20:16  <piscisaureus_>(atleast, not by default, and we never disabled it)
11:20:16  <indutny>it doesn't seem to work for me
11:20:28  <indutny>I've some symbols that are stripped
11:20:39  <piscisaureus_>which one?
11:21:01  <indutny>CRYPTO_set_add_lock_callback
11:21:54  <piscisaureus_>hmm
11:22:15  <piscisaureus_>maybe we define NO_LOCK somewhere?
11:24:14  <indutny>erm
11:24:19  <indutny>it is exported in libopenssl.a
11:28:22  <piscisaureus_>it should be in libeay really
11:28:57  <piscisaureus_>indutny: yeah, it seems that it *should* be exported. I don't know then, mac is a mystery to me :-p
11:29:05  <indutny>piscisaureus_: hahaha
11:29:12  <indutny>piscisaureus_: ok, I'll try to figure this out
11:29:16  <indutny>todah
11:29:26  <piscisaureus_>indutny: but according to tjfontaine an nathan os x doesn't strip so I assumed this was the case
11:29:42  <piscisaureus_>indutny: so if you find a linker flag that fixes it for you, please share :-)
11:30:53  * `3rdEden|BRBchanged nick to `3rdEden
11:32:22  <bnoordhuis>indutny, piscisaureus_: 02dffb0 build: enable DEAD_CODE_STRIPPING on OS X
11:32:30  <piscisaureus_>ah
11:32:37  <piscisaureus_>I think that should be reverted :-)
11:35:18  <rendar>piscisaureus_: i see, but with syncrhonous i/o the callback (e.g. on_read) will be called?
11:35:55  <piscisaureus_>rendar: well file i/o isn't streams
11:36:05  <piscisaureus_>rendar: e.g. for open files you have no on_read
11:36:20  <rendar>hmmm i see
11:36:30  <piscisaureus_>rendar: if you use uv_fs_read and specify NULL for a callback it will be blocking. And no, the callback won't be called :-p
11:38:02  <rendar>i'm following the example in http://nikhilm.github.com/uvbook/filesystem.html -- and i see this line: uv_fs_read(uv_default_loop(), &read_req, open_req.result, buffer, sizeof(buffer), -1, on_read); so (if we are dealing with synchronous i/o) 'on_read' will be called here before uv_fs_read() returns and from the _same_ thread, instead in asynch i/o 'on_read' will be called later from _another_
11:38:02  <rendar>thread and uv_fs_read will return immediately -- right?
11:38:16  <rendar>piscisaureus_: oh i see
11:38:25  <rendar>piscisaureus_: so blocking stuff doesn't have a callback?
11:39:47  <bnoordhuis>rendar: correct
11:40:44  <rendar>bnoordhuis: i see
11:40:58  <rendar>this is because calling a callback with synchronous i/o would be impossible?
11:41:21  <bnoordhuis>rendar: maybe not impossible but pointless
11:41:35  <rendar>bnoordhuis: yeah
11:42:09  <indutny>piscisaureus_: yeah, reverting fixed it
11:42:26  <rendar>bnoordhuis: and often in on_read callbacks there are other calls to uv_fs_read(), so having it with synchronous i/o would make infinite loops, i guess
11:42:39  <bnoordhuis>rendar: that too
11:45:34  <rendar>bnoordhuis: well, the only point it comes to my mind that would be useful, is using the same logic both for sync and async i/o, for example, i code a very long on_read and on_write functions, and i can reuse them also for synchronous i/o
11:46:25  <bnoordhuis>rendar: no one's stopping you from calling your on_read/on_write functions manually :)
11:47:56  <rendar>bnoordhuis: well yeah, of course..the important thing is calling them _after_ uv_fs_read returned (with sync i/o). i guess, right?
11:48:07  <bnoordhuis>rendar: yep
11:50:12  * bnoordhuisis off to lunch
11:50:20  <rendar>see ya
11:50:22  <rendar>:)
11:56:47  <indutny>piscisaureus_: mind pushing revert to the master?
11:56:57  <piscisaureus_>indutny: no, go ahead
11:57:01  <indutny>ok
11:57:20  <MI6>joyent/node: Fedor Indutny master * 46733e8 : Revert "build: enable DEAD_CODE_STRIPPING on OS X" This reverts commit 0 - http://git.io/Zo463Q
12:19:19  * abraxasjoined
12:20:24  * abraxasquit (Remote host closed the connection)
12:45:14  <indutny>bnoordhuis: yt?
12:47:39  * stagasquit (Ping timeout: 244 seconds)
12:48:45  * wolfeidaujoined
12:49:14  * toothrchanged nick to toothrot
12:58:33  * wolfeidauquit (Read error: Connection reset by peer)
12:58:45  * wolfeidaujoined
13:13:16  * sgallaghjoined
13:41:05  <bnoordhuis>indutny: yep
13:41:35  <bnoordhuis>indutny: when you revert a commit, explain why
13:58:45  * sgallaghquit (Remote host closed the connection)
14:02:15  * kazuponjoined
14:04:17  * bradleymeckjoined
14:21:08  * hzjoined
14:21:14  * peterrsquit (Read error: Connection reset by peer)
14:23:31  * peterrsjoined
14:29:23  * sgallaghjoined
14:34:40  <bnoordhuis>You're amongst +285 contributors who are not being employed by Nokia or Digia.
14:34:40  <bnoordhuis>In all fairness, a picture of you should be published too, recognising your
14:34:41  <bnoordhuis>effort making Qt 5 a reality.
14:34:49  <bnoordhuis>^ apparently i've contributed to qt...
14:35:06  <bnoordhuis>the thing is, i don't remember what or when
14:40:32  <tjfontaine>hm when I tested dead_code_stripping here none of the openssl stuff was dropped, I even checked the symbols that weren't used
14:43:49  <bnoordhuis>ab angers me, keep-alive is the default with http/1.1, don't force me to add a connection: keep-alive header >:(
14:46:53  * mjr_joined
15:13:05  <indutny>bnoordhuis: :)
15:13:30  <indutny>bnoordhuis: have you seen my EV_EOF pull request for libuv?
15:13:38  <indutny>bnoordhuis: it crashes for me on osx without it
15:13:40  <bnoordhuis>indutny: yes. i've even commented on it
15:13:47  <indutny>ah
15:13:50  <indutny>I see now
15:14:24  <indutny>bnoordhuis: so what I'm actually curious about
15:14:30  <indutny>is why is it happening
15:14:35  <indutny>I wasn't able to reproduce it...
15:14:46  <bnoordhuis>who reported the issue?
15:15:06  <indutny>bnoordhuis: I
15:15:16  <indutny>well, I wasn't able to reproduce it with test
15:15:20  <indutny>only when doing benchmarks
15:16:00  <bnoordhuis>that's kind of suspicious
15:16:39  <indutny>yes
15:16:44  <indutny>what's interesting
15:16:53  <indutny>is that abort() happening every time code enters this path
15:16:58  <indutny>I mean EV_DELETE
15:17:04  <indutny>both paths, actually
15:17:30  <bnoordhuis>so how can it happen?
15:17:51  <bnoordhuis>kevent() signals read or write readiness for a fd that we're monitoring
15:18:17  <bnoordhuis>but one that's been disabled, i.e. we stopped reading or writing
15:18:35  <bnoordhuis>that in itself is just business as usual
15:18:41  <indutny>yeah
15:18:55  <bnoordhuis>but when it tries to disarm the fd, it's gone
15:19:00  <indutny>indeed
15:19:11  <indutny>it's not just gone
15:19:11  <indutny>i.e. it's not closed
15:19:11  <indutny>or anything
15:19:19  <indutny>because otherwise we probably would get EBADF instead
15:19:20  <piscisaureus_>http://www.godhatestheworld.com/netherlands/filthymanneroflife.html
15:19:30  <indutny>btw "This reverts commit 02dffb063e423688557e2f8004eb817d7626bf41.
15:19:30  <indutny>DEAD_CODE_STRIPPING was stripping out CRYPTO_set_add_lock_callback
15:19:30  <indutny>symbol on which some addons were relying."
15:19:34  <indutny>bnoordhuis: ok? ^
15:19:44  <bnoordhuis>indutny: yes, something like that
15:19:54  <bnoordhuis>s/were/are/
15:20:07  <indutny>ok
15:20:14  <bnoordhuis>present time > past time in commit logs
15:20:28  <indutny>ok
15:20:31  <indutny>s/was/is/ then too
15:20:48  <bnoordhuis>piscisaureus_: intriguing website. paints a very balanced, nuanced image
15:21:00  <bnoordhuis>indutny: yes, indeed
15:21:10  <MI6>joyent/node: Fedor Indutny master * a3877ab : Revert "build: enable DEAD_CODE_STRIPPING on OS X" This reverts commit 0 - http://git.io/CxVChg
15:21:11  <piscisaureus_>hahaha
15:21:52  <piscisaureus_>bnoordhuis: http://www.godhatestheworld.com/netherlands/posterchildrenforsin.html <-- we should get them to add you to that list
15:22:36  <bnoordhuis>piscisaureus_: me? i'm as uncontroversial as it gets
15:23:44  <indutny>bnoordhuis: force pushed fix for ENOENT
15:23:44  <piscisaureus_><s><u>Ben Noorhuis</u></s> has made numerous code contributions to the filthyest application platform in the world. Something with fags and dykes.
15:23:46  <indutny>bnoordhuis: https://github.com/joyent/libuv/pull/656
15:23:52  <indutny>hahahah
15:25:03  <indutny>bnoordhuis: piscisaureus_ http://www.youtube.com/watch?v=24JL9T1PzhM
15:31:05  <indutny>silence
15:35:33  <bnoordhuis>the silence of people working
15:35:39  * kazuponquit (Remote host closed the connection)
15:36:01  <indutny>bnoordhuis: yeah, so about work
15:36:05  <indutny>bnoordhuis: ENOENT?
15:36:10  <indutny>can I land it?
15:36:40  * warzjoined
15:36:41  * warzquit (Changing host)
15:36:41  * warzjoined
15:36:41  <bnoordhuis>indutny: no. i'd like to understand _why_ it's failing with ENOENT
15:36:49  <indutny>great...
15:36:51  <indutny>ook
15:36:58  <indutny>I'll try to figure it out
15:37:20  <indutny>what if... fd was closed, but another fd with the same number was opened? :)
15:37:25  <indutny>somehow
15:37:41  <bnoordhuis>then the question becomes why was that fd closed?
15:37:52  <indutny>yeah
15:40:59  <bnoordhuis>indutny: as a general observation, trying to squelch an error, without understanding why the error happens in the first place, is _always_ the wrong thing to do
15:41:24  <bnoordhuis>it leads to bad, incomprehensible code
15:42:23  <indutny>ok
15:42:37  <indutny>generally, I agree
15:42:46  <indutny>good that you're the guy responsible for pulling such stuff
15:43:00  <indutny>because you don't like pulling such stuff
15:43:28  <bnoordhuis>indeed :)
15:50:28  * c4milojoined
15:52:20  * kazuponjoined
15:52:24  <isaacs>bnoordhuis: there is no way to both "not lose data events" and also "end comes no matter what you do"
15:52:59  <bnoordhuis>isaacs: this is about streams2, right?
15:53:03  <isaacs>bnoordhuis: yeah
15:53:04  * c4milo_joined
15:53:14  * AvianFlujoined
15:53:35  <isaacs>bnoordhuis: if you ever add a data event, or call pause() or resume(), then it works exactly like it used to, except that no data is lost, and pause() works.
15:53:40  * c4miloquit (Read error: Connection reset by peer)
15:53:46  <isaacs>which is universally an improvement
15:53:52  <bnoordhuis>isaacs: yep, that much i understood
15:54:02  <isaacs>so... there is going to be a case where code changes are needed.
15:54:10  <isaacs>if you listen to end, but never consume data.
15:54:34  <bnoordhuis>what's the benefit over streams2 over the old implementation?
15:54:48  <bnoordhuis>i probably missed the original discussion
15:55:02  <isaacs>https://dl.dropbox.com/u/3685/presentations/streams2/streams2-ko.pdf
15:55:30  <isaacs>bnoordhuis: the discussion has been going on since 0.4
15:56:12  <isaacs>the benefits are that it's simpler now to write programs that consume data, and simpler to write programs that implement streams
15:57:07  <isaacs>bnoordhuis: i've never seen a program that listens to end, but doesn't consume data, and isn't a test.
15:57:51  <bnoordhuis>isaacs: fire-and-forget
15:57:51  * jmar777joined
15:58:00  <bnoordhuis>i've written programs that work like that
15:58:14  <bnoordhuis>(and i've a bit of deja vu so we've probably talked about this)
15:58:34  <isaacs>bnoordhuis: if you don't listen to the response event on a request, and don't consume the request data before the response ends in a server, then it'll resume() it for it
15:58:45  <isaacs>ie, if there's no way you could have possibly consumed the data, with http/https
15:59:16  <isaacs>so, http fire-and-forget shoudl still work fine. the problem is that the tests fire-and-forget-but-also-remember
16:05:06  <isaacs>bnoordhuis: any idea why 2433ec8 would drop benchmark/net-pipe.js performance to about 1/4?
16:05:17  <isaacs>er, more like 1/3, i guess
16:05:33  <bnoordhuis>isaacs: on solaris?
16:05:38  <isaacs>solaris and os x
16:06:33  <bnoordhuis>don't know and don't care about os x
16:06:41  <isaacs>there's a drop on ubuntu as well, but it's less significant
16:06:42  <bnoordhuis>but regarding solaris, what kernel and what arch?
16:06:56  <bnoordhuis>yeah, you mentioned ubuntu. for me it's actually a little faster
16:07:07  <bnoordhuis>what arch was that?
16:07:08  <isaacs>benchmark/net-pipe.js is faster?
16:07:23  <isaacs>it was x64 for ubuntu, 32 for smartos
16:07:33  <isaacs>SunOS 828cbda3-e3d6-4744-86bf-8ad6733dcf15.local 5.11 joyent_20120424T232010Z i86pc i386 i86pc Solaris
16:07:44  <isaacs>node: ELF 32-bit LSB executable 80386 Version 1 [FPU], dynamically linked, not stripped
16:08:24  <bnoordhuis>ah no, it's a little slower
16:08:34  <isaacs>x64 is also slower on smartos
16:08:45  <isaacs>like from 3.1 to 1.0
16:09:05  <isaacs>reverting the mmap stuff brings the speed right back
16:09:26  <isaacs>what's that actually for, anyway? is there another way we can fix 4283?
16:09:46  <isaacs>or at least, only do it on Linux, and not on other os's?
16:10:27  <bnoordhuis>the issue - and it probably exists on other platforms as well - is that we alloc buffers in such a way that the memory can't be returned to the os
16:10:54  <bnoordhuis>those memory leaks lots of people complain about? that's because buffers allocated with brk() usually can't be un-brk()'d again
16:11:45  <isaacs>hmm
16:13:02  <isaacs>we should bring this up to the joyent folks and ask them what is the best way to do this on smartos
16:13:20  <isaacs>because using mmap in this way is way too slow
16:13:27  <bnoordhuis>it's probably because mmap performance sucks on solaris
16:13:38  <isaacs>but i hear you, we have to fix the memory leaks
16:13:45  <bnoordhuis>i hadn't thought of it before but when you brought it up i remembered that lots of people complain about it :)
16:14:00  <`3rdEden>like me ;)
16:14:12  <isaacs>bnoordhuis: do you have a test that demonstrates the memory leakings?
16:14:23  <bnoordhuis>yes, there's one in the issue
16:14:27  <isaacs>ok
16:14:31  <bnoordhuis>btw, it's not a straight-up memory leak
16:14:39  <isaacs>right, it's not a js memleak
16:14:48  <bnoordhuis>no, and not a c/c++ memleak either
16:14:49  <isaacs>it's more subtle, as i'm understanding it?
16:14:53  <isaacs>right
16:14:58  <bnoordhuis>the memory is reusable by the malloc() implementation
16:14:59  <isaacs>we alloc and when you free() it doesn't actually free()?
16:15:05  <isaacs>oh, ok
16:15:06  <bnoordhuis>right, it doesn't go back to the os
16:15:10  <isaacs>well, that's good, though, right?
16:15:16  <isaacs>i mean, you used that memory before, yo umight again
16:15:17  <bnoordhuis>yes... but then again not
16:15:38  <bnoordhuis>it's okay when your app does small allocations
16:15:48  <bnoordhuis>because then the memory gets reused
16:16:11  <bnoordhuis>if your app does a lot of small allocations at start-up and then only does large allocations from then on
16:16:17  <bnoordhuis>that memory is _not_ reused
16:16:26  <bnoordhuis>which means it sits unused until the os swaps it out
16:16:29  <bnoordhuis>which is wasteful
16:16:45  <isaacs>hm
16:16:51  <bnoordhuis>it's a fairly subtle issue
16:17:14  * jmar777quit (Remote host closed the connection)
16:17:18  <isaacs>benchmark/net-pipe.js is doing one big alloc followed by lots of medium-sized ones
16:17:37  <bnoordhuis>i'm not sure how much faith to put in that benchmark
16:17:44  * `3rdEdenchanged nick to `3E|DINNER
16:17:45  <bnoordhuis>i get maybe 6.5 gbit throughput
16:17:50  * jmar777joined
16:17:53  <bnoordhuis>while my system can easily do 60 gbit/s
16:18:21  <bnoordhuis>mmap is probably one factor but there must be more
16:18:26  <isaacs>probably
16:18:52  <isaacs>streams1 are slightly faster still, but then there's also all the usability problems. as long as it ends up comparable, i'm happy.
16:18:58  <isaacs>but we should also work on making it faster overall.
16:18:58  * stagasjoined
16:19:33  <isaacs>here's a flamegraph of it: http://static.izs.me/streams2-throughput-flamegraph3.svg
16:19:36  <isaacs>(scroll down)
16:20:10  <isaacs>oh, a bunch of the laggy bits there have already been removed.
16:20:14  <isaacs>i need to make a new one, that's out of date
16:22:18  * jmar777quit (Ping timeout: 264 seconds)
16:22:30  <MI6>joyent/node: Nicolas Chambrier v0.8 * 496c0bd : doc: add Google+ French community - http://git.io/dSA9MA
16:24:52  <isaacs>bnoordhuis: thanks
16:25:17  <isaacs>merge machine :)
16:26:14  <indutny>bnoordhuis: ok, I can definitely say that that fd wasn't closed
16:26:25  <isaacs>we *really* need ci benchmarks
16:26:27  <isaacs>holy crap
16:26:54  <bnoordhuis>we used to have buildbots. they were nice
16:28:15  <indutny>aha
16:28:21  <indutny>so looks like my theory is correct
16:28:37  <indutny>it's closed and then open() returns same fd
16:28:39  * indexzerojoined
16:29:05  <bnoordhuis>indutny: close() should remove the fd from the kqueue set
16:29:16  <bnoordhuis>i say 'should' because it wouldn't surprise me if that's broken on os x
16:29:20  <indutny>hahahaa
16:30:15  <indutny>oh
16:30:18  <indutny>looks like it indeed does
16:30:22  <indutny>hm....
16:30:27  <indutny>why is it happening then
16:30:32  <indutny>time to dig into the xnu
16:31:45  <isaacs>here's what it looks like on the latest master: http://static.izs.me/stacks-a3877ab53.svg
16:35:14  <indutny>bnoordhuis: https://svn.torproject.org/svn/libevent-urz/trunk/kqueue.c
16:35:16  <indutny>just FYI
16:35:19  <indutny>search for ENOENT
16:35:38  <indutny>I can confirm myself that close() removes fd from kqueu
16:35:48  <indutny>but lets try opening socket with the same fd...
16:36:40  <indutny>no
16:36:44  <indutny>it still works as expected
16:38:08  <MI6>joyent/node: Dean McNamee master * 1c265c5 : typed arrays: fix missing type in SizeOfArrayElementForType() When Mikae (+1 more commits) - http://git.io/aAdOYA
16:38:57  * bnoordhuisis off to dinner
16:42:06  * stagasquit (Ping timeout: 250 seconds)
16:43:15  <isaacs>bnoordhuis: aha, the benchmark/net-pipe.js should be about half of benchmark/throughput.js, since you're measuring it in two directions.
16:43:30  <isaacs>yeah, this mmap stuff is not good.
16:43:42  <indutny>bnoordhuis: handle also ENOENT (along EBADF) when kevent fails due to errors
16:43:42  <indutny>in the changelist. ENOENT can be returned in the following valid
16:43:42  <indutny>scenario: fd scheduled for delayed removal from the watched fd
16:43:42  <indutny>list, fd closed (which automatically removes the fd from the
16:43:42  <indutny>kqueue watched list), new opened fd which gets the same number,
16:43:42  <indutny>delayed changes applied (kevent()).
16:57:29  * bradleymeckquit (Quit: bradleymeck)
17:02:42  * jmar777joined
17:04:15  <indutny>bnoordhuis: oooh shit
17:04:23  <indutny>bnoordhuis: I think I'm getting closer to it
17:04:34  <indutny>bnoordhuis: so I place watcher pointer in kevent->udata
17:04:42  <indutny>bnoordhuis: and there two interesting facts that I found
17:04:49  <indutny>1) sometimes udata == NULL
17:05:02  <indutny>2) sometimes udata != loop->watchers[ident]
17:05:09  <indutny>and when it != - it fails
17:08:57  <indutny>I need to think about it...
17:12:17  <indutny>ah
17:12:17  * bradleymeckjoined
17:12:24  <indutny>so sometimes watchers[fd] is NULL
17:12:28  <indutny>and fd is still in kqueue
17:12:31  <indutny>ohhh
17:12:33  <indutny>shit came real :)
17:16:03  * kazuponquit (Remote host closed the connection)
17:21:31  * dapjoined
17:25:35  <indutny>bnoordhuis: what's really interesting...
17:25:45  <indutny>bnoordhuis: is that I'm sure that close() was called on that fd
17:25:49  <indutny>I've debugged it
17:26:00  <indutny>looks like some wierd osx stuff to me
17:26:03  <indutny>probably, not only os
17:26:04  <indutny>osx
17:26:45  <indutny>oh
17:26:47  <indutny>I have an idea
17:27:30  * kazuponjoined
17:28:47  <indutny>bnoordhuis: imagine following scenario
17:28:53  <MI6>joyent/node: isaacs created branch streams2-netwrite - http://git.io/A8N_kA
17:28:57  <indutny>hm...
17:29:03  <indutny>no, probably that's not the case...
17:29:06  <isaacs>piscisaureus_, TooTallNate, indutny, review plz? ^
17:29:20  <indutny>isaacs: it's too late :)
17:29:20  <isaacs>and bnoordhuis when you get back from dinner :)
17:29:24  <indutny>I better og sleeping
17:29:27  <isaacs>indutny: ok
17:30:03  <indutny>generally lgtm
17:30:08  <indutny>but didn't look in details
17:30:31  <indutny>bnoordhuis: so I think that somehow during uv__stream_io callback, fd is closed and reopened
17:30:59  <indutny>and queue from which uv__stream_io was called contains this fd later on
17:31:13  <indutny>I'll narrow this down tomorrow
17:31:29  <indutny>but I think it should be pretty clear to you know
17:33:56  * EhevuTovjoined
17:42:02  * `3E|DINNERchanged nick to `3rdEden
17:49:16  <indutny>bnoordhuis: nailed it down https://github.com/joyent/libuv/pull/656
17:49:17  <indutny>:)
17:49:19  <indutny>ttyl everyone
17:49:22  * indutnysigns off
17:50:30  * warz_joined
17:51:22  * warzquit (Ping timeout: 252 seconds)
17:56:13  * c4milo_quit (Remote host closed the connection)
17:57:02  * peterrsquit (Read error: Connection reset by peer)
17:57:20  * peterrsjoined
17:59:04  * jmar777quit (Remote host closed the connection)
17:59:10  * c4milojoined
18:00:44  * peterrsquit (Client Quit)
18:01:51  <piscisaureus_>bnoordhuis: ?
18:02:17  * isaacsis going to revert the mmap thing
18:02:25  <isaacs>increasing RSS in this scenario is expected behavior.
18:05:38  * EhevuTovquit (Quit: This computer has gone to sleep)
18:14:00  * c4miloquit (Remote host closed the connection)
18:17:30  * lohkeyjoined
18:18:52  * EhevuTovjoined
18:18:59  * c4milojoined
18:24:01  <bnoordhuis>piscisaureus_: what's up?
18:24:10  * bradleymeckquit (Quit: bradleymeck)
18:24:16  <piscisaureus_>bnoordhuis: we're having a call
18:24:25  <bnoordhuis>piscisaureus_: as in now?
18:24:36  <piscisaureus_>bnoordhuis: yep
18:24:40  <bnoordhuis>ah, _now_ the reminder email comes in...
18:24:51  <bnoordhuis>okay, joining
18:24:55  * `3rdEdenquit (Remote host closed the connection)
18:26:12  * c4miloquit (Remote host closed the connection)
18:26:54  * c4milojoined
18:30:24  * TooTallNatejoined
18:34:19  * `3rdEdenjoined
18:46:25  <isaacs>bnoordhuis: chatting with some smartos people about a solution for https://github.com/joyent/node/issues/4283
18:46:36  * felixgequit (Quit: felixge)
18:46:39  <isaacs>bnoordhuis: but the mmap approach can not be made to work without huge costs.
18:46:48  * sblomjoined
18:48:07  <MI6>joyent/node: isaacs master * 6c5356b : Revert "buffer: allocate memory with mmap()" Also Revert "buffer: use MA - http://git.io/XSvaDQ
18:48:15  * kazuponquit (Remote host closed the connection)
18:50:12  <isaacs>sblom, piscisaureus_: Do I need to install something to build the etw stuff? https://gist.github.com/4320822
18:50:47  <sblom>@isaacs: I didn't think so--I'm looking into that GIST now.
18:52:02  <isaacs>sblom: building with noetw noperfcounter works fine
18:52:21  <isaacs>er, noperfctr
18:52:39  <isaacs>maybe i have to install etw and perfcounters?
18:53:26  * AvianFluchanged nick to whatever
18:53:31  * whateverchanged nick to AvianFlu
18:55:33  <sblom>isaacs: Those are both about as built in to Windows as it gets. Lemme see if I can reproduce the build failure. Just building latest v0.8 branch?
18:55:52  <sblom>I'm surprised this is happening in 0.8--I thought the new code was only in master.
18:56:00  <sblom>But I could be crazy.
18:56:13  <bnoordhuis>isaacs: that's pretty lame. it means i'll be fielding "rss leak" issues for years to come :/
18:56:36  <isaacs>sblom: master
18:56:41  <isaacs>sblom: sorry, didn't specify :)
18:56:59  <isaacs>bnoordhuis: well, we can probably work out a way to re-use the memory we've claimed with brk()
18:57:06  <isaacs>bnoordhuis: or punt the issue upstream to the operating systems.
18:57:22  <isaacs>bnoordhuis: (which is what i've just done in the joyent jabber channel)
18:57:41  <bnoordhuis>i'm less optimistic
18:57:42  <sblom>isaacs: Okay--cool. I'll take a look.
18:57:56  <sblom>I'll be away for about an hour, but more news first thing this afternoon.
18:57:57  <bnoordhuis>this particular issue has existed for probably close to 10 years in glibc
18:58:09  <isaacs>bnoordhuis: if we dynamically increased the buffer.poolSize, then that might help things, or if we re-used the SlowBuffer memory when SlowBuffers give them up.
18:58:12  <bnoordhuis>and it's not really a bug either, just a pathological edge case
18:58:18  <isaacs>bnoordhuis: it's a bug in the user.
18:58:24  <isaacs>bnoordhuis: they don't understand how rss works.
18:59:00  <isaacs>but every program in the world has this "bug", because performance generally matters much much more than heap fragmentation or size.
18:59:15  <isaacs>the cure is worse than the disease in this case.
18:59:58  * EhevuTovquit (Read error: Connection timed out)
19:02:01  <piscisaureus_>isaacs: it could be that you need a more expensive version of visual studio to build it. Do you have an msdn subscription? If not, claudio can hook you up with one.
19:02:17  <isaacs>piscisaureus_: aha, that could be it. i think i have express.
19:02:31  <isaacs>claudio gave me some carte blanche msdn access once upon a time.
19:02:39  <isaacs>i'll see if i can dig it up
19:02:47  <piscisaureus_>sblom: I am not thrilled that compiling node now no longer works on express. Is there any way we could detect this and print a warning / build without ETW ?
19:08:51  <isaacs>some child proc regressions on streams2 in glassland
19:09:35  <isaacs>also many windows tests fail when run in a path that has spaces in it
19:09:48  <isaacs>which would not be a worthwhile concern on unix, but windows users generally always have a space in the path somewhere.
19:11:47  * EhevuTovjoined
19:13:05  <MI6>joyent/node: bnoordhuis created branch mmap-reloaded - http://git.io/YnDEFw
19:13:06  * brsonjoined
19:13:20  <bnoordhuis>isaacs: ^ does that work for you?
19:14:09  <piscisaureus_>bnoordhuis: maybe check for >= kPoolSize instead of == kPoolSize
19:14:30  * mikealjoined
19:14:42  <piscisaureus_>ceterum censeo that smartos should just get its shit together :-p
19:15:00  <bnoordhuis>piscisaureus_: yeah, i probably should. this is just a quick fix to fix the benchmark/net-pipe regression
19:15:28  <isaacs>bnoordhuis: that's not so bad.
19:18:02  <bnoordhuis>an interesting observation is that the stream slab allocator clashes with mmap-ed buffers
19:18:17  <bnoordhuis>i.e. performance drops when slabs are mmap-ed instead of new'd
19:20:07  <bnoordhuis>which makes sense, i guess, because mmap'ing large chunks is more expensive than smaller chunks
19:20:15  <isaacs>bnoordhuis: actually it gets slightly faster in mmap-reloaded over master.
19:20:26  <bnoordhuis>yeah, i noticed that too :)
19:20:45  <isaacs>bnoordhuis: how's linux?
19:20:55  * EhevuTovquit (Quit: This computer has gone to sleep)
19:21:01  <isaacs>(really, os x performance is important, but not that much. linux, smartos, and windows are everything)
19:21:02  <bnoordhuis>about the same now
19:21:26  <isaacs>darwin perf is only important for dumb blog post marketing bullshit benchmarks that people run on their laptops :)
19:21:38  <isaacs>(like the kind on blog.nodejs.org)
19:22:25  <bnoordhuis>no mmap: 6.3664 Gbits/sec (6833569792 bits / 999669522 ns)
19:22:26  <bnoordhuis>6.3386 Gbits/sec (6803161088 bits / 999585177 ns)
19:22:26  <bnoordhuis>6.3497 Gbits/sec (6816792576 bits / 999829549 ns)
19:22:26  <bnoordhuis>6.3919 Gbits/sec (6860832768 bits / 999649879 ns)
19:22:37  <bnoordhuis>with mmap-reloaded: 6.4184 Gbits/sec (6888095744 bits / 999482968 ns)
19:22:37  <bnoordhuis>6.4162 Gbits/sec (6887047168 bits / 999669034 ns)
19:22:37  <bnoordhuis>6.4390 Gbits/sec (6911164416 bits / 999608860 ns)
19:22:37  <bnoordhuis>6.4126 Gbits/sec (6883901440 bits / 999769415 ns)
19:23:09  <bnoordhuis>statistically not that significant but it seems to be consistently faster
19:23:28  <isaacs>yes
19:23:35  <isaacs>that's statistically significant.
19:23:45  <isaacs>just did 3 runs, it's consistent and measurable
19:23:54  <isaacs>also, your computer is faster than mine. i'm jealous.
19:24:21  <bnoordhuis>that's not the only thing that's bigger
19:24:24  <bnoordhuis>oh, faster you say?
19:24:39  <bnoordhuis>actually, those numbers are lower than i expect
19:25:04  <bnoordhuis>the numbers with v0.8 are better
19:25:15  <bnoordhuis>to wit: 7.8203 Gbits/sec (8027373488 bits / 955978136 ns)
19:25:15  <bnoordhuis>7.6639 Gbits/sec (8237613056 bits / 1001040111 ns)
19:25:15  <bnoordhuis>7.6723 Gbits/sec (8237613056 bits / 999939702 ns)
19:25:15  <bnoordhuis>7.6193 Gbits/sec (8187281408 bits / 1000750002 ns)
19:25:47  <bnoordhuis>something to look into
19:26:02  <isaacs>yes.
19:26:07  <isaacs>v0.8 is comparable with master-pre-streams2
19:26:21  <isaacs>there's still some strictly js stuff to work out
19:26:27  <isaacs>but that's all just a matter of profiling and experimenting
19:26:33  * isaacs's job for January
19:26:50  <isaacs>the last 10% is polish. but when there's a 70% drop, that's something worth fixing right away.
19:27:00  <bnoordhuis>hm, i wish you'd squashed instead of merged it. kind of hard to backtrack now
19:27:12  <bnoordhuis>oh well, too late now
19:27:29  <isaacs>also, sweet zombie jesus, http-simple is slow as donkeys on smartos.
19:27:31  <isaacs>wtf
19:27:48  * warz_quit
19:28:47  <isaacs>it hasn't regressed since 0.8, so maybe this is just some kind of "ab sucks on smartos" thing
19:28:50  <isaacs>Requests per second: 2725.89 [#/sec] (mean)
19:29:02  <isaacs>(on darwin)
19:29:27  <isaacs>with streams2 on smartos: Requests per second: 702.02 [#/sec] (mean)
19:29:40  <isaacs>pre-streams2 on smartos: Requests per second: 530.95 [#/sec] (mean)
19:29:53  <isaacs>that's terrible.
19:30:48  <isaacs>bnoordhuis: it's easy to revert a merge commit.
19:31:32  <isaacs>and since it's just one big --no-ff (which would otherwise have been a ff) bisecting over it still works as you'd expect.
19:31:53  <bnoordhuis>okay, good
19:33:55  <isaacs>bnoordhuis: i wanted a rip-cord we could pull if it turned out to be a giant waste of time :)
19:34:06  <isaacs>though, that would suck a lot at this point, of course.
19:34:21  <isaacs>piscisaureus_: I'm seeing pre-streams2 master with this: [02:49|% 100|+ 470|- 14]: Done
19:34:28  <isaacs>piscisaureus_: does that gel with your understanding of the world?
19:34:42  <isaacs>piscisaureus_: if so, i'm going to start digging into any regressions.
19:34:48  <piscisaureus_>man, it's been ages since I last run tests
19:34:50  <piscisaureus_>let's see
19:35:02  <isaacs>piscisaureus_: something is busted in child_process, so that'll be my first target.
19:37:50  * isaacsaway for about 2 hours
19:42:35  * EhevuTovjoined
19:42:58  * c4miloquit (Remote host closed the connection)
19:52:57  * warzjoined
19:52:57  * warzquit (Changing host)
19:52:57  * warzjoined
19:56:48  * joshthecoderjoined
20:01:46  <bnoordhuis>yet another interesting observation: prepopulating the page tables with mmap(MAP_POPULATE) gives a 8-10% speedup
20:02:12  <bnoordhuis>those pagefaults are expensive
20:02:34  <bnoordhuis>it probably hurts in low memory situations though
20:18:48  * kazuponjoined
20:20:30  <MI6>joyent/node: Ben Noordhuis mmap-reloaded * 18e5259 : buffer: prefault mmap'ed memory - http://git.io/S0Qpow
20:28:20  * kazuponquit (Ping timeout: 256 seconds)
20:29:37  <sblom>isaacs: This etw build error is starting to come back to me. The idea was that https://github.com/joyent/node/commit/953b049a89f803b0a152e776b8590e78ada180e4 fixed it. But if you're building master and still seeing it, I must not've fixed it as well as I thought.
20:34:37  * c4milojoined
20:36:57  * piscisaureus_quit (Quit: ~ Trillian Astra - www.trillian.im ~)
20:38:15  * piscisaureus_joined
20:45:37  <sblom>piscisaureus_: Do we really no longer build with Express? That build error that isaacs pointed out is due to a missing directory. If you build a second time it should just work (which is also dumb, and I think I know how to fix it).
20:46:18  <piscisaureus_>sblom: oh, right. I was just guessing to be honest, and it appears I guessed wrong.
20:46:25  <piscisaureus_>sblom: I never had that error though...
20:46:52  <piscisaureus_>isaacs: btw, "make test-all" does [39:32|% 100|+ 501|- 47]: Done for me
20:48:11  * EhevuTov_joined
20:48:15  <piscisaureus_>isaacs: so that means that without my parental oversight, "you guys" fuck up the windows build quite badly :-p
20:49:54  <bnoordhuis>piscisaureus_: what tests are failing?
20:51:20  * EhevuTovquit (Ping timeout: 255 seconds)
20:52:43  * benoitcquit (Excess Flood)
20:52:51  <bnoordhuis>before: 175,368 page-faults, after: 7,971 page-faults
20:54:12  * benoitcjoined
20:55:49  <MI6>joyent/node: Ben Noordhuis mmap-reloaded * 7bbbbf7 : buffer: bump kPoolSize, reduce pagefaults by 90% (+1 more commits) - http://git.io/ziFdRA
20:59:16  <sblom>isaacs, piscisaureus_: Looks to me like virgin builds succeed on both master and v0.8 when 953b049a is present, and fail when it's not. I think we just need to back-port it to v0.8 and you should be able to build cleanly.
20:59:37  <sblom>I'm double-checking that our Express build works because I admit that I haven't tried it in a loooong time.
21:00:27  <bnoordhuis>piscisaureus_: did you just get a ton of trac emails?
21:00:35  <bnoordhuis>from something called cloudmosa.com?
21:00:53  <piscisaureus_>bnoordhuis: no?
21:00:56  <piscisaureus_>bnoordhuis: which address?
21:01:05  <piscisaureus_>c9 / sl / private
21:01:06  <piscisaureus_>?
21:01:18  <bnoordhuis>piscisaureus_: my bnoordhuis.nl email address
21:01:39  <bnoordhuis>e.g. Trac Ticket #443: http://apps.facebook.com/battlepirates/ has no screen
21:01:52  <bnoordhuis>which is then signed off with a libuv commit i did...
21:01:57  * sblomaway for lunch.
21:02:03  <bnoordhuis>i guess someone didn't set up his bug tracker right
21:17:00  * sgallaghquit (Remote host closed the connection)
21:24:15  * kazuponjoined
21:28:44  * kazuponquit (Ping timeout: 265 seconds)
21:31:21  * nrajlichjoined
21:32:36  * TooTallNatequit (Read error: Connection reset by peer)
21:32:36  * nrajlichchanged nick to TooTallNate
21:34:35  * wolfeidauquit (Remote host closed the connection)
21:39:57  * stagasjoined
21:44:53  * bnoordhuisquit (Ping timeout: 255 seconds)
21:51:15  * `3rdEdenquit (Quit: Zzzz)
21:52:24  * brsonquit (Ping timeout: 264 seconds)
21:54:00  * brsonjoined
22:00:33  <piscisaureus_>bnoordhuis: I think it's just spam
22:08:25  * warzquit
22:12:53  * brson_joined
22:13:31  * brsonquit (Quit: leaving)
22:21:49  * wolfeidaujoined
22:24:38  * kazuponjoined
22:25:35  * rendarquit
22:28:59  * kazuponquit (Ping timeout: 255 seconds)
22:30:22  * wolfeidauquit (Remote host closed the connection)
22:35:44  * isaacsback
22:39:06  <isaacs>sblom: on master, i get that failure over and over again
22:39:09  <isaacs>sblom: it's not just the first build
22:39:20  <isaacs>'ctrpp' is not recognized as an internal or external command,
22:39:21  <isaacs>operable program or batch file.
22:39:25  <isaacs>sblom: ^
22:39:48  <sblom>Ohhh. I think we're talking about two different issues. I'm talking about this one: https://gist.github.com/4277559
22:40:09  <sblom>isaacs: ^
22:40:13  <isaacs>yeah
22:40:27  <sblom>Sorry about that.
22:40:46  <sblom>Okay--_now_ I'm looking at the right gist, and will let you know what I find out.
22:40:52  <isaacs>sblom: yeah, that's a different thing
22:41:06  <isaacs>sblom: mildly annoying in v0.8, we should probably backport as you say, but not a significant blocker, really
22:41:21  <sblom>isaacs: Okay--will know more shortly.
22:41:45  <isaacs>sblom: i'd like to get perfctr and etw working built in 0.9.4. i know it can do it somehow, but i think my vs express is not doing the job
22:41:59  <isaacs>k thanks
22:43:09  <sblom>isaacs: Do you have \Program Files (x86)\Microsoft SDKs\Windows\v7.0A
22:43:23  <sblom>Might be as simple as needing to add something to your path if you have that.
22:43:25  <isaacs>ircretary: tell bnoordhuis WOW, net-pipe is WAY faster on mmap-reloaded!
22:43:25  <ircretary>isaacs: I'll be sure to tell bnoordhuis
22:43:38  <isaacs>ircretary: tell bnoordhuis (on darwin, about to check smartos)
22:43:38  <ircretary>isaacs: I'll be sure to tell bnoordhuis
22:44:23  <sblom>isaacs: also, what version of VS Express are you running? 2010?
22:44:37  <isaacs>sblom: https://gist.github.com/4323116
22:44:42  <isaacs>sblom: (yes, that path is there)
22:45:01  <isaacs>yes, 2010
22:45:02  <sblom>Yay. Is that path with \Bin in your PATH
22:45:25  <sblom>actually, even better. Is ctrpp.exe in \Program Files (x86)\Microsoft SDKs\Windows\v7.0A\Bin
22:45:26  <isaacs>yes
22:45:40  <isaacs>(it's in the path)
22:46:13  <isaacs>but no ctrpp.exe in there
22:46:14  <isaacs>https://gist.github.com/4323125
22:48:40  <sblom>isaacs: Okay--I'll figure out what the simplest way is to get ctrpp installed.
22:48:44  <sblom>stand by
22:48:55  <sblom>(or not)
22:50:20  * loladiroquit (Quit: loladiro)
22:51:09  <isaacs>k :)
22:51:38  <isaacs>ircretary: tell bnoordhuis net-pipe.js on mmap-reloaded is faster on smartos as well, but not by nearly as large a diff
22:51:39  <ircretary>isaacs: I'll be sure to tell bnoordhuis
22:53:56  * wolfeidaujoined
22:55:38  <isaacs>TooTallNate: you around?
22:55:56  <TooTallNate>isaacs: sup?
22:56:11  <isaacs>TooTallNate: can you take a look at https://github.com/joyent/node/commit/3b949d19b2569281e6e2072fd8e90fc4b0aa7dda
22:56:31  <TooTallNate>isaacs: nice, ok i'll check it out
22:56:35  <isaacs>piscisaureus_: your input would be wonderful as well if you're around ^
22:56:42  <isaacs>piscisaureus_: this is that "sync writes should be sync" thing
22:58:01  <TooTallNate>isaacs: so is that basically the same patch as the other day?
22:58:05  <TooTallNate>looks very similar
22:58:13  <isaacs>TooTallNate: it's exactly that patch, plus a test.
22:58:14  <piscisaureus_>isaacs: why does net.Socket._write call cb() synchronously?
22:58:24  <TooTallNate>kewl
22:58:26  <isaacs>piscisaureus_: that's not the cb() that the user passes into write()
22:58:28  <TooTallNate>piscisaureus_: stderr
22:58:39  * EhevuTov__joined
22:58:43  <piscisaureus_>ah, ok
22:58:47  <isaacs>piscisaureus_: it calls synchronously because the buffer was entirely flushed so it can write more right now if possible.
22:59:13  <isaacs>piscisaureus_: this pulls out the "always wait for req.oncomplete" fuckup that i was doing before.
22:59:24  <piscisaureus_>ah, right
22:59:26  <piscisaureus_>:-)
22:59:27  <piscisaureus_>nice
23:00:11  <isaacs>piscisaureus_: i started down the path of always just creating a new writeReq whether i needed to buffer or not, but it'sa lot of complexity to get the water marks exactly right, and it doesn't actually save anything.
23:00:28  <isaacs>piscisaureus_: you end up with exactly the same number of bytes being buffered either way, just a matter of where.
23:00:42  <TooTallNate>isaacs: what branch is that?
23:00:49  <isaacs>and the same writeReq handling that we had before.
23:00:55  <piscisaureus_>isaacs: so - do you flush to libuv early or late now?
23:00:59  <isaacs>TooTallNate: streams2-netwrite
23:01:30  <isaacs>piscisaureus_: i flush to libuv until the writeQueueLength is > -
23:01:33  <isaacs>er, >0
23:01:56  * EhevuTov_quit (Ping timeout: 252 seconds)
23:02:02  <piscisaureus_>ah I see. hmm.
23:02:39  <isaacs>piscisaureus_: what i'd like to do is shove *everything* into libuv right away, and communicate back to the user using the writeQueueLength instead of _writableState.length
23:02:50  <isaacs>piscisaureus_: but then it's trickier to know when to call the user's cb, emit drain, etc.
23:02:51  * stagas_joined
23:02:53  <isaacs>things get hairier
23:03:29  <isaacs>positively hirsute
23:03:55  <sblom>isaacs: Do you have any other directories in \Program Files (x86)\Microsoft SDKs\Windows?
23:04:18  * stagasquit (Ping timeout: 264 seconds)
23:04:27  <piscisaureus_>isaacs: ok, I can't see any correctness problems with the patch. But you know all the edge cases now, not me.
23:04:53  <piscisaureus_>isaacs: benchmarks will show whether it really looks good to me or not :-)
23:05:51  <isaacs>piscisaureus_: it makes the net stuff a tiny bit faster, but not significantly
23:05:59  <isaacs>and any benchmark is usually pushing bytes fast enough that it doesn't matter.
23:06:08  <isaacs>this is only more efficient in the case of many small writes
23:06:12  * stagasjoined
23:06:31  * wolfeidauquit (Remote host closed the connection)
23:06:36  <piscisaureus_>isaacs: it is really much more complex than that :-)
23:06:42  <txdv>hey girls
23:06:49  * TooTallNatequit (Quit: Computer has gone to sleep.)
23:07:16  * TooTallNatejoined
23:07:26  * wolfeidaujoined
23:07:41  <CoverSlide>hay guurrlll!
23:07:43  <piscisaureus_>isaacs: it depends on whether nagle is on or off, whether you are writing strings or buffers, whether buffers contain unicode or not, etc. Show me a benchmark where it is faster and I'll make one where it is slower.
23:07:55  <piscisaureus_>isaacs: so let's not worry too much now
23:08:04  <piscisaureus_>(I mean, I should not .... )
23:08:26  <isaacs>piscisaureus_: yeah, the point is, it's probably not really a significant perf issue
23:08:34  <isaacs>and if it is, whatver, later.
23:08:37  <isaacs>k, landing it
23:08:38  * stagas_quit (Ping timeout: 250 seconds)
23:08:43  * loladirojoined
23:09:31  <MI6>joyent/node: isaacs master * 82c7c84 : net: Handle sync writable streams synchronously This fixes the case wher - http://git.io/SF_DwQ
23:09:41  <isaacs>we have too many tests that use test/fixtures/ too much
23:09:59  <isaacs>fixtures should not contain test logic
23:10:48  <isaacs>spawn(process.execPath, [__filename, someArg]) > process.spawn(process.execPath, [path.resolve(__dirname, '..', 'fixtures', 'blerg.js')])
23:12:12  <TooTallNate>isaacs: LGTM dawg
23:12:39  <TooTallNate>oh i guess i'm late
23:12:40  <TooTallNate>hahah
23:13:28  * c4miloquit (Remote host closed the connection)
23:13:56  <isaacs>TooTallNate: no, you're right on time. i just have latency compensation in my irc client, so i knew you'd say that.
23:14:17  <TooTallNate>:p
23:14:29  <TooTallNate>"predictive logging"
23:24:59  * kazuponjoined
23:27:21  <isaacs>windows makes me curse more than any other operating system ever.
23:28:11  <isaacs>it's the little things.
23:28:19  <isaacs>like copying a directory, and being told that you can't because there's a file in it
23:28:25  <isaacs>!$@#!#$!@#!$@#$!@
23:29:38  * kazuponquit (Ping timeout: 255 seconds)
23:37:49  <CoverSlide>that's what mice are for
23:45:52  <sblom>isaacs: I'll tell the file system team that you have some feedback for them. :)
23:46:42  <sblom>isaacs: (Things like that and not being able to delete a directory because somewhere I have a command prompte pointed into it drive me nuts.)
23:47:31  <piscisaureus_>unlocker
23:47:53  <sblom>piscisaureus_: Is that the name of a tool that I'm about to fall in love with?
23:48:07  <piscisaureus_>well it is not the prettiest tool in the world
23:48:16  <piscisaureus_>but it makes you quite lazy I can tell you
23:48:29  <piscisaureus_>the only risk is that it doesn't really move stuff to the recycle bin
23:48:45  <piscisaureus_>sblom: http://www.emptyloop.com/unlocker/
23:50:53  <sblom>piscisaureus_: neat
23:52:00  <piscisaureus_>the trick it uses is pretty aweful
23:52:24  <piscisaureus_>it does DuplicateHandle from the process that has the handle open, and specifies DUPLICATE_CLOSE_SOURCE
23:53:00  <piscisaureus_>it actually surprises me that few if any program crashes when that happens
23:53:37  <sblom>impressive
23:54:08  <isaacs>sblom: tell them to be posix.
23:54:37  <isaacs>sblom: it's time to put away that fight. the windows file system paradigm is not different, it's incorrect. posix won.
23:54:52  <isaacs>backslashes? seriously?
23:55:13  <piscisaureus_>isaacs: it's funny that that you pick the least of our concerns ...
23:55:25  <isaacs>how many person-hours has that cost humanity? we could have built another great wall of china by now.
23:55:39  <isaacs>piscisaureus_: yes, indeed! the concerns only grow from there!
23:56:05  <piscisaureus_>sblom: tell them to allow moving files that have been marked for deletion! That shouldn't be too hard, and it would solve almost all problems.
23:56:15  <isaacs>of course, this is like shaking my fists at gravity. good luck arguing with it.
23:56:18  <sblom>isaacs: I delivered the Posix rant to some friends of mine last night, actually. I don't think it can possibly happen, though. That's sort of like saying "Windows should just be Linux". I have a currently shipping solution for you if that's the case...
23:56:19  <isaacs>microsoft is a force of nature.
23:56:50  <isaacs>sblom: well, i mean, the filesystem interface in particular is just hideously antiquated and wrong.
23:57:19  <isaacs>sblom: vms is dead. stop making marionettes with its electrified corpse.
23:57:30  <piscisaureus_>hahaha
23:57:32  <isaacs>windows has some godo stuff in other places.
23:57:48  <piscisaureus_>let's start small
23:57:58  <piscisaureus_>and fix the file locking thing
23:58:02  <isaacs>yes!
23:58:05  <sblom>:)
23:58:16  <piscisaureus_>I think actually the windows file locking model is not that nonsentical
23:58:18  <isaacs>piscisaureus_: oh, i thought you meant "start small" with what you wanted to keep from windows
23:58:34  <piscisaureus_>but if I really want to the file should just be deletable
23:59:17  * loladiroquit (Quit: loladiro)
23:59:49  <isaacs>piscisaureus_: the fact that you can delete it from the command line if the name is too long, that drives me crazy
23:59:58  <isaacs>piscisaureus_: i actually open up a node repl and remove it with rimraf