00:00:00  * ircretaryquit (Remote host closed the connection)
00:00:07  * ircretaryjoined
00:02:45  * indexzerojoined
00:14:47  * sblomquit (Quit: leaving)
00:16:08  <bnoordhuis>those raspberry pis are a bloody scam
00:16:17  <bnoordhuis>buy one anywhere in the world: EUR 35
00:16:26  <bnoordhuis>buy one in mainland europe: EUR 45
00:29:20  <MI6>joyent/libuv: Ben Noordhuis master * 8e3e60f : linux: only pack struct uv__epoll_event on x86_64 On i386, it does not n (+1 more commits) - http://git.io/_NZ-_g
00:29:57  <MI6>joyent/node: Ben Noordhuis master * e4f2a14 : deps: upgrade libuv to 8e3e60f - http://git.io/qukH9Q
00:30:05  * indexzeroquit (Quit: indexzero)
00:31:33  * travis-cijoined
00:31:33  <travis-ci>[travis-ci] joyent/libuv#1018 (master - 8e3e60f : Ben Noordhuis): The build is still failing.
00:31:33  <travis-ci>[travis-ci] Change view : https://github.com/joyent/libuv/compare/9aab5d483794...8e3e60ffbf20
00:31:33  <travis-ci>[travis-ci] Build details : http://travis-ci.org/joyent/libuv/builds/4116785
00:31:33  * travis-cipart
00:35:46  * tomshredsjoined
00:43:07  * EhevuTovquit (Quit: This computer has gone to sleep)
00:43:22  * mikealjoined
00:44:18  * mikealquit (Client Quit)
00:53:10  <Raynos>How do I test for performance regressions for bug fixes / PRs ?
00:57:06  * mikealjoined
00:57:08  * jmar777joined
01:03:40  * mikealquit (Quit: Leaving.)
01:05:24  * tomshredsquit (Quit: Linkinus - http://linkinus.com)
01:08:59  <isaacs>bnoordhuis: you around?
01:09:02  <bnoordhuis>isaacs: yes
01:09:04  <isaacs>bnoordhuis: check this out: https://gist.github.com/4520094
01:09:19  <bnoordhuis>Raynos: make test?
01:09:33  <bnoordhuis>Raynos: or better, python tools/test.py --mode=debug,release simple message pummel
01:09:46  <isaacs>bnoordhuis: basically, streams2 is slightly faster than streams1 js + master's libuv + master's v8
01:09:50  <Raynos>bnoordhuis: so if the tests take longer that count as a performance regression
01:10:13  <isaacs>Raynos: the benchmark that matters the most is http_simple
01:10:18  <isaacs>Raynos: bash benchmark/http.sh
01:10:18  <bnoordhuis>isaacs: really? well done in that case :)
01:10:27  <isaacs>bnoordhuis: but! it still quite a bit slower than V8
01:10:32  <isaacs>*er, than v0.8
01:10:46  <bnoordhuis>ah, okay. the only metric that counts
01:11:02  <bnoordhuis>Raynos: define longer?
01:11:03  <isaacs>and using the master's bindings/libuv/streams1-js + the V8 that v0.8 has, is much much slower.
01:11:20  <isaacs>Raynos: the time the tests take to run is not a metric we care much about
01:11:22  <Raynos>there is a counter in make test that tells me how many seconds it takes to run them I guess
01:11:25  <Raynos>Oh ok, cool.
01:11:34  <isaacs>Raynos: run benchmark/http.sh
01:11:41  <bnoordhuis>isaacs: well... if they take twice as long all of a sudden something is wrong
01:11:43  <isaacs>Raynos: wait at least 120seconds between runs
01:11:47  <isaacs>bnoordhuis: sure, that's true.
01:11:53  * loladiroquit (Quit: loladiro)
01:11:53  <isaacs>bnoordhuis: but usually that just means a test is failing
01:12:00  <isaacs>and http_simple will show msot issues anyway
01:12:16  <bnoordhuis>or that v8 is getting compiled without optimizations again :)
01:12:24  <isaacs>bnoordhuis: yes, that could be
01:12:43  <Raynos>when I run the tests the debugger repl tests timeout some time
01:12:47  <isaacs>but the diff between master js and streams1 js (everything else being the same) makes me think something is fishiy
01:12:56  <Raynos>Not sure whether that's my environment or a regression bug from my patch
01:13:09  <isaacs>Raynos: the debugger repl tests are assholes.
01:13:21  <isaacs>Raynos: that's a separate issue which we've got to fix
01:13:44  <isaacs>Raynos: they work 100% of the time for me when run diretly, about 80% of the time run through `make tes`
01:13:46  <Raynos>isaacs: did you look at https://github.com/joyent/node/issues/4579 ?
01:13:47  <isaacs>t
01:14:56  <isaacs>Raynos: yep. sounds like a bug.
01:15:08  <Raynos>I fixed it, PR soon.
01:15:23  <Raynos>I fixed it by moving responsibility for edge-case into well compliant writable streams
01:18:32  <bnoordhuis>while true; do python tools/test.py --mode=debug,release simple/test-debugger-repl || break; done
01:18:46  <bnoordhuis>it's funny, sometimes it fails on the second run, sometimes it goes on for 50 runs or more
01:19:21  <bnoordhuis>but when it fails, it leaves a child process behind that hogs the port and all is woe
01:19:38  <Raynos>is there way to run a group of tests using the node test tools
01:20:02  <bnoordhuis>Raynos: you mean a subset of e.g. the simple test suite?
01:20:05  <Raynos>like like make `test-group group=simple/test-stream2-*`
01:20:23  <bnoordhuis>you need to specify them by hand
01:20:23  <Raynos>I was more going for wildcard searching files inside simple
01:20:36  <bnoordhuis>chop of test/ and .js and you're good to go
01:20:55  <Raynos>yeah I found that but it was tedious :P
01:21:03  <bnoordhuis>i guess i could make test.py do that automatically
01:21:49  <isaacs>Raynos: i often do something like: for t in test/simple/test-{http,stream,net}*; do ./node $t || break ; done
01:22:14  <isaacs>but a selector in python tools/test.py would be nice
01:22:28  <isaacs>python tools/test.py simple/$glob
01:22:31  <bnoordhuis>you know, it just works
01:22:41  <bnoordhuis>python tools/test.py simple/test-dgram* <- runs just the dgram tests
01:22:46  <bnoordhuis>hah, i never knew
01:22:48  <isaacs>oh, what do you know!
01:22:56  <isaacs>Raynos: guess you got your answer :)
01:23:11  <Raynos>oh.
01:23:12  <Raynos>cool.
01:23:18  <isaacs>bnoordhuis: it'd be nice if `make test` had an arg or something
01:23:27  <isaacs>bnoordhuis: make test TEST=dgram,stream2,net
01:23:40  <bnoordhuis>isaacs: you know makefile syntax right? :)
01:23:49  <isaacs>bnoordhuis: yeah, just playing with ideas.
01:23:53  * indexzerojoined
01:24:04  <isaacs>now that i know that the test runner Just Works that way, i'll probably do something like that
01:24:30  * stagasjoined
01:24:45  <isaacs>bnoordhuis: what do you make of streams1-js+master-uv+master-V8 being so much slower than v0.8?
01:25:02  <bnoordhuis>isaacs: that's on os x?
01:25:16  <isaacs>bnoordhuis: yeah
01:25:58  <bnoordhuis>have you tried downgrading v8 or libuv?
01:26:13  <bnoordhuis>oh, are you testing with or without keepalive?
01:26:31  <isaacs>bnoordhuis: i tried downgrade V8 to v0.8's and things got worse
01:26:42  <isaacs>it's tricky because we changed a bunch of bindings for the V8 update
01:27:04  <isaacs>but, basically, streams 1 = master + `git checkout 77ed12f -- lib` to get the pre-streams2 JS
01:27:49  <bnoordhuis>i'll try upgrading v8 and libuv in v0.8 and see what happens
01:28:02  <bnoordhuis>maybe just libuv first
01:28:14  <isaacs>k
01:28:24  <bnoordhuis>but try benchmarking with keepalive enabled
01:28:28  <isaacs>i mean, if i've managed to make streams2 faster than streams1, then i'll be psyched
01:28:33  <isaacs>that benchmark is with keepalive
01:28:38  <isaacs>without keepalive the numbers are much smaller.
01:28:42  <Raynos>isaacs: https://gist.github.com/d6224c99aad06762726c
01:28:45  <isaacs>like 3000 vs 11000
01:28:46  <Raynos>I can't seem to run that benchmark
01:29:13  <isaacs>Raynos: what os is that?
01:29:29  <Raynos>ubuntu 12
01:30:01  <isaacs>Raynos: weird. works on my ubuntu machine. maybe throw a sudo in front of that?
01:30:11  <isaacs>benchmark/http.sh: line 19: ulimit: open files: cannot modify limit: Operation not permitted
01:30:24  <isaacs>if you can't modify your fd limit, it's going to choke anyway
01:31:27  <Raynos>https://gist.github.com/506c1daa255eb4c73659
01:31:39  <Raynos>its ok ill just not run the benchmark :D
01:31:44  <Raynos>i doubt my patch regresses
01:31:48  <bnoordhuis>ah, the c-ares split... not so convenient now
01:31:59  <isaacs>bnoordhuis: oh?
01:32:14  <bnoordhuis>isaacs: when trying to upgrade libuv in v0.8 to master's version
01:32:41  <isaacs>Raynos: try commenting out the sysctl lines at the top of the http.sh file
01:32:52  <isaacs>bnoordhuis: ah, i see
01:33:04  <bnoordhuis>nothing that can't be fixed though
01:33:11  <Raynos>https://gist.github.com/e11176dc9d5d261398fb
01:33:12  <Raynos>nada
01:34:05  <Raynos>https://gist.github.com/b7664742da96a9cb6f77 <- that's the problem
01:35:38  * EhevuTovjoined
01:36:18  <isaacs>Raynos: so, that drain/finish ondrain() thing
01:36:31  <isaacs>Raynos: really, the readable stream should not be looking at dest._writableState anyway
01:36:42  <isaacs>Raynos: fucking multipipe is the problem
01:36:50  * isaacshate multipipe
01:37:06  <Raynos>isaacs: probably. I presume you do that because you unpipe draining streams
01:37:09  <isaacs>Raynos: he issue is that we dont' track *who* we're awaiting a drain from, only how many drains we're awaiting
01:37:27  <Raynos>Well there's two ways you can do this
01:37:33  <Raynos>either track drain state completely
01:37:48  <Raynos>or tell users that if they unpipe draining streams or close draining streams or finish draining streams that
01:37:54  <Raynos>it is their problem if they do that
01:37:56  <isaacs>you mean, replace the counter with an array of who we're awaiting drain from, and prune the array when it drains?
01:38:22  <isaacs>that'd be abusively slow.
01:38:23  <Raynos>well the issue is removing drain listeners on writable streams when they are in draining state
01:38:39  <Raynos>if pipe does that it's a broken pipe.
01:39:18  <Raynos>i recommend we think about all the situations in which we do that and fix Writable so that it's never in draining state for those situations
01:39:39  <Raynos>if users use writable streams that do not inherit from Writable they can implement and enforce the constraints of draining stream state themself
01:40:15  <Raynos>or when we remove drain listeners we have a way to check whether that stream is in draining state
01:40:48  * stagasquit (Ping timeout: 276 seconds)
01:41:34  * EhevuTovquit (Ping timeout: 246 seconds)
01:42:39  * mikealjoined
01:46:50  * mikealquit (Client Quit)
01:47:24  <Raynos>isaacs: I started https://github.com/joyent/node/pull/4581
01:50:06  <bnoordhuis>isaacs: libuv from master is faster \o/
01:51:14  <bnoordhuis>not by a wide margin but a repeatable 5-8%
01:53:05  <bnoordhuis>isaacs: in case you want to try it yourself: https://github.com/bnoordhuis/node/compare/v0.8...backport-libuv-to-v0.8
01:53:20  <bnoordhuis>Showing 348 changed files with 13,921 additions and 80,707 deletions. <- so sweet
01:53:40  * jmar777quit (Remote host closed the connection)
01:54:17  * jmar777joined
01:56:08  * jmar777_joined
01:56:24  * jmar777quit (Read error: Connection reset by peer)
01:56:38  <isaacs>bnoordhuis: ok... so why is master slower?
01:56:45  <isaacs>bnoordhuis: if V8 is faster, and libuv is faster, and the js is faster?
01:56:52  <isaacs>bnoordhuis: flibby bindings?
01:57:25  <isaacs>Raynos: it calls _read() until the hwm is set.
01:57:32  <isaacs>Raynos: but it doesn't emit 'readable' until the lwm is set
01:57:37  <isaacs>s/set/hit/g
01:57:37  <bnoordhuis>isaacs: good question. guess it's time to compare profiler logs
01:57:56  <Raynos>isaacs: but if you set the lwm to 0 it wont keep calling _read
01:58:06  <Raynos>I think it calls _read until the lwm is set
01:58:08  <isaacs>Raynos: no, if you set hwm to 0 it doesn't keep calling _read
01:58:15  <isaacs>Raynos: it calls _read until the hwm is hit
01:58:33  <bnoordhuis>isaacs: git checkout 77ed12f -- lib <- that actually works?
01:58:36  <Raynos>nope until lwm
01:58:39  * bnoordhuiscrosses his fingers
01:58:44  <isaacs>bnoordhuis: yes, that actually works
01:59:16  <isaacs>Raynos: wrong
01:59:17  <isaacs> if (state.length - n <= state.highWaterMark)
01:59:17  <isaacs> doRead = true;
01:59:18  <Raynos>isaacs: https://github.com/joyent/node/blob/master/lib/_stream_readable.js#L277
01:59:31  <Raynos>isaacs: if you call read() in a tight loop then until highWaterMark
01:59:37  <Raynos>if you call read() once then until lowWaterMark
01:59:58  <Raynos>i.e. if you pull continiously it will go until high water mark, if you pull once it will fill upto low water mark
02:00:06  <isaacs>Raynos: it'll emit 'readable' at lwm, and stop reading at hwm
02:01:28  <Raynos>https://gist.github.com/1e18361d5c9014fd741a three _read because length 3 is > lwm
02:01:30  <isaacs>Raynos: please prove it with a test case.
02:01:57  <Raynos>but maybe im mistaken here
02:03:35  <bnoordhuis>hm. master is faster on some benchmarks, slower on others
02:03:49  <isaacs>Raynos: https://gist.github.com/4521829
02:04:05  <bnoordhuis>ab-hello-world-buffer-1024 and ab-hello-world-string-102400 are notably slower
02:04:06  <isaacs>Raynos: automatically _read() until lwm is hit. you're right.
02:04:15  <isaacs>that's a bug :)
02:04:23  <Raynos>which I think is what you would want
02:04:24  * mikealjoined
02:04:31  <isaacs>no, it is not what you would want
02:04:39  <isaacs>you'd wnat it to buffer up to hwm. because tha'ts what hwm means :)
02:04:46  <Raynos>oh I see
02:04:53  <Raynos>the lwm is just an optimization to let you know when to read it
02:05:02  <Raynos>you set the lwm to say "dont bother me about those tiny chunks"
02:05:14  <Raynos>Actually I dont quite understand what the lwm is for
02:05:48  * TheJHquit (Ping timeout: 255 seconds)
02:06:03  <isaacs>yes, it's an optimization to let you know whwen to read it
02:07:04  <Raynos>isaacs: var r = new Readable(); r.push("foo"); r.read(); // ERROR
02:07:05  <isaacs>Raynos: ohh... wait. no.
02:07:15  <isaacs>Raynos: yeah, you're pushing a string, not a buffer
02:07:19  <isaacs>Raynos: so, here's how it works:
02:07:23  <Raynos>is the user responsible for setting encoding
02:07:31  <isaacs>Raynos: if you read() and it takes you down *below* the hwm, then it triggers a _read()
02:07:37  <isaacs>Raynos: but it still only _read()'s up to the lwm
02:07:52  <Raynos>is there value in detecting for a string and defaulting encoding to "utf8" ?
02:07:53  <isaacs>Raynos: so, lwm=2,hwm=10,buffer=12. read(4) will trigger a _read()
02:08:02  <isaacs>Raynos: sure.
02:08:08  <isaacs>Raynos: or push() could take an encoding arg
02:08:35  <isaacs>so.. yeah, i guess that's fine.
02:08:38  <Raynos>well I would also mean for cb(null, "foo") which also fails if you don't configure encoding
02:08:42  <isaacs>everything seems to be working
02:08:48  * hzquit
02:08:51  <isaacs>yeah
02:09:15  <isaacs>anywya, hwm only matters when deciding whether or not to _read() some more when you call read()
02:09:51  <Raynos>yeah that works fine.
02:10:04  <isaacs>anyway, i guess it'd be good to document
02:10:29  <bnoordhuis>hm. ab-hello-world-string-1024 is a fair bit slower if you crank it up to -k -c 1000
02:10:35  <Raynos>isaacs: is there a valid use-case for wanting a stream to fill up to low water mark without consuming chunks from the buffer. Like peak() or start()
02:11:27  <isaacs>Raynos: start() is called read(0)
02:11:34  <bnoordhuis>master wins by a fair margin on ab-hello-world-buffer-102400 though. interesting
02:12:08  <isaacs>bnoordhuis: when you say "master" you mean "0.8 with master uv"? or "master" master?
02:12:20  <bnoordhuis>isaacs: master master
02:12:26  <bnoordhuis>without streams2
02:12:31  <isaacs>oh. ok
02:12:55  <isaacs>bnoordhuis: any idea why master-streams1 is slower in http.sh?
02:13:42  <bnoordhuis>isaacs: no, but i'm investigating that
02:17:55  * jmar777_quit (Remote host closed the connection)
02:18:23  <Raynos>isaacs: yeah read(0) is nice, thanks :)
02:18:29  * jmar777joined
02:18:33  <bnoordhuis>`perf stat -i -r 5` tells me that v0.8 simply does less, 20.2 vs 21.3 billion cycles
02:18:50  <bnoordhuis>i guess `perf record` is in order
02:22:54  * jmar777quit (Ping timeout: 264 seconds)
02:23:20  <bnoordhuis>that 1 billion extra cycles is all user mode
02:29:56  <bnoordhuis>2.37% node node [.] v8::internal::JSObject::LocalLookupRealNamedProperty(v8::internal::String*, v8::internal::LookupResult*)
02:30:14  <bnoordhuis>we're doing something in master that triggers a lot of calls to LocalLookupRealNamedProperty
02:30:20  <bnoordhuis>and whatever it is, v0.8 isn't doing it
02:30:32  <bnoordhuis>i say "we" but it could be a v8 thing, of course
02:31:44  <bnoordhuis>it's also possible that v8::internal::String::WriteToFlat<char> has become slower
02:31:52  <bnoordhuis>but not by much
02:34:06  <bnoordhuis>LocalLookupRealNamedProperty explains 0.5 of those 1 billion extra cycles
02:34:24  <isaacs>so... we need to... read fewer properties?
02:37:08  <bnoordhuis>isaacs: well... it might be a v8 regression
02:37:21  <bnoordhuis>maybe i should've written that as "regression"
02:37:29  <bnoordhuis>that function now also deals with js proxies
02:37:42  <bnoordhuis>which makes it more expensive
02:38:38  <isaacs>i see
02:39:20  <isaacs>that sucks. i guess i just got so used to newer V8 versions making us faster, i forgot it's not a guarantee
02:40:14  <isaacs>bnoordhuis: how does your master-streams1 compare with master-streams2?
02:40:31  <bnoordhuis>isaacs: ask me that tomorrow or monday, i'm about to sign off :)
02:40:43  <bnoordhuis>it's nearing 4 am here
02:42:06  <bnoordhuis>okay, sleep tight
02:42:12  <isaacs>yikes
02:42:13  <isaacs>g'nite
02:43:23  <Raynos>isaacs: https://gist.github.com/029fd41ce1617ded8a4e
02:44:13  <Raynos>we don't check `state.buffer.length` here ( https://github.com/joyent/node/blob/master/lib/_stream_readable.js#L153 ) in case user calls `read(0)` on purpose
02:44:45  <Raynos>So it should be `n === 0 && nOrig !== 0` or `n === 0 && state.buffer.length === 0`
02:46:44  * bnoordhuisquit (Ping timeout: 248 seconds)
02:47:37  * indexzeroquit (Quit: indexzero)
03:20:04  * mjr_joined
03:24:11  * mikealquit (Quit: Leaving.)
03:26:58  * indexzerojoined
03:29:18  * EhevuTovjoined
03:37:38  * mjr_quit (Quit: mjr_)
03:37:42  * EhevuTovquit (Quit: This computer has gone to sleep)
03:39:14  * trevnorrisjoined
03:44:40  * indexzeroquit (Quit: indexzero)
03:45:09  * indexzerojoined
03:54:29  * mikealjoined
04:04:09  * mikealquit (Ping timeout: 265 seconds)
04:30:45  * mikealjoined
04:38:51  * EhevuTovjoined
04:42:31  * trevnorrisquit (Ping timeout: 245 seconds)
05:03:09  * loladirojoined
05:07:01  * indexzeroquit (Quit: indexzero)
05:09:56  * loladiroquit (Quit: loladiro)
05:10:54  * felixgejoined
05:10:55  * felixgequit (Changing host)
05:10:55  * felixgejoined
05:13:07  * loladirojoined
05:15:14  * loladiroquit (Client Quit)
05:32:40  * felixgequit (Quit: felixge)
05:35:13  * loladirojoined
05:35:16  * indexzerojoined
06:09:42  * EhevuTovquit (Quit: This computer has gone to sleep)
06:10:59  * felixgejoined
06:10:59  * felixgequit (Changing host)
06:10:59  * felixgejoined
06:14:38  * loladiroquit (Quit: loladiro)
06:23:01  * felixgequit (Quit: felixge)
06:23:28  * felixgejoined
06:23:29  * felixgequit (Changing host)
06:23:29  * felixgejoined
06:23:31  * felixgequit (Client Quit)
06:32:52  * brsonjoined
07:00:25  * indexzeroquit (Quit: indexzero)
07:11:01  * mikealquit (Read error: Connection reset by peer)
07:15:56  * mikealjoined
07:18:05  * indexzerojoined
07:23:41  * AvianFluquit
07:56:24  * brsonquit (Read error: Connection reset by peer)
07:57:04  * brsonjoined
08:15:20  * rendarjoined
08:24:37  * stagasjoined
08:30:38  * EhevuTovjoined
08:47:04  * paddybyersjoined
09:07:10  * indexzeroquit (Quit: indexzero)
09:14:24  * indexzerojoined
09:25:18  * paddybyersquit (Ping timeout: 264 seconds)
09:31:49  * paddybyersjoined
09:58:40  * `3rdEdenjoined
10:36:39  * paddybyers_joined
10:40:39  * paddybyersquit (Ping timeout: 276 seconds)
10:40:40  * paddybyers_changed nick to paddybyers
10:55:37  * TheJHjoined
11:12:31  * brsonquit (Ping timeout: 256 seconds)
11:29:51  * EhevuTovquit (Quit: This computer has gone to sleep)
11:37:30  * TheJHquit (Ping timeout: 276 seconds)
11:46:30  * `3rdEdenquit (Remote host closed the connection)
12:03:36  * felixgejoined
12:03:36  * felixgequit (Changing host)
12:03:36  * felixgejoined
12:18:26  * TheJHjoined
12:43:19  * `3rdEdenjoined
12:48:36  * TheJHquit (Ping timeout: 272 seconds)
12:49:40  * hzjoined
12:50:34  * piscisaureus_joined
12:53:20  <piscisaureus_>indutny: congrats man!
12:59:51  * indexzeroquit (Quit: indexzero)
13:01:24  * hzquit (Ping timeout: 264 seconds)
13:04:03  * hzjoined
13:16:38  * piscisaureus_quit (Ping timeout: 240 seconds)
13:21:37  * piscisaureus_joined
13:34:12  * piscisaureus_quit (Ping timeout: 276 seconds)
14:10:10  * piscisaureus_joined
14:24:25  * TheJHjoined
14:34:28  * bnoordhuisjoined
14:47:32  * TheJHquit (Ping timeout: 252 seconds)
15:17:41  * piscisaureus_quit (Ping timeout: 248 seconds)
15:18:14  <bnoordhuis>indutny: fedor, pretty please with sugar on top reapply all patches whenever you upgrade v8
15:18:44  <bnoordhuis>i've just wasted close to an hour because you upgraded v8 and didn't reapply an arm patch...
15:19:29  <bnoordhuis>and now i can recompile v8 yet again
15:23:07  * felixgequit (Quit: felixge)
15:28:57  * TheJHjoined
15:32:44  * piscisaureus_joined
15:39:52  <indutny>bnoordhuis: oooh
15:39:54  <indutny>sorry
15:40:13  <indutny>bnoordhuis: we should store information about this floating patches somewhere inside project
15:40:20  <indutny>have we already did it?
15:40:29  <bnoordhuis>indutny: well, git log --oneline deps/v8
15:40:36  <indutny>heh
15:40:38  <indutny>ok
15:40:40  <indutny>will know
15:40:46  <bnoordhuis>and will do?
15:41:21  <indutny>lets see :)
15:45:52  <bnoordhuis>hm. so i'm backporting v8 and libuv from master to v0.8 for benchmarks
15:45:58  <bnoordhuis>and both seem to make things slower
15:46:01  * `3rdEdenquit (Remote host closed the connection)
15:46:23  <bnoordhuis>but what's odd is that libuv from master seems to regress ab-hello-world-string-102400 and ab-hello-world-buffer-102400 by a fairly substantial margin
15:46:36  <bnoordhuis>for no discernible reason as far as i can tell
15:52:58  <bnoordhuis>umm, or maybe not
15:53:06  <bnoordhuis>the variance is insanely big
15:59:11  <bnoordhuis>okay... i've established that the only regression is in ab-hello-world-string-102400
15:59:39  <bnoordhuis>and it's apparently caused by libuv
16:00:30  <bnoordhuis>the upside is that master's libuv is faster for smaller writes
16:06:06  <bnoordhuis>a curious thing, ab-hello-world-string-1024 is about 7% faster, ab-hello-world-string-102400 is about 9% slower
16:09:21  * felixgejoined
16:09:21  * felixgequit (Changing host)
16:09:21  * felixgejoined
16:12:33  * felixgequit (Client Quit)
16:13:11  <indutny>hm...
16:13:46  <indutny>interesting
16:13:54  <indutny>how could it be possible
16:14:06  <indutny>I mean, how can it affect libuv
16:15:45  <bnoordhuis>i wonder if there's a memory leak in libuv
16:16:04  <bnoordhuis>with master's libuv: 644608k rss, 160650 pagefaults
16:16:21  <bnoordhuis>with v0.8's libuv: 242320 rss, 21906 pagefaults
16:16:32  <indutny>huh
16:16:36  <indutny>160650 pagefaults?
16:16:37  <indutny>ah
16:16:41  <indutny>well
16:16:45  <indutny>it's not proportional to rss
16:16:59  <bnoordhuis>let's see what the buffer benchmark does
16:17:12  <indutny>bnoordhuis: may be better valgrinding it?
16:17:26  <indutny>btw man, my wife has bought me lego mindstorms
16:17:29  <indutny>it's amazing shit
16:17:34  <indutny>in good sense
16:17:56  <bnoordhuis>hm, buffer benchmarks are more or less identical
16:17:58  <indutny>ok, starting VM
16:18:01  <bnoordhuis>indutny: oh?
16:18:09  <indutny>bnoordhuis: lego mindstorms?
16:18:13  <bnoordhuis>yes?
16:18:19  <indutny>http://mindstorms.lego.com/en-us/Products/default.aspx
16:19:01  <bnoordhuis>ah, there's still a difference but it's relatively minor
16:19:07  <bnoordhuis>okay, valgrind it is
16:19:11  <indutny>:)
16:19:21  <indutny>that's what I'm doing right now
16:22:23  <bnoordhuis>valgrind + ab == exercise in frustration
16:22:37  <bnoordhuis>failing all the time with socket timeouts and no way to increase that timeout
16:22:54  <indutny>siege?
16:23:00  <indutny>or
16:23:01  <indutny>-f flag
16:23:12  <indutny>no
16:23:14  <indutny>not, -f
16:23:24  <indutny>-r
16:23:38  <indutny>and -t 10000
16:24:45  <indutny>bnoordhuis: how can I start server with libuv?
16:25:02  <bnoordhuis>indutny: what do you mean?
16:25:05  <indutny>echo-server?
16:25:09  <indutny>well, I want to benchmark libuv
16:25:14  <indutny>with ab
16:26:18  <indutny>or better blackhole server?
16:26:55  * AvianFlujoined
16:26:59  <bnoordhuis>indutny: oh, like that. for example, out/Debug/run-tests tcp_writealot tcp4_echo_server
16:27:16  <indutny>ok, thanks
16:27:39  <bnoordhuis>i'm not really seeing memory leaks
16:27:52  <indutny>so it just allocates more
16:28:04  <bnoordhuis>i guess it could be because of libuv processing a whole lot more events per loop tick
16:28:19  <bnoordhuis>so allocated memory may stay around longer
16:28:34  <bnoordhuis>that's easy to verify
16:31:35  <bnoordhuis>no, that's not it
16:33:58  * piscisaureus_quit (Read error: Connection reset by peer)
16:44:34  <bnoordhuis>i guess i found it
16:44:39  <bnoordhuis> 51.99 0.000809 0 2571 brk
16:45:04  <bnoordhuis>for some reason, libuv is triggering an insane amount of brk() syscalls
16:45:16  <bnoordhuis>libuv from v0.8 doesn't
16:45:34  <tjfontaine>only major change I can think of was ev removal?
16:46:43  <bnoordhuis>yes
16:51:39  * loladirojoined
16:52:59  <indutny>bnoordhuis: brk()?
16:53:06  <indutny>bnoordhuis: I think you need to feed it to massif
16:53:39  <bnoordhuis>indutny: i already did
16:53:43  <indutny>and?
16:53:54  <indutny>who is allocating most?
16:56:28  <bnoordhuis>indutny: well, the heap is stable at about 2.3M with v0.8
16:56:37  <bnoordhuis>that's v0.8's libuv
16:56:55  <bnoordhuis>it grows to ~12MB with master's libuv and then shrinks to ~10M
16:59:36  <bnoordhuis>it looks wildly erratic in massif-visualizer
16:59:43  <bnoordhuis>hah, it's so weird
17:00:13  <bnoordhuis>apparently it's Isolate::EnsureDefaultIsolate() that causes the spikes
17:00:47  <tjfontaine>hrm
17:01:06  <bnoordhuis>let me run it one more time, just in case
17:01:55  <bnoordhuis>you guys want me to upload the massif logs?
17:02:40  <tjfontaine>I'm curious if it's not too much trouble
17:03:57  <indutny>es
17:04:00  <indutny>upload them
17:04:47  <bnoordhuis>let's see if gist.github.com accepts them
17:06:26  <bnoordhuis>https://gist.github.com/5be314ba56eac05232a1
17:09:23  <tjfontaine>interesting
17:12:00  <bnoordhuis>isn't it? but quite inexplicable
17:12:20  <bnoordhuis>i guess libuv and the js code are biting each other
17:12:28  <bnoordhuis>but exactly how is anyone's guess
17:13:29  <bnoordhuis>let's see what happens when i disable the slab allocator
17:13:56  <indutny>haha
17:14:06  <indutny>pretty nice
17:16:37  <indutny>bnoordhuis: correct me if I'm wrong
17:16:45  <indutny>but it seems that most allocations are happening StreamWrap
17:16:48  <indutny>which is expected
17:17:15  <indutny>since that's what really happens
17:17:26  <bnoordhuis>i think so, yes
17:17:56  <bnoordhuis>but it's rather peculiar that the libuv version has so much impact
17:18:16  <indutny>indeed
17:18:57  * piscisaureus_joined
17:19:56  <indutny>piscisaureus_: hey man
17:19:58  <indutny>piscisaureus_: thank you
17:20:07  <piscisaureus_>hey fedor
17:20:16  <indutny>sorry, I was away
17:20:22  <piscisaureus_>indutny: having fun today?
17:21:05  <indutny>erm
17:21:06  <indutny>yeah
17:21:08  <indutny>sort of :)
17:21:21  <bnoordhuis> 97.47 0.002238 37 61 munmap
17:21:26  <indutny>I think I'll catch up with some people at tuesday
17:21:28  <bnoordhuis>^ not sure if that's an improvment :/
17:21:33  <indutny>ahha
17:21:40  <indutny>which remind me about mmap() patch
17:21:51  <indutny>for buffers
17:21:55  <bnoordhuis>yeah, tried that - didn't affect memory usage much
17:21:59  <indutny>hm...
17:22:02  <bnoordhuis>or rather, peak rss
17:22:09  <indutny>what about performance?
17:22:39  <bnoordhuis>didn't check yet
17:23:01  <indutny>I suspect mmap() is much slower than malloc()
17:24:19  <bnoordhuis>yes, but it maps much larger chunks so it's a one-time cost
17:24:29  <bnoordhuis>or maybe not one-time but it's not paid as often
17:24:47  <bnoordhuis>memory usage was lowest with mmap() btw but not by enough
17:25:05  <bnoordhuis>it's still 27M vs 3M
17:25:06  <indutny>ok
17:25:17  <indutny>odd
17:25:30  <indutny>is everything except libuv is the same in your builds?
17:25:40  <indutny>or are you comparing different v8 versions too?
17:27:51  <MI6>joyent/node: yangguo@chromium.org master * 926c90b : v8: Hardfloat does not imply VFPv3, only VFPv2. Raspberry Pi is an examp - http://git.io/XxS8Rg
17:28:10  <bnoordhuis>indutny: no, same version of v8
17:28:15  <indutny>щлбпщщв
17:28:30  <bnoordhuis>i did have to chop up v0.8 a little to get the backport to compile
17:28:47  <bnoordhuis>indutny: https://github.com/bnoordhuis/node/compare/v0.8...backport-libuv-to-v0.8 <- also includes the v8 update
17:29:10  <bnoordhuis>i'm comparing versions of that branch with and without the libuv update
17:29:11  <indutny>yep, I undestand
17:29:21  <indutny>interesting
17:30:13  <bnoordhuis>something to contemplate over dinner
17:30:14  <bnoordhuis>biab
17:32:46  <bnoordhuis>the mmap() patch fixes the bytes/102400 benchmark, curiously enough
17:33:19  * mikealquit (Quit: Leaving.)
17:34:58  <indutny>aha!
17:49:52  <piscisaureus_>how curious is all this
17:55:27  * mikealjoined
17:56:41  * paddybyersquit (Ping timeout: 246 seconds)
17:58:09  * piscisaureus_quit (Read error: Connection reset by peer)
18:01:10  * mikealquit (Quit: Leaving.)
18:01:41  * `3rdEdenjoined
18:02:50  <rendar>piscisaureus_: hi, libuv uses ConnectEx right? then when the connection is completed, it calls uv_process_tcp_connect_req() ?
18:29:16  * mikealjoined
18:50:07  * toothrotquit (Ping timeout: 260 seconds)
18:57:00  * toothrjoined
18:58:20  * toothrchanged nick to toothrot
19:12:21  * mikealquit (Quit: Leaving.)
19:14:53  * piscisaureus_joined
19:18:08  * mjr_joined
19:20:34  * loladiroquit (Quit: loladiro)
19:22:37  * mjr_quit (Client Quit)
19:23:24  * hzquit
19:23:36  * loladirojoined
19:29:36  * EhevuTovjoined
19:33:20  * felixgejoined
19:35:20  * felixgequit (Read error: Connection reset by peer)
19:35:25  * felixge_joined
19:35:25  * felixge_quit (Changing host)
19:35:25  * felixge_joined
19:36:24  * loladiroquit (Quit: loladiro)
19:38:36  <isaacs>bnoordhuis: any promising insights?
19:42:36  * mikealjoined
19:43:35  <dscape>indutny: yt?
19:43:38  <indutny>yes
19:43:40  <dscape>im trying to use clieasy :\
19:43:54  <dscape>how can i write to stdin?
19:43:57  <indutny>erm
19:44:07  <indutny>this is a module I wrote?
19:44:10  <indutny>I take it? :)
19:44:14  <dscape>according to the credits
19:44:14  <dscape>yes
19:44:16  <dscape>:)
19:44:29  <dscape>https://github.com/flatiron/cli-easy
19:44:43  <indutny>ok
19:44:46  <indutny>next question
19:44:51  <indutny>write to stdin!? WTF!
19:45:02  <indutny>ah
19:45:04  <indutny>from other side
19:45:07  <dscape>well people often do that in cli apps
19:45:22  <indutny>well
19:45:25  <indutny>let me see
19:45:48  <dscape>testing clis is terrible, i hoping this to be my salvation
19:45:52  <dscape>normally i just dont test stuff
19:45:52  <dscape>lol
19:46:07  <indutny>it does child_process.exec
19:46:15  <dscape>i manually run commands, or end up straight up using diff
19:46:28  <indutny>so no way of writing to stdin
19:46:29  <indutny>:(
19:46:40  <indutny>I wonder why I hadn't thought about it before
19:47:04  <dscape>indutny: did this module end up not being used then?
19:47:10  <indutny>I suppose so
19:47:15  <indutny>it way like api-easy for clis
19:47:19  <indutny>but demand was pretty low
19:47:21  <indutny>brb
19:47:24  <dscape>thanks indutny
19:49:33  * mjr_joined
19:52:07  <indutny>dscape: you're welcome
19:52:55  <dscape>btw indutny im still using it
19:53:04  <dscape>to my knowledge it is still the best thing out there
19:53:05  <dscape>lol
20:05:53  * indexzerojoined
20:06:40  * loladirojoined
20:06:56  * jmar777joined
20:07:33  * indexzeroquit (Client Quit)
20:09:13  <indutny>dscape: :)
20:09:21  <indutny>dscape: you should add stdin support then
20:17:50  * mjr_quit (Quit: mjr_)
20:29:44  * indexzerojoined
20:30:02  <piscisaureus_>rendar: yes, that's how it works
20:37:50  * jmar777quit (Remote host closed the connection)
20:38:26  * jmar777joined
20:39:20  * jmar777quit (Read error: Connection reset by peer)
20:39:28  * jmar777joined
20:53:25  * paddybyersjoined
20:57:29  * jmar777quit (Remote host closed the connection)
20:58:00  * txdv_joined
20:58:01  * jmar777joined
20:58:03  * piscisaureus_quit (Quit: ~ Trillian Astra - www.trillian.im ~)
20:58:03  * txdv_quit (Client Quit)
20:59:27  * piscisaureus_joined
20:59:51  * paddybyersquit (Ping timeout: 260 seconds)
21:02:30  * jmar777quit (Ping timeout: 264 seconds)
21:07:34  * ryahjoined
21:07:42  <ryah>what's up guys
21:07:59  <ryah>when are buffers going to get backed by typed arrays?
21:08:13  * pooyajoined
21:09:42  * paddybyersjoined
21:10:00  * pooyaquit (Client Quit)
21:10:00  <rendar>piscisaureus_: i see, but what i can't get from libuv sources is: when ConnectEx returns ERROR_IO_PENDING, the request will be completed lately, but what about the connection has errors (e.g. connection refused or connection timed out) ? is IOCP callback called in such case?
21:11:59  <piscisaureus_>ryah: don't know
21:12:02  <piscisaureus_>rendar: yes
21:13:34  <rendar>piscisaureus_: hmm, the system call the iocp callback? and how you know that the completion failed? is this documented somewhere?
21:13:46  <piscisaureus_>rendar: msdn
21:13:57  <piscisaureus_>rendar: have you written anything with iocp so far?
21:14:19  <piscisaureus_>rendar: because the system does not call any callbacks when you use iocp ...
21:15:16  <rendar>well, yes, you're right, it puts the completion and GetQueuedCompletionStatus() returns
21:16:58  <piscisaureus_>rendar: let me copy the relevant portions for you:
21:16:59  <piscisaureus_>Upon failure (the return value is FALSE), those same parameters can contain particular value combinations as follows:
21:16:59  <piscisaureus_>... <snip> ...
21:16:59  <piscisaureus_>If *lpOverlapped is not NULL and the function dequeues a completion packet for a failed I/O operation from the completion port, the function stores information about the failed operation in the variables pointed to by lpNumberOfBytes, lpCompletionKey, and lpOverlapped. To get extended error information, call GetLastError.
21:17:47  <rendar>is this from ConnectEx page?
21:18:59  <piscisaureus_>No the GetQueuedCompletionStatus page
21:21:49  <rendar>piscisaureus_: thanks, finally i saw what i was doing wrong, in my callback i had while (GetQueuedCompletionStatus(..params)) so on ConnectEx error the thread just died
21:23:53  * EhevuTovquit (Quit: This computer has gone to sleep)
21:26:12  * piscisaureus_quit (Read error: Connection reset by peer)
21:26:40  * piscisaureus_joined
21:27:32  <txdv>Hello, i am back :P
21:43:08  * rendarquit
22:01:29  * wolfeidauquit (Remote host closed the connection)
22:02:52  <bnoordhuis>isaacs: promising insights, yes. it's not v8 and it's not really libuv but master's libuv and the slab allocator don't play nice for some reason
22:06:16  * pooyajoined
22:08:58  * brsonjoined
22:13:21  * pooyaquit (Quit: pooya)
22:14:31  * wolfeidaujoined
22:16:05  * Benviequit
22:21:45  * indexzeroquit (Quit: indexzero)
22:27:20  * lohkeyjoined
22:28:32  * lohkeyquit (Client Quit)
22:32:10  * warzjoined
22:32:12  * warzquit (Changing host)
22:32:12  * warzjoined
22:33:46  * EhevuTovjoined
22:35:48  * indexzerojoined
22:56:47  * `3rdEdenquit (Remote host closed the connection)
23:05:03  * wolfeidauquit (Read error: Connection reset by peer)
23:05:10  * wolfeida_joined
23:19:16  <isaacs>bnoordhuis: nice :)
23:30:01  * creationixquit (Ping timeout: 245 seconds)
23:34:46  * qmx|awaychanged nick to qmx
23:38:09  * qmxchanged nick to qmx|away
23:46:16  * creationixjoined