00:00:00  * ircretaryquit (Remote host closed the connection)
00:00:08  * ircretaryjoined
00:13:53  * paddybyers_quit (Ping timeout: 245 seconds)
00:21:11  * TooTallNatequit (Quit: ["Textual IRC Client: www.textualapp.com"])
00:21:40  * pooyaquit (Quit: pooya)
00:58:52  * loladiroquit (Quit: loladiro)
01:05:51  * jmar777joined
01:08:41  * qmx|awaychanged nick to qmx
01:11:10  * brsonjoined
01:28:17  * brsonquit (Remote host closed the connection)
01:29:10  * loladirojoined
02:01:56  * piscisaureus_quit (Ping timeout: 248 seconds)
02:07:39  * piscisaureus_joined
02:07:43  * rvaggjoined
02:21:15  * stagasquit (Ping timeout: 256 seconds)
02:30:34  * AvianFlujoined
02:40:43  * bnoordhuisquit (Ping timeout: 246 seconds)
02:47:56  * erickt_joined
02:47:56  * ericktquit (Read error: Connection reset by peer)
02:47:56  * erickt_changed nick to erickt
02:48:35  * pooyajoined
02:51:01  * qmxchanged nick to qmx|away
02:51:06  * piscisaureus_quit (Quit: ~ Trillian Astra - www.trillian.im ~)
02:58:39  * loladiroquit (Quit: loladiro)
03:01:24  * jmar777quit (Remote host closed the connection)
03:02:02  * jmar777joined
03:06:11  * jmar777quit (Ping timeout: 248 seconds)
03:20:35  * brsonjoined
03:29:34  * lohkeyjoined
03:33:16  * wavdedjoined
03:34:25  <wavded>in libuv, are timers handled on a seperate thread or is epoll, etc. handling that? strace output would make me think there is some threading going on but i could be reading it wrong
03:35:20  * loladirojoined
03:39:54  * pooyaquit (Quit: pooya)
03:59:52  * c4milojoined
04:01:02  <tjfontaine>wavded: at least from the unix side of things it's a min heap, that checks to see if there are any timers to run
04:01:20  <tjfontaine>wavded: so it's a normal part of loop iteration
04:01:58  <tjfontaine>first thing the loop iteration does is update time, then checks for timers that need to be run
04:03:26  <tjfontaine>https://github.com/joyent/node/pull/3872#issuecomment-7804775 has a nice flow chart for timers (and other queue processing) in node
04:06:23  * indexzerojoined
04:09:45  <wavded>tjfontaine: thanks for the link, ahh so epoll_wait (and others) will return when a timer is done?
04:10:51  * lohkeyquit (Quit: lohkey)
04:13:20  <tjfontaine>wavded: uv__timers is called before uv__io_poll (which is what calls epoll_wait), so timers will fire before io (presuming enough time has passed for the timer execution to be valid)
04:13:40  <tjfontaine>https://github.com/joyent/libuv/blob/master/src/unix/core.c#L282
04:18:40  * jmar777joined
04:28:12  * EhevuTovjoined
04:31:04  * jmar777quit (Remote host closed the connection)
04:34:25  <wavded>tjfontaine: ok, but epoll_wait blocks until a OS event comes in, so if a timer is running on that same thread how does epoll not get in the way of that
04:42:20  <wavded>tjfontaine: seems like I found the answer, epoll_wait will a timeout of 0 will return immediately if no events to process
04:42:28  <wavded>will = with
04:44:26  <wavded>tjfontaine: seems like sometimes its called with a timeout and sometimes its 0 from the couple apps I've tried
04:45:55  <wavded>ahh and its in that code you linked :)
04:47:30  <tjfontaine>use the source luke :)
04:47:35  * c4miloquit (Remote host closed the connection)
04:50:25  <wavded>tjfontaine: the source, i use ;) thanks for your help, think I'm wrapping my brain around this more.
04:55:26  * wavdedquit (Quit: WeeChat 0.3.9.2)
05:04:04  * rumpjoined
05:07:13  * qmx|awayquit (Ping timeout: 245 seconds)
05:10:54  * qmx|awayjoined
05:21:04  * pooyajoined
05:56:37  * indexzeroquit (Quit: indexzero)
06:10:45  * indexzerojoined
06:24:01  * rumpquit (Quit: rump)
06:24:21  * ericktquit (Quit: erickt)
06:32:29  * tomshredsjoined
06:35:21  * brsonquit (Ping timeout: 255 seconds)
06:36:24  * pooyaquit (Quit: pooya)
07:26:33  * pooyajoined
07:37:28  * indexzeroquit (Quit: indexzero)
07:38:22  * indexzerojoined
07:42:15  * rendarjoined
07:52:40  * mikealjoined
07:53:48  * indexzeroquit (Quit: indexzero)
07:57:30  * pooyaquit (Quit: pooya)
08:04:07  * pooyajoined
08:10:14  * EhevuTovquit (Quit: This computer has gone to sleep)
08:25:31  * loladiroquit (Quit: loladiro)
08:26:25  * indexzerojoined
08:29:21  * tomshredsquit (Quit: Linkinus - http://linkinus.com)
08:30:18  * loladirojoined
08:36:45  * pooya_joined
08:38:27  * pooyaquit (Ping timeout: 248 seconds)
08:38:28  * pooya_changed nick to pooya
08:48:57  * karupaneruraquit (Excess Flood)
08:52:11  * karupanerurajoined
09:01:59  * pooyaquit (Quit: pooya)
09:26:59  * paddybyersjoined
09:33:47  * AvianFluquit (Remote host closed the connection)
09:59:29  * indexzeroquit (Quit: indexzero)
10:12:39  * indexzerojoined
10:28:59  * indexzeroquit (Quit: indexzero)
10:57:25  * loladiroquit (Quit: loladiro)
11:32:24  * loladirojoined
11:32:30  * loladiroquit (Client Quit)
11:52:25  * stagasjoined
11:59:41  * bnoordhuisjoined
12:13:56  * stagas_joined
12:15:42  * stagasquit (Ping timeout: 256 seconds)
12:15:51  * stagas_changed nick to stagas
12:15:56  * TheJHjoined
12:40:28  <indutny>bnoordhuis: hoya
12:40:32  <bnoordhuis>indutny: heya
12:40:46  <indutny>what do you think about removing POSIX_C_SOURCE from uv.gyp for freebsd?
12:40:54  <indutny>apparently it prevents __BSD_VISIBLE from being set
12:41:02  <indutny>and setgroups() and getgroups() are not available
12:41:05  <indutny>and many other things too
12:41:14  <indutny>like uchar
12:41:43  <indutny>seems to be working fine for me
12:41:52  <indutny>bnoordhuis: thoughts?
12:42:40  <indutny>let me open pull request for it
12:42:58  <bnoordhuis>indutny: that doesn't sound right to me
12:43:08  <indutny>have you seen sys/cdefs.h ?
12:43:11  <indutny>that's the only way
12:43:22  <indutny>and __POSIX_VISIBLE will use default value too
12:43:31  <bnoordhuis>what happens when you define _BSD_SOURCE?
12:44:02  <indutny>everything fails to build
12:44:24  <indutny>please wait for PR
12:44:26  <bnoordhuis>come to think of it, i believe _BSD_SOURCE is actually a linuxism
12:44:41  <bnoordhuis>one that unhides bsd prototypes
12:45:26  <indutny>one sec
12:49:23  <indutny>bnoordhuis: https://github.com/joyent/libuv/pull/691
12:53:10  * paddybyersquit (Ping timeout: 252 seconds)
12:59:49  * TheJHquit (Read error: Operation timed out)
13:11:30  * paddybyersjoined
13:23:33  * `3rdEdenjoined
13:27:22  <bnoordhuis>indutny: still around? i'll look at it in a bit
13:27:31  <indutny>yes
13:29:24  <bnoordhuis>indutny: want to see something joyous?
13:29:25  <MI6>joyent/libuv: bnoordhuis created branch threadpool-rework - http://git.io/3HqJFg
13:29:28  <bnoordhuis>^ there
13:29:35  <indutny>one sec
13:29:37  <indutny>finishing watching film
13:29:58  <bnoordhuis>that threadpool rework would've been finished a month ago if not for all the interruptions :/
13:31:47  <bnoordhuis>i think i broke the tests again with my last touchup :)
13:34:43  * TheJHjoined
13:41:35  <indutny>I'm reading your commits
13:41:39  <indutny>and thinking about one interesting thing
13:41:51  <indutny>what if we could set request priority in libuv
13:42:11  <indutny>and requests with different priorities will operate on different threads with different priorities
13:42:19  * piscisaureus_joined
13:42:20  <indutny>do you think there'll be any difference at all?
13:42:27  <indutny>I'm asking because this might be useful for spdy
14:09:09  <indutny>no response :)
14:13:49  * sgallaghjoined
14:18:17  <indutny>bnoordhuis: still there?
14:20:55  <indutny>so, like I need to play with it
14:39:16  * hzjoined
14:47:43  <bnoordhuis>back
14:47:46  <MI6>joyent/libuv: Ben Noordhuis threadpool-rework * f794d3e : unix: implement a better threadpool Split the threadpool in two: a small - http://git.io/WtA2RQ
14:48:17  <bnoordhuis>indutny: come again re: thread priorities?
14:58:09  <indutny>ok
14:58:12  <indutny>yes
14:58:15  <indutny>thread priorities
14:58:25  <indutny>something that eio has
14:58:31  <indutny>but it wasn't doing it properly
15:00:18  <indutny>bnoordhuis: you follow?
15:00:25  <indutny>add node.js domains to it
15:00:33  <indutny>and voila you can schedule your requests by priority
15:01:02  <bnoordhuis>hmm, maybe
15:01:05  <piscisaureus_>does eio have thread priorities?
15:01:08  <piscisaureus_>I don't think so
15:01:10  <indutny>not threads
15:01:12  <indutny>request priorities
15:01:18  <indutny>but it does it by having separate queues
15:01:28  <indutny>so starvation is possible
15:01:33  <piscisaureus_>I think only libev had that
15:01:36  <indutny>aah
15:01:40  <indutny>yeah, probably
15:01:44  <indutny>anyway
15:01:53  <indutny>now as we've only one thread pool and it's completely in our hands
15:01:55  <indutny>we can think about it
15:02:01  <indutny>that'd be really useful for spdy
15:02:06  <bnoordhuis>how so?
15:02:14  <indutny>SPDY protocol has stream priorities
15:02:27  <piscisaureus_>priorities make sense only for cpubound stuff
15:02:39  <indutny>hm...
15:02:43  <indutny>yes, I was thinking about it
15:02:45  <indutny>are you sure?
15:02:57  <indutny>what if one thread is reading 10gb file
15:03:11  <indutny>idk
15:03:12  <bnoordhuis>indutny: that's why there are lots of i/o threads
15:03:52  <indutny>well, yes
15:04:04  <indutny>so thread priority may help here, isn't it?
15:05:45  <tjfontaine>bnoordhuis: excellent change, now I can just tell
15:05:54  <tjfontaine>people to use .10 when it comes out
15:06:13  <bnoordhuis>tjfontaine: for node-dns?
15:06:23  <tjfontaine>no, for some imagemagick stuff
15:07:01  <tjfontaine>that always had problems with the smaller stack size (at least when i was using the eio custom stuff)
15:07:25  <bnoordhuis>tjfontaine: how large should the stack be for you?
15:07:35  <bnoordhuis>the cpu threads run on smaller stacks as well (256k)
15:07:50  <bnoordhuis>but they don't really have to
15:08:05  <bnoordhuis>so now is the time to speak up :)
15:08:15  <tjfontaine>I didn't really narrow it down, but I can rerun the test that caused it to break and find out :)
15:10:08  <piscisaureus_>smaller stacks dont help
15:10:28  * jmar777joined
15:10:44  <piscisaureus_>this snow storm is pretty crazy
15:10:49  <indutny>you kidding
15:10:51  <indutny>it's insane
15:11:10  <indutny>s/it's/its
15:11:21  <piscisaureus_>in moscow?
15:11:25  <indutny>yeah
15:11:27  <indutny>it was here too
15:11:34  <indutny>not sure if its still there now
15:11:39  <indutny>s/there/here/
15:11:51  <indutny>I was sitting in house for almost two days
15:12:22  <indutny>whoa, I've finally figured out how to build node with dtrace support on freebsd
15:17:07  <indutny>bnoordhuis: https://github.com/joyent/node/pull/4630
15:17:14  <indutny>and that's not all
15:18:06  <indutny>so it builds
15:18:10  <indutny>but nothing is really working
15:18:12  <indutny>yikes!
15:18:16  <indutny>I mean dtrace ustack helper
15:21:06  <indutny>oh
15:21:08  <bnoordhuis>hmm. numcpus * 16 i/o threads makes the fs_stat benchmark flatline with 7+ concurrent requests
15:21:09  <indutny>probably it is
15:21:14  <bnoordhuis>i guess some tuning is in order
15:21:39  <indutny>bnoordhuis: mind pulling some patches?
15:22:57  <bnoordhuis>indutny: if that means reviewing them first, then yes :)
15:23:03  <indutny>:)
15:23:05  <indutny>surely yes
15:23:09  <indutny>one sec
15:23:15  <indutny>I'll add one more commit to that pull request
15:24:06  <bnoordhuis>it's funny, reducing the # of i/o threads to 2, quadruples the throughput
15:24:34  <bnoordhuis>660k reqs/s instead of 125k with concurrency=32
15:24:41  <indutny>scheduling
15:24:43  <indutny>and also
15:24:52  <indutny>kernels don't really work well with concurrent IO
15:24:52  <bnoordhuis>yes, and less locking overhead
15:24:53  <indutny>AFAIK
15:25:08  <indutny>I was writing some db engines years ago
15:25:16  <indutny>concurrent write/read isn't working really well
15:25:25  <indutny>especially to one file
15:25:33  <indutny>not sure if it applies to sockets too
15:25:47  <indutny>bnoordhuis: updated PR https://github.com/joyent/node/pull/4630
15:26:10  <bnoordhuis>the fs_stat stats ".". it's probably causing lots of dirent cache contention
15:26:23  <bnoordhuis>perhaps it's not a really good benchmark
15:28:05  <bnoordhuis>hah, `perf record` actually speeds it up: 100,000 stats (32 concurrent): 0.05s (1,833,719/s)
15:28:16  <indutny>haha
15:28:22  <indutny> elf_begin() failed: Cannot allocate memory
15:28:24  <indutny>nice
15:29:27  <indutny>ah, that's not the reason
15:29:28  <indutny>this is:
15:29:28  <indutny>dtrace: ERROR: open ode/0.9.7/bin/node failed: No such file or directory
15:29:37  <indutny>and this dtrace: ERROR: open ecinfo.so.1 failed: No such file or directory
15:29:47  <indutny>and probably even this trace: ERROR: open s???Y????,?5?y??y??7y???s5???9??_??!?_?y??cϵ4?,?sJ?3h?k?????y??
15:30:02  <bnoordhuis>29.31% run-benchmarks [kernel.kallsyms] [k] __ticket_spin_lock <- guess that explains it
15:30:26  <indutny>ok, back to watching films
15:30:30  <bnoordhuis>do that :)
15:30:38  <indutny>:)
15:30:46  <indutny>can you please pull this two commits
15:30:51  <indutny>and make me happy
15:30:55  <indutny>and proud of myself
15:31:05  <indutny>you can revert them later if you want
15:31:28  <indutny>oh
15:31:30  <indutny>and this too https://github.com/joyent/libuv/pull/691
15:41:48  <bnoordhuis>maybe tonight or tomorrow, i'm afk for a bit
15:42:09  <indutny>ah, ok
15:42:09  <indutny>np
15:51:08  <indutny>bnoordhuis: sorry for bothering you
15:51:13  <indutny>but probably you know what does this mean
15:51:13  <indutny>./out/Release/obj.target/node/src/node_dtrace_ustack.o: ELF 64-bit LSB relocatable AMD64 Version 1 [SSE]
15:51:20  <indutny>"[SSE]" <-
15:51:20  <LOUDBOT>IT WOULD APPEAR THAT YOU DID NOT, INDEED, BREAK THE UNIVERSE.
15:51:47  <indutny>CAPSLOCKBOT: LOUDBOT: hello
15:51:51  <indutny>CAPSLOCKBOT: LOUDBOT: HUH?!
15:51:51  <LOUDBOT>HOW WAS I SUPPOSED TO KNOW IT WAS A REAL HOLIDAY, I MEAN YOM KIP PURE? THAT SOUNDS SO FAKE!
15:55:52  * skebcioquit (Quit: No Ping reply in 180 seconds.)
15:56:42  * skebciojoined
15:57:15  * hzquit
16:00:12  * `3rdEdenquit
16:41:02  * ericktjoined
17:20:29  * rumpjoined
17:22:10  * AvianFlujoined
17:24:08  * sgallaghquit (Ping timeout: 252 seconds)
18:22:38  * TheJHquit (Read error: Operation timed out)
18:25:41  * qmx|awaychanged nick to qmx
18:25:55  * qmxquit (Changing host)
18:25:55  * qmxjoined
18:47:07  * loladirojoined
19:00:57  * loladiroquit (Quit: loladiro)
19:04:59  * TheJHjoined
19:12:52  * loladirojoined
19:16:21  * cjdjoined
19:17:45  <cjd>hi guys, still trying to figure out how to poll on a TUN device with libuv and ran into an issue, TUN is just a char device and when you write() to it, it makes a packet and when you read() from it, it gets one.
19:17:49  <cjd>Now the annoying part
19:18:20  <cjd>They support writev() and readv() but they don't put an array of packets into your iovec, they put parts of a single packet
19:18:58  <cjd>is there any way to abuse the stream API and keep it from using writev(), I see if it only has a single write to do, it will only call write()
19:20:31  <indutny>cjd: I think this is not configurable right now
19:20:49  <indutny>are you writing node.js addon or an application that is using libuv
19:21:23  <indutny>if latter one - I think we may consider some options to fix this problem
19:21:27  <indutny>like another uv API call
19:21:33  <indutny>or stream option
19:21:37  <indutny>probably stream option
19:21:51  <indutny>cjd: if that's ok for you - please ping me or bnoordhuis back tomorrow
19:21:57  <indutny>at the moment we're all away
19:22:08  <indutny>and piscisaureus_ too :)
19:22:27  <cjd>no problem, thanks
19:22:39  <cjd>yeah, my own application so this is pretty flexable
19:22:45  <cjd>thanks for your help
19:23:19  <indutny>np
19:23:24  <indutny>you're welcome
19:26:36  * loladiroquit (Quit: loladiro)
19:31:23  * paddybyersquit (Ping timeout: 245 seconds)
19:34:47  * paddybyersjoined
19:42:05  * `3rdEdenjoined
19:47:22  * brsonjoined
19:47:37  * tomshredsjoined
19:47:37  * tomshredsquit (Remote host closed the connection)
19:47:52  * tomshredsjoined
19:48:58  * tomshredspart
20:13:05  * mikealquit (Quit: Leaving.)
20:14:09  * bnoordhuisquit (Ping timeout: 260 seconds)
20:15:02  * mikealjoined
20:41:08  * qmxchanged nick to qmx|away
21:09:49  * mikealquit (Quit: Leaving.)
21:27:08  * mikealjoined
21:44:26  * rendarquit
21:44:59  * mikealquit (Quit: Leaving.)
21:52:43  * wolfeidauquit (Remote host closed the connection)
22:08:17  * wolfeidaujoined
22:30:54  * c4milojoined
22:32:58  * loladirojoined
22:39:24  * trevnorrisjoined
22:54:32  * trevnorrisquit (Quit: Leaving)
23:07:32  * paddybyersquit (Remote host closed the connection)
23:07:46  * paddybyersjoined
23:14:25  * paddybyersquit (Ping timeout: 260 seconds)
23:22:52  * TheJHquit (Ping timeout: 246 seconds)
23:26:30  * `3rdEdenquit (Quit: Zzzz)
23:44:53  * trevnorrisjoined
23:45:11  * rumpquit (Quit: rump)
23:45:30  * rumpjoined
23:45:43  * rumpquit (Client Quit)