02:45:37  * slurpjoined
02:45:37  * ryahtopic: v0.5.5 release manager is bnoordhuis
02:47:29  <piscisaureus>http://piscisaureus.no.de/
02:47:29  <piscisaureus>Let's see how long this is going to survive
02:49:24  <DrPizza>hah
02:49:25  <DrPizza>nice
02:54:41  <piscisaureus>It will crash as soon something "rare" happens, e.g. ryah kicking someone, CIA-95 doing its fancy color thing
02:54:53  <DrPizza>heh.
02:55:05  <DrPizza>how 'bout some bold
02:55:14  <DrPizza>that works
02:55:14  <piscisaureus>so far so good
02:55:23  <DrPizza>even emboldened in the log
02:55:43  <piscisaureus>mirc formatting is easy
02:55:57  <DrPizza>italicunderlinebold
02:56:09  <DrPizza>cool, it inverts
02:56:21  <DrPizza>as it should
02:56:23  <DrPizza>but this client doesn't
02:56:36  <piscisaureus>heh, mine neither
02:56:42  <piscisaureus>it also support monospace
02:56:45  <DrPizza>this one italicizes
02:57:03  <DrPizza>man this client doesn't even have /ver so I dunno what you're even using
02:57:09  <piscisaureus>but I have not been able to test monospace because my client doesnt support it
02:58:44  <piscisaureus>DrPizza: the irc client is https://github.com/martynsmith/node-irc, the formatting support I hacked together myself
02:58:53  * piscisaureusafw
02:58:58  * piscisaureus*afk
02:59:31  <DrPizza>how robust is this irc library
02:59:39  <piscisaureus>shrug
02:59:41  <DrPizza>writing an irc bot is the main reason that I even looked at node in the first place
03:00:11  <DrPizza>I want an IRC bot that lets people program him, in javascript, in-channel
03:00:20  <DrPizza>to create new binds
03:00:37  <DrPizza>on another network I use, someone has a similar bot using tcl
03:00:40  <DrPizza>but who wants to write tcl
03:05:26  <piscisaureus>I know nothing about tcp
03:05:29  <piscisaureus>*tcl
03:05:35  <DrPizza>lucky you
03:05:56  <piscisaureus>I was able do write this thingie in windows
03:06:03  <DrPizza>excellent
03:06:05  <DrPizza>that's very promising
03:06:15  <DrPizza>that's basically what I wanted
03:06:20  <DrPizza>node that's good enough to write my IRC bot in windows
03:06:24  <DrPizza>before deploying him on lunix
03:06:33  <piscisaureus>the only thing that's annoying is that there's no npm support
03:06:42  <DrPizza>yes
03:06:52  <isaacs>piscisaureus: post knockout
03:06:55  <isaacs>piscisaureus: that's high on my list.
03:07:01  <piscisaureus>the other issue is that I can't check in symlinks with msysgit because msysgit doesn't support symlinks
03:07:13  <piscisaureus>very unfortunate for node development
03:07:43  <piscisaureus>when this node-windows thing is done I should fix git for windows
03:10:52  * isaacsquit (Quit: isaacs)
03:33:07  * johnm1234quit (Ping timeout: 250 seconds)
03:51:20  * johnm1234joined
03:56:57  * piscisaureustopic: test
03:57:10  * piscisaureustopic: v0.5.5 release manager is bnoordhuis
04:05:09  <piscisaureus>ë �
04:08:46  * piscisaureusquit (Quit: ~ Trillian Astra - www.trillian.im ~)
04:11:35  * piscisaureusjoined
04:14:02  * piscisaureustopic: v0.5.5 release manager is bnoordhuis
04:14:58  * piscisaureusquit (Client Quit)
04:17:38  * piscisaureusjoined
04:25:04  * piscisaureusquit (Ping timeout: 258 seconds)
05:05:37  <DrPizza>igorzi: neat :)
05:16:20  <igorzi>DrPizza: what's up?
05:16:42  <DrPizza>the e-mail you were just cced on, nice to know that my thinking isn't completely nuts
05:18:31  <igorzi>oh yeah, right.. just saw it
06:04:56  * mralephjoined
07:01:52  * mralephquit (Quit: Leaving.)
09:52:49  * bnoordhuisjoined
12:05:50  * piscisaureusjoined
12:24:49  * pquernaquit (Read error: Connection reset by peer)
12:24:55  * pquernajoined
12:33:56  <CIA-95>libuv: Ben Noordhuis master * r358d89a / Makefile :
12:33:56  <CIA-95>libuv: build: fix build on systems where `ld --as-needed` is the default
12:33:56  <CIA-95>libuv: Fixes unresolved symbol linker errors from -lm and -lrt. - http://bit.ly/nQcL6c
12:36:50  <bnoordhuis>^ software puritan appeasement >:(
12:52:06  <CIA-95>node: Antranig Basman master * rc05936c / (3 files in 3 dirs):
12:52:06  <CIA-95>node: vm: fix incorrect dispatch of vm.runInContext for argument "filename"
12:52:06  <CIA-95>node: Adds test case and documentation for vm.runInContext and vm.createContext.
12:52:06  <CIA-95>node: Fixes #1140. - http://bit.ly/mPtJGz
13:21:46  <DrPizza>hmm
13:22:01  <DrPizza>I wonder how hard it would be to use cryptoAPI/schannel instead of openssl
13:54:32  <piscisaureus>DrPizza: but why would you want that in the first place?
13:56:52  <DrPizza>it would obviate the need to integrate openssl into the gyp/msvs build,
13:57:33  <DrPizza>it would mean that node automatically used hardware crypto devices, if available
14:00:05  <DrPizza>it would mean that node picked up standard Windows configuration, such as barring the use of weak hashes/encryption algorithms
14:01:22  <DrPizza>it would probably open the door to allowing node to use certificates from the windows certificate store, for both server and client certificates
14:03:23  <DrPizza>it would mean, I believe, that SSL handling were consistent if node were to get http.sys integration
14:26:10  <piscisaureus>DrPizza: it scares me. It would mean that we would have separate implementations of node_crypto and http as well.
14:26:30  <DrPizza>true
14:26:45  <piscisaureus>A good recipe for an unmaintainable clusterfuck
14:28:26  <DrPizza>well
14:28:43  <DrPizza>I don't know
14:28:59  <DrPizza>schannel and openssl with BIO seem to work in a pretty similar way
14:30:14  <piscisaureus>I think selene could have crypto api support
14:30:41  <piscisaureus>but I don't think we can start using that anytime soon
14:30:53  <piscisaureus>(that's what pquerna says anyway)
14:31:45  <piscisaureus>besides, selene implements tls handshake, cypher selection itself. It only uses other libraries for the actual cyphers/hmacs
14:32:29  <DrPizza>interesting
14:33:43  <DrPizza>piscisaureus: is the long-term intent to switch over to selene?
14:34:00  <piscisaureus>undecided
14:34:12  <bnoordhuis>the long-term intent is to make the web a place where encryption isn't necessary
14:34:17  <bnoordhuis>and where the lamb lies with the lion
14:34:48  <bnoordhuis>also: yo nerds
14:34:57  <DrPizza>piscisaureus: using the other libraries only for crypto would certainly make integrating it with other crypto libraries much easier
14:35:13  <bnoordhuis>piscisaureus: how's the stdio stuff coming along?
14:35:45  <piscisaureus>bnoordhuis: don't ask
14:35:56  <bnoordhuis>piscisaureus: just did :)
14:36:02  * jonaslund_joined
14:36:06  <bnoordhuis>ready for thursday or not?
14:36:35  * jonaslundquit (Ping timeout: 260 seconds)
14:36:41  * jonaslund_changed nick to jonaslund
14:37:14  <piscisaureus>bnoordhuis: not as such
14:37:26  <piscisaureus>bnoordhuis: there will be stuff landed that relates to this
14:37:37  <bnoordhuis>piscisaureus: do tell
14:37:41  <piscisaureus>bnoordhuis: but the problem surface was too big for me to solve at once
14:38:22  <piscisaureus>bnoordhuis: let's call
14:38:34  <bnoordhuis>now or tonight?
14:38:50  <bnoordhuis>piscisaureus: ^
14:38:51  <piscisaureus>whatever. I was thinking nwo
14:38:58  <bnoordhuis>okay, let me get a drink first
14:39:06  <bnoordhuis>i can't chat when i'm sober
14:39:13  <DrPizza>lol
14:43:03  <bnoordhuis>piscisaureus: you're not online, you rascal
14:43:09  <bnoordhuis>oh wait, now you are
14:43:15  <piscisaureus>oh you want to skype?
14:43:16  <piscisaureus>aok
14:43:36  <bnoordhuis>yeah, my phone's dead
14:55:11  <piscisaureus>https://github.com/bnoordhuis/libuv/compare/udp
15:01:06  <CIA-95>node: koichik master * r8293bb8 / (26 files in 3 dirs):
15:01:06  <CIA-95>node: test: refactored http test.
15:01:06  <CIA-95>node: Many http tests had used legacy http.Client.
15:01:06  <CIA-95>node: This refactored it to use modern API.
15:01:06  <CIA-95>node: Fixes #1528. - http://bit.ly/nLhELf
15:05:04  * bnoordhuistopic: v0.5.5 release manager is bnoordhuis - this week's feature: dgram support
15:05:15  <piscisaureus>bnoordhuis: what's the purpose of uv_udp_bind?
15:05:29  <bnoordhuis>piscisaureus: bind a socket to an address + port
15:05:46  <piscisaureus>bnoordhuis: then call uv_listen after that?
15:06:07  <bnoordhuis>piscisaureus: no, you don't have to listen
15:06:21  <bnoordhuis>but if you don't bind, you get assigned address INADDR_ANY + a random port
15:07:18  <piscisaureus>bnoordhsui: should we nog make uv_bind generic?
15:07:24  <piscisaureus>oh hmm
15:07:32  <piscisaureus>there's also pipe_bind
15:07:40  <bnoordhuis>i looked at that but there's something of a mismatch
15:07:41  <piscisaureus>but it seems to me that tcp_bind == udp_bind
15:07:47  <piscisaureus>what's that?
15:07:58  <bnoordhuis>yeah, i'm trying to remember what it was
15:08:10  <bnoordhuis>oh, the connection_cb
15:08:22  <bnoordhuis>udp / dgram doesn't have the concept of a connection
15:08:30  <piscisaureus>tcp_bind doesn't take a connection_cb
15:09:08  <bnoordhuis>you're right, that's uv_listen
15:09:17  <piscisaureus>maybe we should call it uv_ip_bind and uv_ip6_bind
15:09:36  <bnoordhuis>oh right, i remember what it is
15:10:16  <bnoordhuis>uv__tcp_bind calls uv__stream_open
15:10:36  <piscisaureus>hmm
15:10:36  <bnoordhuis>it might be possible to merge it
15:10:45  <bnoordhuis>but let's keep it simple for now
15:10:50  <piscisaureus>on windows I think it would be totally the same
15:11:01  <piscisaureus>but yeah ok
15:11:46  <bnoordhuis>one troublesome thing with udp
15:11:55  <bnoordhuis>errors can be emitted at anytime
15:12:16  <bnoordhuis>you try to send a packet and the kernel might return the error from the previous packet
15:12:31  <bnoordhuis>how does that work on windows?
15:16:13  <piscisaureus>bnoordhuis: I have no experience with that
15:16:22  <piscisaureus>http://msdn.microsoft.com/en-us/library/ms740148%28v=VS.85%29.aspx
15:16:25  <piscisaureus>^-- doesn't mention it
15:16:52  <bnoordhuis>piscisaureus: i can fill bookshelves with stuff you don't have experience with
15:16:57  <bnoordhuis>so can your girlfriend for that matter
15:17:30  <piscisaureus>bnoordhuis: ok so
15:17:32  <bnoordhuis>does msdn have something akin to `man 7 udp`?
15:17:43  <piscisaureus>I don't think windows does that
15:18:21  <bnoordhuis>okay, i'll try to think of a way to handle it graciously on the unices
15:25:28  <CIA-95>node: koichik master * r4cf931d / (lib/http2.js test/simple/test-http-legacy.js): (log message trimmed)
15:25:28  <CIA-95>node: http: improve compatibility of legacy API
15:25:28  <CIA-95>node: In http1, legacy http.Client shares one connection with multiple requests.
15:25:28  <CIA-95>node: But in http2, it uses concurrent connections.
15:25:28  <CIA-95>node: With --use-http1, test/simple/test-http-legacy.js passes.
15:25:28  <CIA-95>node: However, it fails without --use-http1 (use http2).
15:25:29  <CIA-95>node: This improves compatibility of legacy http.Client API between http1 and http2.
16:35:33  <ryah>hello!
16:35:40  <ryah>what's new?
16:36:21  <ryah>bnoordhuis: what's the status? stdio, udp?
16:59:06  * isaacsjoined
16:59:39  <piscisaureus>ryah: hello
16:59:42  <ryah>piscisaureus: hello
16:59:55  <piscisaureus>stdio has kind of defeated me
17:00:05  <ryah>piscisaureus: oh no
17:00:11  <ryah>what's happening?
17:00:12  <piscisaureus>I'm going to make a plan and break it up into pieces
17:00:16  <piscisaureus>you should all help me :-)
17:00:24  <ryah>yes we can
17:00:35  <piscisaureus>ryah: it's too big to consider this as one project
17:00:59  <ryah>hm
17:01:11  <ryah>yeah, i didn't want this to become to large ...
17:01:15  * brsonjoined
17:01:27  <ryah>okay - but let's talk about it during the conf call today
17:01:49  <piscisaureus>yes
17:02:25  <ryah>you're attempting to provide an abstracted keystroke layer?
17:02:32  <ryah>is this why it becomes so large?
17:02:44  <piscisaureus>let me put it in a gist
17:03:14  <ryah>maybe we need to reconsider the proposition of this - we dont want to write a termcap library
17:10:00  <piscisaureus>ryah: https://gist.github.com/1149564
17:10:31  <piscisaureus>lots of work
17:12:01  <ryah>piscisaureus: can you put your code up?
17:15:45  * piscisaureusquit (Read error: Connection reset by peer)
17:19:57  <DrPizza>I wonder if there's some suitably-licensed library to deal with this stuff already
17:20:07  <DrPizza>because stuff like treating the different windows handles appropriately has surely been done before
17:20:18  <ryah>yeah
17:20:50  <DrPizza>though I admit, I'm not entirely sure what's out there
17:21:02  <ryah>it would be nice if windows just implemented vt100
17:21:30  <DrPizza>it would be nice if vt100 were better
17:21:55  <DrPizza>The Windows console abstraction is deeply awful
17:22:25  <ryah>yes
17:22:30  <DrPizza>like, there's basically no good way to write something such as screen for Windows.
17:23:10  <ryah>this is why i think our abstraction of windows tty should be encoding keystrokes into vt100
17:23:20  <ryah>yeah - it's overhead - but at least it's a sane abstraction
17:26:46  <DrPizza>although looking at piscis's gist of work, it looks like the things he had planned were lower level than that anyway
17:27:10  <DrPizza>related to doing consistent non-blocking I/O of the console, regardless of whether it's a real console, redirected file, or pipe
17:27:37  <DrPizza>that's all stuff that needs to exist
17:28:18  * brsonquit (Quit: leaving)
17:28:34  * brsonjoined
17:29:13  <DrPizza>ryah: i guess you'd have a kind of dichotomy; "real" console events, which are vt100, and "redirected" events, which are just regular file/pipe input
17:29:34  <DrPizza>I would think you still need to treat real console somewhat differently from the rest
17:30:19  <DrPizza>since real consoles can do colour, overtyping, etc. etc.
17:41:39  * isaacsquit (Quit: isaacs)
17:49:32  <ryah>DrPizza: yeah all the more reason to use VT escape codes :)
17:50:26  <ryah>btw we alredy do the pipe, file, tty switch
17:50:27  <ryah>https://github.com/joyent/node/blob/c05936ca13681059c7aeecfe3a608e4e1afa800a/src/node.js#L232-243
17:50:30  <ryah>just in JS land
17:56:08  <DrPizza>woudl the intent be to hoist that switch into libuv land (or node land I guess, maybe a bit of both?), and give node and/or libuv a higher level "console" notion?
17:58:03  * piscisaureusjoined
17:58:17  <ryah>DrPizza: yeah
17:58:25  <ryah>DrPizza: you'd just have a stdout handle
17:58:31  <ryah>you wouldn't have to worry about what it was
17:58:47  <ryah>with the idea that different platforms can handle that switch differently
17:59:21  <DrPizza>well, you'd want to know if it were a real console handle or not
17:59:27  <DrPizza>since you'd probably structure your output differently
17:59:40  <DrPizza>if you've got a redirected handle you won't try to move the cursor
18:00:35  <ryah>yeah
18:00:41  <ryah>you'd want to be able to query it
18:02:03  <piscisaureus>call?
18:02:28  <bnoordhuis>you're an hour early, piscisaureus
18:02:35  <ryah>bnoordhuis: nope its now
18:02:39  <bnoordhuis>ryah: stdio -> no, udp -> possibly
18:02:48  <piscisaureus>bnoordhuis: now!!
18:02:57  <bnoordhuis>21.00 hours ffs >:(
18:02:59  <bnoordhuis>bot okay
18:03:02  <bnoordhuis>*but
18:04:14  <piscisaureus>bnoordhuis: 0205002500
18:04:14  <piscisaureus>36815906#
18:05:15  <piscisaureus>bnoordhuis: coming?
18:05:18  <bnoordhuis>yes
18:26:18  * isaacsjoined
18:29:17  <piscisaureus>finally
18:29:23  <piscisaureus>... I get to eat the dinner I cooked
18:29:33  <igorzi>ryah: piscisaureus: the combination of no zero read + SetFileCompletionNotificationModes(FILE_SKIP_COMPLETION_PORT_ON_SUCCESS) gives 6% improvement
18:35:19  <ryah>6% is a bit more enticing
18:36:11  <igorzi>it's basically 4 from no zero read + 2 from FILE_SKIP_COMPLETION_PORT_ON_SUCCESS
18:36:33  * ryahis reading about FILE_SKIP_COMPLETION_PORT_ON_SUCCESS
18:36:46  <DrPizza>igorzi: so performing the callbacks in-thread for operations that complete immediately?
18:37:15  <igorzi>no, not to do the callback immediately, just skip going through the IOCP
18:37:25  <igorzi>(if it completes synchronosly)
18:37:40  <DrPizza>what are you using to call the callback then? I thought the callbacks only got called after dequeuing from the IOCP
18:37:54  <igorzi>we still add the request to the queue and let uv_run loop handle it
18:37:59  <DrPizza>oh I see
18:38:15  <ryah>igorzi: how will you call the callback?
18:38:25  <ryah>oh
18:38:25  <ryah>okay
18:38:29  <DrPizza>igorzi: doesn't this mean you're going to have to start locking those queues?
18:38:30  * ryah<-- slow
18:38:40  <DrPizza>since you're going to be making cross-thread modifications of them/
18:38:53  <ryah>DrPizza: there's only one thread atm :)
18:38:54  <igorzi>ryah: why? everything happens on main thread
18:38:54  <DrPizza>(currently I think the req queue only gets touched from the IOCP thread?)
18:39:38  <DrPizza>ryah: there's various bits and pieces that get done in threadpool threads
18:40:32  <DrPizza>I may be wrong, but I thought some of those could feasibly trigger operations that result in reqs getting queued
18:41:02  <igorzi>ryah, for example we issue WSARecv as a result of uv_read_start on main thread. WSARecv could complete synchronosly. If it does we put the req into the queue, and with FILE_SKIP_COMPLETION_PORT_ON_SUCCESS the IOCP packet never gets generated
18:41:20  <ryah>igorzi: yeah that sounds good
18:41:29  <ryah>we should definitely do that
18:41:43  <igorzi>the only downside is that this won't work on 2k3
18:41:48  <ryah>can that be done for WSASend too/
18:42:14  <igorzi>yep, i'm doing it for WSASend and AcceptEx
18:42:27  <igorzi>though i never see AcceptEx complete synchronosly
18:43:08  <igorzi>also, FILE_SKIP_COMPLETION_PORT_ON_SUCCESS seems to improve pump benchmark by about 8%; though that benchmark is very noisy for me
18:43:26  <ryah>^_^b
18:44:18  <ryah>igorzi: so will you able to ifdef out FILE_SKIP_COMPLETION_PORT_ON_SUCCESS for win2k3?
18:44:24  <ryah>is that going to require a lot of work?
18:44:42  <DrPizza>can't ifdef
18:44:45  <DrPizza>need a runtime choice
18:44:56  <ryah>really? okay - well wahtever
18:45:02  <igorzi>ryah: we'll need to do something similar to what we did for GetQueuedCompletionStatusEx (which also doesn't work on 2k3)
18:45:07  <ryah>oh right - you provide executables for all windows
18:45:09  <DrPizza>yes, the same node.exe needs to work on 2003 and 2008
18:45:12  <DrPizza>yeah
18:45:20  <ryah>s/you/i/ :)
18:47:19  <bnoordhuis>piscisaureus: re udp: does windows report send/receive errors?
18:47:50  <bnoordhuis>ryah: for that matter, does darwin?
18:47:58  <bnoordhuis>linux does, freebsd doesn't, not sure about solaris
18:48:06  <bnoordhuis>s/solaris/sunos/
18:48:15  <DrPizza>oh fair enough, all the threadpool operations post back to the IOCP manually to trigger their reqs.
18:48:17  <DrPizza>well that's OK then.
18:48:38  <ryah>bnoordhuis: *shrug*
18:48:55  <ryah>bnoordhuis: how can i test?
18:49:27  <bnoordhuis>hmm, `nc -u` some data to an unreachable host?
18:49:47  <bnoordhuis>no, that's not a good test - i don't think nc waits for udp errors
18:50:43  <ryah>where can i get ab's source code does it come with httpd?
18:50:47  <bnoordhuis>yes
18:50:57  <bnoordhuis>it's in the tools dir
18:51:10  <bnoordhuis>s/tools/support/
18:51:18  <ryah>k
18:52:12  <ryah>okay maybe i was wrong about the pound bencharmk
18:52:17  <ryah>it is running up cpu on my box
18:52:19  <ryah>sorry
18:53:16  <bnoordhuis>ab re-arms the connections btw
18:53:42  <bnoordhuis>line 1775
18:59:41  <ryah>i dont like that everything is shaking on the 20th floor
18:59:45  <ryah>:(
19:00:42  <bnoordhuis>but is it the wind?
19:00:47  <bnoordhuis>or the san andreas fault line?
19:03:05  <ryah>i think its an earthquake
19:05:51  <bnoordhuis>crazy
19:06:01  <bnoordhuis>don't go 'connection reset by peer' on me, ryah
19:17:46  <piscisaureus>Hmm, no mentions of an earthquake on twitter
19:17:51  <piscisaureus>It hasn't happened
19:20:31  <piscisaureus>igorzi: if you use FILE_SKIP_BLABLABLA you need to handle the case where write/connect completes immediately too. Also I'd like to have this as a flag on the handle so we can have something like
19:20:31  <piscisaureus>if (success && handle->flags & HANDLE_IOCP_BYPASS) {
19:20:31  <piscisaureus> uv_insert_pending_req((uv_req_t*) req);
19:20:31  <piscisaureus>}
19:20:50  <piscisaureus>we could even have a macro for that
19:21:45  <piscisaureus>bnoordhuis: what do you mean with "report send/receive errors?"
19:23:47  <piscisaureus>bnoordhuis: you mean that the os might not report errors at all?
19:24:06  * mralephjoined
19:27:37  * isaacs_joined
19:29:28  <bnoordhuis>piscisaureus: freebsd doesn't report udp transmission errors, linux does
19:30:07  <bnoordhuis>i suppose the question is: do we deal with errors?
19:30:34  * isaacsquit (Ping timeout: 246 seconds)
19:31:02  <bnoordhuis>udp is unreliable by nature so you can make a case for "yes we should" and "no we won't"
19:31:21  * isaacsjoined
19:33:43  * isaacs_quit (Ping timeout: 246 seconds)
19:50:15  <DrPizza>yes, but there's a difference between "this packet didn't arrive for some reason or other" and "we got a load of ICMP errors because there's no route to that host". How do those situations manifest?
19:53:10  <DrPizza>or the machine loses its IP address or whatever else
19:53:33  * ryahtopic: https://gist.github.com/1149993 ab.c
19:55:17  <bnoordhuis>DrPizza: you know, i'm not sure
19:55:32  <bnoordhuis>i can see code in the kernel that deals with errors
19:55:40  <bnoordhuis>but i can't actually get it to trigger :)
19:55:44  <DrPizza>I guess they're all different classes of error
19:56:58  <bnoordhuis>the errors you can get are all icmp related (afaict anyway)
19:57:06  <bnoordhuis>no route to host, ttl expired, that kind of thing
19:57:10  <DrPizza>yeah
19:57:22  <piscisaureus>bnoordhuis: should I try to trigger udp errors on windwos?
19:57:33  <DrPizza>naively, one would think that even UDP should be able to trigger that kind of error
19:57:34  <bnoordhuis>on the udp layer the only thing i could find was a bad checksum
19:57:46  <bnoordhuis>piscisaureus: yes, if you can
19:58:49  <piscisaureus>the recent stdio debacle this "if you can" is painful
19:59:10  <bnoordhuis>piscisaureus: ?
19:59:21  <piscisaureus>oh nvm kid kid
20:00:00  <bnoordhuis>he's like the delphi oracle, our piscisaureus
20:00:09  <bnoordhuis>tell me, will i lead a long and happy life?
20:01:18  <ryah>what's this issue with udp errors?
20:01:25  <ryah>are we unsure of the interface to report them?
20:01:43  <bnoordhuis>ryah: unsure if they're actually reported by the kernel
20:02:19  <ryah>how does this change the implementation?
20:02:56  <bnoordhuis>well... if the kernel reports errors, you should probably pass them on to the user
20:03:23  <ryah>why don't we just assume that some kernel will report errors :)
20:03:42  <bnoordhuis>good assumption :)
20:03:50  <bnoordhuis>i'm trying to trigger some so i can test it
20:15:45  <bnoordhuis>piscisaureus: if you want a test program to fool around with: https://gist.github.com/897b6b789872fa731c5c
20:16:04  <bnoordhuis>insert unix-centric warning here :)
20:25:21  <igorzi>piscisaureus: ok, i'll add a flag to the handle that indicates iocp bypass
20:29:24  <igorzi>ryah: piscisaureus: so what about a knob to control the buffer size for overlapped WSARecv? (0-read)
20:35:31  <bnoordhuis>piscisaureus: does windows lets you steal a bound udp port?
20:35:50  * graydonjoined
20:44:12  <piscisaureus>bnoordhuis: I can send udp packets to random addresses, don't get errors
20:44:37  <bnoordhuis>piscisaureus: and what if for example you block outgoing udp traffic?
20:44:54  <piscisaureus>hmm
20:44:57  <piscisaureus>how to do that?
20:45:06  <bnoordhuis>iptables -A OUTPUT -p udp -d -j REJECT --reject-with=icmp-net-unreachable :)
20:45:17  <piscisaureus>har har
20:47:20  <bnoordhuis>piscisaureus: does windows have a SO_REUSEADDR equivalent? if so, what does it do for dgram sockets?
20:47:51  <piscisaureus>bnoordhuis: windows lets you steal all port types - even tcp ones
20:48:10  <bnoordhuis>really? intriguing
20:49:48  <piscisaureus>unless the first application to bind the port sets some vista-only flag that prevents this
20:52:09  <piscisaureus>bnoordhuis: firewall on - no difference
20:52:37  <piscisaureus>can still send to
20:52:37  <piscisaureus>heh
20:52:37  <piscisaureus>udp off == no dns == no internet :-/
20:52:37  <piscisaureus>at least the firewall rule works :-)
20:52:44  <bnoordhuis>haha
20:53:01  <bnoordhuis>do you see return icmp traffic?
20:53:13  <piscisaureus>bnoordhuis: there is no --reject-with=icmp-net-unreachable flag in windows firewall btw
20:53:39  <bnoordhuis>does it drop or ?
20:53:52  <piscisaureus>I don't know what it does
20:54:13  <bnoordhuis>ehm... does tcpdump or wireshark show anything?
20:54:50  <piscisaureus>I have to find a tcpdump that works on windows first
20:56:25  <bnoordhuis>doesn't wireshark? i thought it did
21:01:53  <piscisaureus>hmm
21:06:04  * isaacsquit (Quit: isaacs)
21:08:51  <DrPizza>SO_REUSEADDR is a big clusterfuck of weirdness and bullshit with Windows
21:09:29  <DrPizza>http://msdn.microsoft.com/en-us/library/ms740621%28v=VS.85%29.aspx
21:09:50  <ryah>i think its a clusterfuck everywhere
21:10:08  <DrPizza>ryah: I think you may be right, but the Windows version is particularly demented
21:10:24  <DrPizza>when it says "reuse address" it really means it
21:10:32  <DrPizza>it will steal it away from any program currently using it
21:11:22  <DrPizza>not just if the socket is TIME_WAIT
21:12:36  <DrPizza>So then Microsoft came up with SO_EXCLUSIVEADDRUSE
21:12:41  <piscisaureus>bnoordhuis: I can verify that even if my computer receives an ICMP port unreachable message, WSASendTo reports success
21:12:43  <DrPizza>which prevents anyone from grabbing the socket
21:12:57  <DrPizza>even when it IS time_wait
21:13:04  <DrPizza>so it went from one extreme to the other
21:13:25  <bnoordhuis>piscisaureus: okay
21:13:34  <ryah>DrPizza: wow strange
21:13:41  <bnoordhuis>so does it make sense to silently swallow errors on linux?
21:13:42  <DrPizza>ryah: yeah it's crazy
21:13:52  <bnoordhuis>the bsds don't report them and apparently neither does windows
21:15:31  <DrPizza>ryah: basically, Windows server applications should set SO_EXCLUSIVEADDRUSE, and just forget about ever trying to re-use a 5-tuple, even if it's in TIME_WAIT
21:16:00  <bnoordhuis>piscisaureus: double-check: does windows report the error on the second WSASendTo call?
21:16:03  <DrPizza>I suspect node or libuv should really be setting SO_EXCLUSIVEADDRUSE for server sockets
21:16:07  <bnoordhuis>that's what linux does
21:16:12  <bnoordhuis>sometimes...
21:16:15  <DrPizza>otherwise, anyone can clobber node's listening sockets
21:16:35  <igorzi>ryah: piscisaureus: 0-reads... how about adding an option to uv_tcp_init() that lets the caller decide whether it wants to use a buffer for overlapped WSARecv (less memory vs higher throughput)
21:19:15  <piscisaureus>+1 from my side although I don't consider it very important right now
21:19:52  <piscisaureus>btw - does it necessary need to be a creation-time option? I would guess it is possible to change this in-flight.
21:20:03  <piscisaureus>*necessarily
21:20:34  <piscisaureus>bnoordhuis: I can run WSASendTo in a loop, never an error
21:20:47  <bnoordhuis>piscisaureus: okay thanks, that settles it
21:21:48  <igorzi>piscisaureus: yeah, i guess it would be least disruptive to do it at creation time, but the api could allow it to be modified in-flight if that what we need
21:22:03  <piscisaureus>I don't know if that's what we need
21:22:29  <piscisaureus>think about what the node interface would look like
21:22:58  <piscisaureus>would we pass it to the Socket constructor or would we have a setter for it (like setNoDelay)
21:23:13  <piscisaureus>ryah ^--
21:28:07  <DrPizza>igorzi, piscisaureus: agree/disagree, node or libuv should set SO_EXCLUSIVEADDRUSE on its listening sockets to prevent other apps clobbering node servers? It's NT4+ so no compatibility issues.
21:28:14  <DrPizza>(or at least have an option to set it)
21:29:43  <igorzi>at the very least there should be an option i think
21:30:11  <ryah>igorzi: would the uv_tcp_init specify how large the read buffer would be?
21:30:19  <ryah>igorzi: or just boolean?
21:31:58  <ryah>we need to cut a v0.4.11 release too
21:32:25  <ryah>s/we/i/ :)
21:32:27  <ryah>:(
21:33:27  <igorzi>ryah: a boolean, and then the buffer would be alloc'ed the same way - through alloc_cb
21:35:09  <piscisaureus>DrPizza: I may not know enough to understand the consequences. But I think setting it for listening sockets may be a good idea
21:35:29  <ryah>is there a way we could do this setting dynamically and automatically?
21:35:35  <igorzi>or maybe it should be a buffer size.. piscisaureus,what do you think?
21:35:36  <ryah>i don't feel that users want to think about this
21:36:23  <ryah>like - what if we keep a concurrent connection count?
21:36:45  <igorzi>yeah, if the number of connections goes above N then switch to 0-size buffer
21:36:48  <ryah>yeah
21:37:04  <piscisaureus>that sounds like a neat idea
21:37:04  <ryah>maybe we can count the number of active streams?
21:37:08  <ryah>or active tcp streams
21:37:25  <igorzi>would N be static or settable?
21:37:44  <ryah>static would be fine with me
21:38:26  <ryah>say 20 ?
21:38:28  <piscisaureus>how much wasted memory would be acceptable
21:38:36  <piscisaureus>?
21:38:40  <piscisaureus>16 MB?
21:38:51  <ryah>20 * 64kb = 1.7 MB
21:39:26  <piscisaureus>100 * 64kb = 6,4 MB
21:39:31  <piscisaureus>fine with me
21:39:35  <igorzi>should N be different between 32 and 64 bit?
21:39:40  <piscisaureus>no
21:40:03  <piscisaureus>the kernel locks the user buffers to the nonpaged pool
21:40:13  <ryah>let's do 50 - 3MB of pending read buffers seems reasonable
21:40:25  <ryah>i'd say if you have more than 50 connections you're a high concurrency server
21:40:30  <ryah>most are not
21:41:07  <ryah>i'm slightly worried that 100 is common and so people are going to run benchmarks with 100 and measure single connection memory usage
21:41:14  <ryah>with that
21:41:20  <ryah>:)
21:41:25  <piscisaureus>haha
21:41:28  <piscisaureus>let's do 99 :-)
21:41:39  <ryah>yeah - whatever - some number
21:41:48  <igorzi>right, 64bit windows has much larger nonpaged pool
21:41:48  <piscisaureus>no 50 is fine with me
21:42:02  <piscisaureus>igorzi: what are the sizes of the nonpaged pool?
21:42:18  <piscisaureus>I thought it was 1/4th of the physical memory or something?
21:44:34  <piscisaureus>ah
21:44:35  <piscisaureus> 32-bit Windows Server 2003 with 2GB or more of RAM will have a nonpaged pool limit of 256MB
21:44:35  <piscisaureus> 32-bit Windows Server 2008 will have a nonpaged pool limit of either 2GB or slightly more than 75% of physical memory, whichever is smaller
21:44:35  <piscisaureus> 64-bit Windows Server 2003 will have a nonpaged pool of either 128GB or 40% of physical memory, whichever is smaller
21:44:35  <piscisaureus> 64-bit Windows Server 2008 (or 2008 R2) will have a nonpaged pool limit of either 128GB or slightly more than 75% of physical memory, whichever is smaller
21:46:01  <piscisaureus>so it seems that anything < 10MB is going to be no problem at all
21:47:15  <piscisaureus>igorzi: btw - if you want high throughput, set the kernel send buffer to 64KB or something
21:47:28  <piscisaureus>SO_SNDBUF
21:47:43  <igorzi>piscisaureus: yeah, i will play with that next
21:47:56  <piscisaureus>igorzi: that actually is a big difference (on loopback anyway)
21:49:05  <ryah>someone was asking for a setsockopt binding the other day
21:49:06  <piscisaureus>igorzi: but if you do this for all sockets then you can't create 10K sockets any more (I tried)
21:49:56  <igorzi>ryah: piscisaureus: so, 100 tcp streams?
21:50:12  <piscisaureus>igorzi: go for 50, ryah has a point :-)
21:50:17  <igorzi>ok
21:50:34  <piscisaureus>must.optimize.for.benchmarks >:-)
21:51:13  <igorzi>:)
21:51:19  <ryah>igorzi: it might be nice to add a counter for the closes on each type of uv handle
21:51:29  <ryah>and use the counters to see how many open uv_tcp_t there are
21:51:43  <ryah>or not - just an idea
21:52:55  <piscisaureus>OT i like that linkedin is using node btw. Finally a well-known big company digging it.
21:53:03  <igorzi>ryah: i tried your pound bench modification.. it's running fine on my machine. the numbers are the same. i'll make that work for pipes, and land it, if noone objects
21:55:34  <ryah>piscisaureus: yahoo, facebook, msft, palm don't count?
21:55:50  <piscisaureus>since when is facebook using it?
21:56:08  <ryah>they have an internal benchmarking tool using it
21:56:21  <ryah>and hopefully some production stuff later this year
21:56:26  <piscisaureus>oh ok
21:56:51  <ryah>we need to improve debugging
21:56:56  <ryah>it's losing us users
21:57:05  <ryah>obviously windows first
21:57:36  <ryah>long stack traces are relatively approachable now
21:57:46  <piscisaureus>ryah: debugging as in long stack traces or line stepping?
21:58:08  <piscisaureus>for the latter, they just need to use c9 then :p
21:58:09  <ryah>just generally visability
21:58:21  <ryah>also post-mortem debugging
21:58:27  <ryah>c9?
21:58:32  <ryah>d8?
21:58:36  <piscisaureus>cloud9
21:58:39  <ryah>oh
21:58:58  <ryah>the command line debugger needs to be better
21:59:06  <ryah>no real programmers use IDEs
22:01:23  <DrPizza>:(
22:01:29  <ryah>jk
22:01:55  <DrPizza>long stack traces will make a huge usability difference
22:02:01  * isaacsjoined
22:02:35  <piscisaureus>long stack traces will slow it down
22:02:48  <bnoordhuis>why?
22:02:54  <piscisaureus>or ... well that's just an ungrounded fear
22:03:00  <ryah>it has to record the stack at each event
22:03:03  <piscisaureus>because you need to capture a stack trace all the time
22:03:03  <DrPizza>piscisaureus: I can see ways in which they will
22:03:11  <DrPizza>since it has to capture a stack trace at each event
22:03:18  <ryah>but also ev_walk binding
22:03:23  <ryah>with integration into the debugger
22:03:24  <bnoordhuis>is it such a big performance dent?
22:03:26  <ryah>would be hot
22:03:26  <DrPizza>but it still has a huge usability advantage
22:03:46  <ryah>bnoordhuis: at least 30% on http-simple
22:03:51  <bnoordhuis>oh wow
22:03:56  <ryah>when i measured it
22:04:08  <DrPizza>ryah: what exactly is ev_walk, it enumerates every registered callback in libev somehow?
22:04:12  <DrPizza>and calls a callback on each callback?
22:04:14  <bnoordhuis>any idea why?
22:04:17  <bnoordhuis>DrPizza: yes
22:04:26  <ryah>DrPizza: enumerates the "watchers" = uv_handle_t
22:04:38  <DrPizza>oh, right, yes.
22:04:48  <ryah>we're going to have a binding for libuv too
22:04:53  <ryah>in v0.5.4 - right ben?
22:04:58  <DrPizza>bnoordhuis: stack traces usually aren't optimized much; they're meant to be exceptional
22:05:02  <bnoordhuis>oh right, i'll finish it up
22:05:24  <DrPizza>ryah: so there will be something like a linked list of uv_handle_ts?
22:05:44  <ryah>bnoordhuis: low priority - it can be bumped. let's focus on udp first
22:05:54  <piscisaureus>we should get rid of uv_ref / uv_unref
22:05:54  <ryah>DrPizza: yeah
22:06:09  <DrPizza>ryah: hrm, that sounds a bit yucky
22:06:12  <ryah>DrPizza: piscisaureus we need it...
22:06:21  <ryah>sorry that wasn't for DrPizza
22:06:28  <ryah>piscisaureus: we need it...
22:06:34  <ryah>DrPizza: it's not actually a linked list
22:06:37  * mralephquit (Quit: Leaving.)
22:07:03  <ryah>we'll also want to iterate over pending uv_req_t
22:07:14  <DrPizza>what are the use cases for these, just the debugger?
22:07:15  <ryah>these will make debugging much nicer
22:07:21  <ryah>yeah
22:07:23  <DrPizza>so that you can tell the user all the outstanding activity, etc.
22:07:37  <piscisaureus>ryah: we would be better of if there was a flag to uv_handle_init. like UV_NOCOUNT and UV_COUNTACTIVE
22:07:38  <bnoordhuis>DrPizza: you'll be happy when you them, when you have an event loop that somehow doesn't stop
22:08:04  <DrPizza>bnoordhuis: nothing kill -9 can't fix
22:08:26  <bnoordhuis>DrPizza: quite :) but some people insist on fixing the cause
22:09:09  <piscisaureus>ryah: by default, count the handle towards the refcount. But optionally don't count, or count only as long as uv_is_active(handle).
22:09:30  <piscisaureus>I think that covers 99% of all use cases
22:10:30  <piscisaureus>ryah: the bonus is that if we ev_walk we can tell whether a handle keeps the loop alive, or if it's just always there
22:10:46  <bnoordhuis>^ good point
22:10:54  <ryah>piscisaureus: check out timer_wrap.cc
22:11:05  <piscisaureus>ryah: UV_COUNTACTIVE is for that
22:11:23  <piscisaureus>-> ref() on start, unref() on stop
22:12:55  <ryah>i dont know
22:13:32  * mralephjoined
22:17:04  <DrPizza>wait what is UV_NOCOUNT doing, some kind of weak "don't fire if the loop doesn't exist" or something?
22:17:35  <piscisaureus>I was thinking "this handle doesn't keep the loop alive"
22:17:42  <DrPizza>hm
22:18:56  <piscisaureus>I wonder if we better implement uv_walk in node land btw
22:19:31  <piscisaureus>just have every foo_wrap register it's object in the constructor, then unregister in the close callback
22:19:54  <piscisaureus>then it would be much easier to find the related javascript objects
22:20:41  * piscisaureusin bikeshed mode
22:24:44  <piscisaureus>bnoordhuis: How do we deal with this
22:24:45  <piscisaureus>"Note: The Winsock version of the IP_MULTICAST_LOOP option is semantically different than the UNIX version of the IP_MULTICAST_LOOP option:
22:24:45  <piscisaureus> In Winsock, the IP_MULTICAST_LOOP option applies only to the receive path.
22:24:45  <piscisaureus> In the UNIX version, the IP_MULTICAST_LOOP option applies to the send path."
22:24:54  <DrPizza>lol
22:25:30  <piscisaureus>bnoordhuis: I remember it made tests fail under windows w/ the legacy binding
22:26:30  <bnoordhuis>piscisaureus: how does that work on windows?
22:26:42  <bnoordhuis>i mean, how do you send a multicast packet?
22:28:24  <piscisaureus>bnoordhuis: I don't know, but it was a problem for broadcast as well
22:28:48  <piscisaureus>bnoordhuis: to broadcast, call setsockopt with SO_BROADCAST, then send to a broadcast address
22:29:33  <bnoordhuis>piscisaureus: why does IP_MULTICAST_LOOPO work like that on windows?
22:29:52  <bnoordhuis>i mean, what's the point?
22:30:21  * piscisaureuscalls steve ballmer
22:30:34  <piscisaureus>he doesn't remember
22:30:48  <bnoordhuis>i'll pm bill
22:30:49  <DrPizza>IP multicast, what is the point
22:30:52  <DrPizza>Great idea in theory.
22:30:54  <bnoordhuis>over google talk!
22:31:02  <DrPizza>practical application... nil
22:31:12  <DrPizza>except for fucking apple using fucking multicast DNS for .local domain names argh
22:31:20  <piscisaureus>simple/test-dgram-multicast tests that
22:31:28  <piscisaureus>which reminds me
22:31:56  <piscisaureus>we need to reinstate the .local -> getaddrinfo switch in dns_uv
22:32:19  <DrPizza>to make it use the stupid multicast resolver?
22:32:39  <piscisaureus>to use the system resolver to resolve .local addresses on mac
22:32:48  <DrPizza>does cares not know how to do multicast to port 5353?
22:32:56  <DrPizza>or does the Mac OS X resolver do more than that?
22:33:00  <DrPizza>(I know it does at least that)
22:33:10  <piscisaureus>I think cares doesn't do that
22:33:57  <bnoordhuis>i'm pretty sure it doesn't
22:34:05  <bnoordhuis>but let's go back to udp and multicast
22:34:30  <piscisaureus>cares also doesn't do WINS lookups btw
22:35:04  <bnoordhuis>do we need multicast right now?
22:35:05  <DrPizza>not a bug
22:35:06  <DrPizza>a feature.
22:35:34  <bnoordhuis>or broadcast support?
22:35:49  <piscisaureus>bnoordhuis: we do if we want to pass test-dgram-multicast :-)
22:36:26  <DrPizza>bnoordhuis: I guess having multicast means that you could use node for stuff like bonjour or UPnP services
22:36:51  <bnoordhuis>and for provisioning clustered tomcat instances, yay!
22:37:08  <DrPizza>bnoordhuis: and a node->NAT-PMP/UPnP IGD library would be very cool
22:37:17  <bnoordhuis>but are there realistically using it right now?
22:37:24  <DrPizza>having node servers that punch their own firewall holes
22:37:25  <DrPizza>^_^
22:37:29  <piscisaureus>it could be nice for ryah's node-botnet
22:37:42  <piscisaureus>nodes could automatically discover each other
22:37:59  <DrPizza>bnoordhuis: simplest way of testing; put out some builds in which the functionality is broken
22:38:03  <DrPizza>see how loud the complaints are
22:38:05  <DrPizza>:D
22:38:09  <bnoordhuis>DrPizza: good point :)
22:38:46  <bnoordhuis>piscisaureus: about the udp api
22:38:56  <piscisaureus>ja?
22:39:12  <bnoordhuis>right now, socket creation is deferred until the libuv consumer binds, sends or recvs
22:39:18  <DrPizza>I wonder how hard it would be to for example spider every node-using project on github and perform some kind of simple text analysis to figure out which APIs they're using
22:39:40  <bnoordhuis>recv is a bit troublesome because you don't actually know what type of socket to create, ipv4 or ipv6
22:39:40  <piscisaureus>bnoordhuis: so?
22:40:11  <bnoordhuis>uv_udp_recv and uv_udp_recv6 i hear you say
22:40:22  <DrPizza>ugh
22:40:30  <ryah>now that we have a windows build
22:40:31  <bnoordhuis>yeah, exactly
22:40:36  <ryah>we can start making real botnets
22:40:40  <ryah>russian style
22:40:59  <bnoordhuis>*crickets chirp*
22:41:18  <DrPizza>bnoordhuis: anything with a dual v4/v6 API means that people won't bother supporting v6 which makes the world a sadder place
22:41:27  <bnoordhuis>i kid - keep working on that act, ryah
22:41:56  <bnoordhuis>DrPizza: there's that
22:43:04  <bnoordhuis>on linux, a ipv6 socket can receive ipv4 traffic
22:43:11  <bnoordhuis>i suppose that's true for other os'es as well
22:43:12  <DrPizza>but I don't quite understand the problem, to call recv it has to have some semblance of an address already (it has to have enough to create a socket)
22:43:49  <bnoordhuis>you'd usually call bind first
22:44:12  <bnoordhuis>but you don't have to, a udp socket can send and receive data the moment it's created
22:44:34  <bnoordhuis>it just gets assigned a random port
22:44:48  <bnoordhuis>which might be beneficial for p2p style apps
22:45:02  <DrPizza>bnoordhuis: yes, but the first parameter to recv is still a socket fd/SOCKET, no?
22:45:18  <DrPizza>wait you can create that just totally randomly?
22:45:21  <DrPizza>and receive data on it?
22:45:25  <bnoordhuis>yep
22:45:28  <DrPizza>without specifying anything more?
22:45:31  <DrPizza>but
22:45:35  <DrPizza>how does the other end know where to send?
22:45:37  <DrPizza>I guess you tell it
22:45:38  <DrPizza>huh.
22:45:47  <bnoordhuis>again yep :)
22:46:07  <bnoordhuis>piscisaureus: input please
22:46:17  <DrPizza>bnoordhuis: then I guess just make people set an address to listen on and let them specify or ::
22:46:31  <DrPizza>or some other option for "don't care"
22:46:37  <DrPizza>NULL, say
22:46:52  <ryah>bnoordhuis: i just want to check that uv_udp_t is a subclass of uv_handle_t and uv_msg_t is a subclass of uv_req_t ?
22:46:53  <bnoordhuis>DrPizza: yeah, but you still need to specify AF_INET or AF_INET6 when you create the socket
22:47:02  <piscisaureus>http://www.youtube.com/watch?v=dEkOT3IngMQ&t=0m28s
22:47:12  <bnoordhuis>ryah: ack to the first question, nack (for now at least) to the second
22:47:33  <bnoordhuis>piscisaureus: that'd better be work related
22:47:36  <DrPizza>bnoordhuis: or AF_UNSPEC, can't you?
22:47:48  <DrPizza>or does that not work
22:47:52  <ryah>bnoordhuis: ok
22:48:18  <piscisaureus>bnoordhuis: I'm trying to understand the problem
22:49:44  <bnoordhuis>DrPizza: i don't think AF_UNSPEC works for creating a socket
22:49:55  <piscisaureus>no I don't think so either
22:50:17  <DrPizza>bnoordhuis: hmm, it looks like it does for SOCK_RAW and for ICMP (at least on Windows) but not for SOCK_DGRAM
22:50:30  <DrPizza>figures.
22:50:47  <bnoordhuis>AF_UNSPEC?
22:50:54  <bnoordhuis>then i learned something today
22:50:54  <DrPizza>yes
22:50:59  <DrPizza>at least in winsock
22:51:09  <piscisaureus>bnoordhuis: so what is the problem? you want to listen for ipv4 and ipv6 at the same time?
22:51:14  <piscisaureus>or?
22:51:29  <bnoordhuis>piscisaureus: socket creation is deferred
22:51:52  <bnoordhuis>so when the consumer calls uv_udp_recv you don't know what type of socket to create
22:51:53  <piscisaureus>oh we don't know what it is going to be heh
22:51:57  <bnoordhuis>right
22:52:04  <piscisaureus>hmm
22:52:12  <bnoordhuis>so you can split up the api in uv_udp_recv and uv_udp_recv6
22:52:18  <piscisaureus>okay so why don't we have this problem with tcp?
22:52:25  <piscisaureus>oh because you always bind first then
22:52:28  <bnoordhuis>yep
22:52:31  <piscisaureus>aha
22:52:38  <bnoordhuis>or connect, same thing
22:52:41  <ryah>forget ipv6
22:52:49  <ryah>no one cares about that fictional protocol
22:52:52  <piscisaureus>I don't like recv and recv6
22:52:53  <piscisaureus>/me pondering
22:53:18  <bnoordhuis>ryah: better to do it now and do it right than having to retrofit it later
22:53:53  <ryah>http://www.google.com/intl/en/ipv6/statistics/
22:53:57  <piscisaureus>bnoordhuis: we can receive ipv4 packets on an ipv6 socket
22:54:02  <piscisaureus>only on vista+
22:54:12  <piscisaureus>but ipv6 on xp/2003 is fictional anyway
22:54:12  <ryah>if ipv6 traffic goes above 0.5% we can worry about it
22:54:19  <ryah>doubtful that will ever happy
22:54:21  <ryah>happen
22:54:45  <piscisaureus>bnoordhuis: http://msdn.microsoft.com/en-us/library/bb513665%28v=vs.85%29.aspx
22:55:23  <ryah>(im just kidding - kind of - im glad you're thinking about it)
22:56:28  <DrPizza>piscisaureus: make it a flag then
22:56:40  <DrPizza>ryah: I'm starting to see ipv6 connections on bittorrent from time to time
22:56:43  <DrPizza>it's definitely growing
22:56:43  <DrPizza>slowly
22:57:00  <ryah>DrPizza: do you have ipv6 connectivity?
22:57:18  <DrPizza>not native, only tunnelled
22:57:29  <DrPizza>also youtube is now streaming over ipv6
22:57:59  <bnoordhuis>piscisaureus: yeah, linux has that same feature
22:58:39  <piscisaureus>bnoordhuis: but - I don't understand exactly how it works :-/
22:58:46  <DrPizza>would it be unacceptable to say that ipv6 only works on dualstack OSes?
22:59:05  <bnoordhuis>piscisaureus: one drawback is that ipv4 addresses are reported as :::ffff:
22:59:16  <bnoordhuis>bound to raise complaints
22:59:16  <ryah>DrPizza: the last 99% is always the hardest userbase to assimilate
22:59:22  <piscisaureus>bnoordhuis: hmm
22:59:30  <DrPizza>ryah: haha
22:59:32  <ryah>sorry 99.7%
22:59:45  <piscisaureus>bnoordhuis: let's just bind to if the user doesn't explicitly bind
22:59:58  <bnoordhuis>ipv4? sounds reasonable
23:00:19  <DrPizza>bnoordhuis: ist here any functional difference between binding to :: and then calling recv, and just calling recv on an unbound socket?
23:00:33  <piscisaureus>no
23:00:37  <DrPizza>an unbound AF_INET6 socket, that is
23:00:38  <bnoordhuis>the alternative is to let the user specify upfront what kind of socket they want
23:00:43  <DrPizza>oh in that case, force people to bind
23:00:44  <DrPizza>it's harmless
23:00:48  <piscisaureus>just force them to bind
23:01:02  <piscisaureus>they can bind to either '' port 0 or '::' port 0
23:01:24  <ryah>they should not need to bind
23:01:35  <DrPizza>ryah: for ipv6 I mean
23:01:46  <piscisaureus>for ipv4 they don't have to
23:01:51  <DrPizza>to let them override the ipv4 default
23:02:03  <DrPizza>it's either that or to set a flag or something on the first uv_udp_recv call
23:02:03  <bnoordhuis>piscisaureus: to recap: you propose to auto-bind to
23:02:04  <ryah>sure
23:02:21  <piscisaureus>bnoordhuis: yeah
23:02:40  <piscisaureus>ryah: we could skip the actual bind syscall if possible
23:03:03  <piscisaureus>that is - if the user tries to bind to [::]:0 then we could as well not call bind() at all
23:03:10  <bnoordhuis>yeah, what piscisaureus is saying is equivalent to creating a ipv4 udp socket
23:03:12  <piscisaureus>just record that the user wants an ipv5 socket
23:04:41  <bnoordhuis>piscisaureus: to be sure: what we're saying here is: default to ipv4 unless the user says otherwise
23:04:49  <piscisaureus>yez
23:05:03  <bnoordhuis>i can live with that
23:05:15  <bnoordhuis>next point of order: how to pass in addresses to send and recv
23:05:29  <bnoordhuis>sockaddr_in and sockaddr_in6 differ in size
23:05:50  <piscisaureus>pass an address to recv() ?
23:05:56  <bnoordhuis>as a string?
23:06:06  <piscisaureus>hmm no
23:06:18  <piscisaureus>bnoordhuis: why pass an adress to recv() ?
23:06:35  <bnoordhuis>so you can tell who sent the packet
23:06:39  <bnoordhuis>but actually...
23:06:49  <bnoordhuis>that's one of the things i created that cookie param for
23:06:50  <piscisaureus>bnoordhuis: that's gotta be sent to the callbacl
23:07:10  <bnoordhuis>right
23:07:29  <bnoordhuis>uv_udp_send takes an address though - and has to
23:07:40  <piscisaureus>for send we could just pass it as struct sockaddr*
23:08:17  <bnoordhuis>i suppose so
23:08:22  <bnoordhuis>does windows have socklen_t?
23:08:42  <piscisaureus>no
23:08:56  <bnoordhuis>okay, we'll use int
23:09:38  <piscisaureus>bnoordhuis: can't look at sockaddr.sa_family?
23:09:42  <piscisaureus>why use socklen at all?
23:10:05  <bnoordhuis>technically, the offset of sa_family may differ
23:10:59  <piscisaureus>also I think it's harmless to have send and send6 btw. I mean we also have connect and connect6
23:11:22  <bnoordhuis>right
23:11:30  <bnoordhuis>ugly but functional
23:11:46  <piscisaureus>does it make the binding more difficult?
23:11:54  <bnoordhuis>no, not at all
23:12:26  * mralephquit (Quit: Leaving.)
23:12:46  <bnoordhuis>piscisaureus: last point: how to tell the recv callback the type of sockaddr?
23:13:01  <bnoordhuis>param family in [AF_INET, AF_INET6]?
23:13:20  <piscisaureus>hmm
23:13:46  <ryah>guys we're making --use-uv default after v0.5.4 - k?
23:13:52  <bnoordhuis>ryah: yes
23:15:06  * ryahtopic: https://gist.github.com/1149993 ab.c | --use-uv default after v0.5.4 | v0.5.4 release manager bnoordhuis, UDP
23:16:17  <piscisaureus>bnoordhuis: hmm. I don't have a good idea for that. I think a family param is okay if you say that sockaddr.sa_family doesn't work
23:17:02  <piscisaureus>bnoordhuis: thinking about it - maybe it would be nice if we could use dualstack sockets for unbound sockets
23:17:29  <piscisaureus>bnoordhuis: but we can always do that later
23:17:45  <bnoordhuis>piscisaureus: well... give me a good reason
23:17:56  <bnoordhuis>it's easy to implement so time is not an issue
23:18:20  <piscisaureus>bnoordhuis: so people don't have to think about bound, let them just call send and send6
23:18:28  <piscisaureus>*bind
23:18:50  <bnoordhuis>i can live with that
23:19:04  <bnoordhuis>but the :::ffff: address issue still stands
23:19:21  <piscisaureus>bnoordhuis: I guess we would want automatic mapping between ipv4 and v4-mapped-v6
23:19:28  <piscisaureus>so the user doesn't have to worry about that
23:19:54  <DrPizza>yes, if you just detect such addresses and strip off the ::ffff:
23:19:55  <bnoordhuis>hmm, i have to digest that one
23:20:02  <piscisaureus>so if the socket is dualstack, send(4) would map the address to ipv6 then call send6
23:20:03  <DrPizza>that would seem harmless and safe in general
23:20:28  <piscisaureus>and recv would - indeed - strip the ::ffff: and tell the callback that it's ipv4
23:21:02  <bnoordhuis>that'd work
23:21:09  <bnoordhuis>there's probably a lot wrong with it
23:21:23  <bnoordhuis>i just can't think of anything yet
23:22:27  <bnoordhuis>btw, sexy baby alert: http://www.flickr.com/photos/digilijn/6050012453/in/set-72157627449190374
23:22:33  <bnoordhuis>she's cute, isn't she?
23:22:43  <piscisaureus>ah she just looks like drpizza
23:22:50  <DrPizza>wt
23:22:51  <DrPizza>f
23:22:53  <DrPizza>wut
23:23:20  <bnoordhuis>^ my daughter :)
23:23:27  <piscisaureus>drpizza: sorry, you're the victim here, bnoordhuis and I are insulting each other all the time
23:23:33  <DrPizza>;_;
23:26:30  <bnoordhuis>piscisaureus: let me run a few dual-stack tests here
23:27:56  <DrPizza>in an ideal world you'd just use dual-stack everything, and only make the v4/v6 difference manifest during name/address resolution/specification.
23:28:35  <piscisaureus>doesn't node do that?
23:28:56  <DrPizza>piscisaureus: perhaps but it seems feasible to me that uv could do so too
23:29:06  <DrPizza>at least, if it weren't for win2k3
23:29:16  <piscisaureus>perhaps
23:29:23  <piscisaureus>maybe we should pass ip addresses as strings
23:29:33  <piscisaureus>like, always
23:30:24  <piscisaureus>solid dual-stack support for tcp could be a pretty neat
23:30:32  <piscisaureus>when ipv6 finally takes off
23:30:38  <DrPizza>yes
23:30:46  <DrPizza>also it's good for ryah's botnet
23:30:55  <DrPizza>since you could give each bot a unique endpoint
23:31:06  <DrPizza>even when stuck behind corporate NATs
23:31:10  <DrPizza>they can just teredo out or whatever
23:32:01  <piscisaureus>basically what you'd want is uv_bind_dual(port)
23:32:13  <piscisaureus>bind to any ipv4 + ipv6 interface
23:37:47  <bnoordhuis>okay, dual stack should work
23:38:14  <bnoordhuis>one narrow use case we can't service: have server A listening on ipv4 and server B on ipv6
23:38:57  <piscisaureus>bnoordhuis: eh?
23:39:01  <bnoordhuis>on linux anyway, ipv6 is by default dual stack
23:39:02  <piscisaureus>server A / server ?
23:39:04  <piscisaureus>server A / server B?
23:39:30  <bnoordhuis>imagine you want to redirect incoming ipv4 traffic to your ipv6 site
23:40:16  <piscisaureus>bnoordhuis: you can't turn off dualstack on linux?
23:40:24  <bnoordhuis>yeah, but you need to set a flag (IPV6_V6ONLY)
23:40:43  <piscisaureus>bnoordhuis: on windows too. But that flag is on by default
23:41:03  <bnoordhuis>and it's off by default on linux - awesome eh? :)
23:41:08  <bnoordhuis>not sure what the bsds do
23:41:19  <piscisaureus>you should turn it on on linux
23:41:30  <piscisaureus>but yeah check the bsds first
23:43:46  <bnoordhuis>freebsd more or less follows linux
23:44:58  <piscisaureus>bnoordhuis: so we want to enable dualstack by default?
23:45:30  <bnoordhuis>well... let me check opensolaris
23:45:31  <ryah>bnoordhuis: nice looking baby
23:45:35  <piscisaureus>bnoordhuis: so currently with node you can't create a tcp4 server and a tcp6 server on the same port
23:45:40  <bnoordhuis>ryah: she's cute eh? :)
23:46:14  <bnoordhuis>piscisaureus: that's correct
23:46:58  <piscisaureus>how old is mees now btw?
23:47:06  <bnoordhuis>almost nine months
23:47:25  <bnoordhuis>not really a baby anymore :(
23:47:34  <igorzi>bnoordhuis: my son is 9.5 months :)
23:47:42  <bnoordhuis>hey congrats, igorzi
23:47:48  <DrPizza>bnoordhuis: maybe the best option would be to make all the udp functions where it matters (and others, I guess), have a shitty flag to specify "v4", "v6", "both"
23:47:50  <igorzi>bnoordhuis: thanks, you too :)
23:47:53  <ryah>we should have NAT piercing in node-core
23:47:54  <bnoordhuis>how do you like being a parent?
23:47:58  <DrPizza>babies mainly just look weird to me
23:48:00  <DrPizza>ryah: lol
23:48:17  <igorzi>bnoordhuis: it was kind of crazy at first, but i like it a lot
23:48:26  <igorzi>bnoordhuis: you?
23:48:28  <DrPizza>ryah: like upnp igd/nat-pmp?
23:48:30  <bnoordhuis>igorzi: yeah, same here :)
23:48:46  <bnoordhuis>igorzi: and it's actually her mom that did most of the work in the early days
23:49:25  <ryah>DrPizza: like STUN
23:49:55  <igorzi>bnoordhuis: i ended up doing a lot of work.. (his mom had to go back to work after ~2 months) :(
23:50:10  <DrPizza>doesn't stun need an external stun server?
23:50:22  <bnoordhuis>igorzi: ow, that sucks - babies are more than a handful
23:50:35  <bnoordhuis>i don't think i'd get anything done if i'd have to babysit mees
23:52:16  <CIA-95>libuv: Ben Noordhuis tcp_perf * r084f681 / Makefile :
23:52:16  <CIA-95>libuv: build: fix build on systems where `ld --as-needed` is the default
23:52:16  <CIA-95>libuv: Fixes unresolved symbol linker errors from -lm and -lrt. - http://bit.ly/oT9NyT
23:52:17  <CIA-95>libuv: Igor Zinkovsky tcp_perf * rd763581 / (4 files): Skip IOCP for tcp operations that complete synchronously - http://bit.ly/nd3IsB
23:52:35  <igorzi>piscisaureus: -----^
23:55:55  <piscisaureus>igorzi: does it improve perf?
23:56:07  <piscisaureus>igorzi: btw - lgtm
23:56:30  <ryah>he said 2%
23:56:33  <ryah>before?
23:56:58  <piscisaureus>although it wasn't really necessary to move the `reqs_pending++` inside the if btw
23:57:08  <piscisaureus>s/wasn't/isn't/
23:58:15  <piscisaureus>igorzi: btw
23:58:29  <piscisaureus>for me pAcceptEx6 == pAcceptEx for me
23:58:38  <piscisaureus>also for ConnectEx etc
23:59:43  <igorzi>piscisaureus: yeah, it seems to make it more readable (so that there's a common pattern -> if (UV_SUCCEEDED_WITHOUT_IOCP) else if(UV_SUCCEEDED_OUT_IOCP) else..)