00:00:05  <piscisaureus>igorzi: yeah fair enough
00:00:21  <DrPizza>is there no way to statically bind go ConnectEx, AcceptEx, etc.
00:00:29  <bnoordhuis>piscisaureus: sunos too
00:00:30  <DrPizza>are thy simply not exported, so that you MUST look them up by GUID?
00:00:43  <bnoordhuis>piscisaureus: so it seems windows is the outlier
00:00:51  <igorzi>piscisaureus: yeah, i remember this, and i also remember henry claiming that it wasn't the same..
00:01:32  <piscisaureus>igorzi: in theory it could be different, we should check on an xp machine w/ ipv6 support installed
00:02:32  <piscisaureus>igorzi: in theory machines could have a different lsp/driver for ipv6 but I have yet to encounter the first machine that doesn't use msafd for tcp
00:02:32  <igorzi>(if we do need both) we should still probably stash pAcceptEx or pAcceptEx6 into a static or something (to avoid the branch every time)
00:03:21  <piscisaureus>igorzi: in a static? like in a socket field?
00:04:02  <piscisaureus>igorzi: I don't think it would have much benefit. It would just move the branch point.
00:05:30  <igorzi>make "LPFN_ACCEPTEX pAcceptExFamily" global, and just have one branch during uv_winsock_startup()
00:06:10  <piscisaureus>igorzi: eh how would that work?
00:06:58  <piscisaureus>igorzi: we still need to choose either pAcceptEx or pAcceptEx on a per-socket basis or?
00:07:08  <piscisaureus>* pAcceptEx or pAcceptEx6
00:07:41  <igorzi>piscisaureus: never mind.. for some reason i thought that UV_HANDLE_IPV6 is a global flag.. need to go get some food
00:08:32  <bnoordhuis>piscisaureus: so unbound sockets: ipv4 or dual stack?
00:08:45  <piscisaureus>hmm
00:08:52  <piscisaureus>use ipv4 for the moment
00:08:56  <bnoordhuis>ipv4 is easier yes
00:09:06  <piscisaureus>we'll deal with dualstack later
00:09:15  <bnoordhuis>one more thing
00:09:26  <bnoordhuis>what if you call uv_udp_send on a ipv6 socket?
00:09:36  <bnoordhuis>or vice versa, uv_udp_send6 on a ipv4 socket?
00:09:39  <bnoordhuis>UV_EINVAL?
00:09:39  <igorzi>piscisaureus: sorry, i was having a brain fart.. does the rest of the change look fine?
00:09:57  <piscisaureus>bnoordhuis: uv_eafnosupport?
00:10:07  <bnoordhuis>okay
00:10:13  <piscisaureus>igorzi: heh. yes the rest looks good to me
00:11:25  <piscisaureus>bnoordhuis: we'll deal with dualstack later, in the selene era
00:11:36  <piscisaureus>or the http.sys era
00:11:41  <DrPizza>heh
00:12:47  <piscisaureus>why does no.de run 0.4.1?
00:17:04  <bnoordhuis>piscisaureus: like this then? https://gist.github.com/cc11217bf271d8a897c7
00:17:18  <bnoordhuis>the uv_af_t enum somehow doesn't sit right with me
00:17:37  <bnoordhuis>otoh passing in sizeof(struct sockaddr*) sucks too
00:18:13  <DrPizza>a type is much better than a size
00:18:27  <bnoordhuis>that was my thinking too
00:18:33  <piscisaureus>maybe UNIX should be PIPE btw
00:18:47  <bnoordhuis>piscisaureus: i'll probably remove it anyway
00:19:32  <piscisaureus>why do we still have udp_bind?
00:19:35  <DrPizza>whoa, pre3 is launched and only £299?
00:19:41  <DrPizza>I am almost tempted to buy one just to check it out
00:19:51  <piscisaureus>oh that's because of the type
00:20:02  <bnoordhuis>piscisaureus: yeah, uv_stream_t vs uv_udp_t
00:20:48  <bnoordhuis>you'd have to merge uv_tcp_bind and uv_pipe_bind too
00:20:56  <bnoordhuis>but uv_pipe_bind has a different prototype
00:21:10  <piscisaureus>bnoordhuis: we're saying that the struct sockaddr* is valid until the callback ends?
00:21:27  <bnoordhuis>piscisaureus: what function?
00:21:36  <piscisaureus>uv_udp_recv_cb
00:21:42  <bnoordhuis>yeah
00:22:01  <piscisaureus>yeah - fine
00:22:07  <piscisaureus>looks good
00:22:16  <piscisaureus>let ryan have a look too
00:22:21  <DrPizza>doesn't a sockaddr_in know its own type?
00:22:28  <DrPizza>can't you just query sin_family?
00:22:44  <piscisaureus>I was saying that too but bnoordhuis says it can't be done
00:22:51  <DrPizza>huh really?
00:23:27  <bnoordhuis>well... the offset of sin_family may be different in sockaddr_in and sockaddr_in6
00:23:53  <bnoordhuis>that's not the case on linux and probably not on the bsds either
00:23:57  <bnoordhuis>but still...
00:24:00  <DrPizza>it isn't in windows either
00:24:11  <piscisaureus>I thought the whole purpose of struct sockaddr* was to have a base class for sockaddr_* types
00:24:20  <DrPizza>you would have to be a vindictive bastard to do that IMO
00:24:36  <DrPizza>piscisaureus: yes me too
00:24:41  <DrPizza>just sa_family and then opaque data
00:24:48  <piscisaureus>yes
00:24:50  <DrPizza>and then you can cast the pointer to sockaddr_in or sockaddr_in6
00:24:59  <bnoordhuis>i suppose i can live with that
00:25:11  <piscisaureus>on windows it's the same offset as well
00:25:19  <bnoordhuis>first field in the struct?
00:25:21  <DrPizza>yes
00:25:23  <piscisaureus>yes
00:25:42  <bnoordhuis>count of hands? all in favour raise your hand?
00:25:42  <piscisaureus>it may even be in the berekely sockets specification
00:25:51  <piscisaureus>+1 for sa_family
00:25:53  <bnoordhuis>yeah, there's a rfc that specifies it
00:25:55  <DrPizza>+1 for sa_family
00:25:58  <bnoordhuis>+1 too
00:26:01  <DrPizza>lol
00:26:15  <bnoordhuis>okay, let me rework it
00:26:28  <piscisaureus>so I assume the others disagree
00:26:53  <piscisaureus>that makes -10 (not counting the bots)
00:27:11  <DrPizza>I think more likely they're just not paying attention
00:27:38  <CIA-95>libuv: Igor Zinkovsky master * re67b9a3 / (4 files): Skip IOCP for tcp operations that complete synchronously - http://bit.ly/nD82FW
00:28:11  <ryah>igorzi++
00:28:42  <ryah>igorzi: does that break the win2k3 build?
00:29:07  <piscisaureus>it shouldn't
00:29:09  <igorzi>igorzi: no, it shouldn't
00:29:16  <igorzi>ryah: it souldn't
00:29:28  <piscisaureus>igorzi: get some food :-)
00:29:38  <igorzi>yeah, that's about time :)
00:30:10  <bnoordhuis>piscisaureus: maybe merge uv_udp_send and uv_udp_send6 too?
00:30:18  <bnoordhuis>if we're going to use sa_family...
00:30:42  <bnoordhuis>no wait, the convention is to pass by value, not reference
00:30:47  <piscisaureus>bnoordhuis: yeah why not. you'd have to switch to passing by reference then btw
00:31:05  <bnoordhuis>right, but that flies in the face of tcp
00:31:18  <piscisaureus>bnoordhuis: why?
00:31:35  <piscisaureus>you mean, consistency-wise?
00:31:39  <bnoordhuis>piscisaureus: yes
00:32:02  <bnoordhuis>and it's slightly annoying you can't use uv_ipv4_addr anymore
00:32:07  <bnoordhuis>not directly anyway
00:32:16  <piscisaureus>bnoordhuis: maybe we should invent struct uv_address that accommodates either ipv6 or ipv4
00:32:44  <bnoordhuis>that thought crossed my mind
00:32:50  <DrPizza>char[max(sizeof(struct sockaddr_in), sizeof(struct sockaddr_in6] ;_;
00:32:51  <bnoordhuis>but that's layering things on top
00:33:09  <bnoordhuis>plus you'd probably be copying stuff in and out all the time
00:33:28  <DrPizza>or just cast a pointer
00:33:38  <DrPizza>unions too
00:33:49  <DrPizza>I don't know which is more illegal
00:33:54  <DrPizza>accessing the "wrong" union member
00:33:58  <DrPizza>or casting a char[]
00:34:09  <bnoordhuis>strict aliasing rules you mean?
00:34:16  <bnoordhuis>you can always cast to char
00:34:21  <DrPizza>yes true
00:34:22  <bnoordhuis>er, alias to char
00:34:25  <DrPizza>yes
00:35:16  <DrPizza> maybe accessing the wrong union if the declarations overlap is allowed anyway
00:35:20  <DrPizza>it's certainly safe
00:36:21  <bnoordhuis>hah, it's technically a violation but all compilers allow it
00:36:31  <piscisaureus>we could just have
00:36:32  <piscisaureus>struct uv_address {
00:36:32  <piscisaureus> enum uv_family family;
00:36:32  <piscisaureus> union {
00:36:32  <piscisaureus> struct sockaddr_in in;
00:36:32  <piscisaureus> struct sockaddr_in6 in6;
00:36:32  <piscisaureus> }
00:36:33  <piscisaureus>}
00:37:19  <bnoordhuis>piscisaureus: rather: union uv_address { struct sockaddr_in in; struct sockaddr_in6 in6; }
00:38:08  <DrPizza>union uv_address { u_short family; struct sockaddr_in in; struct sockaddr_in6 in6; }
00:38:17  <piscisaureus>bnoordhuis: #define UV_FAMILY(addr) (addr).in.sin_family
00:38:29  <DrPizza>addr.family
00:38:44  <bnoordhuis>like that
00:38:44  <piscisaureus>DrPizza is probably right
00:39:01  <bnoordhuis>i wonder if you'll run into alignment issues
00:39:17  <bnoordhuis>no
00:39:45  <DrPizza>shold be a u_short for all 3 union members, so one would hope it has the right alignment for a u_short
00:40:22  <bnoordhuis>but let's go with uv_udp_send and uv_udp_send6 for now
00:40:39  <bnoordhuis>rewriting everything to use uv_address is a pretty big change in itself
00:41:01  <piscisaureus>uv_address uv_inet_pton(const char* s) <- parses either ipv4 and ipv6
00:41:01  <piscisaureus>int uv_inet_ntop(uv_address address, char* buf, size_t len)
00:41:01  <bnoordhuis>and i want to get this stuff done before thursday night
00:42:16  <bnoordhuis>piscisaureus: https://gist.github.com/cc11217bf271d8a897c7
00:43:42  <bnoordhuis>piscisaureus: one more issue btw
00:43:52  <bnoordhuis>the send methods may need to save state
00:44:05  <bnoordhuis>and that should be alloc'ed by the libuv consumer
00:44:13  <piscisaureus>hmm
00:44:19  <piscisaureus>uv_send_t?
00:44:26  <piscisaureus>uv_sendto_t?
00:44:47  <bnoordhuis>yes, probably
00:45:01  <bnoordhuis>i think ryah wanted it to be a subclass of uv_req_t
00:45:06  <piscisaureus>bnoordhuis: I think it should be recvfrom_start and recvfrom_stop
00:45:09  <piscisaureus>I agree
00:45:40  <bnoordhuis>piscisaureus: explain
00:46:11  <piscisaureus>oh hmm maybe not stop()
00:46:39  <piscisaureus>in doubt again
00:46:45  <piscisaureus>we have read_start and read_stop
00:47:19  <piscisaureus>but we don't have listen_stop - it never stops listening, but just accepts until the server is closed
00:47:25  <piscisaureus>so yeah recv is okay
00:47:43  <bnoordhuis>well... it's a fair point
00:47:47  <bnoordhuis>ponder, ponder
00:47:52  <piscisaureus>bnoordhuis: one more question
00:48:03  <piscisaureus>how does the user know it has got the *complete* message?
00:48:12  <bnoordhuis>piscisaureus: you don't
00:48:41  <bnoordhuis>it's up to the user to implement some kind of message protocol
00:49:14  <piscisaureus>bnoordhuis: so it is the user's responsibility to supply a sufficiently big buffer when alloc_cb is called?
00:49:52  <bnoordhuis>piscisaureus: doesn't matter, we can recvmsg() until the kernel signals EAGAIN
00:50:06  <bnoordhuis>besides, dgrams can't be larger than the MTU
00:50:21  <bnoordhuis>so the buffer is never very large
00:50:28  <piscisaureus>bnoordhuis: but - so how will the user know where the boundary between messages is?
00:50:33  <piscisaureus>or it doesn't?
00:50:37  <bnoordhuis>it doesn't :)
00:50:49  <piscisaureus>oh ok
00:50:58  <DrPizza>will I be able to do some kind of uv_udp_recv(..., malloc(my_fixed_message_length), my_fixed_message_length, my_callback), and only have my callback called whne the buffer is filled?
00:51:07  <bnoordhuis>DrPizza: no
00:51:23  <piscisaureus>no
00:51:28  <DrPizza>boo.
00:51:33  <bnoordhuis>use tcp :)
00:51:43  <piscisaureus>DrPizza: but it's sufficiently easy to implement though
00:52:14  <piscisaureus>the alloc cb should keep some state
00:52:15  <DrPizza>piscisaureus: oh I mean I know I can manage the buffers myself to make sure they're the right size before trying to process them
00:52:20  <DrPizza>it's just nice to not have to
00:52:33  <DrPizza>or rather, it's nice to have that fairly common task standardized
00:52:51  <bnoordhuis>DrPizza: you'd almost never use udp in that case
00:53:04  <DrPizza>true
00:53:07  <DrPizza>though it'd be nice for tcp too!
00:53:13  <bnoordhuis>ask ryah :)
00:53:26  * bnoordhuisis getting better and better in delegating
00:53:29  <DrPizza>haha
00:54:01  <piscisaureus>DrPizza: you should have come to nodecamp in june.
00:54:09  <piscisaureus>This was set in stone there muhahaha
00:54:09  <DrPizza>piscisaureus: why
00:54:13  <DrPizza>oh
00:54:14  <DrPizza>:(
00:54:38  <piscisaureus>DrPizza: but again - writing an alloc_cb that does this is not hard
00:54:51  <DrPizza>hmm
00:55:02  <DrPizza>do alloc_cbs have anywhere useful to stash state?
00:55:18  <piscisaureus>alloc_cbs get the uv_handle as an argument
00:55:30  <DrPizza>oh so they can stuff it in the void*
00:55:42  <piscisaureus>so you can use void* uv_handle.data to stash whatever data you want
00:55:47  <DrPizza>yes
00:56:32  <piscisaureus>DrPizza: we selected this interface so the user has maximum flexibility to layer it's own memory management on top
00:56:41  <piscisaureus>for node we do a slab allocator
00:57:29  <bnoordhuis>piscisaureus: uv_recv_start and uv_recv_stop maybe isn't an altogether bad idea
00:57:50  <bnoordhuis>with uv_udp_recv you can start a listener for incoming data
00:58:00  <bnoordhuis>but you can't stop it until you uv_close the handle
00:58:30  <piscisaureus>bnoordhuis: which is also true for uv_listen btw
00:58:40  <piscisaureus>the question: is recv more like read or more like listen :-)
00:58:41  <bnoordhuis>then again, what use is a socket where you don't accept incoming data?
00:58:49  <piscisaureus>bnoordhuis: send data
00:59:13  <bnoordhuis>right, right
00:59:26  <bnoordhuis>i suppose we should have start and stop functions then
00:59:31  <piscisaureus>I wonder why create a socket to send dgrams in the first place
00:59:41  <piscisaureus>but that also is a theoretical question
00:59:52  <bnoordhuis>it's kind of silly, isn't it?
00:59:57  <piscisaureus>yes it's stupid
01:00:17  <piscisaureus>bnoordhuis: but yay, +1 on recv_start/stop
01:00:29  <bnoordhuis>okay, let's go with that
01:00:42  <bnoordhuis>i'll update the gist
01:01:41  <piscisaureus>bnoordhuis: MSG_PARTIAL?
01:01:47  <piscisaureus>or is that not for upd?
01:01:54  <bnoordhuis>yes, it's for udp
01:02:11  <bnoordhuis>eh wait, i'm thinking of MSG_TRUNC
01:02:46  <piscisaureus>maybe that's a windows-only thing
01:02:55  <piscisaureus>it's for detecting message boundaries
01:03:16  <bnoordhuis>piscisaureus: maybe it's the same
01:03:27  <bnoordhuis>MSG_TRUNC returns the real size of the incoming packet
01:03:33  <piscisaureus>oh
01:03:40  <bnoordhuis>which might be smaller or larger than the buffer you passed to the kernel
01:04:43  <piscisaureus>bnoordhuis: windows sets a MSG_PARTIAL flag if there's more to retrieve in a subsequent read operation
01:04:53  <bnoordhuis>oh right
01:05:09  <bnoordhuis>that's MSG_TRUNC too - but in the returned flags field
01:05:17  * graydonquit (Quit: Leaving.)
01:05:24  <piscisaureus>I wonder if we could pass that to the user somehow
01:05:34  <piscisaureus>so it knows the boundaries
01:05:58  <bnoordhuis>hmm, that cookie thing
01:06:12  <bnoordhuis>if (uv_udp_truncated(cookie)) { }
01:06:46  <bnoordhuis>or pass a flags field / struct field to the recv callback
01:06:46  <piscisaureus>bnoordhuis: why not use uv_udp_truncated(handle) then?
01:07:08  <bnoordhuis>yeah, that's just as easy
01:07:12  <piscisaureus>a flags field will suffice
01:07:27  <bnoordhuis>UV_UDP_PARTIAL?
01:07:40  <piscisaureus>yeah why not?
01:07:54  <bnoordhuis>for parity's sake i want a UV_UDP_IMPARTIAL flag too
01:08:03  <DrPizza>hah
01:08:04  * isaacsquit (Quit: isaacs)
01:09:30  <bnoordhuis>piscisaureus: what does read_start do when there's already a read callback registered?
01:09:37  <bnoordhuis>disarm the old one, continue with the new one?
01:11:53  <piscisaureus>bnoordhuis: no, this: http://www.youtube.com/watch?v=R7fFCJ5ek6A&feature=related
01:12:23  <piscisaureus>piscisaureus: it returns UV_EALREADY I think
01:12:37  <piscisaureus>bnoordhuis: it returns UV_EALREADY I think, like listen() and read_start()
01:12:50  <piscisaureus>at least that's what happens on windows
01:13:26  <bnoordhuis>oh, that's different from unix right now
01:14:16  <bnoordhuis>uv-unix just updates the read_cb and alloc_cb fields
01:14:31  <bnoordhuis>(potential memory leak that)
01:14:44  <ryah>http://jobs.nodejs.org/a/jbb/job-details/544636 <-- justin.tv looking for node.js people
01:14:55  <ryah>bnoordhuis: can we split up uv-unix.c ?
01:14:55  <piscisaureus>oh, i'll apply
01:14:56  <piscisaureus>by all
01:14:58  <bnoordhuis>i'm applying!
01:14:59  <piscisaureus>*bye
01:15:04  <bnoordhuis>ryah: yes please :)
01:15:25  <ryah>is it going to fuck it up if i do that now?
01:15:29  <bnoordhuis>yes
01:15:30  <ryah>er
01:15:33  <ryah>fuck you up
01:15:35  <ryah>okay
01:15:40  <ryah>let's do it after the udp
01:15:49  <bnoordhuis>let's
01:16:06  <bnoordhuis>is justin.tv special?
01:16:11  <piscisaureus>finally some sanity at the unix front
01:16:14  <ryah>kind of big
01:16:58  <bnoordhuis>linkedin's bigger i think?
01:17:19  <bnoordhuis>more mainstream anyway
01:17:26  <ryah>https://github.com/joyent/libuv/issues/147
01:17:26  <bnoordhuis>and let's not forget github!
01:18:10  <ryah>crazy fog just surrounded my building
01:18:19  <ryah>now its gone
01:18:31  <bnoordhuis>you should write a book, ryah
01:18:34  <piscisaureus>ryah: so you really had an earthquake in the morning?
01:18:37  <bnoordhuis>you're a born story teller
01:18:39  <ryah>:)
01:18:48  <ryah>piscisaureus: not sure - the whole building was shaking
01:19:14  <piscisaureus>ryah: if its not an earthquake you should leave the building
01:19:18  <ryah>nothing on the map
01:19:25  <ryah>i guess not
01:19:44  <ryah>when there's an earthquake you can feel the building swing a bit
01:20:35  <piscisaureus>I would not stay if my apartment flat did that
01:21:10  <piscisaureus>well, not if it was heavily shaking without a reasonable explanation anyway
01:21:49  <bnoordhuis>what's a reasonable explanation for heavy shaking in a building besides an earthquake?
01:22:06  <DrPizza>a REALLY big truck passing outside.
01:22:13  <bnoordhuis>haha
01:22:32  <DrPizza>or a huge Anonymous BART protest
01:22:49  <ryah>:)
01:28:30  <bnoordhuis>piscisaureus: typedef void (*uv_udp_recv_cb)(uv_udp_t* handle, ssize_t nread, uv_buf_t buf, struct sockaddr* addr, int flags);
01:28:43  <bnoordhuis>should that become a request struct as well? it's rather unwieldy this way
01:29:16  <bnoordhuis>ryah: ^
01:30:42  <ryah>no
01:31:03  <bnoordhuis>explain
01:31:03  <ryah>the req is somehow about i/o that will happen
01:31:19  <bnoordhuis>okay, not that has happened
01:32:14  <ryah>yeah
01:32:22  <ryah>each message can have flags?
01:32:40  <ryah>what sort of flags?
01:33:24  <bnoordhuis>ryah: PARTIAL for example
01:33:26  <ryah>http://dtrace.org/blogs/rm/2011/08/16/visualizing-kvm/
01:33:31  <bnoordhuis>that's MSG_TRUNC on unices
01:33:33  <ryah>^-- rmustacc
01:33:52  <bnoordhuis>that looks sweet
01:34:06  <bnoordhuis>i'm beginning to cave in
01:34:11  <piscisaureus>bnoordhuis: read and listen doesn't use a req. recv shouldn't either
01:34:27  <bnoordhuis>ryah piscisaureus: battle it out
01:34:39  <bnoordhuis>oh wait, recv
01:34:41  <bnoordhuis>no okay
01:34:58  <piscisaureus>bnoordhuis: also - multiple send's can be queued up but not recv's
01:35:03  <piscisaureus>there's only one recv at a time
01:35:14  <bnoordhuis>yep
01:35:25  <bnoordhuis>might still be worthwhile to wrap the arguments in a struct
01:36:28  <piscisaureus>bnoordhuis: I don't think it's necessary. But if you do, just pass the struct as an argument
01:36:34  <piscisaureus>or pointer
01:36:40  <bnoordhuis>right, as a pointer
01:36:49  <bnoordhuis>the reason i mention a struct
01:37:02  <bnoordhuis>is that on unices we'll probably want to pass aux data
01:43:20  <bnoordhuis>piscisaureus: https://gist.github.com/cc11217bf271d8a897c7
01:54:48  <piscisaureus>bnoordhuis: nice
01:54:55  <piscisaureus>bnoordhuis: final nit
01:55:05  <piscisaureus>bnoordhuis: what type do we use for a flags field?
01:55:23  <bnoordhuis>piscisaureus: you have something in mind?
01:55:40  <bnoordhuis>it's an int now btw
01:55:48  <piscisaureus>bnoordhuis: u....... int?
01:56:01  <bnoordhuis>piscisaureus: ?
01:56:07  <piscisaureus>ok
01:56:10  <piscisaureus>make it unsigned
01:56:23  <bnoordhuis>sure - but why?
01:57:13  <piscisaureus>well ok
01:57:18  <piscisaureus>maybe it doesn't matter
01:57:24  <DrPizza>because signed types are generally annoying
01:57:50  <piscisaureus>bitwise arithmetic and signed numbers are not good style
01:57:59  <bnoordhuis>bit shifting no
01:58:07  <bnoordhuis>bit or doesn't matter
01:58:17  <bnoordhuis>i don't mind making it unsigned though
01:58:23  <DrPizza>I think it's pretty bad style to use signed types for flags
01:58:42  <bnoordhuis>yeah? why?
01:59:03  <bnoordhuis>i don't see the problem as long as you don't bit shift
01:59:20  <DrPizza>because it means you can't bitshift
02:00:06  <bnoordhuis>some people see that as a feature
02:20:09  <piscisaureus>ok. let's do something relaxing for 15 minutes
02:35:49  * brsonquit (Ping timeout: 246 seconds)
02:48:25  <piscisaureus>does somebody already support gzip-ing http responses in node?
03:01:41  * bnoordhuisquit (Ping timeout: 260 seconds)
03:40:14  <ryah>piscisaureus: we want gzip support in core
03:41:15  <piscisaureus>ryah: I know, I was just wondering how people dealt with this so far
03:41:32  <piscisaureus>I mean, gzip-ing web pages is pretty common and useful too
03:49:01  <piscisaureus>anyway, im gone
03:49:05  <ryah>later
05:50:30  * isaacsjoined
05:52:11  * isaacsquit (Client Quit)
09:44:00  <CIA-95>libuv: Ryan Dahl master * red11883 / include/uv.h : Improve docs - http://bit.ly/oygeNE
12:48:00  <ryah>im 70% throughuv_loop_t patch
12:48:00  <ryah>no sharing of handles
12:48:00  <ryah>just namespacing
12:48:00  <ryah>unix only
14:00:00  <CIA-95>libuv: Jeroen Janssen master * r0883b3d / src/win/pipe.c : remove unused variables - http://bit.ly/qcpG0X
14:00:00  <CIA-95>libuv: Jeroen Janssen master * r507c4cc / (3 files in 2 dirs): fixes for unsigned/signed comparison - http://bit.ly/o9JBGm
14:00:00  <CIA-95>libuv: Jeroen Janssen master * ra51f376 / src/win/internal.h : added missing prototype for uv_process_pipe_shutdown_req - http://bit.ly/n3S1Il
14:00:00  <CIA-95>libuv: Jeroen Janssen master * r6b27bf9 / src/win/req.c : uv_process_pipe_shutdown_req call should match prototype - http://bit.ly/nGKrXJ
14:42:00  <CIA-95>libuv: Bert Belder master * rf71acf1/ AUTHORS : Update AUTHORS - http://bit.ly/pU0bA0
15:00:00  <CIA-95>libuv: Bert Belder master * rcc5aa51/ AUTHORS : Small correction to AUTHORS - http://bit.ly/rjbGc6
17:00:32  <ryah>piscisaureus: thanks for doing the authors
17:02:29  <piscisaureus>np
17:03:17  <piscisaureus>node's ATHORS needs some love as well I think
17:06:25  <piscisaureus>`git log --date-order --reverse --pretty=format:"%an <%ae>" --branches=v0.2,v0.4,master | uniq`
17:06:41  <ryah>yeah
17:06:48  <ryah>piscisaureus: tools/updateAuthors.awk
17:07:40  <ryah>piscisaureus: im doing multiplicity
17:07:47  <piscisaureus>oh, nice
17:08:00  <piscisaureus>ryah: what's the interface?
17:08:06  <ryah>https://gist.github.com/fb8fafb1c3684e7d04a7
17:08:38  <ryah>maybe it's better to see a diff
17:08:56  <piscisaureus>eh, yeah :-)
17:09:11  <ryah>https://gist.github.com/3e59c81f98f21ccd72a7
17:10:21  <piscisaureus>ryah: will multiplicitly be optional or always on?
17:10:32  <ryah>always on
17:14:19  <ryah>do you think that's bad?
17:17:42  <piscisaureus>I dont know
17:17:56  <piscisaureus>Why does Marc Lehman think it's bad?
17:18:35  <piscisaureus>Maybe we should have a uv_idle benchmark
17:19:01  <piscisaureus>How many loop iterations can we process per second?
17:25:10  <ryah>piscisaureus: because he's insane
17:25:30  <ryah>the overhead as far as i can tell is 1 byte to each stack frame
17:25:53  <ryah>possibly some extra pointer derefs
17:26:10  <ryah>EV_MULTIPLICITY is crazy talk
17:30:47  <ryah>my main problem is how to get eio results back into the correct thread
17:31:31  <ryah>it doesn't seem possible
17:31:37  <ryah>this might be the end of the road for eio
17:32:42  <DrPizza>does the unix side have any analogue to the req queue? can the eio worker threads post a req on the req queue so that it will be picked up by the correct thread?
17:33:11  <ryah>they can all post thread pool jobs
17:33:16  <ryah>the problem is getting the back
17:33:26  <ryah>the eio_req isn't associated with any particular
17:33:28  <ryah>thread
17:35:24  <DrPizza>ryah: so do you need some kind of uv_post_req(uv_loop_t*, uv_req_t) that can be safely called cross-thread, so that the workers can communicate their results back to the main thread and have it run callbacks etc. correctly?
17:35:41  <ryah>DrPizza: yeah
17:35:57  <ryah>the response queues need to be per thread
17:36:20  <DrPizza>pre thread or per loop?
17:36:52  * rmustaccpart
17:37:01  <igorzi>ryah: speaking of eio, what do we want to do with eio_custom? just create an equivalent API in libuv for now?
17:41:10  <ryah>igorzi: yes, please
17:44:36  <ryah>piscisaureus: we should definitely measure it - but im going to be surprised if it makes a diff
18:09:23  <CIA-95>libuv: Ryan Dahl multiplicity2 * r3461acc / (35 files in 3 dirs): multiplicity unix attempt 2 - http://bit.ly/oCRKv7
18:10:21  <ryah>^-- piscisaureus what do you think
18:11:24  <ryah>and everyone else
18:20:42  <piscisaureus>ryah: looks good
18:20:54  <piscisaureus>(honestly, I hate it but I know we have no choice)
18:21:13  <piscisaureus>ryah: only I'm not sure about uv_default_loop()
18:21:50  <piscisaureus>Since users will need that often we should avoid letting them make a call every time
18:22:34  <piscisaureus>maybe have `extern uv_loop_t* uv_default_loop` in uv.h?
18:23:46  <piscisaureus>ryah: also - do we let people call uv_init() once before everything else. I'd like that.
18:24:48  <piscisaureus>we could make it safe to call it more than once
18:27:48  <piscisaureus>http://jobs.nodejs.org/a/jbb/job-details/545251
18:28:13  <piscisaureus>^-- look at the minimum requirements. omg these people exist?
18:32:31  <DrPizza>piscisaureus: what I'm after are the job specs demanding "5 years experience with node.js"
18:36:43  <piscisaureus>they're not saying that
18:37:18  <piscisaureus>But:
18:37:18  <piscisaureus>Strong kernel dev experience
18:37:18  <piscisaureus>Strong network server dev experience
18:37:18  <piscisaureus>Experience with VM design, hotspot optimization, JIT and method JIT, AOT, etc.
18:37:18  <piscisaureus>Experience with Node.js and server side JavaScript.
18:37:19  <piscisaureus>^ -- pretty unreasonable
18:39:56  <CIA-95>node: Ben Noordhuis master * rd72c694 / benchmark/http_simple.js : bench: make http_simple send chunked encoding if requested - http://bit.ly/rs9zvR
18:51:56  <ryah>* Experience with VM design, hotspot optimization, JIT and method JIT, AOT, etc.
18:51:58  <ryah>hah
18:52:13  <ryah>those people actually do not exist
18:52:31  <ryah>there are like 100 of them
18:52:44  <bnoordhuis>but they're leaving oracle en masse now
18:53:01  <bnoordhuis>so who knows, maybe yahoo will scoop them up
18:53:08  <ryah>there are crazy good programmers out there
18:53:20  <ryah>but pretty few and far between :)
18:53:33  <ryah>e.g. my boss is pretty insanely good
18:53:44  <bnoordhuis>your boss is mark m?
18:53:52  <ryah>bryan cantrill
18:54:05  <bnoordhuis>oh right
18:54:16  <bnoordhuis>there aren't many of those
18:55:38  <ryah>so piscisaureus - default loop needs to be initialized
18:55:45  <ryah>on libev
18:56:08  <ryah>we should probably add a const to the return value
18:56:17  <ryah>hopefully the compiler can inline
18:56:28  <bnoordhuis>https://github.com/joyent/node/issues/1546 <- easy win, 15% speed up on http_simple
18:56:42  <piscisaureus>ryah: your compiler doesn't inline across objects by default
18:56:58  <piscisaureus>can't you just init the default loop on uv_init?
18:57:05  <ryah>bnoordhuis: sounds good
18:57:24  <ryah>bnoordhuis: can you post numbers in that thread
18:57:40  <bnoordhuis>ryah: yep, will do
18:57:54  <ryah>piscisaureus: do we need a uv_init() ?
18:57:59  <ryah>i think we should just init loops
18:58:08  <piscisaureus>ryah: well -
18:58:20  <piscisaureus>ryah: I nee to pickup pointers to kernel functions and winsock extensions etc
18:58:29  <piscisaureus>just once, not once for every loop
18:58:50  <ryah>you can do that lazily
18:59:12  <bnoordhuis>call time?
18:59:19  <piscisaureus>yes but it requires mutexes etc because ppl might initialize a loop from another thread?
18:59:40  <ryah>piscisaureus: use mutexes then
18:59:51  <bnoordhuis>piscisaureus: read up on rcu
19:00:13  <bnoordhuis>but only if it's a potential bottleneck, else don't bother
19:00:35  <ryah>piscisaureus: i think static volatile bool uv_has_been_inited;
19:00:39  <ryah>would work
19:00:51  <ryah>nm - you need mutex
19:01:18  <ryah>but it's fine
19:01:39  <DrPizza>rcu is patented
19:02:31  <ryah>there's nothing wrong with mutex locks when used properly - this is a fine use
19:03:24  <ryah>my question is: do we add a thread api to libuv?
19:03:30  <ryah>i think we're going to have to
19:03:31  <DrPizza>no
19:03:33  <DrPizza>ugh
19:03:49  <ryah>how can we test multiplicity without starting threads?
19:04:07  <ryah>and how can we start threads without an API?
19:04:50  <DrPizza>doesn't the test harness already have a thread API, or is that ust the win32 test harness
19:04:57  <DrPizza>https://github.com/joyent/libuv/blob/master/test/runner-win.c#L300
19:04:59  <bnoordhuis>DrPizza: it's very common in the linux kernel
19:05:10  <bnoordhuis>so i suspect patents aren't an issue
19:05:10  <DrPizza>bnoordhuis: IBM has given them a patent grant
19:05:15  <bnoordhuis>oh, is that it
19:05:30  <bnoordhuis>damn yanks and their patent system
19:06:10  <DrPizza>at least, that's my understanding
19:09:45  <piscisaureus>Are we going to have the call?
19:09:50  <ryah>gah sorry
19:09:51  <ryah>yes
19:09:52  <piscisaureus>I'd like to have dinner
19:10:01  <piscisaureus>we can also wait 30 minutes if that's preferrable
19:10:12  <ryah>let's jsut do it real quick im sorry
19:11:05  <ryah>bnoordhuis: skype
19:11:27  <piscisaureus>bnoordhuis: call
19:11:43  <ryah>fuck
19:11:45  <piscisaureus>wrong
19:11:45  <bnoordhuis>i'm there guys, where are you?
19:21:14  <ryah>piscisaureus: can you link me to the lock api?
19:21:21  <ryah>or just the name
19:21:43  <ryah>http://msdn.microsoft.com/en-us/library/ms682530(v=vs.85).aspx ?
19:21:48  <ryah>CRITICAL_SECTION
19:22:11  <DrPizza>that's the closest equivalent to a standard (non-shared) pthread mutex, if that's what you're after
19:24:25  <piscisaureus>ryah:
19:24:25  <piscisaureus>critical_section - http://msdn.microsoft.com/en-us/library/ms682530%28v=vs.85%29.aspx -> mutex that must be freed by the aquiring thread.
19:24:25  <piscisaureus>mutex - http://msdn.microsoft.com/en-us/library/ms684266%28v=vs.85%29.aspx -> more flexible mutex, can be used with wait functions etc
19:29:53  <ryah>yeah maybe we do need a uv_init...
19:29:56  <ryah>annoying
19:30:51  * graydonjoined
19:32:57  <piscisaureus>ryah: Maybe we could use Interlocked functions to achieve the same
19:38:40  <ryah>we need to change uv_getaddrinfo_t* handle into a uv_req_t
19:41:04  <ryah>https://twitter.com/#!/ben_howes/status/103913485334884352
19:41:11  <ryah>drupal inspired?
19:42:14  <piscisaureus>I don't like drupal
19:42:38  <ryah>i have never used it but people tell me its awesome
19:42:51  <piscisaureus>I have made many websites with drupal
19:43:30  <piscisaureus>It is probably the least bad ph-based cms
19:43:34  <piscisaureus>*php
19:44:43  <piscisaureus>the recommended way of using drupal is applying ugly hacks
19:48:35  <ryah>"Have you been the type to declare that those who don't use Android are cylons who are under mind control from Cupertino? Or are you Peter Bright, turning up your nose at all of us while you wax on about the unappreciated genius of the Windows 7 Phone?"
19:48:39  <ryah>:)
19:53:22  <piscisaureus>DrPizza: you work at ars technica?
19:54:16  <bentkus>interesting that the most interesting nodejs dev talk happens in the libuv channel
19:55:24  <piscisaureus>Yes this channel is much less noisy
19:55:58  <piscisaureus>But I used to kind of follow what is going on in #node, but no more
19:57:06  <bentkus>too much hype
19:58:06  <piscisaureus>bnoordhuis: have you ever heard of the "kinds of code" conference?
19:58:16  <bnoordhuis>piscisaureus: no
20:04:52  <ryah>bnoordhuis: i can't measure a difference by removing that debug() call
20:05:24  <ryah>were you testing v0.4 or master?
20:05:43  <bnoordhuis>ryah: you need to use /bytes/256/chunked and this patch:
20:05:43  <bnoordhuis>--- a/benchmark/http_simple.js
20:05:44  <bnoordhuis>+++ b/benchmark/http_simple.js
20:05:44  <bnoordhuis>@@ -92,7 +92,8 @@ var server = http.createServer(function (req, res) {
20:05:44  <bnoordhuis> "Content-Length": content_length });
20:05:44  <bnoordhuis> }
20:05:47  <bnoordhuis>
20:05:49  <bnoordhuis>- res.end(body);
20:05:51  <bnoordhuis>+ res.write(body);
20:05:53  <bnoordhuis>+ res.end();
20:05:55  <bnoordhuis>res.end(body) triggers the hot path
20:06:05  <bnoordhuis>on master btw
20:06:20  <ryah>ah
20:06:24  <ryah>great thanks.
20:14:20  <ryah>diff is minimal on v0.4
20:15:32  <piscisaureus>https://gist.github.com/1152491
20:15:32  <piscisaureus>ryah bnoordhuis igorzi: ^-- comments?
20:17:04  <bnoordhuis>piscisaureus: is there any chance uv_init() gets called more than once or from more than one thread?
20:18:10  <piscisaureus>bnoordhuis: the idea of this is that it can be called many times but uv__init is run exactly once. And uv_init blocks until uv__init is done, even if another thread calls it.
20:18:44  <bnoordhuis>piscisaureus: volatile doesn't work the way you think it does
20:18:53  <piscisaureus>how do I think it works?
20:18:56  <ryah>bnoordhuis: https://github.com/joyent/node/issues/1546#issuecomment-1830159
20:19:46  <bnoordhuis>ryah: i tried it with different concurrency levels and response body sizes
20:19:56  <bnoordhuis>always at least a 10% speed up
20:20:26  <bnoordhuis>but never higher than 15%
20:20:54  <bnoordhuis>piscisaureus: you think that a write to a volatile is atomic and immediately visible to other threads or cpu's
20:21:07  <piscisaureus>bnoordhuis: no
20:21:16  <piscisaureus>or - I secretly updated the gist
20:21:20  <piscisaureus>now I don't rely on this
20:21:37  <CIA-95>node: Ryan Dahl v0.4 * r9cd5108 / lib/http.js : Fixes #1546. Remove expensive debug call. - http://bit.ly/nlxZfz
20:21:53  <piscisaureus>bnoordhuis: but - I *do* assume that cmpxchg is immedately visible to other threads/cpus, but that is true right?
20:22:17  <igorzi>piscisaureus: bnoordhuis: it looks fine to me.. InterlockedCompareExchangePointer generates a full mem fence..
20:22:49  <piscisaureus>I am only concerned about mingw's support for InterlockedCompareExchangePointer
20:22:56  <ryah>bnoordhuis: im merging v0.4 into master
20:23:11  <bnoordhuis>ryah: go for it
20:23:23  <bnoordhuis>piscisaureus: cmpxchg is not, lock cmpxchg is
20:23:45  <piscisaureus>yes, that's what i mean
20:24:03  <piscisaureus>(although - doesn't cmpxchg add lock implicitly?)
20:24:10  <bnoordhuis>no
20:24:25  <bnoordhuis>uv_initialized still isn't used right
20:24:45  <piscisaureus>bnoordhuis: it's not a problem
20:25:14  <igorzi>piscisaureus: looks like InterlockedCompareExchangePointer is there in mingw winbase.h
20:25:15  <piscisaureus>bnoordhuis: if a thread sees an updated uv_initialized (so it sees 0) it will hit the WaitForSingleObject branch
20:25:35  <bnoordhuis>piscisaureus: does InterlockedCompareExchangePointer only swap when uv_init_event is NULL?
20:25:39  <piscisaureus>yes
20:25:48  <piscisaureus>and it returns the old value
20:26:09  <piscisaureus>(or just the value if uv_init_event != NULL)
20:26:16  <bnoordhuis>right, then it's safe
20:28:15  <piscisaureus>bnoordhuis: igorzi: thanks for your critical review :-)
20:28:58  <CIA-95>node: Ryan Dahl master * r9cd5108 / lib/http.js : Fixes #1546. Remove expensive debug call. - http://bit.ly/nlxZfz
20:28:59  <CIA-95>node: Ryan Dahl master * r8320af7 / (4 files in 2 dirs):
20:28:59  <CIA-95>node: Merge remote branch 'origin/v0.4'
20:28:59  <CIA-95>node: Conflicts:
20:28:59  <CIA-95>node: doc/api/tls.markdown - http://bit.ly/re9WEV
20:29:09  <piscisaureus>... this kind of stuff is always a minefield
20:30:17  <ryah>piscisaureus: we can just add a uv_init()
20:30:22  <ryah>piscisaureus: if this is too hairy
20:30:37  <ryah>and just say that it's not thread safe
20:30:44  <bnoordhuis>ryah: https://github.com/joyent/node/issues/1536
20:30:52  <piscisaureus>ryah: well... I kind of figured it out
20:31:21  <piscisaureus>ryah: but still I don't like making an actual call all the time when calling uv_default_loop()
20:31:32  <piscisaureus>maybe we can check if this gets optimized out
20:31:47  * isaacsjoined
20:32:14  <ryah>bnoordhuis: i think we're removing --vars
20:32:27  <ryah>doesn't make sense x-platform
20:32:38  <bnoordhuis>ryah: it's sometimes useful when people report bugs
20:32:57  <ryah>yes i want to know what he's doing with it
20:33:43  <piscisaureus>what does node --vars do?
20:33:57  <bnoordhuis>piscisaureus: prints the build flags node was compiled with
20:34:14  <bnoordhuis>piscisaureus: what's the deal with that 'kinds of code' conference?
20:35:05  <piscisaureus>bnoordhuis: oh it's 'kings of code'
20:36:01  <piscisaureus>bnoordhuis: they want me to do a talk on node
20:36:19  <piscisaureus>I guess ryah is too expensive
20:37:08  <bnoordhuis>kings of code i've heard of
20:37:10  <bnoordhuis>amsterdam right?
20:38:53  <piscisaureus>yeah
20:39:08  <piscisaureus>is that a fine conference or a bullshit one?
20:41:09  <bnoordhuis>iirc a colleague of mine attended in 2009 and it was alright
20:43:20  <bnoordhuis>ryah: https://github.com/bnoordhuis/node/commit/b816fea <- worthwhile addition to http_simple?
20:43:34  <CIA-95>node: isaacs master * r711a356 / doc/api/url.markdown : Close #1544 Document slashesDenoteHost flag in url.parse - http://bit.ly/o9OuLF
20:46:12  <CIA-95>node: isaacs v0.4 * r307a95f / doc/api/url.markdown : Close #1544 Document slashesDenoteHost flag in url.parse - http://bit.ly/r8PZ7i
20:47:34  <DrPizza>piscisaureus: yes, I write for Ars Technica
20:54:05  <CIA-95>node: Ben Noordhuis master * r8e8f36f / lib/http2.js : Fix #1546 some more. Remove expensive debug call. - http://bit.ly/nn4e1E
20:54:58  <bnoordhuis>piscisaureus: suggestions on udp things you want tested?
20:55:47  <piscisaureus>bnoordhuis: everything
20:56:03  <CIA-95>libuv: Igor Zinkovsky zero_reads * r4898d40 / (include/uv-win.h src/win/internal.h src/win/tcp.c): Windows: Pre-allocate buffers for overlapped WSARecv if the number of active tcp streams is below 50. - http://bit.ly/r1EKQp
20:56:11  <bnoordhuis>piscisaureus: enumerate
20:56:25  <igorzi>piscisaureus: please review ----^
20:56:56  <piscisaureus>bnoordhuis: send, receive packets. MSG_TRUC support. send with/without bind. Check sender address/port.
20:57:37  <piscisaureus>bnoordhuis: send never returns error
20:57:44  <piscisaureus>what else?
20:57:52  <bnoordhuis>piscisaureus: double bind
20:58:00  <piscisaureus>yeah
20:58:08  <piscisaureus>recv_stop
20:58:13  <piscisaureus>double recv_start
20:58:21  <bnoordhuis>send of packet > mtu (but maybe that's what you meant with MSG_TRUNC)
20:58:39  <piscisaureus>bnoordhuis: no that's not what I mean
20:58:57  <DrPizza>ryah: oh lol, I just saw your quote about me. Where did you getthat?!
20:59:11  <piscisaureus>bnoordhuis: I mean, send a 10kb packet but have alloc_cb return a 5k buffer.
20:59:35  <DrPizza>omg
20:59:37  <DrPizza>it was written on my own site
20:59:38  <DrPizza>;_;
20:59:39  <bnoordhuis>piscisaureus: you can't send 10 kb packets :) but i get what you mean
20:59:41  <DrPizza>trolled by jacqui
20:59:42  <DrPizza>lol
21:01:36  <piscisaureus>igorzi: put the "50" in a constant
21:01:55  <igorzi>piscisaureus: ok, will do
21:04:05  <piscisaureus>the comment /* Post another 0-read if still reading and not closing. */ is no longer accurate
21:04:42  <igorzi>oh right
21:05:24  <piscisaureus>igorzi: also - if read_stop is called while a non-0 read is in flight uv_process_tcp_read_req hits the nonblocking path again
21:05:51  <piscisaureus>it won't harm thouh
21:06:56  <piscisaureus>hmm
21:07:56  <igorzi>piscisaureus: won't it skip the while loop?
21:08:02  <piscisaureus>yeah probably
21:08:09  <piscisaureus>but what if the user calls read_start again after that
21:08:19  <piscisaureus>do we correctly report that buffer to the user after that?
21:08:23  <piscisaureus>also
21:09:43  <piscisaureus>igorzi: suppose
21:09:43  <piscisaureus>- the user calls read_start, libuv does a non-0 read
21:09:43  <piscisaureus>- the user calls read_stop,
21:09:43  <piscisaureus>- the read completes, libuv doesn't call read_cb but hangs on to the buffer
21:09:43  <piscisaureus>- the user calls uv_close
21:09:44  <piscisaureus>where are we giving the user the buffer back so it can free it?
21:09:58  <piscisaureus>ryah: this is also a question for you
21:10:17  <DrPizza>maybe make it so that read_stop prevents the issuance of new reads, but can still allow outstanding reads to complete
21:10:24  <DrPizza>(and so call their callback)
21:10:30  <piscisaureus>yes I think that would be best
21:10:33  <piscisaureus>ryah: do you agree?
21:11:51  <piscisaureus>it may be unfortunate for node users that depend on the non-behaviour that pause() immediately stops all data events
21:12:32  <piscisaureus>api design is hard
21:13:20  <igorzi>hmm.. i'd assume that the caller will know that since because it never got read_cb it'll need to free the buffer when it does uv_close
21:13:45  <piscisaureus>igorzi: that is so painful
21:14:07  <DrPizza>that sounds like a displeasing API
21:14:12  <piscisaureus>igorzi: right now in the unlikely event that we end up with an unused buffer we call read_cb with nread=0
21:14:50  <piscisaureus>but read_stop - right now - blocks further read_cbs so we can't use that
21:16:00  <igorzi>the problem with the current api is that read_cb is also used as an indicator to free the buffer.. something has to give.. either we're allowed to do read_cb after read_stop, or we need some other way to notify caller that the buffer needs to be freed
21:16:01  <ryah>bnoordhuis: regarding chunked addition - sure. it doesn't affect the existing tests, right?
21:16:21  <bnoordhuis>ryah: nope
21:16:38  <igorzi>(or the caller needs to track it, which i agree is painful)
21:17:33  <piscisaureus>we better solve this problem right because we will have the same problem with file streams (incl. linux)
21:17:52  <ryah>piscisaureus: read_stop does should not mean that uv_read_cb isn't called
21:17:53  <piscisaureus>because that also requires preallocated buffers.
21:18:32  <piscisaureus>bnoordhuis: igorzi: DrPizza: we all agree with ryah?
21:18:35  <piscisaureus>I do
21:18:41  <DrPizza>yes
21:18:54  <igorzi>yes
21:18:55  <DrPizza>read_stop just means no new read attempts are made, it shouldn't influence outstanding ones
21:19:03  <bnoordhuis>oh right, yes
21:19:07  <igorzi>ok great - we have a solution then
21:19:23  <piscisaureus>igorzi: ok, can you make it so it works that way?
21:19:26  <ryah>i still do not love our alloc_cb and read_cb API
21:19:31  <ryah>but i cannot think of anything better
21:19:51  <igorzi>piscisaureus: yep - i'll make that work
21:19:56  <igorzi>bbl
21:20:59  <ryah>https://github.com/joyent/node/blob/c05936ca13681059c7aeecfe3a608e4e1afa800a/src/stream_wrap.cc#L188-190
21:21:02  <ryah>^-- this scares me
21:21:22  <ryah>it works most of the time but we have the possibility for exploding memory
21:21:52  <ryah>https://github.com/joyent/node/blob/c05936ca13681059c7aeecfe3a608e4e1afa800a/src/stream_wrap.cc#L143-168
21:27:25  <CIA-95>node: Ben Noordhuis master * r63607a0 / benchmark/http_simple.js : bench: make number of response body chunks configurable in http_simple - http://bit.ly/nLX7kh
21:44:26  <piscisaureus>ryah: doesn't the node 0.4 slab allocator have the same properties?
21:52:28  <ryah>piscisaureus: yeah - i guess
21:54:27  <piscisaureus>although I think that pre-allocating reads is going to make this problem worse
21:55:08  <piscisaureus>because it will make (handle_that_last_alloced == handle) almost never true
21:56:54  <piscisaureus>it would be so awesome if v8 could just give us some space on its heap
21:57:02  <piscisaureus>and then resize it heh
22:00:00  <ryah>yes :)
22:06:04  <piscisaureus>The Buffer.slice api was a bad idea
22:09:35  <bnoordhuis>why?
22:11:37  <piscisaureus>It makes it much more difficult to improve this situation in the future. You can have 2 objects that are guaranteed to reference the same memory. I don't think we can ever achieve the same with typedarrays or something.
22:13:25  <piscisaureus>Although the major showstopper for referencing the v8 heap is the inability to lock objects into position
22:18:02  <ryah>typedarrays have a similiar slice thing
22:18:47  <piscisaureus>can typedarrays be locked into memory?
22:19:28  <DrPizza>Does v8 not expose its allocator?
22:22:39  <ryah>piscisaureus: yes
22:22:45  <ryah>but they're not in v8
22:22:54  <ryah>they're outside the heap
22:22:59  <ryah>we have typed arrays now btw
22:23:01  <piscisaureus>hmm
22:23:42  <piscisaureus>so this typed array implementation is not going to give us much
22:23:53  <ryah>> t = new Int8Array(5); t[0] = 0xff
22:23:54  <ryah>255
22:24:15  <ryah>more or less the same as buffers
22:24:23  <piscisaureus>yeah
22:24:27  <ryah>but we're going to slowly replace the buffer API with them
22:25:00  <DrPizza>so typed arrays are chunks of C++/slab allocator memory, and the v8 GC just tracks the object references, not the memory itself?
22:25:08  <bnoordhuis>piscisaureus: udp: do we *want* to be able to steal already bound ports?
22:25:12  <ryah>DrPizza: yes
22:25:20  <piscisaureus>bnoordhuis: *shrug* no
22:25:43  <bnoordhuis>piscisaureus: good
22:25:55  <piscisaureus>bnoordhuis: but it would be nice if we could steal ports that are 'occupied' by a TIME_WAIT socket
22:26:17  <bnoordhuis>piscisaureus: you don't have TIME_WAIT with udp
22:26:22  <ryah>DrPizza: https://github.com/joyent/node/blob/66f77963a368f8696c26db7b42e510fc0b2aebbf/deps/v8/include/v8.h#L1634-1658
22:26:27  <piscisaureus>how does String::AsciiValue lock it's target into memory?
22:26:34  <ryah>DrPizza: this is how they are interfaced into V8
22:27:00  <ryah>https://github.com/joyent/node/blob/66f77963a368f8696c26db7b42e510fc0b2aebbf/deps/v8/include/v8.h#L1634-1648
22:28:04  <DrPizza>so v8 has a kind of built-in concept of objects that own arrays that are kept external to it
22:28:16  <ryah>yeah
22:28:33  <ryah>it's just a bit of magic to allow you to define square bracket operator
22:28:36  <DrPizza>heh
22:28:49  <DrPizza>does it still have a notion of the memory usage incurred by the object?
22:29:09  <piscisaureus>no
22:29:32  <piscisaureus>well it tracks amountOfExternallyAllocatedMemory or something
22:29:41  <DrPizza>yeah
22:29:48  <piscisaureus>but I don't think it understands the memory that a particular object takes
22:29:48  <DrPizza>to allow external objects to fake memory pressure
22:30:09  <ryah>there are some interesting new callbacks in here...
22:30:18  <ryah>https://github.com/joyent/node/blob/66f77963a368f8696c26db7b42e510fc0b2aebbf/deps/v8/include/v8.h#L2996-3008
22:30:22  <ryah>havent seen this before
22:31:10  <ryah>that would be a nice place for a dtrace probe
22:31:36  <bnoordhuis>this one will make pquerna happy: https://github.com/joyent/node/blob/66f77963a368f8696c26db7b42e510fc0b2aebbf/deps/v8/include/v8.h#L3045
22:33:06  <bnoordhuis>piscisaureus: i think i want a uv_udp_init6 function
22:33:17  <bnoordhuis>one that takes a flag UV_UDP_IPV6ONLY or something
22:33:51  <bnoordhuis>added benefit: you can create the socket in the uv_udp_init* function itself, no need to defer
22:34:23  <piscisaureus>hmm ok.
22:34:33  <piscisaureus>I don't like it very much
22:34:50  <piscisaureus>but since I say this all the time maybe I'll shut up
22:34:52  <piscisaureus>So
22:35:45  <piscisaureus>every time you call String::Utf8Value v8 just mallocs that memory and copies the string there
22:36:36  <piscisaureus>i never realized that
22:36:52  <ryah>really?
22:36:56  <piscisaureus>yes
22:36:59  <ryah>literally mallocs?
22:37:12  <piscisaureus>well it calls new char[len + 1]
22:37:14  <ryah>or allocs in its heap
22:37:19  <piscisaureus>no it calls new
22:37:22  <ryah>oh
22:37:25  <ryah>man...
22:37:42  <ryah>i didnt realize that either - i thought it was in the heap
22:37:53  <DrPizza>It still could be.
22:37:55  <piscisaureus>the object "wrapper" is in the heap
22:38:01  <DrPizza>They might have a global operator new overload.
22:38:04  <piscisaureus>so it gets garbage collected
22:38:06  <piscisaureus>they don't
22:38:10  <DrPizza>heh
22:38:11  <ryah>DrPizza: i dont think so
22:40:22  <piscisaureus>ryah: https://github.com/joyent/node/blob/5b58473b4060094b07b4cbd1a7e77ea2338773d5/deps/v8/src/api.cc#L5104-5122
22:40:22  <piscisaureus>and https://github.com/joyent/node/blob/5b58473b4060094b07b4cbd1a7e77ea2338773d5/deps/v8/src/allocation.h#L83-94
22:40:53  <bnoordhuis>old news is old!
22:41:07  <piscisaureus>bnoordhuis: ?
22:41:26  <bnoordhuis>piscisaureus: that string::utf8value mallocs memory
22:41:53  <piscisaureus>well... I knew that it maks a copy, but I didn't know that it mallocs
22:42:15  <bnoordhuis>where else would it get the memory from?
22:42:22  <bnoordhuis>not the v8 heap
22:42:23  <piscisaureus>from it's own heap
22:42:28  <piscisaureus>why not?
22:43:18  <bnoordhuis>not 100% sure
22:43:40  <bnoordhuis>but i suspect that it's not always safe or easy to manipulate the v8 heap
22:43:59  <DrPizza>maybe there's no point for objects whose lifetimes are totally irrelevant to the GC
22:44:16  <piscisaureus>it *is* relevant to the gc
22:44:19  <DrPizza>maybe it's actually bad to do so, since it means it can't then compact those objects
22:44:23  <DrPizza>is it?
22:44:30  <DrPizza>how can it be, if it lives outside it?
22:44:55  <piscisaureus>my theory is that v8 wants to move its objects around but doing so would break the pointer you get from Utf8Value::Operator*
22:45:06  <DrPizza>I assume v8 is a compacting gc
22:45:19  <DrPizza>but that means it can't let raw pointers leak out
22:45:31  <DrPizza>and perhaps they found it useful to have raw pointers leak out for strings
22:45:39  <bnoordhuis>piscisaureus: good point
22:46:34  <piscisaureus>bnoordhuis: that was why i was checking it in the first place :-) I wanted to find out how v8 was able to lock stuff to its place in memory, but as it turns out, it doesn't.
22:46:43  <DrPizza>heh
22:55:44  * mralephjoined
23:22:31  <igorzi>piscisaureus: what would be a good last_error.code for when we dispatch read_cb with -1? (to indicate free buffer)
23:22:57  <piscisaureus>igorzi: we already do under some circumstances
23:23:03  <piscisaureus>I think it's uv_eagain
23:23:09  <piscisaureus>I'm going to check
23:23:49  <ryah>yes
23:24:15  <ryah>1sec - let me link to some code
23:24:18  <piscisaureus>igorzi:
23:24:18  <piscisaureus>https://github.com/joyent/libuv/blob/master/src/win/tcp.c#L816-818
23:24:26  <piscisaureus>igorzi: that maps to UV_EAGAIN
23:24:45  <igorzi>piscisaureus: ok, thanks
23:25:21  <piscisaureus>this uv_set_sys_error(WSAEWOULDBLOCK); is pretty backwards btw - we should be able to set err.code directly and still provide some usefull description of the error
23:25:38  <ryah>piscisaureus, igorzi: https://github.com/joyent/libuv/blob/cc5aa51fdf115bd15eb422a3e483d8bc84823626/test/echo-server.c#L87
23:25:54  <ryah>^-- EAGAIN
23:27:27  <igorzi>yep, looks like WSAEWOULDBLOCK gets translated into UV_EAGAIN
23:28:35  <ryah>read_cb with nread == 0 is possible
23:29:12  <piscisaureus>ryah: btw I'm afraid that we will have more loc than Boost:ASIO at some point
23:29:28  <DrPizza>lol
23:29:30  <ryah>piscisaureus: yeah perhaps :)
23:29:39  <ryah>i thought that too
23:29:45  <DrPizza>I find that hard to believe
23:29:50  <DrPizza>boost::ASIO is pretty damn big
23:30:00  <piscisaureus>Boost ASIO – works correctly but is unacceptably massive (300 files
23:30:00  <piscisaureus>and 12000 semicolons). Doesn’t support regular files.
23:30:09  <DrPizza>heh
23:30:13  <piscisaureus>OK 300 files is still far out
23:30:24  <piscisaureus>12000 semicolons is not so much though
23:30:35  <DrPizza>no regular files in ASIO? sad.
23:30:53  <DrPizza>I have a treat in store for the weekend
23:30:56  <piscisaureus>DrPizza: I don't think it provides an abstraction of any kind btw
23:30:58  <DrPizza>the new libdispatch code
23:31:21  <piscisaureus>DrPizza: it just wants you to decide between ReadFile and read() etc
23:31:24  <DrPizza>piscisaureus: no? just C++ wrappers around various OS-specific functionality? I thought it was more abstract than that
23:31:27  <piscisaureus>not really an x-platform layer
23:31:29  <DrPizza>huh
23:31:33  <DrPizza>that seems pointless if true
23:31:34  <ryah>~/projects/libuv% find . | grep -v .git | wc -l 3685
23:31:41  <ryah>3685
23:32:24  <piscisaureus>wc -l?
23:32:34  <ryah>count lines
23:32:48  <piscisaureus>300 files and 12000 semicolons -> 40 semicolons / file
23:32:51  <piscisaureus>must me small files
23:33:02  <DrPizza>piscisaureus: lots of angle brackets, no doubt
23:33:24  <piscisaureus>maybe they have just #define SEMICOLON ;
23:33:38  <piscisaureus>then printf("hello") SEMICOLON
23:33:50  <piscisaureus>that sounds like reasonable in a C++ world
23:33:56  <ryah>okay libuv has currently 444 files
23:33:57  <DrPizza>;_;
23:34:02  <ryah>approximately
23:34:03  <DrPizza>444 O_O
23:34:06  <DrPizza>that seems a lot
23:34:09  <ryah>mostly due to pthread-win32
23:34:12  <DrPizza>oh
23:34:14  <DrPizza>well
23:34:19  <DrPizza>pthread-win32 has one file per function
23:34:21  <DrPizza>it's crazy.
23:34:39  <DrPizza>it's weird, it does some few complex things
23:34:56  <DrPizza>but seems to have been developed by someone who doesn't really understand how modern C compilers work
23:36:17  <piscisaureus>hmm, I may be wrong about asio not abstracting stuff
23:36:23  <piscisaureus>it at least does for sockets
23:36:35  <DrPizza>I thought it did
23:37:07  <piscisaureus>http://www.boost.org/doc/libs/1_47_0/doc/html/boost_asio/reference.html
23:39:22  <DrPizza>hm
23:39:48  <DrPizza>select, Solaris /dev/poll, epoll, kqueue, and IOCP
23:40:06  <piscisaureus>all you ever need
23:40:41  <ryah>ryan@mac1234:~/projects/libuv% find src | grep -v ".git" | wc -l 177
23:41:03  <ryah>we have a lot of crap with ares and ev autoconf files
23:41:09  <ryah>but 177 files in src/
23:42:37  <DrPizza>my files are slightly out of date
23:42:37  <piscisaureus>ryah: http://www.boost.org/doc/libs/1_47_0/doc/html/boost_asio/reference/io_service/run/overload1.html
23:42:42  <DrPizza>but 28 files for core libuv
23:42:49  <piscisaureus>"Multiple threads may call the run() function to set up a pool of threads from which the io_service may execute handlers. All threads that are waiting in the pool are equivalent and the io_service may choose any one of them to invoke a handler. "
23:43:00  <DrPizza>136 files for ares + eio + ev
23:45:37  <ryah>piscisaureus: so they must use a single iocp
23:45:42  <piscisaureus>yes
23:45:58  <ryah>so should we :)
23:47:09  <piscisaureus>asio supports tcp/udp/raw sockets, timers, ssl, serial ports
23:47:15  <piscisaureus>we can do better than that
23:47:28  <DrPizza>ssl in uv?
23:47:34  <piscisaureus>-1
23:47:38  <DrPizza>lol
23:47:39  <piscisaureus>serial ports -10
23:50:35  <ryah>do they do dns?
23:50:47  <piscisaureus>I think so but I can't find documentation so far
23:51:04  <piscisaureus>i think its somewhere inside tcp::protocol::resolver
23:51:33  <ryah>god i can't stand boost
23:51:41  <piscisaureus>why?
23:51:41  <ryah>the docs are really annoying
23:51:52  <ryah>it's too OO
23:51:52  <DrPizza>the docs are enormously inconsistent in style and presentation
23:52:11  <ryah>template<
23:52:11  <ryah> typename InternetProtocol,
23:52:11  <ryah> typename ResolverService = resolver_service<InternetProtocol>>
23:52:11  <ryah>class basic_resolver :
23:52:12  <ryah> public basic_io_object< ResolverService >
23:52:18  <ryah>groan...
23:52:35  <piscisaureus>http://www.boost.org/doc/libs/1_47_0/doc/html/boost_asio/reference/ip__basic_resolver/async_resolve.html
23:52:36  <DrPizza>meh, I don't care about that stuff, as long as I never have to type it
23:52:44  <DrPizza>it's a legitimate implementation strategy IMO
23:53:07  <DrPizza>wreaks havoc on compile times
23:53:13  <DrPizza>but hey, no runtime penalty
23:53:41  <ryah>i'm not against templates if used in moderation
23:55:12  <piscisaureus>http://stackoverflow.com/questions/6701435/scalable-server-framework-in-c
23:56:19  <ryah>"have been unable to find someone yet who can do it within my budget (2 or 300 US dollars)"
23:56:22  <ryah>yeah...
23:58:17  <DrPizza>ryah: I would love a libuv++ wrapper, that did just enough wrapping to let me use c++ lambdas instead of function pointers
23:59:24  <piscisaureus>I wonder if we should add a way for people to leave out parts of libuv
23:59:39  <ryah>DrPizza: don't c++ lambda have a different type than function pointers?
23:59:44  <ryah>you wouldn't be able to sue the directly
23:59:52  <DrPizza>ryah: yeah, you'd have to stash them in the void*
23:59:53  <piscisaureus>e.g. if you're not interested in console io then #define UV_NOCONSOLE