00:32:05  * mhicksquit (Quit: Leaving.)
00:46:34  * trentmquit (Quit: Leaving.)
01:20:00  * ed209quit (Remote host closed the connection)
01:20:08  * ed209joined
02:05:10  * [0xAA]quit (Ping timeout: 250 seconds)
02:21:46  * bahamatquit (Quit: Leaving.)
02:24:36  * mhicksjoined
02:25:28  * dap_quit (Quit: Leaving.)
02:27:04  * mhicksquit (Client Quit)
02:27:14  * mhicksjoined
02:27:34  * jhendricksquit (Read error: Connection reset by peer)
02:28:06  * jhendricksjoined
02:30:30  * mellocquit (Quit: Leaving.)
02:35:53  * jhendricksquit (Ping timeout: 245 seconds)
04:45:03  * mhicksquit (Quit: Leaving.)
04:48:55  * mhicksjoined
05:48:41  * mhicksquit (Quit: Leaving.)
06:03:59  * mhicksjoined
06:04:09  * mhicksquit (Client Quit)
06:09:55  * mhicksjoined
10:20:00  * ed209quit (Remote host closed the connection)
10:20:07  * ed209joined
10:40:02  * mariusp_joined
14:53:11  * pmooneyjoined
16:16:28  * chorrelljoined
16:20:56  * mariusp_quit
16:41:16  * chorrellquit (Read error: Connection reset by peer)
16:42:08  * chorrelljoined
16:52:03  * chorrellquit (Quit: My MacBook has gone to sleep. ZZZzzz…)
17:07:31  * mellocjoined
17:12:19  * dap_joined
17:14:15  * trentmjoined
17:15:39  * jhendricks1joined
17:18:16  * trentmquit (Client Quit)
18:01:22  * bahamatjoined
18:35:14  * chorrelljoined
18:52:00  * [0xAA]joined
18:54:04  * headw01joined
19:24:58  * headw01quit (Remote host closed the connection)
19:28:17  * headw01joined
19:42:27  * jen20__joined
19:43:25  <jen20__>Hi all! Hopefully a simple question: is it possible to stream stdout of a process running in the reduce phase of a manta job?
19:45:26  <dap_>Stream it to where?
19:48:40  <jen20__>well, actually stream is probably not the correct term - what I'm actually looking to do is have access to it for use with e.g. mget, but in a partial state
19:49:13  <jen20__>it looks like mtee should be able to do that, but had no luck with that approach so far
19:50:48  <dap_>Ah. There's no built in way to do that today. It would be possible to wire something together yourself.
19:51:20  <jen20__>effectively just redirect the output of the command to another file in the same container, I'd guess?
19:51:43  <dap_>If that's sufficient...I was assuming you'd need some way to get it out of the container too
19:52:29  <dap_>in which case you'd need to pipe it to something that was sending it to some other network service you've got. You could cobble together an "nc" with a "curl" or something to do that...but it's not trivial.
19:53:02  <dap_>One reason for this is that Manta can always execute tasks multiple times (if we have internal failures), so the output is not committed until the task fully completes.
19:53:23  <dap_>Are you doing this for debugging? Or a progress report?
19:54:02  <jen20__>Oh, yeah that makes sense. Progress reporting - basically I want a way to monitor a running job (eventually exposed to a UI over HTTP)
20:20:00  * ed209quit (Remote host closed the connection)
20:20:07  * ed209joined
20:29:30  * chorrellquit (Quit: Textual IRC Client: www.textualapp.com)
20:46:38  * chorrelljoined
21:03:02  <dap_>jen20__: one thing we've often talked about is having long jobs like that periodically PUT a little status file to /$MANTA_USER/jobs/$MANTA_JOB_ID/stor/status.txt or something. But that'd be something the task itself would have to do. We could build in some commands to assist in doing that.
21:08:54  * jhendricks1quit (Ping timeout: 250 seconds)
21:22:26  * echelog-1quit (Quit: Ctrl-C at console.)
21:23:01  * echelog-1joined
21:35:44  <jen20__>that would likely work, I hadn't considered that as an option
21:35:48  <jen20__>thanks @dap_
21:35:58  <jen20__>thanks @dap
21:36:04  <jen20__>ugh, IRC and lack of edit :p
21:51:36  * chorrellquit (Quit: Textual IRC Client: www.textualapp.com)
21:57:43  * headw01quit (Remote host closed the connection)
22:22:56  <dap_>No problem.
22:23:52  <jen20__>Actually, do interactive jobs created with mlogin have the same behaviour with regard to multiple executions?
22:29:04  * [0xAA]quit (Ping timeout: 250 seconds)
22:29:14  <dap_>jen20__: No. If they fail for any reason, they're basically done. (Technically, they get retried too, but they're implemented so that subsequent attempts fail immediately)
22:32:22  <jen20__>I suspect I can abuse that to actually get exactly what I want - thanks!
22:46:06  * [0xAA]joined
22:57:27  * dap_quit (Quit: Leaving.)
22:58:57  * jhendricksjoined
23:00:52  * jhendricks1joined
23:02:58  * jhendricksquit (Ping timeout: 245 seconds)
23:39:34  * rmustacctopic: Manta: Big Data Unix | Now Open Source! -- https://github.com/joyent/manta | http://apidocs.joyent.com/manta/ | http://logs.libuv.org/manta/latest
23:40:08  * rmustacctopic: Manta: Big Data Unix | Now Open Source! -- https://github.com/joyent/manta | http://apidocs.joyent.com/manta/ | http://logs.libuv.org/manta/latest
23:50:22  * rmustacctopic: Manta: Big Data Unix | Now Open Source! -- https://github.com/joyent/manta | http://apidocs.joyent.com/manta/ | http://logs.libuv.org/manta/latest