00:17:46  * lloyddequit (Remote host closed the connection)
00:18:22  * lloyddejoined
00:19:05  * yunongquit (Quit: Leaving.)
00:19:11  * fredkquit (Quit: Leaving.)
00:19:44  * nfitchquit (Quit: Leaving.)
00:22:21  * lloyddequit (Ping timeout: 240 seconds)
00:56:07  * dapquit (Quit: Leaving.)
00:56:28  * dapjoined
00:56:38  * dapquit (Client Quit)
00:59:44  * yunongjoined
01:01:01  * yunongquit (Client Quit)
01:01:03  * bixujoined
01:03:42  * ins0mniaquit (Ping timeout: 256 seconds)
01:12:22  * yunongjoined
01:12:53  * _Tenchi_joined
01:22:06  * yunongquit (Quit: Leaving.)
01:28:25  * abraxasjoined
01:36:32  * bixuquit (Read error: Connection reset by peer)
01:37:05  * bixujoined
01:44:58  * fredkjoined
01:45:27  * fredkquit (Client Quit)
02:14:34  * echelog-1quit (Disconnected by services)
02:17:52  * echelog-1joined
02:20:41  * echelog-1quit (Remote host closed the connection)
02:22:29  * echelog-1joined
03:32:47  * lloyddejoined
04:37:55  * yunongjoined
04:38:27  * yunongquit (Client Quit)
05:09:00  * marsellquit (Quit: marsell)
05:09:59  * marselljoined
05:11:35  * marsellquit (Client Quit)
05:29:50  * lloyddequit (Ping timeout: 240 seconds)
05:31:06  * lloyddejoined
05:42:06  * lloyddequit (Remote host closed the connection)
05:42:32  * lloyddejoined
05:46:48  * lloyddequit (Ping timeout: 245 seconds)
05:50:24  * marselljoined
06:04:59  * yunongjoined
06:13:04  * lloyddejoined
06:21:23  * lloyddequit (Ping timeout: 245 seconds)
06:55:54  * marsellquit (Quit: marsell)
07:18:01  * lloyddejoined
07:22:06  * lloyddequit (Ping timeout: 245 seconds)
07:44:21  * yunongquit (Quit: Leaving.)
08:11:11  * wpreulquit (Read error: Connection reset by peer)
09:19:05  * lloyddejoined
09:22:56  * ins0mniajoined
09:24:06  * lloyddequit (Ping timeout: 276 seconds)
09:26:16  * marselljoined
09:50:46  * ghostbarquit (Remote host closed the connection)
10:19:23  * lloyddejoined
10:23:26  * lloyddequit (Ping timeout: 240 seconds)
11:19:43  * lloyddejoined
11:23:49  * lloyddequit (Ping timeout: 246 seconds)
11:24:20  * abraxasquit (Remote host closed the connection)
12:20:03  * lloyddejoined
12:24:27  * lloyddequit (Ping timeout: 256 seconds)
12:54:03  * echelog-1quit (Ping timeout: 240 seconds)
12:56:46  * echelog-1joined
13:20:23  * lloyddejoined
13:24:24  * lloyddequit (Ping timeout: 241 seconds)
13:43:27  * lloyddejoined
13:47:56  * lloyddequit (Ping timeout: 245 seconds)
14:06:12  * lloyddejoined
14:11:58  <nahamu>random idea: browser extension that correctly handles the response when you GET a directory and renders it like a regular web server index page.
14:35:30  * chorrelljoined
14:42:16  * fennokin_joined
14:44:37  * paulfryzeljoined
14:45:32  * lloyddequit (Remote host closed the connection)
15:06:41  * lloyddejoined
15:38:33  * wpreuljoined
15:42:49  * wpreulquit (Ping timeout: 246 seconds)
15:45:50  * wpreuljoined
15:46:37  <wpreul>are there any good manta jobs examples that use node that someone can point me to?
15:49:06  * nfitchjoined
16:01:49  <rmustacc>wpreul: http://mcavage.me/blog/2013/07/19/using-node-modules-in-manta/
16:01:51  <rmustacc>That help?
16:02:12  <wpreul>yes, thank you
16:09:16  * fredkjoined
16:21:39  * paulfryzelquit (Remote host closed the connection)
16:22:02  * yunongjoined
16:22:11  * wpreulquit (Remote host closed the connection)
16:26:11  * dapjoined
17:21:04  * yunongquit (Quit: Leaving.)
17:42:54  * ghostbarjoined
17:59:20  * ArtyFactsjoined
17:59:47  <ArtyFacts>hiya
17:59:58  <rmustacc>Hello
18:00:29  <ArtyFacts>i think tj is sick of me bugging him with questions so he's kicking me over to you ;-)
18:00:35  <tjfontaine>:)
18:00:39  <ArtyFacts>nice to meet you rmustacc
18:01:06  <rmustacc>Well, TJ knows the answers better than I do.
18:01:18  <tjfontaine>well, maybe or maybe no :)
18:01:26  <ArtyFacts>hehe
18:01:38  <ArtyFacts>ok, so, the big question of the moment is...
18:02:09  * yunongjoined
18:02:46  <ArtyFacts>i have a very small/simple benchmark i'm trying to do, and the performance i'm seeing from manta is not really what I'd expect...
18:03:04  <ArtyFacts>so this is likely a "what am i doing wrong here?" sort of thing
18:03:48  <ArtyFacts>echo /me/stor/kittens.txt | time mjob create -w 'msplit -n 10' ^ 'xargs -l1 -i sh -c "curl -sL {} > /dev/null"'
18:04:06  <ArtyFacts>kittens.txt is just a big list of urls from google image search
18:04:59  <rmustacc>Okay.
18:05:27  <ArtyFacts>if i run this on my piddly home connection, with "cat | xargs curl" it takes about 1.5 minutes
18:05:41  <ArtyFacts>if i run it on manta on 10 parallel mappers... it takes about 1.5 minutes
18:06:35  <ArtyFacts>(and those are numbers from late last night... re-running the test just now it takes more like 2.5 minutes)
18:06:50  <rmustacc>You mean reducers and not mappers right?
18:07:29  <ArtyFacts>no? the job is just a pipeline of 2 maps, one to split up the urls and one to curl them into /dev/null
18:08:23  <rmustacc>msplit is explicitly for reducers.
18:08:48  <ArtyFacts>hrmm, this just raises further questions!
18:08:53  <rmustacc>At least per http://apidocs.joyent.com/manta/msplit.html
18:08:59  <ArtyFacts>first, if msplit is only for reducers, why does what i'm doing work at all? ;-)
18:09:15  <nfitch>ArtyFacts: Can you give a recent job id?
18:09:28  <ArtyFacts>sure, sec
18:09:32  <nfitch>I'd like to see how that actually translated to a job.
18:10:22  <rmustacc>I'm not really sure, but my guess is that you created a single map task.
18:12:04  <ArtyFacts>69d0a9a0-a6a7-4bb6-917d-8cfd952e910f
18:12:24  <ArtyFacts>rmustacc: if there were true, it shouldn't run 10 times slower with -n 1 should it?
18:12:29  <ArtyFacts>which it does.
18:13:19  <rmustacc>ArtyFacts: I'm just guessing, I'm not really that familiar, sorry.
18:13:55  <nfitch>Here's your job phases:
18:13:55  <nfitch> "phases": [
18:13:55  <nfitch> {
18:13:55  <nfitch> "exec": "msplit -n 10",
18:13:55  <nfitch> "type": "map"
18:13:56  <nfitch> },
18:13:56  <nfitch> {
18:13:56  <nfitch> "exec": "xargs -l1 -i sh -c \"curl -sL {} > /dev/null\"",
18:13:57  <nfitch> "type": "map"
18:13:57  <nfitch> }
18:13:58  <nfitch> ]
18:14:02  <ArtyFacts>yup
18:14:17  <nfitch>You aren't specifying the number of reducers (-n 10 might do it).
18:14:37  <rmustacc>nfitch: Since I tell everyone else, I might as well tell you, don't paste blobs of text in the channel. ;)
18:14:40  <nfitch>So, yes this is a single map to a single map… actually, I'm not sure what that would do :)
18:14:54  <ArtyFacts>well, it *seems* to what i expect :-)
18:15:07  <ArtyFacts>just... much more slowly than i'd expect
18:15:08  <nfitch>rmustacc: ok, I won't do that anymore
18:15:19  <ArtyFacts>*to do
18:16:14  <ArtyFacts>ok, so here is the second part of this question:
18:17:05  <nfitch>That's because it's only running in a single compute zone. The second phase of your job should have a "count": 10 and a "type": reduce. See: http://apidocs.joyent.com/manta/jobs-reference.html#multiple-reducers-count-property
18:18:35  <ArtyFacts>hrmm
18:19:19  <ArtyFacts>ok, that is counter-intuitive to me - i would expect my mappers to be allocated with some equidistribution between zones - but i will try it
18:20:06  <ArtyFacts>ok, anyway, part b of that question (and the answer might be related)
18:20:24  <dap>ArtyFacts: All "msplit" does is partition your output randomly by line into 10 outputs, and emit those to the next phase. The intended use case is for the next phase to be reducers, and we should probably make that error out, but in your case, you probably have 10 map tasks generated.
18:20:35  <dap>partition your input*
18:20:51  <ArtyFacts>if i change the second map phase to do something more like "curl -sL {} | mput /me/stor/images/...." it adds several minutes to the overall time
18:21:03  <ArtyFacts>this is only 44 images totalling ~18MB
18:21:49  <dap>ArtyFacts: What's the input file, exactly?
18:22:01  <dap>*is wishing he'd cleaned up jobex and made it public about now*
18:22:14  <ArtyFacts>the really odd part of this is if i instead let the job just pipe onto outputs it takes several minutes on top of that!! :-)
18:22:21  <ArtyFacts>i can pastebin the input file, sec
18:22:42  * ins0mniaquit (Read error: Operation timed out)
18:22:42  <dap>Is it just a list of plaintext URLs?
18:22:53  <tjfontaine>that's my understanding yes
18:22:53  <ArtyFacts>http://pastebin.com/XsUsbg7a
18:22:55  <ArtyFacts>yup
18:23:03  <ArtyFacts>google image search results on the word "kitten"
18:24:20  <dap>I'm trying to remember what the difference is between "-l1" and "-n1"
18:24:23  <dap>For xargs
18:24:53  * ArtyFactsquit (Remote host closed the connection)
18:25:16  <tjfontaine>I htink his mlogin just got reset :)
18:25:33  <tjfontaine>also, it's cute that irssi is available in the manta platform image
18:25:52  * ArtyFactsjoined
18:26:13  <ArtyFacts>sorry about that, crappy twc connection likes to drop intermittently
18:26:41  <dap>ArtyFacts: Is it okay with you if I look at the data produced by your job?
18:27:00  <ArtyFacts>yes, please do
18:27:35  <ArtyFacts>-l breaks the input by a maximum of lines, -n breaks the input by a maximum of arguments
18:27:39  <ArtyFacts>so
18:27:49  <ArtyFacts>in this case I suspect they would have the same behavior?
18:28:12  <ArtyFacts>wait that shouldn't have been a question...
18:28:19  <ArtyFacts>in this case I believe they would have the same behavior. :-)
18:30:08  <dap>Your job definitely saw some slowness in the stack itself (i.e. not really a result of what you were doing)
18:30:50  <dap>Have you run it more than once?
18:31:32  <ArtyFacts>yup, several times last night and once or twice now today
18:33:26  <dap>Yeah, it seems kind of sluggish. I'm looking into it.
18:38:44  * ArtyFactsquit (Remote host closed the connection)
18:39:54  * ArtyFactsjoined
18:47:25  <dap>ArtyFacts: While the job you mentioned did hit some slowness with the system that I'm still debugging, it also occurs to me that your job hits lots of random servers on the internet, whose latency is likely highly variable (and not necessarily independent over time, if you're hitting some multiple times in parallel).
18:48:36  <ArtyFacts>sure, and i certainly considered that...
18:49:30  * ArtyFactsquit (Remote host closed the connection)
18:51:05  * ArtyFactsjoined
18:52:09  <ArtyFacts>terrible connection here... makes IRCing from a manta job a bit annoying
18:52:19  <ArtyFacts>i'll have to set up a bnc script job later
18:53:11  <dap>Oh, TJ wasn't kidding!
18:53:49  <ArtyFacts>about?
18:53:54  <tjfontaine>maybe LeftWing needs to do resumable mlogin sessions
18:54:02  <dap>You dropped because your mlogin job ended
18:54:40  <LeftWing>You're running an IRC client via mlogin?
18:54:43  <tjfontaine>:)
18:55:39  <ArtyFacts>dap: i've been trying to addinputs to the job to keep that from happening, but it doesn't appear to help :-)
18:55:52  <ArtyFacts>afk a min
18:56:30  * ArtyFactsquit (Quit: brb)
19:03:51  * paulfryzeljoined
19:21:55  * ArtyFactsjoined
19:22:11  <ArtyFacts>it saddens me that i can't reliably irrsi from mlogin
19:22:19  <ArtyFacts>it saddens me even more that it is for two reasons
19:22:21  <ArtyFacts>:-)
19:23:50  <nahamu>why would you run irssi in an mlogin session?
19:24:09  <ArtyFacts>why not?
19:24:27  <ArtyFacts>pound for pound it is cheaper than running it from linode
19:24:33  <nahamu>wouldn't the smallest regular JPC instance be cheaper?
19:24:42  <ArtyFacts>(but still more expensive than elsewhere, ask tj about my thoughts on all of that, heh)
19:24:59  <ArtyFacts>yes
19:25:00  <ArtyFacts>but
19:25:05  <nahamu>wait... it's cheaper to leave an mlogin session running than it is to run a linode of comparable size?!?!?!
19:25:35  * abraxasjoined
19:26:51  <ArtyFacts>well, it is a bit of an apples to oranges comparison... but for "what you get" i consider it cheaper. There is a cost to managing lower level instances. :-|
19:28:40  <nahamu>I guess part of it depends on whether you generally leave IRC running, or generally only run it from time to time...
19:29:08  <ArtyFacts>also true
19:29:37  <ArtyFacts>anyway, yah, resumable mlogin sessions would be great for a lot of reasons
19:29:55  * abraxasquit (Ping timeout: 240 seconds)
19:37:25  <nahamu>ArtyFacts: make your job launch a screen session and e.g. an ssh server.
19:38:12  <nahamu>but in general I'd think that's a bad idea because you're consuming a slot on a box where other people might also need to run jobs.
19:38:46  <nahamu>(although if there were a pool of machines for accepting mlogin sessions that aren't accessing an underlying object.......
19:40:19  <nahamu>right, you could write a job that pulls down a public key file, sets up an appropriate ssh config, and then execs sshd
19:40:33  <nahamu>it wouldn't terminate until you killed the sshd and you could log in via ssh.
19:41:12  <nahamu>(and perhaps sets up a cron to email you every hour to remind you that it's still running...)
19:42:12  <ArtyFacts>or, in other words, just build a better mlogin ;-)
19:42:17  <nahamu>and then you get mosh running in there so that it's less sucky from your sucky connection.
19:42:24  <ArtyFacts>which i might do, but first i just want to fetch and process images in reasonable time XD
19:42:54  <nahamu>ArtyFacts: how long are you running these sessions (in a perfect world)?
19:43:07  <nahamu>if mlogin leveraged something like mosh, would that be enough?
19:45:38  <ArtyFacts>in my perfect world, i'd be able to leave arbitrary jobs running indefinitely? :-)
19:46:15  <nahamu>jobs that were invoked interactively, though.
19:46:33  <ArtyFacts>in my perfect world, there wouldn't be much difference
19:47:08  <ArtyFacts>but yah, i may want to interactively set up some parts of my pipeline, and then let them just sit out there
19:47:25  <ArtyFacts>i think tj called such a practice "squatting" but i think that has inappropriate negative connotations
19:47:37  <nahamu>it is squatting if you do it currently.
19:48:05  <nahamu>But in theory they could beef up the system to change squatting into an alternate way to fire up a zone on a non-Manta server.
19:48:34  <ArtyFacts>in my perfect world, idle jobs would sleep, without cost, and there'd be a free processing tier
19:48:47  <ArtyFacts>in my perfect world, the pricing model would be more along the lines of heroku
19:48:52  <nahamu>normally you either run an ephemeral job in Manta on an object stored in Manta on the server where the object is stored. Or you run an instance in the JPC.
19:49:17  <ArtyFacts>but in the meantime, i might not mind paying to have some pipeline portion continuously up and "primed" for jobs.
19:49:20  <nahamu>If one could use the same tools to launch a zone that looks identical to a Manta zone, but on a non-manta server, then it's no longer a problem.
19:49:34  <ArtyFacts>"an instance in the jpc" is probably something i will not ever do
19:49:57  <nahamu>but you do every time you use mlogin. ;)
19:50:00  <ArtyFacts>and yes, if the manta stack was available even in some sandboxy form.... heh
19:50:04  <nahamu>you're just using a special one on a special server.
19:50:21  <ArtyFacts>yes... i will probably not use joyent offerings that do not have this specialness
19:50:33  <ArtyFacts>this specialness is the only reason i'm now a joyent customer at all :-P
19:50:55  <nahamu>So they've baited you, now they need to set the hook. ;)
19:51:00  <ArtyFacts>sure.
19:51:08  <ArtyFacts>please hook the crap out of me
19:51:53  <ArtyFacts>but don't do it with jpc, that will not work - i don't see value there over other offerings
19:52:02  <nahamu>dap / LeftWing : if someone invokes mlogin without specifying an object, you should run their session in a zone on a box that isn't storing objects.
19:52:37  <nahamu>it eliminates "squatting" issues, and lets you charge people for something they want. :)
19:53:05  <nahamu>I have no idea how hard that would be to implement...
19:53:34  <nahamu>but it seems like a logical thing to have as you could also run reduce jobs on such machines.
19:54:40  <nahamu>ArtyFacts: is it that mlogin launches so much faster than provisioning a zone in the JPC?
19:56:09  <ArtyFacts>nope, it is strictly the higher level interface
19:59:17  <nahamu>"higher level interface"?
19:59:58  <ArtyFacts>if i intend to be primarilly using joyent for the object storage and mapreduce provisioning i'm going to use the same tooling for asset deployment, reporting metrics, etc as much as possible because... why wouldn't i? If i have the option of interfacing the same stuff through "low level api A" and "higher level api B" to do the same things, I don't see a lot of motivation for "developing against both"
20:01:10  <ArtyFacts>(particularly when I feel there are better offerings for places where low-level-api-A style interface would be either a benefit or strictly necessary)
20:03:43  <dap>ArtyFacts, nahamu: We have talked about it. You know, priorities :)
20:03:56  <nahamu>dap: indeed, ACLs come first. :)
20:04:18  <ArtyFacts>lol
20:04:24  <nahamu>ArtyFacts: sounds like they'll hook you when they get the chance. :)
20:04:25  <dap>You've nailed it :)
20:04:40  <dap>Well, the "ACLs come first" bit ;)
20:04:41  <ArtyFacts>nahamu: i'm sure they will. they also have an unfair advantage.
20:04:49  <ArtyFacts>ACLs should come second
20:05:03  <ArtyFacts>a way to actually see my spend without writing my own script to do it (that i'm not sure is entirely correct, heh)....
20:05:26  <ArtyFacts>is probably the most critical problem of the system right now, IMO. heh.
20:05:26  <dap>Yes, that too.
20:05:49  <dap>Would you be okay with a script that *we* wrote that just goes over your usage logs?
20:06:00  <ArtyFacts>yes
20:06:01  <ArtyFacts>although
20:06:06  <dap>That's all our billing does anyway :)
20:06:10  <ArtyFacts>i will probably just use it as a spec to fix my own script, at this point :-P
20:06:15  <dap>Well, except, not over *your* copy of the logs.
20:06:54  <nahamu>dap: are our copies of the logs just snaplinks to Joyent's copy?
20:07:09  <nahamu>(such a damn elegant system.....)
20:07:58  * wpreul_joined
20:08:00  <dap>I don't think we keep per-user copies of the logs, but IIRC your logs are generated by partitioning the overall logs by user
20:08:25  <dap>Metering (including delivery of reports to users) is just a bunch of jobs run by the system
20:08:49  <wpreul_>I am trying to create a job manifest that takes objects from one folder, validates them, saves them in another folder... this is the manifest I have so far:
20:09:40  <wpreul_>{
20:09:40  <wpreul_> "phases": [
20:09:41  <wpreul_> {
20:09:42  <wpreul_> "exec": "mfind -t o /$MANTA_USER/stor/private/input/"
20:09:44  <wpreul_> },
20:09:46  <wpreul_> {
20:09:48  <wpreul_> "exec": "node index",
20:09:50  <wpreul_> "assets": ["/$MANTA_USER/stor/private/assets/validator.tar"],
20:09:52  <wpreul_> "init": "tar -xf /assets/$MANTA_USER/stor/private/assets/validator.tar"
20:09:54  <wpreul_> },
20:09:56  <wpreul_> {
20:09:58  <wpreul_> "exec": "mpipe /$MANTA_USER/stor/private/output/$MANTA_INPUT_FILE"
20:10:00  <wpreul_> }
20:10:02  <wpreul_> ]
20:10:04  <wpreul_>}
20:10:28  <wpreul_>this works when spread out into a longer command:
20:10:51  <wpreul_>$mfind -t o /$MANTA_USER/stor/private/input/ | mjob create -o -s /$MANTA_USER/stor/private/assets/validator.tar --init 'tar -xf /assets/$MANTA_USER/stor/private/assets/validator.tar' -m 'node index.js' -r 'mpipe /$MANTA_USER/stor/private/output/file'
20:11:29  <wpreul_>any thoughts on why the manifest is wrong, or the variable to use for mpiping unique file names?
20:12:41  <ArtyFacts>can the output filenames be random?
20:12:57  <wpreul_>yes
20:13:08  <wpreul_>thats fine, just don't want to overwrite anything that already exists in outputs
20:13:32  <tjfontaine>shouldn't you specifying your phase type?
20:14:09  <ArtyFacts>i've found myself doing a bit of stuff like "mpipe /.../\$RANDOM\$RANDOM...."
20:14:11  <wpreul_>is that the problem? if so, should probably have the job be validated and throw an error
20:14:24  <ArtyFacts>well heh
20:14:30  <ArtyFacts>your job is arguably valid, it is just 3 mappers
20:14:33  <ArtyFacts>which is not what you mean.
20:15:39  <ArtyFacts>well, might be what you mean, depending on how you tackle the filenaming question
20:15:47  <wpreul_>in the docs, http://apidocs.joyent.com/manta/job-patterns.html type is not required
20:15:52  <ArtyFacts>but yah, your job manifest does not match your commandline
20:16:09  <ArtyFacts>because your commandline specifies mappers and reducers and your jobs do not
20:16:29  <wpreul_>I see, so need to add reduce to the last task I assume?
20:16:47  <tjfontaine>yes, the last one should be "type": "reduce"
20:16:57  <tjfontaine>but I'm not sure that $MANTA_INPUT_FILE will mean much to you at that point
20:17:05  <tjfontaine>which is what ArtyFacts was trying to get at
20:17:09  <ArtyFacts>right
20:18:06  <wpreul_>is $RANDOM always a valid variable at that point?
20:18:19  <ArtyFacts>$RANDOM is always random
20:18:32  <ArtyFacts>make sure to escape it or else you will generate the number locally and give it to all your machines
20:18:40  <tjfontaine>wpreul_: do you actually want to accumulate the results from all map validations into a single file, or to overwrite the existing file?
20:19:00  <wpreul_>accumulate into a single file is fine
20:19:05  <tjfontaine>ok
20:20:22  <ArtyFacts>if you really want a good filename to output, use something like a sha sum of the content
20:20:48  <ArtyFacts>in theory, even $RANDOM$RANDOM$RANDOM$RANDOM$RANDOM$RANDOM.file will collide eventually, and probably much sooner than sha512sum :-D
20:20:50  <tjfontaine>fwiw, you don't necessarily need to mpipe to a storage object with a random id, you could just let it output and then get it from /jobs/<uuid>/outputs
20:21:05  <ArtyFacts>yes
20:21:06  <ArtyFacts>or
20:21:07  <ArtyFacts>better yet
20:21:09  <ArtyFacts>snaplink ;-)
20:22:57  <ArtyFacts>tjfontaine: speaking of sequences of mappers...
20:23:14  <ArtyFacts>tjfontaine: maybe there should be a tunning parameter for flattening map phases?
20:23:59  <ArtyFacts>*tuneing
20:24:11  <tjfontaine>ArtyFacts: I'm not sure entirely what you mean
20:25:15  <ArtyFacts>well right now i can easily (and even dynamically at runtime) adjust the breadth of my pipeline's parallelism, as in my mpipe -n10 into curl fetch
20:25:33  <ArtyFacts>but if i want to adjust the depth of my pipeline, i have to do so statically in the job definition by combining/seperating mappers
20:25:50  <ArtyFacts>*separating
20:26:36  <wpreul_>any tip on deleting all of current jobs ? :)
20:26:42  <wpreul_>I have a bunch at this point
20:26:53  <ArtyFacts>mjob list -s running | xargs mjob cancel
20:26:54  <ArtyFacts>maybe?
20:27:09  <tjfontaine>or do you mean: mrm -r /$MANTA_USER/jobs :)
20:27:12  <ArtyFacts>or do you mean all 'done' jobs/
20:27:13  <ArtyFacts>?
20:27:17  <ArtyFacts>yah, what tj said
20:27:18  <ArtyFacts>heh
20:27:20  <ArtyFacts>brb
20:27:44  <wpreul_>all done jobs
20:29:15  <wpreul_>it would be nice if I could also start a job
20:29:21  <wpreul_>like a job I want to repeat
20:29:52  <wpreul_>as it is, looks like I have to keep creating new jobs that are basically all the same
20:30:26  <ArtyFacts>just add inputs to an existing job
20:30:28  <ArtyFacts>no?
20:30:39  <tjfontaine>wpreul_: you should be able to massage the output from `mjob get <old uuid>` to kick a new one
20:31:02  <ArtyFacts>tjfontaine: ooh that would make a nice handy utility script, actually
20:31:20  <ArtyFacts>mjob recreate [id]
20:32:10  <tjfontaine>that can probably be done
20:38:56  <tjfontaine>wpreul_: there's a bit of a bug, you should be able to see the result from `mjob list -s done` and use that to delete, but for now if you: mjob list, mjob list -s running, uniq -u you'll see jobs not running
20:41:53  <nfitch>ArtyFacts: File a gh issue? https://github.com/joyent/node-manta
20:46:07  <ArtyFacts>#116
20:48:57  <nfitch>Thanks!
20:53:38  <paulfryzel>is it possible to have mput create the directory for you if it doesn't already exist on file upload?
20:55:33  <tjfontaine>paulfryzel: https://github.com/joyent/node-manta/issues/86
20:58:00  <paulfryzel>tjfontaine: ah ok, thanks a lot. i'll just mark's solution there until something along those lines makes its way into the SDK
21:28:35  * chorrellquit (Quit: My MacBook Pro has gone to sleep. ZZZzzz…)
22:09:56  * AvianFlujoined
22:19:13  * bixuquit (Remote host closed the connection)
22:29:19  <dap>ArtyFacts: We think we have a handle on why things are slow, and we're testing a solution. Sorry about it.
22:30:00  <dap>It may end up being Monday.
22:30:46  <ArtyFacts>cool
22:32:29  <dap>Thanks for letting us know… it looks like it's been a gradual degradation, and I've been distracted fixing other stuff.
22:33:11  <ArtyFacts>i'm sure i'll be bringing up some more points as well soon ;-)
22:46:36  * bixujoined
22:53:38  * lloyddequit (Remote host closed the connection)
22:54:15  * lloyddejoined
22:58:21  * lloyddequit (Ping timeout: 245 seconds)
23:12:06  * wpreul_quit (Ping timeout: 245 seconds)
23:27:10  * fennokin__joined
23:28:28  * fennokin_quit (Ping timeout: 245 seconds)
23:28:28  * fennokin__changed nick to fennokin_
23:33:16  * fennokin_quit (Quit: fennokin_)