00:20:08  <rvagg>Trott: what do you need?
00:20:25  <rvagg>going to be taking the Pi's out for a few minutes, DON'T PANIC!
00:27:17  * node-ghjoined
00:27:17  * node-ghpart
00:27:49  * node-ghjoined
00:27:49  * node-ghpart
00:48:23  * node-ghjoined
00:48:23  * node-ghpart
00:56:38  * node-ghjoined
00:56:38  * node-ghpart
02:23:46  <Trott>rvagg: If you have a GUI login and sudo password, the following four commands:
02:24:34  <Trott>https://www.irccloud.com/pastebin/HWwBmHKw/
02:25:20  <Trott>Line 1 will (unless GUI login changes everything) take anywhere from 5 seconds to 2 minutes to run. Which is a problem. It should take about 250ms.
02:25:48  <Trott>Line 2 and 3 will disable ReportCrash, and then line 4 is line 1 all over again but hopefully doesn't take so long to run.
02:26:02  <Trott>Trying to figure out if disabling ReportCrash solves the problem we've been seeing with OS X in CI.
02:27:16  <Trott>The thing is that `launchctl unload` won't work with an SSH login. Need to be logged in via the GUI interface.
02:27:43  <Trott>And line 3 needs `sudo`. So you need to be logged in via the GUI/window interface with a login that can use sudo. Whee!
02:29:53  <Trott>Oh, and those commands should be run from the project directory in node-test-commit-osx. Shouldn't interfere with other tests running at the same time, but you know, uh, no guarantees I guess....
02:30:37  <Trott> /Users/iojs/build/workspace/node-test-commit-osx/nodes/osx1010 specifically....
02:32:02  <Trott>If you want, just run lines 2 and 3 and I'll do the rest in as non-intrusive a way as I can.
03:18:11  <Trott>rvagg: All the Raspberry Pi devices seem stuck at the same git checkout. Oldest runs are 90 minutes and still just this far:
03:18:14  <Trott>https://www.irccloud.com/pastebin/COV1ZC9y/
03:18:25  <Trott>Problem? Or expected after the switcheroo you did?
04:06:40  <Trott>2.5 hours and still nothing moving on the Raspberry Pi devices. I'll terminate the currently-running tests and hope that everything is self-correcting...here we go....
04:12:10  <Trott>No luck. Pi devices all seem to stall out on build as above. Help? /cc rvagg jbergstroem
04:47:11  <rvagg>Trott: I took away the SSD from the NFS mounts, just a normal HDD now; I didn't anticipate it'd be a _huge_ difference since the network and their own power seems to be the biggest problem for them all, I might have underestimated
04:47:35  <rvagg>Trott: the SSD is coming back, I'm just moving things around a lot, trying to decomission some of the hardware and replace with some newer things
04:51:10  <rvagg>Trott: working on the OSX stuff you've given above, will let you know
04:51:18  <Trott>thanks!
04:54:02  <rvagg>Trott: running first command, got the abort() and stacktrace fairly quickly, waiting now for whatever comes next, so first data point is that the delay is happening after the abort, if that helps
04:54:47  <Trott>Yeah, that first command can take 2 minutes sometimes. :-|
04:55:41  <rvagg>2m20s for it to finish
04:57:43  <rvagg>Trott: you hit the nail on the head, ran those two unloads and it's done in 220ms
04:58:12  <rvagg>nice sleuthing, I don't imagine that was easy to work out
04:58:21  <Trott>That is fantastic news. Now I have to look up how to make those two launchctl changes permanent. (Or maybe they already are.)
04:58:41  <rvagg>we need to document this for our our ansible work for the new MacStadium builds
04:59:35  <rvagg>I'll work on the Pi's, I might just need to expedite my network shuffle and get the SSD back in play