00:21:18  * chorrellquit (Quit: My Mac has gone to sleep. ZZZzzz…)
00:22:07  * chorrelljoined
01:06:42  * chorrellquit (Quit: Textual IRC Client: www.textualapp.com)
01:20:00  * ed209quit (Remote host closed the connection)
01:20:07  * ed209joined
04:44:08  * cxe1_quit (Quit: This computer has gone to sleep)
04:49:36  * echelog-1quit (Ping timeout: 240 seconds)
04:49:54  * cxe1_joined
04:50:35  * echelog-1joined
05:00:03  * cxe1quit (Quit: This computer has gone to sleep)
08:14:00  * bahamas10quit (Ping timeout: 256 seconds)
08:16:04  * bahamas10joined
08:26:27  * cxe1_quit (Quit: This computer has gone to sleep)
08:52:16  * bixuquit
08:56:24  * happy-dudequit (Quit: Connection closed for inactivity)
09:05:11  * bixujoined
10:20:00  * ed209quit (Remote host closed the connection)
10:20:07  * ed209joined
12:17:40  * xmerlin_joined
13:32:44  * therealkoopajoined
13:36:33  * cxe1_joined
13:42:38  * therealkoopaquit (Remote host closed the connection)
13:52:55  * chorrelljoined
14:35:08  * xmerlin__joined
14:35:15  * xmerlin_quit (Ping timeout: 250 seconds)
14:39:27  * xmerlin_joined
14:41:29  * xmerlin__quit (Ping timeout: 246 seconds)
14:51:35  * chorrellquit (Quit: Textual IRC Client: www.textualapp.com)
15:00:14  * chorrelljoined
15:22:28  * pmooneyjoined
15:35:03  * happy-dudejoined
15:49:12  * ryancnelsonjoined
15:58:07  * fredkjoined
16:04:33  <cxe1_>for two CN manta deployment which services supose to run where. Any working examples of the file?
16:37:37  * eVikingjoined
17:02:17  * chorrellquit (Quit: My Mac has gone to sleep. ZZZzzz…)
17:02:34  * eViking_joined
17:06:09  * eVikingquit (Ping timeout: 276 seconds)
17:09:40  * ryancnelsonquit (Quit: Leaving.)
17:29:45  * chorrelljoined
18:15:34  * eViking_quit (Remote host closed the connection)
18:36:24  * eVikingjoined
18:46:53  * eVikingquit (Remote host closed the connection)
18:52:57  * ryancnelsonjoined
19:01:49  * dap_joined
19:28:53  * pmooneyquit (Ping timeout: 248 seconds)
19:29:50  * eVikingjoined
20:20:01  * ed209quit (Remote host closed the connection)
20:20:08  * ed209joined
20:23:16  * chorrellquit (Quit: Textual IRC Client: www.textualapp.com)
20:33:44  * pmooneyjoined
20:48:23  <cxe1_>manta-marlin take 5 hours to download and I have a T1 . Is there another place to get it from?
20:50:45  <rmustacc>A standard copper T1?
20:55:56  <_Tenchi_>quaint
21:03:59  <cxe1_>how equivalent of T1
21:04:18  <cxe1_>Anybody saw that : Stderr from zfs receive: cannot receive: invalid stream (bad magic number)
21:04:18  <cxe1_>Uncaught Error: write EPIPE
21:04:18  <cxe1_>FROM
21:04:22  <cxe1_>when installing manta
21:04:33  <cxe1_>day number 10 of trying to install it .
21:07:05  <rmustacc>That generally means that some part of a downloaded file was corrupted.
21:07:30  <cxe1_><rmustacc> I checked SHA and size and compared it with manifest
21:07:32  <rmustacc>Though there could be other errors that occurred that lead to that happening.
21:07:44  <rmustacc>So, what was the exact series of steps you took?
21:08:11  <cxe1_>this is last step of manta-deploy
21:09:10  <cxe1_>I downloaded file.bz2 for manta-marlin bb9264e2-f134-11e3-9ec7-478da02d1a13 and imported file with sdc-imgadm
21:10:01  <rmustacc>How did you download it? The more precise you can be about the series of commands you run, the easier it will be to help.
21:10:25  <cxe1_>I downloaded it from https://updates.joyent.com/images/bb9264e2-f134-11e3-9ec7-478da02d1a13/file
21:11:09  <rmustacc>Using some tool and placed it somewhere?
21:11:13  <cxe1_>I used adminui and CLI (several times) to import the file
21:11:20  <cxe1_>from CLI I used
21:12:10  <cxe1_>sdc-imgadm add-file bb9264e2-f134-11e3-9ec7-478da02d1a13 -f file.bz2
21:12:38  <cxe1_>got same error exactly
21:13:32  <cxe1_>I tried it from the UI with finding image bb9264e2-f134-11e3-9ec7-478da02d1a13 and clicking on "add file"
21:15:34  <cxe1_>sdc-imgadm get bb9264e2-f134-11e3-9ec7-478da02d1a13 is showing right "sha1" and right size
21:15:43  <cxe1_>I guess this is all
21:20:50  * klyntonjoined
21:22:12  <cxe1_><rmustacc> can it be related : svc:/smartdc/application/ufds-master:ufds-1393 (UFDS Master Service)
21:22:12  <cxe1_> Zone: baf1ed6d-f76e-4796-8bd2-f7e998657b40
21:22:12  <cxe1_> Alias: ufds0
21:22:12  <cxe1_> State: maintenance since April 20, 2015 07:51:08 PM UTC
21:22:12  <cxe1_>Reason: Restarting too quickly.
21:22:13  <cxe1_> See: http://illumos.org/msg/SMF-8000-L5
21:22:15  <cxe1_> See: /zones/baf1ed6d-f76e-4796-8bd2-f7e998657b40/root/var/svc/log/smartdc-application-ufds-master:ufds-1393.log
21:22:18  <cxe1_>Impact: This service is not running.
21:22:20  <cxe1_>?
21:22:48  <rmustacc>Is there a reason that you're using add-file as opposed to import?
21:23:19  <cxe1_><rmustacc> always falling. take 3-4 hours to download it
21:23:38  <cxe1_>the only way is to download it elsewhere over night and import it
21:23:53  <rmustacc>But sdc-imgadm import takes the files on the command line.
21:24:13  <rmustacc>eg. you would do sdc-imgadm import -m manifest -f file.
21:24:24  <cxe1_>yes. and create a job which is just timing out over 3-4 hours
21:24:50  <ryancnelson>that's not true.
21:25:06  <cxe1_>package manifest is getting created by manta-deploy-coal
21:25:13  <cxe1_>(it is Coal)
21:25:30  <cxe1_>which part is not true?
21:26:00  <rmustacc>There are two different ways one can invoke imgadm import which is where the confusion is here between you and ryancnelson
21:26:34  <cxe1_>I already have image with manifest with file. Image is not activated. You can activate it once file is there
21:27:02  <cxe1_>File is 13Gb and take hours on end to download it from updates.joyent.com
21:27:54  <cxe1_>sorry. Mean to say "I already have image with manifest without the file"
21:28:53  <cxe1_>getting this error when trying to run manta-adm update -y /var/tmp/lab-config.json
21:29:57  <rmustacc>So, here's what we'll do. First, we'll go diagnose the UFDS error, then after that, give what I'm suggesting with imgadm import a shot, but note that it's different from what you previously had.
21:30:04  <rmustacc>And if that fails, then it's in a state I don't understand.
21:30:15  <cxe1_>UFDS resolved by restarting the zone
21:30:33  <rmustacc>Well, one can always clear the service, but that does not give us the chance to root cause and make sure it's a known issue.
21:30:37  <rmustacc>Oh well.
21:31:19  <cxe1_>"sdc-imgadm import -m manifest -f file"
21:31:27  <cxe1_>shall I delete and use this?
21:31:52  <ryancnelson>"manifest" and "file" should be the actual filenames of the file, and the manifest file
21:32:43  <cxe1_>of course
21:32:57  <cxe1_>ran sdc-imgadm delete bb9264e2-f134-11e3-9ec7-478da02d1a13
21:33:43  <cxe1_>running "sdc-imgadm import -m /tmp/manifest.json -f file /var/tmp/image.bz2"
21:33:59  <cxe1_>took manifest from "https://updates.joyent.com/images/bb9264e2-f134-11e3-9ec7-478da02d1a13"
21:34:10  <cxe1_>and file from https://updates.joyent.com/images/bb9264e2-f134-11e3-9ec7-478da02d1a13/file
21:34:13  <ryancnelson>how much disk space do you have?
21:34:34  <ryancnelson>never mind. that's not the issue
21:34:58  <cxe1_>checked the space. I can run NASA DC from this machine
21:35:20  <ryancnelson>i thought you were using coal?
21:35:39  <cxe1_>Yes. I modified the machine to act exactly like VM image
21:35:56  <cxe1_>same IP . Just change config to use bnx0 interfaces.
21:36:03  <cxe1_>Coal just a name for now
21:36:38  <cxe1_>I mean I used coal config and modified it. I have no CN nodes for now
21:37:08  <cxe1_>Just to make sure. Real CoaL is failing with EXACTLY same error
21:37:34  <cxe1_>so I thought I am short on space, so I took a bigger machine. Same error
21:38:41  <rmustacc>Okay and that error was again that it was an invalid ZFS checksum?
21:39:18  <cxe1_>{"name":"imgadm","req_id":"2cb0e200-e7a4-11e4-a325-fdfa605c185c","hostname":"headnode","pid":4734,"level":30,"uuid":"bb9264e2-f134-11e3-9ec7-478da02d1a13","downFile":"/var/tmp/.imgadm-downloads/bb9264e2-f134-11e3-9ec7-478da02d1a13.file","msg":"using pre-downloaded image file","time":"2015-04-20T21:29:26.765Z","v":0}
21:39:18  <cxe1_>{"name":"imgadm","req_id":"2cb0e200-e7a4-11e4-a325-fdfa605c185c","hostname":"headnode","pid":4734,"level":30,"uuid":"bb9264e2-f134-11e3-9ec7-478da02d1a13","downloadInfo":{"path":"/var/tmp/.imgadm-downloads/bb9264e2-f134-11e3-9ec7-478da02d1a13.file"},"msg":"downloaded image","time":"2015-04-20T21:29:26.768Z","v":0}
21:39:18  <cxe1_>{"name":"imgadm","req_id":"2cb0e200-e7a4-11e4-a325-fdfa605c185c","hostname":"headnode","pid":4734,"level":20,"msg":"done downloads","time":"2015-04-20T21:29:26.769Z","v":0}
21:39:20  <cxe1_>Stderr from zfs receive: cannot receive: invalid stream (bad magic number)
21:39:22  <cxe1_>Uncaught Error: write EPIPE
21:39:31  <cxe1_>Sorry for the long output
21:40:11  <cxe1_>Added file "/var/tmp/image.bz2" (compression "gzip") to image bb9264e2-f134-11e3-9ec7-478da02d1a13
21:40:11  <cxe1_>Activated image bb9264e2-f134-11e3-9ec7-478da02d1a13
21:40:18  <cxe1_>All kosher
21:44:47  <cxe1_>trying to deploy
21:45:05  * eVikingquit (Remote host closed the connection)
21:52:39  * eVikingjoined
21:53:08  <cxe1_><rmustacc> where do I send 6 packs?
21:53:42  <rmustacc>No need.
21:53:59  <cxe1_><rmustacc> would be my undivided pleasure.
21:55:23  <cxe1_>how hard would it be to introduce two CN and move manta from the HN?
21:57:59  <cxe1_><rmustacc> It worked!!!!
21:58:45  <cxe1_>Kill me I don;t know what is the difference. Why import with sdc-imgadm import -m /tmp/manifest.json -f file /var/tmp/image.bz2 works and adding file after the fact doesn't
22:04:58  <rmustacc>I don't know off hand, you could follow up on list, but the add-file flow is not one I'm particularly familiar with. I suspect there's some missing piece of information or something other expectation that's not being taken into account.
22:05:20  <rmustacc>eg. it may rely on the manifest to take compression into account, etc.
22:10:18  <cxe1_><rmustacc> what is my easiest way to test it? install node.js ?
22:11:06  <rmustacc>To test manta?
22:12:18  <cxe1_>yes
22:13:52  <rmustacc>I would certainly start with the sdk taht way.
22:13:53  <rmustacc>*that
22:15:53  <cxe1_>is my MANTA_URL ip of the webapi?
22:16:05  <cxe1_>Cannot find instructions for CoaL
22:20:04  <rmustacc>Not the webapis, no. Usually it's the load balancer services.
22:22:02  <cxe1_>not cloudAPI?
22:24:21  * bahamas10quit (Ping timeout: 248 seconds)
22:25:58  <rmustacc>No, it's not cloudapi.
22:26:02  <rmustacc>Manta uses a different front door.
22:26:06  <cxe1_>gotch
22:26:10  <cxe1_>found the LB
22:28:21  <cxe1_>what was the remedy for this "mls: Error: self signed certificate"
22:28:31  <cxe1_>some environmental variable
22:45:09  <rmustacc>cxe1_: Did you read the manual pages for the software?
22:45:33  <rmustacc>It says to either use -i or set MANTA_TLS_INSECURE
22:57:28  * bahamatjoined
23:05:24  * eViking_joined
23:09:28  * eVikingquit (Ping timeout: 272 seconds)
23:38:50  <cxe1_>where one can see the error for the falling job (mjob create: error:)
23:39:28  <cxe1_><rmustacc> sorry . I ran "man mls" and only after realized there is no classic manual pages for this
23:41:13  <ryancnelson>there is documentation, though, and you need to read it. https://github.com/joyent/manta/blob/master/docs/manta-ops.md
23:49:56  <rmustacc>cxe1_: We do have classic manual pages for the tools.
23:50:15  <rmustacc>They're shipped, though I don't know how npm delivers them.
23:50:24  <cxe1_>ok. I must have not installed them
23:50:52  <rmustacc>Or npm delivers them somewhere that's not useful.
23:56:17  * eViking_quit (Remote host closed the connection)
23:57:35  * eVikingjoined