01:18:31  * dap_quit (Quit: Leaving.)
01:20:00  * ed209quit (Remote host closed the connection)
01:20:07  * ed209joined
02:14:02  * jhendricksquit (Quit: Leaving.)
02:14:33  * jhendricksjoined
02:14:58  * jhendricksquit (Client Quit)
02:15:16  * jhendricksjoined
02:16:03  * jhendricksquit (Client Quit)
02:16:47  * jhendricksjoined
02:16:47  * jhendricksquit (Client Quit)
03:00:43  * chorrelljoined
03:04:17  * bahamatquit (Quit: Leaving.)
03:08:49  * jayschmidtquit (Quit: Leaving.)
03:18:25  * chorrellquit (Quit: Textual IRC Client: www.textualapp.com)
03:32:56  * therealkoopaquit (Remote host closed the connection)
03:33:32  * therealkoopajoined
03:37:51  * therealkoopaquit (Ping timeout: 258 seconds)
03:47:49  * jayschmidtjoined
04:01:02  * jhendricksjoined
04:06:29  * jhendricksquit (Quit: Leaving.)
04:11:14  * therealkoopajoined
04:13:14  * jhendricksjoined
04:19:03  * therealkoopaquit (Remote host closed the connection)
04:29:03  * jhendricksquit (Quit: Leaving.)
04:30:47  * jhendricksjoined
04:34:55  * jhendricksquit (Client Quit)
04:40:50  * jhendricksjoined
04:50:50  * jhendricksquit (Quit: Leaving.)
04:56:57  * jhendricksjoined
05:20:09  * therealkoopajoined
05:24:38  * therealkoopaquit (Ping timeout: 245 seconds)
05:28:29  * jhendricksquit (Quit: Leaving.)
05:30:09  * jhendricksjoined
05:42:05  * jhendricksquit (Quit: Leaving.)
05:52:01  * jhendricksjoined
06:20:55  * therealkoopajoined
06:25:30  * therealkoopaquit (Ping timeout: 244 seconds)
06:36:21  * jhendricksquit (Quit: Leaving.)
06:37:10  * mellocquit (Quit: Leaving.)
07:05:24  * bahamatjoined
07:21:39  * therealkoopajoined
07:27:40  * therealkoopaquit (Ping timeout: 250 seconds)
07:37:44  * mellocjoined
07:39:05  * trentmquit (Quit: Leaving.)
07:43:56  * mellocquit (Read error: No route to host)
07:44:20  * mellocjoined
08:06:25  * mellocquit (Quit: Leaving.)
08:16:19  * therealkoopajoined
08:20:33  * therealkoopaquit (Ping timeout: 252 seconds)
08:33:13  * mellocjoined
08:33:58  * mellocquit (Client Quit)
08:59:02  * mhicksquit (Quit: Leaving.)
09:17:02  * therealkoopajoined
09:20:30  * bahamatquit (Quit: Leaving.)
09:22:28  * therealkoopaquit (Ping timeout: 258 seconds)
10:18:46  * therealkoopajoined
10:20:00  * ed209quit (Remote host closed the connection)
10:20:07  * ed209joined
10:23:55  * therealkoopaquit (Ping timeout: 268 seconds)
11:19:31  * therealkoopajoined
11:24:22  * therealkoopaquit (Ping timeout: 258 seconds)
12:20:26  * therealkoopajoined
12:25:24  * therealkoopaquit (Ping timeout: 268 seconds)
13:21:14  * therealkoopajoined
13:25:52  * therealkoopaquit (Ping timeout: 260 seconds)
13:36:20  * therealkoopajoined
14:32:39  * chorrelljoined
14:50:10  * bahamatjoined
14:58:05  * mhicksjoined
15:00:21  * headw01joined
15:18:05  * chorrellquit (Quit: Textual IRC Client: www.textualapp.com)
15:19:23  * chorrelljoined
15:23:20  * chorrellquit (Client Quit)
15:23:42  * chorrelljoined
15:51:53  * jhendricksjoined
15:53:05  * jhendricksquit (Client Quit)
15:55:56  * jhendricksjoined
15:59:13  * chorrellquit (Quit: My MacBook has gone to sleep. ZZZzzz…)
16:00:54  * jhendricksquit (Quit: Leaving.)
16:13:13  * chorrelljoined
16:22:27  * chorrellquit (Quit: Textual IRC Client: www.textualapp.com)
16:22:32  * jhendricksjoined
16:23:47  * jhendricksquit (Client Quit)
16:44:01  * trentmjoined
17:24:37  * metsukejoined
17:24:46  * mellocjoined
17:27:06  <metsuke>I'm trying to use manta-sync to upload ~2TB of data into Joyent's cloud Manta. However, I consistently get an error: "FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory" I tried increasing the max_old_space_size to no avail. My box has 32GB of memory. Do I need more memory, or is there a way to tweak node?
17:30:11  <bahamat>metsuke: node.js internally is 32-bit and it can only allocate about 1.6GB of RAM to a single process.
17:30:54  <bahamat>metsuke: If there are any core files dumped from your manta-sync attempts, we'd like to be able to look at them to try to resolve the issue.
17:30:57  <metsuke>bahamat: so manta-sync is limited to smaller subsets of data?
17:31:06  <metsuke>yea, I have lots of core dumps =)
17:31:16  <bahamat>metsuke: It *should* be possible to upload objects of arbitrary size.
17:31:28  <bahamat>The fact that it's having an error means that there's a bug of some kind.
17:33:42  * headw01quit (Remote host closed the connection)
17:33:48  <bahamat>metsuke: Can you file a bug in the github repo and I'll ping bahams10 to take a look at it.
17:39:54  * dap_joined
17:45:00  * bahamas10joined
17:46:09  <metsuke>my node.core is 1.8G
17:53:45  * jhendricksjoined
18:05:51  <metsuke>here's the issue I submitted: https://github.com/bahamas10/node-manta-sync/issues/27
18:23:26  <bahamas10>metsuke: i'm checking it out now
18:24:04  <metsuke>awesome, thanks!
18:24:55  <bahamas10>np. my initial thought is that the simple act of storing an array with every source file and dest file name (as strings) is exhausting the memory
18:27:48  <bahamas10>metsuke: how many files are there total? I know it's 1.5T but i'm thinking the number of actual files is the problem here
18:28:02  <bahamas10>try $ find . -type f | wc -l, where . is the dir you're trying to upload
18:32:09  <metsuke>777303
18:32:38  <bahamas10>great thanks, i'll do some testing locally... i'm also pulling the core dump now as we speak to analyze
18:32:54  <metsuke>thank you!
18:35:22  <bahamas10>also, can you try running with `-v` and then perhaps `-vv` it increase the verbosity? this may shed some light on the issue
18:43:57  <metsuke>bahamas10: https://gist.github.com/metsuke0/9d1c53bd6141c1b84922c3ce9e9459e6
18:44:29  <metsuke>I get some errors in the beginning, but it doesn't seem affect the rest of it. I was able to do smaller uploads fine
18:45:40  <bahamas10>yeah i've seen those dtrace errors before, they shouldn't be affecting manta-sync
18:46:04  <bahamas10>also, what version of manta-sync are you using? `manta-sync -V` will tell you
18:46:30  <metsuke>0.4.1
18:47:07  <bahamas10>ok cool, that's the latest then
18:47:21  * jhendricksquit (Quit: Leaving.)
18:48:19  <bahamas10>so my latest theory right now is number of subdirectories... i'm wondering if i have some bad logic somewhere in `finder.js` to find all files recursively
18:48:33  <bahamas10>i'm attempting to recreate your issue by creating a large number of local files to sync
18:50:02  * jhendricksjoined
18:59:55  * trentmquit (Quit: Leaving.)
19:00:14  * trentmjoined
19:49:26  <bahamas10>so just an update: it's failing while building the source file list. manta-sync used to work by just gathering a list of files (an array of string filenames), but it has since changed (with the introduction of --reverse) to create `LocalFile` and `MantaFile` objects depending on direction of sync
19:49:46  <bahamas10>the source gathering logic for local files (non-reverse sync) is here https://github.com/bahamas10/node-manta-sync/blob/master/lib/localfile.js#L108
19:50:22  <bahamas10>I haven't successfully recreated the issue yet, but i'm working on a couple test programs to try and isolate the issue if metsuke you wouldn't mind running them
19:51:25  <bahamas10>i'm busy with other priorities atm, but will have more time tonight to isolate this issue. for the time being, what operating system and what version of node are you using? sorry if you've stated that before, I joined the IRC when you said "my node.core is 1.8G"
19:52:45  * melnaiem_liajoined
19:53:29  <bahamas10>right now, for every file found, a LocalFile and MantaFile object are created and held in an array... for a large number of files this might be the source of the memory bloat. My current theory is that, by only storing a string of the filename, and creating the LocalFile and MantaFile objects on demand when needed, this could reduce memory bloat as the GC could eliminate these objects when they fall out of
19:53:31  <bahamas10>scope
19:55:31  <metsuke>bahamas10: OS - 11.0-RELEASE-p3 node - v6.9.1
19:56:34  <metsuke>I can run anything on this server. It was set up to test the manta sync
19:56:52  * jhendricksquit (Quit: Leaving.)
20:01:30  * jhendricksjoined
20:07:15  * jhendricksquit (Quit: Leaving.)
20:20:01  * ed209quit (Remote host closed the connection)
20:20:08  * ed209joined
20:27:41  * melnaiem_liaquit (Ping timeout: 260 seconds)
20:47:47  <metsuke>bahamas10: sorry, I forgot to add that the OS is FreeBSD
21:54:38  * metsukequit (Quit: http://www.kiwiirc.com/ - A hand crafted IRC client)
21:57:52  * metsukejoined
21:58:17  * metsukequit (Client Quit)
21:58:29  * metsukejoined
22:02:03  * jhendricksjoined
22:03:13  * metsukequit (Client Quit)
22:03:49  * metsukejoined
22:12:20  * metsukequit (Quit: http://www.kiwiirc.com/ - A hand crafted IRC client)
22:15:01  * metsukejoined
22:20:15  * metsukequit (Quit: http://www.kiwiirc.com/ - A hand crafted IRC client)
22:20:45  * metsukejoined
22:27:32  * bahamatquit (Ping timeout: 258 seconds)
22:29:24  * bahamatjoined
23:19:24  * jhendricksquit (Quit: Leaving.)
23:21:17  * therealkoopaquit (Remote host closed the connection)
23:21:53  * therealkoopajoined
23:25:57  * therealkoopaquit (Ping timeout: 240 seconds)
23:34:34  * therealkoopajoined
23:46:10  * therealkoopaquit (Remote host closed the connection)
23:46:44  * therealkoopajoined
23:48:08  * trentmquit (Read error: Connection reset by peer)
23:48:34  * trentmjoined
23:49:09  * jhendricksjoined
23:50:57  * therealkoopaquit (Ping timeout: 240 seconds)