18:07:08  * thlorenzquit (Ping timeout: 240 seconds)
18:08:18  * dguttmanjoined
18:11:25  * kenperkinsquit (Remote host closed the connection)
18:12:13  * kenperkinsjoined
18:12:50  * thlorenzjoined
18:32:48  * jxsonquit (Remote host closed the connection)
18:35:36  * thealphanerdjoined
18:37:35  * jhizzlejoined
18:46:26  * phatedjoined
18:54:21  * thealphanerdquit (Ping timeout: 255 seconds)
18:55:40  * jhizzlequit (Remote host closed the connection)
18:57:34  * yorickquit (Ping timeout: 240 seconds)
18:58:10  * yorickjoined
19:00:20  * nfroidurejoined
19:01:37  * mikolalysenkoquit (Ping timeout: 240 seconds)
19:02:57  * thealphanerdjoined
19:04:03  <rowbit1>Hourly usage stats: [developer: 0, free: 38]
19:08:33  * jxsonjoined
19:13:49  * ednapiranhaquit (Remote host closed the connection)
19:17:42  * ednapira_joined
19:18:52  * ednapira_changed nick to ednairanha
19:18:54  * kriskowal_joined
19:18:57  * ednairanhachanged nick to ednapiranha
19:20:32  * kriskowalquit (Ping timeout: 255 seconds)
19:20:33  * kriskowal_changed nick to kriskowal
19:31:47  * addisonjjoined
19:33:38  * jxson_joined
19:36:49  * jxsonquit (Ping timeout: 240 seconds)
19:38:01  * jxson_quit (Ping timeout: 240 seconds)
19:38:55  * parsonsquit (Remote host closed the connection)
19:47:05  * jxsonjoined
19:47:54  * mikolalysenkojoined
19:49:46  * toddselfquit (Remote host closed the connection)
19:52:09  * thlorenzquit (Remote host closed the connection)
19:56:44  * kenperkins_joined
19:59:14  * kenperkinsquit (Ping timeout: 240 seconds)
19:59:50  * jxsonquit (Remote host closed the connection)
20:00:49  * airportyhquit (Quit: airportyh)
20:02:53  * jxson_joined
20:04:03  <rowbit1>Hourly usage stats: [developer: 1, free: 35]
20:08:01  * ins0mniaquit (Ping timeout: 240 seconds)
20:08:57  * jxson_quit (Remote host closed the connection)
20:11:41  * parsonsjoined
20:11:45  * parsonsquit (Remote host closed the connection)
20:12:50  * parsonsjoined
20:16:29  * phatedquit (Remote host closed the connection)
20:18:23  * dominictarrjoined
20:20:11  * toddselfjoined
20:21:55  * toddself_joined
20:23:32  * phatedjoined
20:24:14  * toddselfquit (Ping timeout: 240 seconds)
20:26:14  * toddself_quit (Ping timeout: 240 seconds)
20:27:08  * airportyhjoined
20:29:11  * yorickquit (Read error: Operation timed out)
20:32:20  * phatedquit (Remote host closed the connection)
20:35:59  * calvinfoquit (Quit: Leaving.)
20:40:39  * parsonsquit (Remote host closed the connection)
20:41:11  * peutetrequit (Quit: peutetre)
20:44:47  <jjjohnny_>ogd: should I be able to pipe ldjson-stream.serialize() to ldjson-stream.parse()?
20:45:44  <ogd>jjjohnny_: yea I think so
20:45:47  * parsonsjoined
20:46:14  <jjjohnny_>going csv | binary-csv() | ldjson.serialize | ...parse() I get a error for non-string/buffer
20:47:33  <jjjohnny_>perhaps thats a dumb way to go from csv to a single json object?
20:48:50  <jjjohnny_>seems to happen when I pipe the ldj.parse() to fs.writeStream
20:49:30  <mikolalysenko>ogd: do you think that it might be useful in dat to have a fast suffix trie implementation?
20:49:58  <mikolalysenko>I have been thinking about this a bit lately, mostly out of curiosity
20:50:26  <mikolalysenko>you can do a lot of things pretty efficiently though using suffix tries: http://en.wikipedia.org/wiki/Suffix_tree#Functionality
20:51:47  * mrmrsjoined
20:54:59  <jjjohnny_>i'll try concatn
20:55:17  * airportyhquit (Quit: airportyh)
20:56:35  * parsonsquit (Remote host closed the connection)
20:56:58  * pfallenopquit (Ping timeout: 268 seconds)
21:00:15  <jjjohnny_>csv | binary-csv | ljdjson.serialize | ldjson.parse | concat-stream(writeFile)
21:01:06  <jjjohnny_>non stream ending is probably not ideal for large data sets tho
21:01:52  * ferossjoined
21:03:22  * wolfeida_changed nick to wolfeidau
21:03:43  * calvinfojoined
21:04:03  <rowbit1>Hourly usage stats: [developer: 7, free: 42]
21:07:35  * parsonsjoined
21:09:25  <chrisdickinson>mikolalysenko: i've been poking at ukkonen's algorithm a bit over the last month or so -- especially w/r/t diffing non-string sequences
21:09:54  <mikolalysenko>chrisdickinson: ukkonen is not a good way to build suffix trees I think
21:10:01  <chrisdickinson>oh?
21:10:19  <chrisdickinson>O(kn) build time, I thought
21:10:19  <mikolalysenko>after researching it some, I found a much better/simpler approach that was published some time ago
21:10:24  <chrisdickinson>ooo
21:10:26  <mikolalysenko>you can do it in O(n)
21:10:32  <mikolalysenko>and cache efficient too
21:10:35  <chrisdickinson>well, `k` in this case is the size of the alphabet
21:10:46  <chrisdickinson>so, ukkonen's is O(n)
21:10:49  <mikolalysenko>the algorithm I found is O(n + log log(k))
21:10:59  <chrisdickinson>interesting
21:11:03  <mikolalysenko>and it is way simpler
21:11:06  <chrisdickinson>do you have a link to the paper?
21:11:09  <mikolalysenko>yes
21:11:23  <mikolalysenko>it technically builds a suffix array, not a suffix tree
21:11:33  <mikolalysenko>but you can conver it to a suffix tree trivially via a cartesian tree
21:11:41  <mikolalysenko>http://www.cs.helsinki.fi/u/tpkarkka/publications/icalp03.pdf
21:11:42  <mikolalysenko>that paper
21:11:55  <mikolalysenko>tldr, ukkonen's method is probably obsolete
21:12:14  <mikolalysenko>though it has the one kind of nice property that you can incrementally insert new strings with it, but it is not a good approach for static data
21:12:51  <jjjohnny_>ogd: its the JSON.parse that causes the error when streaming to fs.writeStream
21:13:09  <jjjohnny_>ogd: this fixes that without breaking old usage https://github.com/NHQ/ldjson-stream/blob/f9c9c6385129f12fba78ecf37197022b0fc1d560/index.js
21:13:14  <mikolalysenko>this one though does O(n/B log_M/B n/B) work in the cache oblivious model
21:13:31  <mikolalysenko>whereas ukkonen's method uses pointer arithmetic and does n cache hits
21:13:58  <jjjohnny_>i guess tho I could have simply used 'split'
21:14:06  <dominictarr> http://www.wonder-tonic.com/geocitiesizer/
21:14:12  <dominictarr>^ best thing ever
21:14:18  <mikolalysenko>you can reduce the factor of k a bit if you apply something like signature sort, but it probably isn't worth it imo
21:14:59  <jjjohnny_>dominictarr: i shall investigate thy hipster speak hyperbole
21:15:53  <mikolalysenko>chrisdickinson: it also has some code too, that should be easy enough to port to js
21:16:03  <chrisdickinson>awesome!
21:16:15  * toddselfjoined
21:16:19  <mikolalysenko>it is super short, only 50 lines
21:16:50  <mikolalysenko>if you decide to go ahead and implement it, I would recommend maybe using typedarray-pool for the memory allocation steps
21:16:59  <mikolalysenko>but other than that it looks pretty easy
21:17:24  <jjjohnny_>dominictarr: fail http://www.wonder-tonic.com/geocitiesizer/content.php?theme=1&music=4&url=substack.neocities.org
21:20:44  <mikolalysenko>I also wonder if it might be possible to port their algorithm to the gpu...
21:21:01  * toddselfquit (Ping timeout: 268 seconds)
21:21:25  <mikolalysenko>though that is starting to get a bit crazy
21:21:30  * jirwinjoined
21:22:34  <mikolalysenko>yep. apparently it is already done
21:22:43  <mikolalysenko>http://dl.acm.org/citation.cfm?id=2442536
21:23:57  <mikolalysenko>haha, here is a student project that does it: http://www.cs.cmu.edu/afs/cs/user/chaominy/www/418_project/finalreport.html
21:24:36  * parsonsquit (Remote host closed the connection)
21:25:19  * jxsonjoined
21:25:31  <ogd>jjjohnny_: fs.writeStream only accepts buffers, you cant write objects to it
21:25:38  * toddselfjoined
21:25:38  * pfallenopjoined
21:26:33  * parsonsjoined
21:27:58  * calvinfoquit (Quit: Leaving.)
21:29:21  <jjjohnny_>ogd: yeah thats why ldjson-stream.parse() can't pipe directly to a writeStream. replaced with split. but it seems ilke you may want to pipe .parse() to a readstream, so....
21:29:33  * jxsonquit (Client Quit)
21:29:34  <jjjohnny_>i mean writeSteam
21:29:46  <ogd>jjjohnny_: you can pipe .stringify() to a writeStream
21:30:10  <ogd>jjjohnny_: a fs.createWriteStream that is. you can pipe .parse() to an object mode writeSTream
21:32:44  * jxsonjoined
21:33:30  <jjjohnny_>as in JSON.stringify?
21:34:27  <dominictarr>jjjohnny_, looks great!
21:35:11  <jjjohnny_>dominictarr: its didnt rly do anything with the webpage data at all!
21:35:37  <jjjohnny_>cheap lies!
21:38:47  <jjjohnny_>how do I object mode an fs.writeStream?
21:40:06  * jxsonquit (Remote host closed the connection)
21:41:34  <jjjohnny_>docs say no streams in node core are object mode
21:42:33  * jxsonjoined
21:42:46  <substack>you have to write bytes to files
21:42:49  <substack>not objects
21:43:46  <dominictarr>jjjohnny_, there isn't much stuff on that page ...
21:44:26  * contrahaxquit (Quit: Sleeping)
21:44:28  <jjjohnny_>substack: thats what im sayinh these people calling me a fool!
21:44:38  <dominictarr>jjjohnny_, http://www.wonder-tonic.com/geocitiesizer/content.php?theme=3&music=6&url=github.com/NHQ
21:44:56  * tmcwjoined
21:45:10  * toddselfquit (Remote host closed the connection)
21:46:17  * toddselfjoined
21:46:39  * yorickjoined
21:48:13  * calvinfojoined
21:48:45  * jxsonquit (Remote host closed the connection)
21:50:33  <rowbit1>substack, pkrumins: These encoders are STILL down: 50.57.174.105(dev-ie8-2)
21:50:58  <rowbit1>/!\ ATTENTION: (default-local) alec@... successfully signed up for team_2 browserling plan ($30). Cash money! /!\
21:50:58  <rowbit1>/!\ ATTENTION: (default-local) paid account successfully upgraded /!\
21:51:11  * parsonsquit (Remote host closed the connection)
21:51:41  <pkrumins>team plan
21:51:54  * parsonsjoined
21:51:54  <substack>\o/
21:52:03  <pkrumins> |
21:55:41  <jjjohnny_>\/ \
21:55:43  <jjjohnny_>DAMNITY
21:56:08  * parsonsquit (Ping timeout: 240 seconds)
22:02:20  * dominictarrquit (Ping timeout: 268 seconds)
22:02:54  * parsonsjoined
22:04:03  <rowbit1>Hourly usage stats: [developer: 11, free: 23]
22:06:27  * parsonsquit (Remote host closed the connection)
22:06:57  * parsonsjoined
22:13:47  * ednapiranhaquit (Remote host closed the connection)
22:14:21  * ednapiranhajoined
22:15:45  * jhizzlejoined
22:15:54  * mikolalysenkoquit (Ping timeout: 268 seconds)
22:16:35  * parsonsquit (Remote host closed the connection)
22:17:43  * naneauquit (Quit: Quit.)
22:18:39  * ednapiranhaquit (Ping timeout: 255 seconds)
22:21:39  * calvinfoquit (Ping timeout: 252 seconds)
22:21:49  * dominictarrjoined
22:22:05  * jhizzlequit (Remote host closed the connection)
22:23:13  * calvinfojoined
22:30:04  * phatedjoined
22:33:59  * ednapiranhajoined
22:34:20  * phatedquit (Remote host closed the connection)
22:40:02  * tmcwquit
22:40:15  * phatedjoined
22:44:50  * yorickquit (Read error: Connection reset by peer)
22:45:45  * contrahaxjoined
22:53:57  * calvinfoquit (Quit: Leaving.)
22:54:20  * naneaujoined
22:57:27  * mikolalysenkojoined
23:04:03  <rowbit1>Hourly usage stats: [developer: 5, free: 32]
23:14:03  <rowbit1>substack, pkrumins: Encoders down: 184.106.106.66 (dev-ie7-1)
23:16:33  <rowbit1>/!\ ATTENTION: (default-local) hbsmiley7@... successfully signed up for developer browserling plan ($20). Cash money! /!\
23:16:33  <rowbit1>/!\ ATTENTION: (default-local) paid account successfully upgraded /!\
23:27:48  * mrmrsquit (Remote host closed the connection)
23:28:17  * mrmrsjoined
23:28:43  * contrahaxquit (Quit: Sleeping)
23:29:43  * mrmrs_joined
23:29:49  <jesusabdullah>nexxy: halp should I feel at all sorry for Eich y/n
23:30:03  <rowbit1>substack, pkrumins: Encoders down: 184.106.106.66 (dev-ie7-1)
23:30:22  <jesusabdullah>nexxy: (I think it's good he stepped down as CEO but he seems bummed, and I always feel bad for bummed people)
23:30:57  * calvinfojoined
23:31:07  <substack>he did? o_O
23:31:53  <jesusabdullah>Yeah
23:31:57  <jesusabdullah>it's all over the news
23:32:10  <jesusabdullah>stepped down and apparently decided to take a sabbatical
23:32:25  * mrmrsquit (Ping timeout: 240 seconds)
23:41:14  * defunctzombie_zzchanged nick to defunctzombie
23:51:53  * ednapiranhaquit (Remote host closed the connection)
23:52:25  * ednapiranhajoined
23:56:49  * ednapiranhaquit (Ping timeout: 240 seconds)
23:58:36  <jesusabdullah>substack: I had someone email me about data center tycoon
23:58:52  <jesusabdullah>substack: speaking of, you know anything about building economics simulations?
23:59:53  <substack>just make it up as you go along
23:59:57  <substack>economics is made up anyways