00:02:01  * dominictarrjoined
00:16:29  <mbalho>just forked that and added a cli mode
00:43:42  * timoxleyjoined
00:57:32  * thl0quit (Remote host closed the connection)
01:07:31  * ralphtheninjaquit (Quit: Lost terminal)
01:15:00  * thl0joined
01:20:06  * thl0quit (Remote host closed the connection)
02:20:32  * dominictarrquit (Quit: dominictarr)
02:22:01  * dominictarrjoined
02:25:07  * dominictarrquit (Client Quit)
02:26:34  * wolfeidauquit (Remote host closed the connection)
02:57:13  * mreviljoined
03:06:55  * thl0joined
03:13:54  * dominictarrjoined
03:39:49  * thl0quit (Remote host closed the connection)
05:00:06  * wolfeidaujoined
05:10:41  * mrevilquit (Remote host closed the connection)
05:40:50  * dominictarrquit (Quit: dominictarr)
06:21:06  * mreviljoined
06:25:45  * mrevilquit (Ping timeout: 268 seconds)
09:26:55  * rescrvquit (Ping timeout: 264 seconds)
09:28:39  * rescrvjoined
10:07:22  * st_lukejoined
10:17:49  * st_lukequit (Remote host closed the connection)
10:24:51  * st_lukejoined
10:41:04  * mcollinajoined
11:12:32  * st_lukequit (Remote host closed the connection)
11:39:50  * ralphtheninjajoined
12:24:37  * Acconutjoined
12:29:29  * ralphtheninjaquit (Quit: leaving)
12:33:42  * Acconutquit (Quit: Acconut)
12:43:35  * st_lukejoined
13:19:10  * dominictarrjoined
13:28:01  * ralphtheninjajoined
13:53:24  <mbalho>http://macwright.org/2012/11/14/indexing-searching-big-static-data.html
13:59:58  * thl0joined
14:00:59  <st_luke>this is awesome - http://bl.ocks.org/tmcw/4063830
14:04:54  * st_lukequit (Remote host closed the connection)
14:55:46  * ralphtheninjaquit (Quit: Lost terminal)
16:00:46  * Acconutjoined
16:00:50  * Acconutquit (Client Quit)
16:11:45  * ramitosjoined
16:20:14  * werlejoined
16:24:35  * werlequit (Ping timeout: 246 seconds)
16:34:43  * mcollinaquit (Read error: Connection reset by peer)
16:35:30  * thl0quit (Remote host closed the connection)
17:06:43  * brianloveswordsquit (Excess Flood)
17:07:37  * brianloveswordsjoined
17:22:16  * mcollinajoined
18:01:59  <mbalho>dominictarr: did you consider adding a stemmer to your inverted index?
18:02:24  <dominictarr>would be happy to merge such a pull request
18:02:53  <dominictarr>it also needs weighting, by for example, closeness to top of file
18:03:21  <mbalho>well if you store positions in the original document then you can derive that
18:03:44  <mbalho>if you store full words in a trie or whatever it is more data
18:03:50  <mbalho>but i think the point of stemming is that you can reduce index size
18:03:58  <mbalho>but i think you'd have to stem the search queries too?
18:04:05  <dominictarr>yes
18:04:05  <mbalho>i havent thought it through yet, was wondering if you had
18:04:21  <mbalho>is the tradeoff that you cant get autocomplete suggestions that are full words?
18:04:27  <dominictarr>so, there is a stemmer that looks good on npm
18:04:30  <dominictarr>natural
18:04:34  <dominictarr>I think is the name
18:04:45  <dominictarr>so, you just have an index that is
18:04:59  <mbalho>also https://github.com/fortnightlabs/snowball-js
18:05:08  <dominictarr>stemmed_key:rank:doc_hash -> original dos
18:05:24  <dominictarr>and then you stream the index, and retrive the docs
18:05:37  <dominictarr>this would allow you to filter it a second time
18:05:48  <dominictarr>so, you could do multiword searches like that
18:05:49  <mbalho>ah right
18:05:56  <dominictarr>basically, a join
18:06:19  <mbalho>ah so without stemming you cant, i see
18:06:53  <dominictarr>currently all I have is uppercase each word, so it's case insensitive
18:07:19  <dominictarr>also dscape told be about indexing word pairs
18:07:22  <mbalho>if you have a trie can you store it in a way that is 1D streamable for a cetain prefix??
18:07:31  <dominictarr>then you can do phrase searches
18:07:44  <dominictarr>hmm.
18:08:07  <dominictarr>well, actually, leveldb already has an optimization that doesn't restore common prefixes
18:08:33  <dominictarr>so, you might have that already
18:23:33  * thl0joined
18:56:06  * julianduquejoined
19:12:14  * Acconutjoined
19:12:22  * Acconutquit (Client Quit)
19:14:45  * timoxleyquit (Quit: Computer has gone to sleep.)
20:26:57  * ralphtheninjajoined
20:28:57  * mreviljoined
21:17:24  * thl0quit (Remote host closed the connection)
21:18:28  * thl0joined
21:24:14  * thl0quit (Remote host closed the connection)
21:41:50  * mcollinaquit (Read error: Connection reset by peer)
21:50:54  * thl0joined
21:51:05  * thl0quit (Remote host closed the connection)
21:52:34  * julianduquequit (Ping timeout: 276 seconds)
22:01:01  * julianduquejoined
22:08:35  * julianduquequit (Remote host closed the connection)
22:35:50  * mrevilquit (Remote host closed the connection)
22:43:15  * eugenewarequit (Remote host closed the connection)
23:15:18  * eugenewarejoined
23:23:55  * thl0joined
23:43:24  * ralphtheninjaquit (Ping timeout: 240 seconds)
23:46:15  * mreviljoined
23:47:13  * thl0quit (Remote host closed the connection)
23:50:42  * mrevilquit (Ping timeout: 256 seconds)