00:13:13  * mhernandez1joined
00:14:00  * mhernandez1quit (Remote host closed the connection)
01:12:22  * njjjoined
01:12:56  <njj>Hi all, is it possible to make the value of an leveldb entry and array? So I can push to a certain key
01:13:48  <njj>or would it be best to store some hash + key: value, then split on the hash?
01:15:09  <substack>njj: that depends on how big you expect the arrays to get
01:15:18  <substack>and what kinds of queries you'll need to perform
01:15:20  <njj>@substack could be pretty big
01:15:35  <substack>njj: in that case, I would store each item as a separate key/value pair
01:16:09  <substack>njj: how are your entries ordered?
01:16:21  <njj>I'm still trying to figure that out I suppose
01:16:29  <njj>basically what I'm doing is storing info based on geolocation
01:16:46  <njj>and I want to be able to query people with a common geolocation, without querying everything and then filtering down
01:17:16  <substack>https://en.wikipedia.org/wiki/K-nearest_neighbors_algorithm
01:17:42  <njj>hmm interesting
01:17:44  <substack>there are a few of these, like https://www.npmjs.com/package/level-nearby-stream
01:17:52  <substack>if your data isn't so terribly big it should be ok
01:18:13  <substack>or rather, https://www.npmjs.com/package/level-places
01:18:41  <njj>what I'm trying now is something like https://jsfiddle.net/2nvmxxpx/1/
01:18:51  <njj>where `gridA` could be the key
01:18:53  <substack>there are better algorithms, but you start getting into areas of active research quickly
01:19:08  <njj>the data value is simple, its just text
01:20:02  <substack>njj: what order of magnitude is your dataset?
01:20:15  <njj>@substack this last link you sent is pretty great, I think I could leverage this
01:21:14  <substack>yeah, that one should work ok if you have a dataset in the hundreds of thousands or less
01:21:22  <njj>I imagine the data set could be pretty big at some point, there are simple ways to do stuff - but I'm concerned about scalability
01:21:37  <substack>the size overhead of leveldb gets to be a bit much for super big datasets
01:22:07  <njj>I see
01:22:14  <substack>if you don't yet have much data it's ok to forego worrying very much about it right now
01:22:34  <njj>yeah I guess its more proof of concept
01:22:56  <substack>I've got an experimental, non-level file format https://github.com/substack/mddf but it needs proper balancing and some refactoring to be practical
01:26:01  <njj>@substack I think I could use the level-places, and just store the data in there versus a city name or whatever they use in the example
01:26:07  <njj>for now..
01:30:36  <njj>@substack yeah this works great man, thanks!
01:46:32  * njjquit (Remote host closed the connection)
02:34:55  * mhernandez1joined
02:53:26  * mhernandez1quit (Remote host closed the connection)
07:23:18  * bonswouarjoined
09:07:02  * chiltsquit (Ping timeout: 246 seconds)
09:07:48  * chiltsjoined
12:45:21  * mhernandez1joined
14:58:54  * jerrysvjoined
15:19:03  * bonswouarquit (Remote host closed the connection)
15:34:14  * mhernandez1quit (Remote host closed the connection)
15:40:44  * mhernandez1joined
16:59:01  * jerrysv_joined
17:01:28  * jerrysvquit (Ping timeout: 256 seconds)
17:07:43  * jerrysv_changed nick to jerrysv
17:50:49  * jerrysv_joined
17:54:32  * jerrysvquit (Ping timeout: 244 seconds)
18:00:22  * sethvincentjoined
18:43:43  * sethvincentquit (Ping timeout: 260 seconds)
19:31:36  * mhernandez1quit (Remote host closed the connection)
19:32:08  * mhernandez1joined
19:52:57  * sethvincentjoined
20:57:09  * BruNeX__joined
20:57:17  * BruNeX__quit (Client Quit)
22:32:06  * mhernandez1quit (Remote host closed the connection)
22:45:48  * jerrysv_quit (Remote host closed the connection)