00:43:08  * jondelamottequit (Ping timeout: 245 seconds)
00:45:01  * eugenewarequit (Remote host closed the connection)
00:45:28  * eugenewarejoined
00:47:36  * mrevilquit (Remote host closed the connection)
00:49:49  * eugenewarequit (Ping timeout: 256 seconds)
01:05:41  * mikealquit (Quit: Leaving.)
01:19:07  * mikealjoined
01:48:18  * mreviljoined
01:48:50  * jondelamottejoined
01:54:44  * thl0joined
01:56:21  * mrevilquit (Ping timeout: 248 seconds)
01:57:50  * eugenewarejoined
02:01:43  <levelbot>[npm] level-userdb@0.1.6 <http://npm.im/level-userdb>: LevelDB-backed user database for things like web applications (@niallo)
02:04:09  * eugenewarequit (Ping timeout: 246 seconds)
02:17:33  * dominictarrjoined
02:29:46  * mikealquit (Quit: Leaving.)
03:00:00  * mikealjoined
03:03:02  * mreviljoined
03:07:34  * thl0quit (Remote host closed the connection)
03:07:49  * mrevilquit (Ping timeout: 248 seconds)
03:16:19  * julianduquequit (Quit: leaving)
03:22:57  * jondelamottequit (Ping timeout: 264 seconds)
03:48:53  * mikealquit (Ping timeout: 240 seconds)
04:14:29  * mikealjoined
04:14:54  * mreviljoined
04:18:12  * eugenewarejoined
04:28:45  * werle1quit (Quit: Leaving.)
04:53:52  * brianloveswordsquit (Excess Flood)
04:55:44  * brianloveswordsjoined
05:04:51  * mikealquit (Read error: Connection reset by peer)
05:11:33  * mikealjoined
05:33:38  * mikealquit (Quit: Leaving.)
05:42:40  * mrevilquit (Remote host closed the connection)
05:49:02  * st_lukequit (Remote host closed the connection)
06:32:14  * dominictarrquit (Quit: dominictarr)
06:54:39  * ramitosquit (Quit: Textual IRC Client: www.textualapp.com)
07:18:41  * eugenewarequit (Read error: Connection reset by peer)
07:19:06  * eugenewarejoined
07:37:26  * juliangruberquit (Ping timeout: 240 seconds)
07:38:39  * brianloveswordsquit (Ping timeout: 240 seconds)
07:43:35  * juliangruberjoined
07:45:14  * brianloveswordsjoined
07:48:16  * eugenewarequit (Ping timeout: 240 seconds)
08:20:09  * mcollinajoined
08:42:55  * mcollinaquit (Read error: Connection reset by peer)
09:02:52  * st_lukejoined
09:29:48  * No9joined
09:31:47  * nathan7quit (Quit: leaving)
09:39:33  * nathan7joined
09:49:57  * ralphtheninjaquit (Ping timeout: 264 seconds)
10:05:46  <substack>almost ready to publish my first level module, hooray!
10:07:06  * mcollinajoined
10:40:51  * missinglinkjoined
10:42:25  * mcollinaquit (Remote host closed the connection)
10:47:36  <rvagg>w00t!
11:00:48  * mcollinajoined
11:06:21  * mcollinaquit (Ping timeout: 256 seconds)
11:14:03  * mcollinajoined
11:21:49  * num9joined
11:22:04  * st_lukequit (Remote host closed the connection)
11:33:39  * Acconutjoined
11:35:10  * Acconutquit (Client Quit)
11:43:33  * mcollinaquit (Ping timeout: 248 seconds)
11:48:24  * jxsonquit (Read error: Connection reset by peer)
11:49:09  * jxsonjoined
11:53:11  * eugenewarejoined
12:05:42  * mcollinajoined
12:07:56  * julianduquejoined
12:08:31  * julianduquequit (Remote host closed the connection)
12:09:15  * julianduquejoined
12:13:57  * paulfryzeljoined
12:14:04  * paulfryzelquit (Remote host closed the connection)
12:14:40  * paulfryzeljoined
12:16:09  * werlejoined
12:23:51  * thl0joined
12:35:34  * num9quit (Ping timeout: 246 seconds)
12:37:24  * dominictarrjoined
12:44:07  * mcollinaquit (Ping timeout: 260 seconds)
12:48:47  * mcollinajoined
12:53:27  * mcollinaquit (Ping timeout: 256 seconds)
13:29:06  * dominictarrquit (Quit: dominictarr)
13:34:35  * mreviljoined
13:38:46  * mrevilquit (Ping timeout: 240 seconds)
13:47:07  * missinglinkquit (Ping timeout: 260 seconds)
13:53:38  * ednapiranhajoined
14:32:34  * ralphtheninjajoined
15:09:18  * tomerdjoined
15:37:57  * julianduquequit (Ping timeout: 264 seconds)
15:42:02  * mreviljoined
15:48:05  * paulfryz_joined
15:51:01  * paulfryzelquit (Ping timeout: 248 seconds)
15:57:17  * jondelamottejoined
15:57:32  * werlequit (Quit: Leaving.)
15:58:45  * jxsonquit (Remote host closed the connection)
15:59:08  * tomerdquit (Remote host closed the connection)
15:59:42  * tomerdjoined
15:59:58  * tomerdquit (Read error: Connection reset by peer)
16:00:26  * tomerdjoined
16:00:39  * jxsonjoined
16:04:53  * jxsonquit (Ping timeout: 240 seconds)
16:05:31  * tomerdquit (Remote host closed the connection)
16:06:04  * tomerdjoined
16:10:23  * tomerdquit (Ping timeout: 260 seconds)
16:13:34  * mrevilquit (Remote host closed the connection)
16:18:14  <hij1nx>substack: awesome what is it?
16:23:09  * paulfryz_quit (Remote host closed the connection)
16:23:47  * paulfryzeljoined
16:24:49  * paulfryzelquit (Remote host closed the connection)
16:25:23  * paulfryzeljoined
16:36:20  * mreviljoined
16:38:57  * werlejoined
16:41:41  * thl0_joined
16:42:49  * werle1joined
16:45:34  * jondelamottequit (Ping timeout: 276 seconds)
16:46:06  * thl0_quit (Remote host closed the connection)
16:50:58  * werlequit (*.net *.split)
16:50:59  * thl0quit (*.net *.split)
16:51:00  * sveisveiquit (*.net *.split)
16:57:30  * sveisveijoined
17:01:05  * ednapira_joined
17:01:32  * juliangruber_joined
17:02:07  * jondelamottejoined
17:02:18  * sveisvei_joined
17:03:08  * jondelamotte__joined
17:04:23  * juliangruberquit (Ping timeout: 246 seconds)
17:04:23  * juliangruber_changed nick to juliangruber
17:04:23  * sveisveiquit (Ping timeout: 246 seconds)
17:04:24  * ednapiranhaquit (Ping timeout: 246 seconds)
17:04:24  * brianloveswordsquit (Ping timeout: 246 seconds)
17:04:33  * sveisvei_changed nick to sveisvei
17:07:01  * jondelamottequit (Ping timeout: 271 seconds)
17:09:44  * brianloveswordsjoined
17:17:34  * jxsonjoined
17:17:56  * mikealjoined
17:51:56  * mikealquit (Quit: Leaving.)
17:52:56  * soldairjoined
17:53:07  * mikealjoined
18:02:16  * jondelamotte__quit (Ping timeout: 240 seconds)
18:04:26  * mikealquit (Quit: Leaving.)
18:04:36  * jondelamotte__joined
18:05:32  * jondelamotte__changed nick to jondelamotte
18:07:23  * mikealjoined
18:53:09  * paulfryzelquit (Remote host closed the connection)
19:01:02  * thl0joined
19:01:03  * dominictarrjoined
19:08:21  * mrevil_joined
19:11:16  * mrevilquit (Ping timeout: 240 seconds)
19:36:53  * mikealquit (Quit: Leaving.)
19:40:45  * tomerdjoined
19:42:34  * nodjoined
19:42:35  <nod>hi
19:43:20  <nod>curious - I'm wanting to use levelup. I'm wondering if there's a model lib that uses leveldb?
19:47:15  * tomerdquit (Remote host closed the connection)
19:47:48  * tomerdjoined
19:52:03  * tomerdquit (Ping timeout: 256 seconds)
19:54:06  <soldair>nod: there is scuttlebutt and level-scuttlebutt but that is probably not exactly what you are talking about.. though its fun and awesome
19:55:04  <nod>soldair: i'll take a look. I'm probably just going to throw together a simple mixin for my models that handles the serialization and storage
19:55:50  <soldair>nod: a lot depends on how and what attributes of those you need to query
19:57:04  <soldair>nod: if you just need key value lookups for objects that can be json its very very straight forward
20:02:04  <nod>soldair: ya, i think that's why i'll go the simple mixin route. thx for the pointer on scuttlebutt. i'll have to check that out
20:07:16  * mikealjoined
20:07:19  * julianduquejoined
20:11:30  * mmckeggquit (Ping timeout: 264 seconds)
20:12:06  * mmckeggjoined
20:13:50  * jondelamotte__joined
20:17:02  * jondelamottequit (Ping timeout: 240 seconds)
20:18:55  * tomerdjoined
20:20:31  * werle1quit (Quit: Leaving.)
20:20:48  * werlejoined
20:22:29  * mrevil_quit (Remote host closed the connection)
20:23:07  * mreviljoined
20:26:18  * tomerdquit (Remote host closed the connection)
20:26:53  * tomerdjoined
20:30:44  * tomerd_joined
20:30:48  * tomerdquit (Read error: No route to host)
20:35:33  * tomerd_quit (Remote host closed the connection)
20:36:06  * tomerdjoined
20:36:36  * dominictarrquit (Quit: dominictarr)
20:40:47  * tomerdquit (Ping timeout: 256 seconds)
20:42:29  * mrevilquit (Remote host closed the connection)
20:42:31  * jondelamottejoined
20:44:05  * mreviljoined
20:46:03  * jondelamotte__quit (Ping timeout: 256 seconds)
20:52:17  * tomerdjoined
21:02:59  * jondelamotte__joined
21:03:47  * tomerdquit (Remote host closed the connection)
21:04:22  * tomerdjoined
21:05:00  * dominictarrjoined
21:05:50  * jondelamottequit (Ping timeout: 240 seconds)
21:09:09  * tomerdquit (Ping timeout: 264 seconds)
21:10:02  * jondelamotte__changed nick to jondelamotte
21:11:59  * jondelamottequit (Quit: Leaving)
21:21:31  * brianloveswordsquit (Excess Flood)
21:22:08  * mikealquit (Quit: Leaving.)
21:23:15  * brianloveswordsjoined
21:25:38  * mikealjoined
21:32:04  * mikealquit (Quit: Leaving.)
21:35:44  * mikealjoined
21:39:31  * ednapira_quit (Remote host closed the connection)
22:05:27  * st_lukejoined
22:11:14  * mrevilquit (Remote host closed the connection)
22:12:14  * mreviljoined
22:20:35  * mikealquit (Quit: Leaving.)
22:20:42  * thl0quit (Remote host closed the connection)
22:25:16  * glougloujoined
22:25:28  <glouglou>hello all
22:27:40  <glouglou>i want to use a fast database to store live computer network representation, even storing events about packets going on on the network. it means adding a small record in the db every time there is a new packet on the network
22:28:06  <glouglou>this means there can be hundreds and even thousands of records every second
22:28:38  <glouglou>there will be writes (from the network probes) and reads (from the visualisation clients)
22:29:35  <glouglou>the database will run on a dedicated normal server (2Ghz / 3 GB ram)
22:29:43  <glouglou>do you think leveldb can keep up ?
22:30:02  <nod>one leveldb instance has limitations on being opened by one process
22:30:05  <glouglou>maybe i need an in-memory db ?
22:30:41  <glouglou>i plan to use leveldb-server for multi process access (https://github.com/srinikom/leveldb-server)
22:30:53  <nod>if you're storing pcap data, it's going to grow really quickly
22:31:21  <glouglou>it's not the packet itself, only the ID (reference) of the connection, and size of packet
22:32:01  <nod>you're still talking about an ingress rate of millions of packets a day on a moderately slow network
22:32:44  <glouglou>yep, i will need to have to purge old records, it's not a problem, but thx for noticing
22:32:46  <nod>maybe partition your db files based on time? create a new ldb every 24h or week or something?
22:32:55  <nod>ldb scales up really nicely
22:32:58  <nod>and writes very fast
22:33:08  <glouglou>ok
22:33:38  <glouglou>there will be even more read than writes (multiple visualisation clients at the same time)
22:34:38  <glouglou>so you don't see issues about the load (number of read / writes per second) ?
22:35:22  <nod>(sorry trying to leave for the weekend, it's quitting time here)
22:35:31  <nod>i'm not sure if you'll have problems, honestly
22:35:39  <brycebaril>glouglou: I'd suggest seeing what kind of throughput you can get with leveldb on your hardware
22:36:10  <nod>yeah. it all depends on your rate of writes. you say you'll read more than write, but i doubt that's true.
22:36:19  <nod>networks are chatty beasts.
22:37:09  <glouglou>it's opensource project so it will run anywhere ;)
22:38:01  <glouglou>note that the packet events will propagate through the db from the network probes to the visualisation clients
22:40:05  <brycebaril>glouglou: of course, but the hardware could make a huge difference here, e.g. lets say you wanted to do this with a mesh network of Raspberry Pis with XBee.
22:40:31  <glouglou>i was thinking about unqlite for in-memory database, do you know about it ?
22:41:21  <glouglou>at server exit, i could save the db to a file to provide persistence across restarts of the server, that is not critical
22:41:52  <nod>why not just use redis?
22:42:17  <brycebaril>Nope, I haven't heard of that one
22:42:57  <nod>matter of fact, pubsub might would work well for you. pub your packet details and have visualizations sub to those channels
22:42:59  <brycebaril>Well, unless he's using unix sockets that'll add to his network activity :)
22:43:06  <nod>heh good point
22:43:09  <glouglou>nod: mh i didn't know it was in-memory ... i'm checking their website right now thanks !
22:44:32  <brycebaril>glouglou: what language are you thinking for implementation?
22:44:59  <glouglou>my server is in C
22:45:33  <glouglou>i was planning to access the db via API calls or IPC
22:45:44  <brycebaril>this particular irc community is fairly geared toward node.js, but we can probably obliquely help you out, especially rvagg I'd guess
22:46:11  <brycebaril>yeah, I was looking at leveldb-server that you linked
22:46:28  <mbalho>theres is #leveldb for the db itself, this channel is generally for leveldb in node.js
22:46:35  <mbalho>(hence the double hashtag)
22:47:02  <glouglou>mh your hints there are already helping me ;)
22:47:59  <nod>Question - given a key formed like 'foo~3452323' (where the int is a unix timestamp), other than a readstream with limit of 1 and rever=true, is there a better way of getting the most recent entry w/o knowing the timestamp?
22:49:13  <brycebaril>nod: that's the way I've done it as well.
22:50:02  <nod>Raynos: thx
22:50:10  <nod>hmm that was a weird tab complete. take 2, action!
22:50:12  <nod>brycebaril: thx
22:50:18  <nod>Raynos: sorry, mistab.
23:04:46  * dominictarrquit (Quit: dominictarr)
23:04:56  * mrevilquit (Remote host closed the connection)
23:06:58  * mikealjoined
23:10:39  * werlequit (Quit: Leaving.)
23:12:48  * dominictarrjoined
23:36:30  * thl0joined
23:38:53  * jondelamottejoined
23:40:57  * jondelamottequit (Client Quit)
23:55:31  <substack>hij1nx: it uses level-search to expose a queryable interface so you can expose your leveldb data over http
23:55:54  <substack>so you can do things like curl -sg 'http://localhost:4000/?search=["location",/Oakland/]'
23:56:09  <substack>and curl -sg 'http://localhost:4000/?sort=repos'