00:00:38  * shamaquit (Quit: (╯°□°)╯︵ɐɯɐɥs)
00:15:26  * contrahaxjoined
00:41:05  * contraha_joined
00:42:24  * contrahaxquit (Ping timeout: 276 seconds)
01:12:17  * contraha_quit (Read error: Connection reset by peer)
01:12:23  * contrahaxjoined
01:17:30  * contrahaxquit (Ping timeout: 276 seconds)
01:20:47  * contrahaxjoined
01:24:06  * phatedjoined
01:28:51  * phatedquit (Ping timeout: 250 seconds)
03:05:11  * terinjokesquit (Ping timeout: 240 seconds)
03:06:47  * milkandtangquit (Ping timeout: 244 seconds)
03:09:06  * terinjokesjoined
03:12:32  * phatedjoined
03:12:43  * milkandtangjoined
03:17:01  * phatedquit (Ping timeout: 252 seconds)
03:36:46  * phatedjoined
03:40:51  * phatedquit (Ping timeout: 240 seconds)
05:59:52  * fotoveritequit (Ping timeout: 244 seconds)
06:00:33  * contrahaxquit (Ping timeout: 240 seconds)
06:45:27  <substack>https://github.com/substack/o5m-decode
10:51:52  * thealphanerdquit (Quit: farewell for now)
10:52:23  * thealphanerdjoined
11:39:47  * ahdinosaurjoined
12:05:36  * contrahaxjoined
12:09:01  * fotoveritejoined
15:43:21  * contrahaxquit (Quit: Sleeping)
16:25:02  * contrahaxjoined
17:22:57  * phated_joined
17:42:38  * shamajoined
17:47:17  * warbrett_joined
17:50:49  * warbrett_quit (Client Quit)
17:51:01  * warbrettjoined
17:55:07  * contrahaxquit (Read error: Connection reset by peer)
17:55:31  * contrahaxjoined
18:54:59  * ralphtheninjajoined
19:42:02  * phated_quit (Remote host closed the connection)
20:00:56  <mikolalysenko>substack: nice
20:01:20  <mikolalysenko>though the way o5m does regions via relations is really weird
20:05:54  <substack>yes well
20:06:17  <substack>osmconvert can partition them
20:06:25  <substack>and then I can convert them into tiles
20:07:28  <substack>currently fighting geojson-vt for some osm-p2p things since existing tools like ideditor support vector tiles
20:08:02  <substack>but it would be nice to have a map lib that was more of a clean break from all of that legacy noise
20:09:35  <substack>I think it's also going to be important to have a system for farming out these indexing jobs among many computers
20:09:44  <substack>more for the long-term longevity of these systems
20:10:16  <substack>if somebody wants to donate resources to the indexing pool, it should be easy for them to do so
20:11:08  <substack>but systems are going to have different capabilities in terms of cpu, disk, and network capacity
20:18:56  * toddself_joined
20:19:07  * toddself_quit (Client Quit)
20:19:13  * toddself1joined
20:19:33  * toddself1changed nick to toddself_
20:21:37  <mikolalysenko>yeah
20:22:18  <mikolalysenko>there is so much stuff built on top of the current osm stack that making a big change is kind of scary
20:23:05  <substack>the main challenge is liberating the data from these crummy formats
20:23:28  <mikolalysenko>it is because the tools out there all communicate using this stuff
20:23:34  <mikolalysenko>but really the main consumer is the tile renderer
20:23:55  <mikolalysenko>if you are willing to put up the all the effort to roll a new tile renderer maybe you could change it all
20:24:00  <substack>I have some scripts ready to go to convert all of planet osm to o5m tiles, I just need a computer with enough disk space that won't overheat like my laptop in this desert sun :(
20:24:40  <substack>I think making a custom tile format from the partitioned o5m tiles should be pretty feasible
20:24:41  <mikolalysenko>once everything is chopped into o5m tiles will it be easier to do local updates?
20:24:45  <substack>yes
20:24:51  <mikolalysenko>ok, cool
20:25:05  <substack>here is the script https://github.com/substack/peermaps/blob/master/scripts/planet.sh
20:25:12  <mikolalysenko>yeah processing o5m tiles does seem easier than working with the whole osm blob
20:25:56  <substack>I think a partitioned dataset hosted on ipfs,hyperdrive,webtorrent would be so useful
20:26:02  <substack>because you could do ad-hoc extracts very easily
20:26:22  * contrahaxquit (Read error: Connection reset by peer)
20:26:23  <substack>which means that you can do incremental processing to generate tiles without having a super powerful machien
20:26:41  <substack>and you don't need to worry about API keys or rate limits if you fetch the data from a p2p network
20:26:58  * contrahaxjoined
20:27:20  <substack>and *all* the hosted tile servers on the internet are rate-limited because their architecture can't scale without $$$
20:29:08  <substack>people would experiment so much more with maps if there weren't tolls everywhere you look
20:30:05  <substack>whenever you make something with maps, you've got to worry that it might cost a lot of money if it gets popular
20:30:16  <substack>or the map service might change its API for no reason
20:30:29  <substack>and so your old stuff that was working fine just breaks
20:32:27  <mikolalysenko>yeah
20:33:01  <mikolalysenko>it should be possible to do incremental updates of the o5m tiles too, though maintaining all the relations might get messy
20:33:15  <mikolalysenko>still that might be less work than writing a brand new tile renderer or whatever
20:33:37  <mikolalysenko>also: are the o5m tiles are all cut at uniform resolution or is it adaptive?
20:33:40  <substack>o5m data isn't tile data, it's a more efficient encoding than protobuf
20:33:54  <substack>but you can partition the planet osm data using that script I wrote into o5m data
20:34:03  <substack>you could do that with pbf data too, but o5m is better
20:34:30  <substack>the o5m tiles divide when they are larger than a threshold file size
20:34:39  <substack>for example 1M
20:34:45  <substack>and that is after compression with gzip
20:35:17  <substack>and the division is based on approximately equal area instead of dividing by lat and lon linearly
20:35:36  <substack>which is dumb because in real world cases people mostly throw away coordinates past a certain maximum latitude
20:36:15  <substack>but you can just as well take the arcsin of the latitude to compute an approximately equal area grid
20:37:20  <substack>which is important for the future when people start farming on antarctica
20:37:21  <mikolalysenko>substack: I can get you a remote server to do this
20:37:31  <substack>that would be A+ amazing
20:37:34  <substack>I need about 70G
20:37:57  <substack>was going to buy this from digital ocean's new block storage, but if you have something ready to go that would be ideal
20:38:37  <mikolalysenko>nah, don't bother with it
20:38:57  <mikolalysenko>a while back I got the ipfs project to provision a server for map experiments
20:39:01  <substack>I've got the scripts ready to go to generate the o5m partitioned data
20:39:10  <mikolalysenko>and once I get settled in here we can get you on there and run it
20:39:21  <substack>sweet
20:39:32  <mikolalysenko>though probably can't do it tonight, will have to be tomorrow
20:39:46  <substack>no problem, I'm still figuring out some other parts of the puzzle
20:39:50  * phatedjoined
20:40:08  <substack>like, when are we going to need the underlying o5m data (for editing)
20:40:15  <mikolalysenko>yeah
20:40:26  <substack>and when will we just need the tile data, and what parts of the data do we need in the tiles?
20:40:33  * phated_joined
20:40:55  <mikolalysenko>conceptually we can think of everything else as different filters/materialized views of the raw osm tile data
20:41:01  <substack>does ipfs have a webrtc thing yet? I don't really care about which p2p network this thing uses, so long as I can get the data from a web browser
20:41:18  <mikolalysenko>there is something, though I don't know much about it yet
20:41:35  <mikolalysenko>https://github.com/ipfs/js-ipfs
20:42:31  <mikolalysenko>at any rate though, this problem is sort of independent of ipfs vs webtorrent vs whatever else
20:42:57  <mikolalysenko>I still haven't though through the whole conversion process from o5m data to renderable vector tiles
20:43:11  <mikolalysenko>but ideally that part should be all done in js and should be purely functional
20:43:27  <mikolalysenko>so then you can turn incremental changes in o5m data -> incremental vector updates
20:43:31  <substack>yes, and if the o5m files are capped at 1M it doesn't even need to be very performant
20:43:40  <mikolalysenko>yeah
20:43:54  <mikolalysenko>so what I am wondering now is if there is some system out there written in js that already does this
20:44:07  <substack>for what output format?
20:44:10  <substack>there is geojson-vt
20:44:21  <substack>which produces vector tiles https://github.com/mapbox/vector-tile-spec
20:44:23  * phatedquit (Ping timeout: 244 seconds)
20:44:28  <mikolalysenko>hmm
20:44:40  <substack>these are probably good enough™ although I don't like them
20:45:25  <mikolalysenko>my sentiments as well
20:45:38  <substack>do you plan on making your own renderer?
20:45:45  <substack>if so, we might as well make a better format
20:45:56  <substack>it'd probably be best to generate vector tiles plus the new thing
20:46:09  <substack>because that would integrate with legacy systems better
20:46:18  <substack>(even though vector tiles are pretty new)
20:47:21  <mikolalysenko>I'm really tempted to write one
20:47:24  <substack>do it
20:47:42  <substack>the main thing I'm unclear about is how text labels would fit into things
20:48:18  <mikolalysenko>still need to think about it a bit more and make sure that this is really justifiable
20:48:36  <mikolalysenko>like if the mapbox stuff is good enough and works then this is a big project to launch
20:48:53  <mikolalysenko>but on the other hand, I really do want to just roll my own
20:49:05  <mikolalysenko>anyway there is a bunch of commotion here and it looks like I need to go
20:49:16  <mikolalysenko>talk to you tomorrow, I'll get you that server somehow
22:46:50  * phated_quit (Remote host closed the connection)
22:47:24  * phatedjoined
22:51:29  * phatedquit (Ping timeout: 244 seconds)
23:45:28  * phatedjoined
23:50:12  * phatedquit (Ping timeout: 272 seconds)