maybe getting the access point from 100 Mbit to 1000 will fix some flakiness

Show thread

Bought a $50 tool to fix one end of the ethernet cable when it was the other end in the keystone block

For reference, my hardware is an 8700K, 1TB 970 Pro NVMe, and 64 GB DDR4 RAM. I ran these tests on Ubuntu Desktop 21.04 with Postgres 13.3, and osm2pgsql 1.4.1.

I would also like to look into optimizations for the tile serving process, which will require different Postgres settings for a very read-heavy workload.

Show thread

I also tried a full planet file import, just to see how long it would take. In 2014 at work we used 2010 server hardware to do a planet import in about 5 days. Back then the planet was about 24 GB; now it is 62 GB.

I choose a 40 GB node cache for the import and disabled autovacuum in PG. The main import and indexing took 14 hours, and an additional 6 hours for a manual "VACUUM FULL VERBOSE" afterwards.

That's pretty good! The postgres dataset in ZFS is 395 GB. For the entire planet import.

Show thread

Optimizing RAM usage will provide better import performance gains than your choice of filesystem.

I was surprised to find that ZFS compression didn't help, as I recall it used to help about 5 years ago on a server at work.

The OSM planet file is available by a torrent since Nov 2020, btw.

Show thread

I've been doing some testing of the OpenStreetMap database import process and trying to find what "tweaks" actually work to make osm2pgsql faster. After some testing with the Canada extract region (2.6G), I have a short list.

* Use SSDs, preferably NVMe drives
* Use a good node cache size (I found 2x extract size to be good)
* Use a flat node cache
* Optimize postgresql memory settings
* Set your CPU governor to performance mode
* ZFS needs tweaking to match performance of XFS and ext4

Finished my condo Ethernet wiring a week ago. Im happy with being able to shove this stuff in a crawlspace and be out of sight.

Now I have four large drywall holes to fix

I've been pronouncing it "eff-stab" in my head for years and only recently realized it's "eff-ess-tab"

Hmm, I sure have a lot of side projects I want to finish and maybe blog about.

my brain: let's rewire the condo with ethernet instead

I just love logging in. Entering passwords every hour forever is the best. Having to enter a code emailed to me? Nothing better.

Did you know? Sending JSON to API Gateway with the former character without specifying the charset in the Content-Type will result in the latter character being sent to upstream servers!

hard-coding my AMI IDs to prevent terraform from blowing away completely fine instances.

devops is so much extra work

Poor quality glue on this steel series mouse, the thumb pads have been loose for over a year. After using superglue to put them back on, they are sticking but the side buttons are weird now. :AngeryCat:

Going to recycle this junk and go back to the G5 I kept in storage. Ill keep looking for an ambidextrous mouse that is properly made.

Is the new M1 MacBook Air also fast at switching resolutions on an external 2K display? YES.

Switching refresh rate? Takes a few seconds.

(This is the base model MBA with no upgrades.)

For some reason Safari 13 will reset my local settings for devdocs.io every few days. Doesn't happen in FireFox 83. 仄

Show older
Mastodon

Server run by the main developers of the project It is not focused on any particular niche interest - everyone is welcome as long as you follow our code of conduct!