The project is now uploaded to https://hackaday.io/project/175832-minimal-raspberry-pi-cm-4-carrier
and the PCB files to https://github.com/PTS93/Minimal-RPi-CM-4-Carrier
Please keep in mind this is nothing but a quick and dirty demonstration. Especially the routing is far from good and completely optimized for CNC milling, nothing you wanna copy
Yes the connectors are not super easy to solder, you need magnification and a steady hand but a 30$ stereo-microscope totally does the job here. If you did SMD before and feel ok about it you will have no issue.
Below is a trick to get the alignment right when hand soldering.