Yes the connectors are not super easy to solder, you need magnification and a steady hand but a 30$ stereo-microscope totally does the job here. If you did SMD before and feel ok about it you will have no issue.
Below is a trick to get the alignment right when hand soldering.
The project is now uploaded to https://hackaday.io/project/175832-minimal-raspberry-pi-cm-4-carrier
and the PCB files to https://github.com/PTS93/Minimal-RPi-CM-4-Carrier
Please keep in mind this is nothing but a quick and dirty demonstration. Especially the routing is far from good and completely optimized for CNC milling, nothing you wanna copy
What makes these hard is that they are very long. To make this easier align it on one end and tack down just one single pin, you can still rotate it quite a bit. Go to the other side, align, tack down a pin and you are good to go with just drag soldering along the whole thing.