Okay, I have started to play around with trivial-gamekit (#commonlisp game framework). Sounds fun, but I have already run into an issue (https://github.com/borodust/trivial-gamekit/issues/30).
We will see where it goes.