This is a problem that I face too. I have not yet figured out how to smoothly move over my terminal workflows to vterm (running ZSH in vi mode).
I even made a post here asking for suggestions but I have not found a graceful solution to it yet.
@AusatKeyboardPremi @kelvie If you use doom-Emacs, you just hit C-z to go to Emacs-state and hit it back again to go back to Evil-mode after working in a mode you think Emacs-state is more productive...