OP here. I am only using Jellyfin only for videos, and for that, Fermata is actually a pretty good app to let you access Jellyfin via Android Auto to watch videos.
I am pretty sure Jellyfin AA works as it should (for audio only, which I did not know about) otherwise there would have been lots of posts on Reddit prior to the exodus…
Because I have no audio content in my Jellyfin server.