It would make my life easier as well so I'm interested to see if there is a way. So far I haven't found it.
Currently, the "Role Import" feature only supports adding roles via usernames. It is a bit of extra work, but I will run a User Report to match the email addresses with usernames and then put together the CSV for role imports. Your idea is great, and would definitely serve in improving the experience.
To help us work towards improving our Community product, I recommend submitting your idea to our Product Ideas exchange!