Moving from one album to another was taking too long, so I finally used my profiler to see why and decide if I should do anything about it. It turns out that significant improvements could be made. They are in the 0.99.02.014 release.
Saving from the tag editor is still too slow for my liking, but there a lot of time is spent on rescanning the modified files, and that can't be improved too much. So I'm seriously considering adding a "Fast save" option, which would not rescan the files immediately, but just mark them for automatic scanning the next time the program starts (or the user presses the "Reload" button.)