Hello,
thank you for reporting the this issue. Yes, the speed of switching was decreased in Big Sur in comparison with previous version 10.15. It is a result of changes in system. As for hanging if click during switch - unfortunately there is no workaroung for it yet. The only way to avoid it is...