If it's the permissions thing as mentioned, there is a fix in beta 2 for a bug in beta 1 which potentially caused many duplicate permission combination entries to be created, which would specifically explain a much longer permission rebuild time. The B2 upgrade script should be removing the duplicate permission combinations from users and the permission rebuilder should be removing any unused permission combinations.
I wouldn't expect the permission rebuild speed in B2 to be significantly different from the speed in 1.x for a given number of permission combinations.