This is now fixed. As it turns out, there are 3 different behaviors in browsers in RTL (they're consistent in LTR). Chrome was behaving as expected, but Safari and Firefox act a different way, while IE/Edge act a totally different way as well.
I've now implemented a system that normalizes the scroll positions (scrollLeft) and allows us to interact with them fairly consistently and that resolves this issue.