The data in the database must be assumed to be legitimate and thus trusted. If an attacker can modify your database to perform an object injection, then you have very big problems in general
Generally, what you say about XF1 isn't entirely true. We did use it in some situations as a fail safe check, but it should not be relied on. Further, both XF1 and XF2 have locations where objects are serialized into the DB and thus full unserialization is needed.
So essentially, there isn't really anything unsafe given that the data here is trusted.
On a side note though, we do recommend that you use JSON for new columns where appropriate. (You can change existing ones, but you need to convert all of the data.)