EDIT: This was very strange, this line was in config.php file:
// FIX IP ADDRESS FOR PROXY
$_SERVER['REMOTE_ADDR'] = $_SERVER['108.162.230.161'];
After I removed it everything was working smoothly, that was scary...
BTW, what the specified line do?
Problem solved, also HTTPS seems to work perfectly after my VPS provider installed a trusted certificate.