I will premise this with - I'm not a server tech, just been doing this stuff for close to 30 years privately. Purely my opinion, and I hope server techs here can steer this if needed.
Is this because your system is using swap? If so, possibly why your CPU is max'ed out. I would also ask, is that just a NGINX setup? Or is there a user interface involved, like cPanel? The first one is lightweight, the second adds overhead you just don't need. There are other factors I don't know about obviously, like what addons you're using, are you using the latest MariaDB version and is it tuned to your setup, etc etc.
I have 12 sites on my server, I only have 6 cores because its cheaper to have that option than having 2 cores with 16GB RAM and the HDD I need. In all honesty, mine could easily run on 2 cores, and I have a lot more going on than your one site does. I need the RAM to get everything into memory. I shifted away from Linode for this reason, as all they offered me was package increases, paying exorbitant amounts for stuff I didn't need. Linode 16GB RAM is nearly USD$100 monthly. I found BinaryLane here in Australia which allow tailoring your memory and storage, making it super cheap. To illustrate what I was talking about, I am running at about 12GB used RAM, so its cheaper for me to pick the 6 core package on my provider because if I choose less cores and up the RAM, it costs me more than the default package for 6 cores. I actually have my storage at 100GB which brings my monthly cost to $52AUD. Mine will easily run on 2 cores, but I need the RAM and HDD, and that 2 core package with 16GB RAM and 100GB HDD ends up at $70 a month. Sometimes packages are better for price, sometimes not.
Yours should easily run on one core with 2GB RAM. Don't add swap. If caching is setup correct at CF, that takes load from your server. CF caching should be at about 60% for XF, if not, then you have gains available to you right there. If your DB is tuned and you aren't running poorly coded addons, that will remove load.
