I am talking about a host with 8 cores. If you assign 4 cores to the VM, the performance is often worse than if you assign 2 cores to the VM. This is with VirtualBox. The reason given on the Virtualbox forum is that the guest needs to constantly calculate which core(s) to use, and that slows it down.
I can't tell a difference between 2 cores or 4 cores and most activities on a 6 core Intel high-end system (with SSD). When compiling code, 4 cores is noticeably faster.
With VMs, they are notorious for odd behaviors across hardware and platforms. They (along with their updates) are the bane of admins. VMs on AMD hardware is an even greater mess than on Intel hardware.
On both VirtualBox and VMWare support forums, they're all volunteers - even the odd Oracle or VMWare employee that just might drop in and make a comment. I've found that 90+ % of the time, on the more nebulous issues the discussions end up being bunk.
However, I'm certain your experience it is what you describe it to be. Nothing is a surprise in IT.