Advice Request Is a NVMe SSD better for running virtual machines?

[blackice]

Thinking more about it for real use the first thing I noticed was the space and cable mess savings of an m.2 drive (not that SSDs take a lot of space). The form factor is almost more helpful, so maybe even a SATA m.2 fits that benefit.

 

[Vasudev]

It really depends on what SATA and what NVMe SSD you had. NVME SSDs can reach blazing speeds beyond 2000MB/s which is faster than RAM on computers before NVMe SSDs. However you can also just make a cheap NVMe drive (many exist and are sold as gaming drives — large capacity lower speed) and they wouldn’t be remarkable. Same with SATA, some of the SATA SSDs are excellent but remember too that the first few SSDs on SATA were not that remarkable and couldn’t even go over 200-300MB/s consistently and some 10k RPM enterprise hard drives can beat them today.

I don't know where you got that info that NVMe is faster than RAM. RAM is usually 10-20x faster than any IO. Even DDR3 has 18GB/s bandwidth in dual channel mode and with bit of RAM OC you can push to 20GB/s.
@shmu26 Yeah Nvme can immensely speedup IO tasks. Other than that no difference. Do note NVMe throttle to below SATA Speed once 70-85C is reached so less temps mean more consistent performance.

 

[shmu26]

Yeah Nvme can immensely speedup IO tasks. Other than that no difference.

So my question is what makes a virtual machine feel less responsive. Is it because of IO?
I am not talking about running heavy tasks in the VM, just typical light use.

 

[ForgottenSeer 85179]

Do note NVMe throttle to below SATA Speed once 70-85C is reached so less temps mean more consistent performance.

My NVMe is between 30 - 40°C

 

[mazskolnieces]

So my question is what makes a virtual machine feel less responsive. Is it because of IO?
I am not talking about running heavy tasks in the VM, just typical light use.

The main reason is that any disk drive runs slower than RAM. Now I know that's a pat answer and basically relegates the reason to the obvious. However, the disk drive itself is not usually the real bottleneck. Buses and drivers, and a host of other things, can be the culprit to slow VMs. The other more common problem is that a particular OS build just doesn't work well with the Virtualization software that you're using. For example, Linux distro X build Y is sluggish whereas the next build, build Y works noticeably faster. Heck, even Oracle and VMWare can't figure out the causes of slow VMs.

As far as VBox this guide on how to tweak a VM for speed this guide rings true:

The Complete Guide to Speeding Up Your Virtual Machines (howtogeek.com)

 

[sepik]

"If a CPU that only has 16 PCIe lanes is configured with a graphics card and an M.2 SSD, it will reduce the number of lanes being used by the graphics card to eight, to free up four lanes for the SSD."

 

[Local Host]

"If a CPU that only has 16 PCIe lanes is configured with a graphics card and an M.2 SSD, it will reduce the number of lanes being used by the graphics card to eight, to free up four lanes for the SSD."

With no noticeable performance impact, since PCIe 3.0 at x8 is faster than PCIe 2.0 at 16x, which is more than enough for GPUs still (no need to even mention PCIe 4.0).

Plus this only affects Intel CPUs, since AMD CPUs have dedicated lanes for periphericals (like M.2).

So my question is what makes a virtual machine feel less responsive. Is it because of IO?
I am not talking about running heavy tasks in the VM, just typical light use.

Speeding up the processes of a virtual machine will always be an ongoing endeavor, try not to waste too much money on that.

 

[sepik]

@Local Host
True, that's why some prefers AMD CPUs over Intel ones. Back in time (486 era) there were (mostly in AMI BIOS feature) to setup AGP/PCI bus latency times. Especially when using old soundblaster soundcards. I managed to get, about 3-4 more FPS by reducing bus based latency times. Those were the times...and a challenge with my friends: Who can free most of the MS-DOS 640k(free memory blocks) after loading mouse, keyboard, etc drivers. No Shadow RAM used. Managed to get 634k but lost to Quarterdeck QEMM memory manager, which win the competition with 640k. Damit.

 

[MacDefender]

I don't know where you got that info that NVMe is faster than RAM. RAM is usually 10-20x faster than any IO. Even DDR3 has 18GB/s bandwidth in dual channel mode and with bit of RAM OC you can push to 20GB/s.
@shmu26 Yeah Nvme can immensely speedup IO tasks. Other than that no difference. Do note NVMe throttle to below SATA Speed once 70-85C is reached so less temps mean more consistent performance.

For newer generation devices, absolutely RAM is faster. I was more talking about mid-2000's CPU to RAM bandwidth, Memory Performance - Intel Core 2 Duo: Memory Performance Part Deux (anandtech.com)

It wasn't that long ago when CPUs can barely push 3GB/s of memory to memory performance, though yeah today RAM is much faster than NVMe and with more and more of the move towards integrated memory controllers, real world CPU to RAM performance has gotten a lot better too.


Note too that NVMe based SSDs with SR-IOV are right around the corner: Samsung PM1733 and PM1735 PCIe Gen4 SSDs | ServeTheHome SR-IOV will be a huge step up in performance, as it basically moves the virtualization onto the SSD and the SSD provides to the VM what looks like a native SSD. So there will be zero IO overhead from a VM, and the host isn't involved at all. And that one is rated for 8000MB/s read performance....

 

[mazskolnieces]

Note too that NVMe based SSDs with SR-IOV are right around the corner: Samsung PM1733 and PM1735 PCIe Gen4 SSDs | ServeTheHome SR-IOV will be a huge step up in performance, as it basically moves the virtualization onto the SSD and the SSD provides to the VM what looks like a native SSD. So there will be zero IO overhead from a VM, and the host isn't involved at all. And that one is rated for 8000MB/s read performance....

This along with other major hardware improvements are the reason I am putting off the next system upgrade for a year or two. I know Gen 4 has been here already for a while, but still need kinks ironed out and prices to come down.

The hardware improvements over the next year or two will be above what is typical as far as improvements that we get.

 

[MacDefender]

This along with other major hardware improvements are the reason I am putting off the next system upgrade for a year or two. I know Gen 4 has been here already for a while, but still need kinks ironed out and prices to come down.

The hardware improvements over the next year or two will be above what is typical as far as improvements that we get.

Yeah, can't wait for PCIe Gen4 to become more widespread. Between that and the card-based x4/x8 PCIe SSDs coming down to a good price, it's a very exciting time to be in the market for hardware.

Now, if high end GPUs would come back in stock.....

 

[mazskolnieces]

Now, if high end GPUs would come back in stock.....

I can live without discrete high end GPUs. Lucky me. But I agree with you for those that need that class of GPU.

 

[Vasudev]

For newer generation devices, absolutely RAM is faster. I was more talking about mid-2000's CPU to RAM bandwidth, Memory Performance - Intel Core 2 Duo: Memory Performance Part Deux (anandtech.com)

It wasn't that long ago when CPUs can barely push 3GB/s of memory to memory performance, though yeah today RAM is much faster than NVMe and with more and more of the move towards integrated memory controllers, real world CPU to RAM performance has gotten a lot better too.


Note too that NVMe based SSDs with SR-IOV are right around the corner: Samsung PM1733 and PM1735 PCIe Gen4 SSDs | ServeTheHome SR-IOV will be a huge step up in performance, as it basically moves the virtualization onto the SSD and the SSD provides to the VM what looks like a native SSD. So there will be zero IO overhead from a VM, and the host isn't involved at all. And that one is rated for 8000MB/s read performance....

Even then, SSD or NVMe wasn't viable tech for consumer. I had LGA 775 with HDD maxing out at 120MB/s seq. R/W(4k perf. 0.001MB/s R/W on CDM 1 or 2.x.) and RAM was still faster with 2x 2GB DDR2 800MHz dual channel which peaked at 8GB/s and migrating to DDR3L(Ivybridge) gave me improved memory speeds consuming 1.3V and faster clock freqs.
I still feel Optane had better 4k R/W speeds which matters the most. Sure the OS first install and setting up environment is faster on Samsung PM1733 but is it faster at sustained 4k w/o using dynamic turboWrite or something; In other words, pseudo-SLC cache.

So my question is what makes a virtual machine feel less responsive. Is it because of IO?
I am not talking about running heavy tasks in the VM, just typical light use.

Besides initial OS install times, speed difference becomes thin or not noticeable on either SATA/NVMe SSDs. Do note NVMe drive take longer time to POST in either AHCI/RAID and makes up for lost time by raw IO speed. I've a delay of 5-10sec even with Crucial P1.

My NVMe is between 30 - 40°C

Nice. My ambients are above 30-45C on avg so every 1C drop will improve 4k speeds.

 

[ForgottenSeer 85179]

Do note NVMe drive take longer time to POST in either AHCI/RAID and makes up for lost time by raw IO speed. I've a delay of 5-10sec even with Crucial P1.

Ah that's why my Post is slow. Thanks!

 

[shmu26]

Will NVMe help significantly with speed in compressing video files?

 

[MacDefender]

Will NVMe help significantly with speed in compressing video files?

No, probably not. Video encoding even with GPU acceleration is not IO bound, it is compute bound. Now if you spend a lot of time copying video files around that could be different.

 

[Vasudev]

Ah that's why my Post is slow. Thanks!

I didn't know you changed your username. I had to check your pixel 3a thread. Haha...
That POST is sort-of minimal POST and uses last POST state to reduce the boot time otherwise w/o you need to wait a long time.

 

[Da_iaji]

I actually have a PM1733, but its SR-IOV doesn't seem to work for me.
I have to pass it into VM as a block device, which causes great additional I / O overhead.