I've used VMWare Player for quite a while now and have been happy with it, but the latest release has some issues that annoy me.
(Resumed machines don't seem to want to poll the host for the correct time, and sometimes machines open with the window stuck at the size of the VM manager window instead of maximizing)
So I decided to play around with VirtualBox. I've been pretty pleased so far.
VB has been able to open every .vdmk file I've created with VMWare Player except for the clone of my physical Windows machine. I can only assume that is due the difference in (virtual) hardware. It doesn't give any error message other than "Unable to start virtual machine." Sadly, this also kills the machine if you then try to open it in VMWare Player.

(That's what backups are for, right?)
I like the fact that VB can even create a machine in the .vmdk format so you can use the machine in both VB and VMWare. (It will also do VirtualPC and Parallels files)
I also like the fact that VB has full snapshot capability. With VMWare Player, I was limited to a single snapshot. (Suspend/resume)
As for speed, I honestly can't tell a difference whether I have the machine running in VMWare or VirtualBox. VMware Player does suspend/resume slightly quicker, but not enough that I'd consider that a strike against VB.
It's considerably easier on some Linux distros to install the guest tools in VirtualBox than on VMWare. In fact, on Ubuntu/Kubuntu, all you have to do is click on Install Guest Additions and the virtual CD is mounted and the OS asks if you want to run the auto-run content. You say yes and it does everything else. Quite a bit easier than mounting VMWare-tools, extracting the archive and manually running the installer through the command-line. There are even a few distros that I've never actually been able to make the VMWare tools support copy/paste functionality between host and guest work, while with VB tools, I can copy/paste just fine. VMWare tools also corrupted the kernel of Fedora 16, (oddly, the compiler didn't have any errors...) while VB tools did not.
VB is a bit less resource intensive with a VM running. RAM and CPU usage aren't drastically different going by the numbers, but the the host machine is more responsive with a VM running in VirtualBox than when I'm running a VM in VMWare Player.
I'm still using VMWare Player for my Windows applications testing, but I've started running my Linux machines exclusively in VirtualBox.