You want a lot from your virtual desktops, but they ask for a lot in return. Keep CPU, memory, storage and display processing in mind as you build a virtual desktop system, or else certain resources could be your downfall.
Desktop virtualization is nothing like performing virtualization on a remote server or in the cloud. It's tempting to think of virtualization as "just another application," but that's a mistake. VM software, especially when run side-by-side with other programs in a host OS, is far more demanding than most other applications. For that reason, you need to take special care when creating a virtual desktop system.
Here are a few of the most important desktop resources and components you need to think about.
More desktop virtualization project resources
Your guide to planning a VDI project
Comparing VDI and Remote Desktop Services
How to succeed with VDI
The more memory, the better -- right? This guideline has been the case with computing for a long time, but it goes double for desktop virtualization. Unlike a server virtualization solution with a hypervisor, which has little overhead (depending on the hypervisor, of course), desktop VMs run along with whatever other applications are running in a conventional OS.
The easiest solution is just to add as much memory as possible to the virtual desktop system. These days, most desktop-grade motherboards max out at 32 GB or so, which should give you plenty of memory to work with. Any memory VMs do not actually use the host OS will use for things like I/O caching, which can help improve performance on all guest machines.
Again, the more the better, but the type of storage you choose for desktop virtualization is vital as well. My VM workhorse has three drives: a solid-state drive (SSD) for storing the host OS and the VM software itself; a physical disk for storing the disk images for the guests; and another physical disk for swap space or other storage.
If you'll have multiple guests running simultaneously, set things up so that the running VMs will, whenever possible, have their disk images stored on different physical spindles. This improves parallelism and reduces I/O contention. If you plan on devoting individual hard disks to individual VMs via pass-through, segregated disk images are the best solution, but make sure you have enough I/O channels in the system to support that many disks to begin with.
Note that SSDs come in roughly two grades these days: consumer-grade SSDs, which plug into an existing SATA controller like a regular drive, and server-grade SSDs, which come with their own direct-to-PCI interface and custom controller drivers. The latter are faster but also pricier, so they're not always worth it for a virtual desktop scenario.
These days, virtualized systems often use hardware-assisted virtualization, courtesy of instruction sets found in both Intel and AMD processors (Intel VT-x and VI-I, and AMD-V). Those processors allow a feature called Second Level Address Translation (SLAT), which reduces overhead during address mapping for virtual machines -- thus keeping performance up, especially in clusters.
Bear in mind that not every CPU out there will support SLAT functionality. If you plan on repurposing a notebook as virtual desktop, for instance, there's a chance the processor won't support SLAT. It won't be impossible to run a VM on that machine, but it won't yield the best performance. You can use programs such as CoreInfo to determine if a machine's processor supports SLAT.
Display hardware and GPUs
In most cases you can get by with relatively stock GPU and display hardware. Desktop virtualization products are sophisticated enough now to make proper use of multiple displays, so if you want a given VM to use more than one monitor, you can do that. Multiple displays are also useful if you want to use the guest OS desktop on your main display and have a host running on a secondary display, as an organizational measure.
It's still more difficult to run software in a guest OS that makes use of GPU-accelerated computational functions, such as NVIDIA's CUDA. Video driver support for CUDA would need to be added to the guest OS tools for the VM software for this to work, and although there has been some interesting research in this direction for a while now, there's no one-size-fits-all solution. Anything that requires CUDA or similar functionality (for example, Adobe Photoshop) needs to run on the host.
Something else to keep in mind is that if you want to use any host hardware with a guest OS in a VM, there must be support for it. Simply installing the driver in the guest won't do anything if the VM software doesn't know how to present the device to its guests. Generic device types, such as anything hosted over USB, should work fine, but the more cutting-edge the device type, the less likely you're going to have integration support for it.