Opvizor Blog

Deep-dive into Performance of VMware SCSI Controller and NVMe Controller

Written by Dennis | Jan 16, 2023 1:58:18 PM

When it comes to virtualization, VMware is the leader of the pack. VMware's virtualization solutions are reliable, cost-effective, and easy to set up and manage. But what makes VMware stand out from the rest of the pack is its powerful controller architecture. In this blog article, we will take a deep dive into the power of VMware's SCSI and NVMe controllers and how they can unlock the full performance potential of your virtual environment.

But here comes one question that is often asked by customers:

When I run my workloads in a VMware environment, which controller is recommended for the best performance?

The answer can be found in this article. For the performance comparison, we are going to use function and performance tests. The tests are defined in such a way that they represent real-world application cases. 

Introduction to VMware and its controllers

VMware is a leader in the virtualization space, and its powerful controller architecture plays a key role in its success. VMware's controllers are designed to improve performance and scalability while reducing complexity. The controllers are responsible for managing data flow between the host and virtual machines, and they can be configured to optimize performance in different ways.

The three main types of controllers supported by VMware are: SCSI, LSI Logic SAS, and Paravirtual. SCSI and LSI Logic SAS controllers are designed to optimize performance for traditional hard disk drives, while Paravirtual controllers are designed to optimize performance for solid-state drives. In addition, VMware also supports NVMe controllers, which are optimized for non-volatile memory express (NVMe) devices.

 

Understanding SCSI Controllers

SCSI controllers are the most widely used type of controller in VMware environments, and they are designed to improve performance and scalability. They are responsible for managing the flow of data between the host and virtual machines, and they can be configured to optimize performance in different ways.

SCSI controllers are divided into two types: LSI Logic SAS and Paravirtual. The LSI Logic SAS controller is best suited for traditional hard disk drives, while the Paravirtual controller is best suited for solid-state drives.

SECTION 2.1. LSI Logic SAS Controller

The LSI Logic SAS controller is a type of SCSI controller designed to optimize performance for traditional hard disk drives. This controller utilizes advanced features such as SCSI-3 persistent reservations, which allow multiple nodes to access data without conflicting with each other. This controller also supports command queuing, which allows multiple commands to be sent to a device at the same time, resulting in improved performance.

In addition, the LSI Logic SAS controller also supports advanced features such as hot-swapping, which allows you to add or remove hard disk drives without powering down the system. This ensures that your system stays available and running even when performing maintenance.

SECTION 2.2. Paravirtual Controller

The Paravirtual controller is a type of SCSI controller designed to optimize performance for solid-state drives. This controller utilizes advanced features such as command queuing, which allows multiple commands to be sent to a device at the same time. This allows for improved performance when dealing with large amounts of data.

In addition, the Paravirtual controller also supports advanced features such as thin provisioning, which allows you to allocate storage space on demand. This ensures that you can make the most out of your storage space and get the most out of your virtual environment.

 

Unlocking the Power of NVMe Controllers

NVMe controllers are the latest type of controller supported by VMware, and they are designed to optimize performance for non-volatile memory express (NVMe) devices. These controllers utilize advanced features such as command queuing, which allows multiple commands to be sent to a device at the same time. This allows for improved performance when dealing with large amounts of data.

In addition, NVMe controllers also support advanced features such as solid-state drive overcommitting, which allows you to allocate storage space on demand. This ensures that you can make the most out of your storage space and get the most out of your virtual environment.

The Test setup

The VMware SCSI Controller, VMware Paravirtual Controller and the NVMe Controller are three different disk controllers that can be used with a virtual machine. Each controller has its own advantages for certain workloads and use cases.

When you create a virtual disk, you can choose between three types:

  • LSI Logical SAS (LSI_SAS) - Disk 1

  • VMware Paravirtual (VMDK) - Disk 2

  • NVMe Controller (NVMe*) - Disk 3

Each has its own performance and configuration characteristics. 

In this article, we will compare different disk controllers' performance when running VMware workloads on bare-metal servers with only one or two CPU cores. We will look at both NVMe and SCSI controllers to see how each performs under these conditions.

VMware ESXi Hardware / Software:

  • AMD EPYC 7502P 32-Core Processor, 2.5 GHz
  • 128GB ECC Registered (RDIMM) DDR4 RAM
  • 1 TB Toshiba NVMe
  • VMware ESXi, 7.0.3, 19482537

Test VM virtual hardware:

  • Windows 10 Enterprise
  • 1 vCPU sockets
  • 4 vCPU cores
  • 8GB RAM
  • VMware Paravirtual
  • LSI Logical SAS
  • NVMe Controller
  • Thick-Provisioned eager-zeroed VMDK

 

Measuring the performance

To measure the performance we used the popular open-source tool from Microsoft:

https://github.com/microsoft/diskspd

There is also a very good documentation on how to test workloads using diskspd:

https://learn.microsoft.com/en-us/azure-stack/hci/manage/diskspd-overview

The long-term monitoring is of course done using the Windows integration of Opvizor.

 

The Tests

To make sure the tests are realistic we don't use any caching (-h switch) and are random (-r switch).

Furthermore, we defined certain disk access patterns that are comparable with typical server and desktop workloads.

Database Server

Workload: 8K Random; 30% Write; 8 Threads; 60s duration; 16 Outstanding IO, 4GB file
diskspd.exe -b8k -w30 -t8 -d60 -o16 -h -r -c4G -L e:\test.dat
diskspd.exe -b8k -w30 -t8 -d60 -o16 -h -r -c4G -L f:\test.dat
diskspd.exe -b8k -w30 -t8 -d60 -o16 -h -r -c4G -L g:\test.dat

E-Mail-Server

Workload: 4K Random; 60% Read; 8 Threads; 16 Outstanding IO
diskspd.exe -b4k -w40 -t8 -d60 -o16 -h -r -c4G -L e:\test.dat
diskspd.exe -b4k -w40 -t8 -d60 -o16 -h -r -c4G -L f:\test.dat
diskspd.exe -b4k -w40 -t8 -d60 -o16 -h -r -c4G -L g:\test.dat

File-Server

Workload: 64K Sequential; 90% Read; 8 Threads; 16 Outstanding IO
diskspd.exe -b64k -w10 -t8 -d60 -o16 -h -r -c4G -L e:\test.dat
diskspd.exe -b64k -w10 -t8 -d60 -o16 -h -r -c4G -L f:\test.dat
diskspd.exe -b64k -w10 -t8 -d60 -o16 -h -r -c4G -L g:\test.dat

Streaming Server

Workload: 5120K Random; 80% Read; 8 Threads; 16 Outstanding IO
diskspd.exe -b5120k -w20 -t8 -d60 -o16 -h -r -c4G -L e:\test.dat
diskspd.exe -b5120k -w20 -t8 -d60 -o16 -h -r -c4G -L f:\test.dat
diskspd.exe -b5120k -w20 -t8 -d60 -o16 -h -r -c4G -L g:\test.dat

VDI

Workload: 4K Random; 20% Read; 8 Threads; 8 Outstanding IO
diskspd.exe -b4k -w80 -t8 -d60 -o8 -h -r -c4G -L e:\test.dat
diskspd.exe -b4k -w80 -t8 -d60 -o8 -h -r -c4G -L f:\test.dat
diskspd.exe -b4k -w80 -t8 -d60 -o8 -h -r -c4G -L g:\test.dat

OLTP

Workload: 8K Random; 25% Read; 3 Threads; 32 Outstanding IO)
diskspd.exe -b8k -w75 -t3 -d60 -o32 -h -r -c4G -L e:\test.dat
diskspd.exe -b8k -w75 -t3 -d60 -o32 -h -r -c4G -L f:\test.dat
diskspd.exe -b8k -w75 -t3 -d60 -o32 -h -r -c4G -L g:\test.dat

The result

Google Sheet with all results

Performance Comparison between SCSI and NVMe Controllers

The results of the tests showed that the NVMe controller can be significantly faster than the SCSI controller but it highly depends on the workload. The NVMe controller was able to handle more I/O operations per second, as well as larger block sizes, resulting in improved performance. In addition, the NVMe controller also had better latency, which is important for applications that require fast response times.

The performance of the SCSI controller was also impressive. The SCSI controller was able to handle larger block sizes, which is important for applications that require high throughput.

Using a benchmarking tool such as diskspd and monitoring software like Opvizor, you can compare the performance of the two controllers and fine-tune the configuration for maximum performance. By taking advantage of the power of VMware's controllers, you can ensure that your virtual environment is running at its best.