Microsoft Azure IAAS – storage benchmarks, comparison with on-premises (part 5)

This article is part of a series of 9 where I am talking about storage benchmarks performed on Microsoft Azure and on-premises. For a better understanding please make sure you read also the other parts:

  1. Describe what does the Iometer benchmarks will consist of.
  2. Describe how I configured the Azure VM and the benchmark results for the A3 Standard VM using a single disk hosted by Standard Locally Redundant Storage (LRS).
  3. Describe how I configured the Azure VM and the benchmark results for the A3 Standard VM using multiple (eight) disks in a RAID-0 (striped) configuration hosted by Standard Locally Redundant Storage.
  4. Describe how I configured the Azure VM and the benchmark results for the DS3 Standard VM using a single Premium P10 disk hosted by Premium Locally Redundant Storage (LRS).
  5. Describe how I configured the Azure VM and the benchmark results for the DS3 Standard VM using a single Premium P30 disk hosted by Premium Locally Redundant Storage (LRS).
  6. Describe how I configured the Azure VM and the benchmark results for the DS3 Standard VM using multiple (two) disks in a RAID-0 (striped) configuration hosted by Premium P30 Locally Redundant Storage.
  7. Describe how I configured the Azure VM and the benchmark results for the DS3 Standard VM using multiple (eight) disks in a RAID-0 (striped) configuration hosted by Standard Locally Redundant Storage.
  8. Describe how I configured the Hyper-V on-premises VM and the benchmark results using local HDD and SSD.
  9. IOPS benchmark detailed conclusions (on-premises and Azure IaaS).

 

DS3 Standard VM using Premium P30 Locally Redundant Storage (LRS) – setup and benchmark results

An important aspect of Azure is that the IOPS and the throughput are limited by the VM series and size. The MS Azure Standard Tier DS series Standard_DS3 VM is limited to 128 MBps throughput, 12 800 IOPS and maximum 8 disks (premium or standard). Another very important aspect is the premium storage is divided in three types of premium disks:

  • P10: all the premium disks up to 128 GiB (Gibibytes), limited to 500 IOPS and 100 MBps throughput;
  • P20: all the premium disk higher than 128 GiB and up to 512 GiB (Gibibytes), limited to 2300 IOPS and 150 MBps throughput;
  • P30: all the premium disk higher than 512 GiB and up to 1024 GiB (Gibibytes), limited to 5000 IOPS and 200 MBps throughput;

For this type of VM and storage, the default setting of the virtual disk host caching is set to Read Only.

 

The following PowerShell commands have been used to provision an DS3 Standard VM with a single premium P30 disk hosted by Premium Locally Redundant Storage (benchmark setup).

Add-AzureAccount
$subscriptionName = '<YOUR SUBSCRIPTION NAME>'
$VMOSPremiumStorageAccountName = 'azioospremiumdisks'
$IOStressPremiumStorageAccountName = 'aziostresspremiumdisks'
$location = 'West Europe'
$adminName = '<YOUR ADMIN USER NAME>'
$adminPassword = '<YOUR PASSWORD>'
$vmName ='AZIODS3'
$imageFamily = 'Windows Server 2012 R2 Datacenter'
$vmSize ='Standard_DS3'

Select-AzureSubscription -SubscriptionName $subscriptionName -Current

New-AzureStorageAccount -StorageAccountName $VMOSPremiumStorageAccountName -Location $location -Type 'Premium_LRS'
New-AzureStorageAccount -StorageAccountName $IOStressPremiumStorageAccountName -Location $location -Type 'Premium_LRS'

$imageName = Get-AzureVMImage | where { $_.ImageFamily -eq $imageFamily } | sort PublishedDate -Descending | select -ExpandProperty ImageName -First 1
Set-AzureSubscription -SubscriptionName $subscriptionName -CurrentStorageAccount $VMOSPremiumStorageAccountName
$OSDiskPath = "https://" + $VMOSPremiumStorageAccountName + ".blob.core.windows.net/vhds/" + $vmName + "_OS.vhd"
$vm = New-AzureVMConfig -Name $vmName -ImageName $imageName -InstanceSize $vmSize -MediaLocation $OSDiskPath
Add-AzureProvisioningConfig -Windows -VM $vm -AdminUsername $adminName -Password $adminPassword
New-AzureVM -ServiceName $vmName -VMs $VM -Location $location

Set-AzureSubscription -SubscriptionName $subscriptionName -CurrentStorageAccount $IOStressPremiumStorageAccountName
#This will attach a new data disk configured to use the default host caching (read only) for Premium_LRS
Get-AzureVM $vmName -Name $vmName | Add-AzureDataDisk -CreateNew -DiskSizeInGB 1023 -DiskLabel "main" -LUN 0 | Update-AzureVM

 

The above PowerShell commands created two Premium Locally Redundant Storage accounts:

  • azioospremiumdisks – used to host the OS disk of the VM.
  • aziostresspremiumdisks – used to host the additional disk mapped to the VM.

The IOPS benchmarks are performed on the additional mapped disk. The aziostresspremiumdisks storage account will only host the additional virtual disk (F:\ volume) – so in this way the benchmark tests are not influenced by other workloads.

 

Benchmark results (DS3 Standard VM, F: volume on single P30 disk, hosted by Azure Premium Locally Redundant Storage)

The Iometer results can be downloaded from here.

 

DS3_Standard_VM_F_volume_on_single_P30_disk_hosted_by_Azure_Premium_Locally_Redundant_Storage_Throughput
In the above chart you can see the registered throughput. Very important to be taken in consideration is the fact the Azure Premium storage I/O unit size is 256 KB. Ideally the application who is consuming this type of storage should work with I/Os in size of 256 KB (or higher) -> to be able to use at maximum the Azure Premium storage. However, in general the existing server software solutions are working with lower size I/Os. The second very important fact is the Read Only host caching setting (the default settings for Premium disks). I quote from the Microsoft Azure site “Cache-hits are not limited by the allocated IOPS/Throughput of the disk. That is, when you use a data disk with ReadOnly cache setting on a DS-series VM or GS-series VM, Reads that are served from the cache are not subject to Premium Storage disk limits. Hence you could get very high throughput from a disk if the workload is predominantly Reads. Note that, cache is subject to separate IOPS / Throughput limits at VM level based on the VM size. DS-series VMs have roughly 4000 IOPS and 33 MB/sec per core for cache and local SSD IOs.”

 

OK, it gets a bit more complicated here, but let’s try to explain the benchmark results.

  • Because of the 5000 IOPS limit per P30 disk, in the benchmark we couldn’t reach writing speeds higher than 41.80 MBps at 100.39 ms average latency (10 times the throughput registered with P10 disks). In case you think “Azure Premium storage is based on SSDs, why is that slow?”, the answer is the Azure Premium storage I/O unit size is 256 KB and our benchmarks are using 8 KB I/Os. Azure Premium storage if not filling the remaining 248 KB with other 8 KB IOPS. So, if you reach the 5000 IOPS limit with 8 KB I/Os, then you get ≈40 MBps writing speed. If your IOPS have been in size of 256 KB, then you would get in theory 1250 MBps writing speed, but you will not record such speed due to the VM threshold (128 MBps throughput). Remember, in general the existing server software solutions are working with I/Os with unit size <256 KB.
  • The results obtained for the exclusive read operations benchmarks have been influenced by the caching settings. As noticed in the chart and also mentioned by the Microsoft Azure site, in this case the speed limits are actually caused by the VM threshold.
  • If we take in consideration the 20 ms recommended latency, we can see all the time the registered throughput is close to the maximum registered value, which indicates clearly the limits are mainly coming from the QOS limits applied by Azure (not caused by the physical storage).

 

DS3_Standard_VM_F_volume_on_single_P30_disk_hosted_by_Azure_Premium_Locally_Redundant_Storage_IOPS_and_average_latency_distribution
The above chart indicates the IOPS and average latency distribution across different workloads (# of Outstanding I/Os). If we take in consideration the 20 ms recommended latency, we can say:

  • for read-only operations whatever is higher than 256 outstanding I/Os (heavy workloads) will result in application performance degradation. As can be seen the average latency for read-only operations is growing (up to 62.74 ms) and gets just a bit outside the chart limits.
  • for write-only operations whatever is higher than 128 outstanding I/Os (relative heavy workloads) will result in application performance degradation. As can be seen the average latency for write-only operations is growing (up to 200 ms) and gets outside the chart limits.

Combined with the results from the previous chart we can conclude an application who is using this type of storage & VM setup will provide optimal performance if:

  • the expected throughput for write operations is ≈42 MBps;
  • the expected throughput for read operations is ≈134 MBps;

 

For this storage setup (single P30 disk hosted by Azure Premium LRS):

  • I recommend it for hosting any kind of heavy production VMs;
  • I definitely not recommend it for hosting DEV and TEST VMs (due to high costs);

 

Continue with Microsoft Azure IAAS – storage benchmarks, comparison with on-premises (part 6).

 

Leave a Reply