Microsoft Azure IAAS – storage benchmarks, comparison with on-premises (part 4)

This article is part of a series of 9 where I am talking about storage benchmarks performed on Microsoft Azure and on-premises. For a better understanding please make sure you read also the other parts:

  1. Describe what does the Iometer benchmarks will consist of.
  2. Describe how I configured the Azure VM and the benchmark results for the A3 Standard VM using a single disk hosted by Standard Locally Redundant Storage (LRS).
  3. Describe how I configured the Azure VM and the benchmark results for the A3 Standard VM using multiple (eight) disks in a RAID-0 (striped) configuration hosted by Standard Locally Redundant Storage.
  4. Describe how I configured the Azure VM and the benchmark results for the DS3 Standard VM using a single Premium P10 disk hosted by Premium Locally Redundant Storage (LRS).
  5. Describe how I configured the Azure VM and the benchmark results for the DS3 Standard VM using a single Premium P30 disk hosted by Premium Locally Redundant Storage (LRS).
  6. Describe how I configured the Azure VM and the benchmark results for the DS3 Standard VM using multiple (two) disks in a RAID-0 (striped) configuration hosted by Premium P30 Locally Redundant Storage.
  7. Describe how I configured the Azure VM and the benchmark results for the DS3 Standard VM using multiple (eight) disks in a RAID-0 (striped) configuration hosted by Standard Locally Redundant Storage.
  8. Describe how I configured the Hyper-V on-premises VM and the benchmark results using local HDD and SSD.
  9. IOPS benchmark detailed conclusions (on-premises and Azure IaaS).

 

DS3 Standard VM using Premium P10 Locally Redundant Storage (LRS) – setup and benchmark results

An important aspect of Azure is that the IOPS and the throughput are limited by the VM series and size. The MS Azure Standard Tier DS series Standard_DS3 VM is limited to 128 MBps throughput, 12 800 IOPS and maximum 8 disks (premium or standard). Another very important aspect is the premium storage is divided in three types of premium disks:

  • P10: all the premium disks up to 128 GiB (Gibibytes), limited to 500 IOPS and 100 MBps throughput;
  • P20: all the premium disks higher than 128 GiB and up to 512 GiB (Gibibytes), limited to 2300 IOPS and 150 MBps throughput;
  • P30: all the premium disks higher than 512 GiB and up to 1024 GiB (Gibibytes), limited to 5000 IOPS and 200 MBps throughput;

For this type of VM and storage, the default setting of the virtual disk host caching is set to Read Only.

 

The following PowerShell commands have been used to provision an DS3 Standard VM with a single premium P10 disk hosted by Premium Locally Redundant Storage (benchmark setup).

Add-AzureAccount
$subscriptionName = '<YOUR SUBSCRIPTION NAME>'
$VMOSPremiumStorageAccountName = 'azioospremiumdisks'
$IOStressPremiumStorageAccountName = 'aziostresspremiumdisks'
$location = 'West Europe'
$adminName = '<YOUR ADMIN USER NAME>'
$adminPassword = '<YOUR PASSWORD>'
$vmName ='AZIODS3'
$imageFamily = 'Windows Server 2012 R2 Datacenter'
$vmSize ='Standard_DS3'

Select-AzureSubscription -SubscriptionName $subscriptionName -Current

New-AzureStorageAccount -StorageAccountName $VMOSPremiumStorageAccountName -Location $location -Type 'Premium_LRS'
New-AzureStorageAccount -StorageAccountName $IOStressPremiumStorageAccountName -Location $location -Type 'Premium_LRS'

$imageName = Get-AzureVMImage | where { $_.ImageFamily -eq $imageFamily } | sort PublishedDate -Descending | select -ExpandProperty ImageName -First 1
Set-AzureSubscription -SubscriptionName $subscriptionName -CurrentStorageAccount $VMOSPremiumStorageAccountName
$OSDiskPath = "https://" + $VMOSPremiumStorageAccountName + ".blob.core.windows.net/vhds/" + $vmName + "_OS.vhd"
$vm = New-AzureVMConfig -Name $vmName -ImageName $imageName -InstanceSize $vmSize -MediaLocation $OSDiskPath
Add-AzureProvisioningConfig -Windows -VM $vm -AdminUsername $adminName -Password $adminPassword
New-AzureVM -ServiceName $vmName -VMs $VM -Location $location

Set-AzureSubscription -SubscriptionName $subscriptionName -CurrentStorageAccount $IOStressPremiumStorageAccountName
#This will attach a new data disk configured to use the default host caching (read only) for Premium_LRS
Get-AzureVM $vmName -Name $vmName | Add-AzureDataDisk -CreateNew -DiskSizeInGB 10 -DiskLabel "main" -LUN 0 | Update-AzureVM

 

The above PowerShell commands created two Premium Locally Redundant Storage accounts:

  • azioospremiumdisks – used to host the OS disk of the VM.
  • aziostresspremiumdisks – used to host the additional disk mapped to the VM.

The IOPS benchmarks are performed on the additional mapped disk. The aziostresspremiumdisks storage account will only host the additional virtual disk (F:\ volume) – so in this way the benchmark tests are not influenced by other workloads.
Microsoft_Azure_IAAS_storage_benchmarks_comparison_with_on_premises_11
Microsoft_Azure_IAAS_storage_benchmarks_comparison_with_on_premises_12
Microsoft_Azure_IAAS_storage_benchmarks_comparison_with_on_premises_13

Benchmark results (DS3 Standard VM, F: volume on single P10 disk, hosted by Azure Premium Locally Redundant Storage)

The Iometer results can be downloaded from here.

 

DS3_Standard_VM_F_volume_on_single_P10_disk_hosted_by_Azure_Premium_Locally_Redundant_Storage_Throughput
In the above chart you can see the registered throughput. Very important to be taken in consideration is the fact the Azure Premium storage I/O unit size is 256 KB. Ideally the application who is consuming this type of storage should work with I/Os in size of 256 KB (or higher) -> to be able to use at maximum the Azure Premium storage. However, in general the existing server software solutions are working with lower size I/Os. The second very important fact is the Read Only host caching setting (the default settings for Premium disks). I quote from the Microsoft Azure site “Cache-hits are not limited by the allocated IOPS/Throughput of the disk. That is, when you use a data disk with ReadOnly cache setting on a DS-series VM or GS-series VM, Reads that are served from the cache are not subject to Premium Storage disk limits. Hence you could get very high throughput from a disk if the workload is predominantly Reads. Note that, cache is subject to separate IOPS / Throughput limits at VM level based on the VM size. DS-series VMs have roughly 4000 IOPS and 33 MB/sec per core for cache and local SSD IOs.”

 

OK, it gets a bit more complicated here, but let’s try to explain the benchmark results.

  • Because of the 500 IOPS limit per P10 disk, in the benchmark we couldn’t reach writing speeds higher than 4.18 MBps. In case you think “Azure Premium storage is based on SSDs, why is that slow?”, the answer is the Azure Premium storage I/O unit size is 256 KB and our benchmarks are using 8 KB I/Os. Azure Premium storage if not filling the remaining 248 KB with other 8 KB IOPS. So, if you reach the 500 IOPS limit with 8 KB I/Os, then you get 4000 KBps (≈4 MBps) writing speed. If your IOPS have been in size of 256 KB, then you would get 100 MBps throughput (because this is a P10 disk limited to maximum 100 MBps throughput). Remember, in general the existing server software solutions are working with I/Os with unit size <256 KB.
  • The results obtained for the exclusive read operations benchmarks have been influenced by the caching settings. As noticed in the chart and also mentioned by the Microsoft Azure site, in this case the speed limits are actually caused by the VM threshold.
  • If we take in consideration the 20 ms recommended latency, we can see all the time the registered throughput is close to the maximum registered value, which indicates clearly the limits are mainly coming from the QOS limits applied by Azure (not caused by the physical storage).

 

DS3_Standard_VM_F_volume_on_single_P10_disk_hosted_by_Azure_Premium_Locally_Redundant_Storage_IOPS_and_average_latency_distribution
The above chart indicates the IOPS and average latency distribution across different workloads (# of Outstanding I/Os). If we take in consideration the 20 ms recommended latency, we can say:

  • for read-only operations whatever is higher than 256 outstanding I/Os (heavy workloads) will result in application performance degradation. As can be seen the average latency for read-only operations is growing (up to 62.82 ms) and gets just a bit outside the chart limits.
  • for write-only operations whatever is higher than 8 outstanding I/Os (very light workloads) will result in application performance degradation. As can be seen the average latency for write-only operations is growing (up to 2 seconds) and gets way outside the chart limits.

Combined with the results from the previous chart we can conclude an application who is using this type of storage & VM setup will provide optimal performance if:

  • it is a read intensive application (needs to handle heavy read-only workloads);
  • the expected throughput for read-only operations is ≈134 MBps;
  • the expected throughput for write-only operations is low (≈4 MBps);

 

For this storage setup (single P10 disk hosted by Azure Premium LRS):

  • I recommend it for hosting:
    • search based solutions;
    • web servers (site files) for systems with very high usage;
    • MS SQL servers (database data and transaction log files) – ideally for systems focused to provide very fast access to read-only databases, archives, legacy data;
    • file shares with moderate usage (where the throughput for write operations is not high);
    • test & development VMs where speed is essential;
  • I definitely not recommend it for hosting:
    • VMs with heavy write storage usage;

 

Continue with Microsoft Azure IAAS – storage benchmarks, comparison with on-premises (part 5).

 

Leave a Reply