Recently, I’ve been exploring high availability iSCSI targets and using them as virtual machine storage. I have always been a bit weary of iSCSI performance over gigabit networks due to some not-so-great experiences many years ago. iSCSI technology has progressed quite a bit since then. FreeNAS has an excellent implementation of iSCSI, as well as Nexenta. I wanted to get a good grasp on how well everything was performing, so I decided to run some basic benchmarks.
The virtual machine I’m working with has Ubuntu 15.04 installed, but these commands will work on just about any linux distribution in existence. The hyperviser is running VMware ESXi 6, with this particular virtual machine stored on an iSCSI target served from a FreeNAS virtual machine running on another VMware ESXi hyperviser. The FreeNAS virtual machine has been given PCI Passthrough to a 3ware 9650SE-16ML, connected to 4x1TB Hitachi SATA Hard Drives exported as JBOD (individual drives, no RAID). FreeNAS configured a RAID10 with the four drives, which is where the iSCSI target resides. The network is gigabit, with an Adtran NetVanta 1524ST switch. I have not enabled Jumbo Frames. The theoretical maximum transfer speed on a gigabit network is about 125MB/s. Of course, with the overhead associated with TCP/IP and iSCSI encapsulation, a single link should be a little less. Lets get started.
To test WRITE speed of hard disk using the DD command:
# sync; dd if=/dev/zero of=tempfile bs=1M count=1024; sync 1024+0 records in 1024+0 records out 1073741824 bytes (1.1 GB) copied, 10.8496 s, 99.0 MB/s
So, this command writes a bunch of zeros to a file called tempfile, with a size of 1024MB. If you want to use a larger test file, you can change “1024” to a higher number. For instance, changing 1024 to 10000 would write a 10GB temp file. This command only tests the write speed. As you can see, the reported write speed was 99MB/s, not too shabby.