slow writes - nfs datastore

Similar Questions

• Poor ESXi 4 NFS Datastore Performance with various NAS systems

  Hello!

  In testing, I found that I get between a half and a quarter of the e/s inside a guest performance when ESXi 4 systems connect to the using NFS data store if the clients connect to the exact same NFS share.  However, I don't see this effect if the data store using iSCSI or local storage.  This has been reproduced with different systems running ESXi 4 and NAS systems.

  My test is very simple.  I created naked CentOS 5.4 minimum installation (completely updated 07/04/2010) with VMware Tools loaded and the creation time of a file of 256 MB using jj.  I create the file on the root (a VMDK stored in different data warehouses) partition or a directory of the NAS mounted via NFS, directly in the comments

  My crucial test configuration consists of a single test PC (Intel 3.0 GHz Core 2 Duo E8400 CPU with a single Intel 82567LM-3 Gigabit NC and 4 GB RAM) running ESXi 4 connected to a printer HP Procurve 1810 - 24 G, which is connected to a VIA EPIA M700 NAS system running OpenFiler 2.3 with two 1.5 to 7200 tr / MIN SATA disks configured in front of software RAID 1 and dual Gigabit Ethernet NIC.  However, I have reproduced it with different ESXi PC and NAS systems.

  This is a release of one of the tests.  In this case, the VMDK is a store of data stored on the NAS via NFS:

  -

  ---

  root@iridium /sync #; Sync; Sync; time {dd if = / dev/zero of = bs = 1 M count = test.txt 256 sync; sync; sync ;}}
  2560 records in
  2560 records out
  268435456 bytes (268 MB) copied, 0,524939 seconds, 511 MB/s
  Real 0m38.660s
  user 0m0.000s
  sys 0m0.566s
  root@iridium /# mount/mnt 172.28.19.16:/mnt/InternalRAID1/shares/VirtualMachines
  root@iridium /# cd/mnt
  synchronization of the # mnt root@iridium; Sync; Sync; time {dd if = / dev/zero of = bs = 1 M count = test.txt 256 sync; sync; sync ;}}
  2560 records in
  2560 records out
  268435456 bytes (268 MB) copied, 8,69747 seconds, 30.9 MB/s
  Real 0m9.060s
  user 0m0.001s
  sys 0m0.659s
  mnt root@iridium#.

  -

  ---

  The first dd is a VMDK stored in a connected via NFS data store.  The dd ends almost immediately, but the synchronization takes nearly 40 seconds!  It's less than 7 MB per second transfer rate: very slow.  Then I get the exact same NFS share that ESXi is used to store data directly in the comments and repeat the DD.  As you can see, the SD is longer and the synchronization takes no real time (as befits a NFS share with active sync), and the whole process takes less than 10 seconds: this is four times faster!

  I don't see these results on data warehouses mounted via NFS.  For example, here is a test on the guest even running from a mounted via iSCSI data store (using the exact same SIN):

  -

  ---

  root@iridium /sync #; Sync; Sync; time {dd if = / dev/zero of = bs = 1 M count = test.txt 256 sync; sync; sync ;}}
  2560 records in
  2560 records out
  268435456 bytes (268 MB) copied, 1,6913 seconds, 159 MB/s
  Real 0m7.745s
  user 0m0.000s
  sys 0m1.043s

  
  root@iridium /# mount/mnt 172.28.19.16:/mnt/InternalRAID1/shares/VirtualMachines
  root@iridium /# cd/mnt
  synchronization of the # mnt root@iridium; Sync; Sync; time {dd if = / dev/zero of = bs = 1 M count = test.txt 256 sync; sync; sync ;}}
  2560 records in
  2560 records out
  268435456 bytes (268 MB) copied, 8,66534 seconds, 31.0 MB/s
  Real 0m9.081s
  user 0m0.001s
  sys 0m0.794s
  mnt root@iridium#.

  -

  ---

  And the same comments linking internal SATA drive of the PC ESXi:

  -

  ---

  root@iridium /sync #; Sync; Sync; time {dd if = / dev/zero of = bs = 1 M count = test.txt 256 sync; sync; sync ;}}
  2560 records in
  2560 records out
  268435456 bytes (268 MB) copied, 6,77451 seconds, 39.6 Mbps
  Real 0m7.631s
  user 0m0.002s
  sys 0m0.751s
  root@iridium /# mount/mnt 172.28.19.16:/mnt/InternalRAID1/shares/VirtualMachines
  root@iridium /# cd/mnt
  synchronization of the # mnt root@iridium; Sync; Sync; time {dd if = / dev/zero of = bs = 1 M count = test.txt 256 sync; sync; sync ;}}
  2560 records in
  2560 records out
  268435456 bytes (268 MB) copied, 8,90374 seconds, 30.1 MB/s
  Real 0m9.208s
  user 0m0.001s
  sys 0m0.329s
  mnt root@iridium#.

  -

  ---

  As you can see, the performance of NFS direct comments for each of the three are very consistent.  ISCSI and the performance of the store local data disk are both a bit better than that - as I expect.  But the mounted via NFS data store gets only a fraction of the perfomance of the any of them.  Obviously, something is wrong.

  I was able to reproduce this effect with an Iomega Ix4 - 200 d as well.  The difference is not as dramatic, butalways important and consistent.  Here is a test of a guest of CentOS using a VMDK stored in a data store provided by an Ix4 - 200 d via NFS:-.

  root@palladium /sync #; Sync; Sync; time {dd if = / dev/zero of = bs = 1 M count = test.txt 256 sync; sync; sync ;}}
  2560 records in
  2560 records out
  268435456 bytes (268 MB) copied, 11,1253 seconds, 24.1 Mbps
  Real 0m18.350s
  user 0m0.006s
  sys 0m2.687s
  root@palladium /# mount/mnt 172.20.19.1:/nfs/VirtualMachines
  root@palladium /# cd/mnt
  synchronization of the # mnt root@Palladium; Sync; Sync; time {dd if = / dev/zero of = bs = 1 M count = test.txt 256 sync; sync; sync ;}}
  2560 records in
  2560 records out
  268435456 bytes (268 MB) copied, 9,91849 seconds, 27.1 MB/s
  Real 0m10.088s
  user 0m0.002s
  sys 0m2.147s root@palladium mnt-#.

  -

  ---

  Once more, the direct NFS mount gives very consistent results.  But using the diskette provided by ESXi on a mounted NFS datastore gives still worse results.  They are not as terrible as OpenFiler test results, but they are constantly between 60% and 100% longer.

  Why is this?  What I've read, NFS performace is supposed to be within a few percent of the iSCSI performance, and yet I see between 60% and 400% worse performance.  And this isn't a case of the SIN is not able to provide correct NFS performance.  When I connect to the NAS via NFS directly inside the guest, I see much better than when ESXi connects to the same NAS (the same proportion!) via NFS.

  The configuration of ESXi (network and network cards) is 100% stock.  There is no VLAN in place, etc., and ESXi system has only one

  Single Gigabit adapter.  It is certainly not optimal, but it doesn't seem to me to be able to explain why a virtualized guest is able to get a lot better performance NFS as ESXi itself to the same NAS.  After all, they both use the same exact suboptimal network configuration...

  Thank you very much for your help.  I would be grateful any idea or advice, you might be able to give me.

  Hi all

  It is very definitely a performance O_Sync problem. It is well known that NFS VMware shops still use O_Sync for writes little matter what share put for a default value. VMware uses a custom file locking system so you really can't compare it to a normal NFS share connection to a different NFS client.

  I have validated that the performance will be good if you have an SSD cache or storage target with enough reliable battery backup.

  http://blog.laspina.ca/ubiquitous/running-ZFS-over-NFS-as-a-VMware-store

  Kind regards

  Mike

  vExpert 2009

• Is it possible create Oracle RAC on NFS datastore?

  Hello

  Is it possible create Oracle RAC on NFS datastore?   With the VMFS data store, we use VMDK files as the Oracle RAC shared virtual disks with the Comptrollership and multi-writer SCSI Paravirtual, what about the NFS datastore? is the controller SCSI Paravirtual and writer multi function supported on the NFS datastore?

  Unless I'm missing something, this is not supported on NFS.

• NFS datastore = > no host not connected, impossible to remove

  Hello

  I have a NFS datastore (it was an ISO repository), I need to delete.   So I deleted all records from this share

  My problem is I disassembled it all hosts and the data store is still visible in my inventory and I am unable to remove it.

  When I try "Datastore Mounte to... additional host", the Wizard run in an endless loop and does not load the list of hosts.

  On my hosts, the NFS share is not visible. So nothing stuck because of a file in use.

  Have you already encountered this problem?

  Sorry found the culpit... instant on the virtual machines (with mapped CD-ROM).

• For NFS Datastore vSphere alarms

  So I would like to create an alarm that corresponds to a (not state) event that fires when a NFS data store is disconnected.  I found the trigger "Interruption of the connection to the NFS server", but it doesn't seem to work at all.  Also, I would only triggers the action when the host is not in Maintenance Mode, because that would be very annoying for an outgoing call because a host has restarted for patches and generated alarm type disconnected "NFS Datastore.

  use triggers esx.problem.storage.apd. *.

  When NFS disconnects you will get official messages in the file vmkernel.log on the host

• Windows do not install ISO in the NFS datastore

  Hi all

  I searched this forum for a few days and tried to suggestions from different positions without success. I recently installed ESXi 5.1 update 1. I have setup a NFS datastore on the same computer by using a USB external hard drive. I was able to install RHEL6 using an iso from the NFS data store. The problem is that I can't install Windows by using an iso of Windows 7. Whenever the virtual computer is booted, it aims to achieve crashes and boot TFTP. No iso standard is detected. I tried the following:

  1. guaranteed 'Connected' and 'Connect at Power On' options for CD/DVD are verified. However, I have noticed that when the virtual machine starts, the 'Connected' for Windows option becomes not controlled. This is not the case for the Linux VM.

  2. change the boot order in BIOS to the first boot from CD/DVD.

  3. uncontrolled ' connect at power on "for network adapters.

  Even after these changes, VM trying to do a start-up network and TFTP.

  The next thing I did:

  4 network cards removed from the BIOS (by changing configuration).

  For the moment, VM does not network boot attempt, but complains that the operating system was not detected.

  Few details on the NFS datastore:

  1. 1 TB external USB with 2 Configuration of partitions ext4 as an NFS share to the RHEL6 server on the same machine.

  2. NFS configured correctly because I can install from an iso RHEL6 very well.

  Am I missing something? Nothing wrong with iso of Windows. I used it elsewhere. Also tried a different iso Windows without success. Help, please. Thanks in advance for your time.

  Kind regards.

  As the ISO for the operating system files are big and sometimes take a considerable amount of clusters on the hard drive make a control office (or a scan of the drive) can fix corrupt ISO file. and to make sure that your ISO is not corrupted try to open it with Winrar and extract a file from it.

  Yours,
  Mar Vista

• Cannot remove/remove NFS datastore because its use, but is not

  I am trying to remove an old NFS data store, but I get an error message saying that it is in use. I find the virtual machine that he believes is used, stepped through all the parameters of the virtual machine and there is nothing pointing to this NFS datastore. I also tried to remove the store of data from the command line using 'esxcfg-nas - d < NFS_Datastore_Name >. That returned an error saying "unknown, cannot delete file system. Even 30 minutes before I was access and moving data out of this data to another ESX host store. I don't know what else to try. Can someone please help?

  It also still shows up on top of the virtual machine as used storage an idle...

  The virtual machine has an active overview that was created while the virtual machine was on the NFS datastore?

  André

• Moving to VM - NFS Datastore - no vmotion - invalid

  Hey people,

  Having a problem here to move a virtual machine from one ESX host to another (with VC).  First of all, let me tell you that I don't have vmotion (working on fixing that), but I have no shared storage (NFS datastore).

  If the virtual machine is hosted by esx1 on this NFS data store.  I stop the virtual machine and remove it from the inventory.  Then, I go to esx2 and browse the data store.  Find the vmx file and add to the inventory.  Then, the virtual machine appears in the inventory, but is grayed out with (invalid) beside him.

  I'm sure I could add a new virtual machine and use the existing vmdk files like discs, but I would rather simply add to the inventory with the existing configuration.

  Is this possible?

  Thank you very much

  Grant

  -

  ---

  Without vmotion you should always be able to migrate cold the VM - Power Down the VM - right click on the virtual machine name, select migrates - select another ESX host - you can change the storage or leave it where it is at.

  This will allow to get cold migrate the virtual computer with the ing to remove and re add to the inventory of VC.

  If you find this or any other answer useful please consider awarding points marking the answer correct or useful

• SSD on MPB early 2011 (13 inches) slow write speed

  Hey all,.

  I installed an SSD Crucial of MX300 in my MPB 2011 (13 inches). I get that I have probably will not hit the read/write 510/510, said the player, but right now I get only a writing speed of 240 vs a reading of 470.

  Any ideas on why it's so slow?

  Thank you.

  What is it connected to internally? If we replace the optical drive, then it may be because it's a 3 Gb/s bus rather than the primary drive SATA 6 GB/s bus connects to.

  Do you have your repartition and reformat the SSD drive before using it? It is not unusual to see some write speeds slower than reading speeds. The MX300 is a new economic model that may be slower than expected in real use.

• Slow write on a CD - R

  I'm on XP SP3 Home edition on a Compaq Presario.  I have 2 HARD drive on my system (C: and F :).)  I archive data to the CD (E :).  I do this by collecting all current data in one folder (c: or F :), then using Windows Explorer I SEND the file to the CD (standard 700 MB CD - R).)

  In my view, it is the process that happens in 3 steps: Windows puts all the files in a folder called Local Settings\Application Data\Microsoft\CD Burning, then creates an image of the CD on this basis and finally then writes this image on a CD.

  If I do this with a 100 MB file (maybe 10 subfolders, 200 files), copy files set / transfer (stage 1 I think, but maybe Stadium 2 - certainly not of stage 3) happens quickly: the names of flash files just straight up.

  If I do it with a large file (say 600 MB, 1000 files and subfolders maybe 50) process occurs MUCH more slowly.  Sometimes it starts fast and then slows down just so that 1 MB files take SECONDS to process.

  I tried to swap the drive Image is either the same or different in the source folder. makes little difference.  Tried to move the folder C: to F: source; makes little difference.  Switch off anti-virus (AVG Free 9.0): no real difference.  When the copy process is running slowly, Windows Task Manager indicates the low CPU usage - so I think it's some kind of lock on the copied files.  When the Copy task works I tried to close the Windows Explorer where is the source of the lock: makes no difference.  A new start after all the preparation with no other apps running - makes no difference, so it is not one of the preparation. measures or another app that keeps the lock.

  I use the write disk caching, but the drive cache size does not seem to be user variables.  The downturn is without a doubt before burning (seeing CD from the drive not lit; Regular light HARD drive IT) but, for the record, CD player settings are on fast write.  The system has 768 MB of RAM and the paging file is set to 2204 / 2204 MB (on F :).

  Any ideas?

  Well, I found a work around if it isn't a solution.  And it was necessary because the wizard was telling me that the first step (prep.) would take up to 7 days to complete!

  Copy the files that need to be archived in a folder on the C: drive, said in My Documents.
  Cut / paste in C:\Documents and utilisateur\Local Settings\Application Data\Microsoft\CD Burning Settings\nom it takes about 5 seconds only pointers are rewritten
  The "Ready for writing the files" icon appears.  The actual burning of the CD takes the usual time, but it was the preparation. a step that took forever.

  Should attempt to multi-threaded code that was not based on multi-thread, I guess.

• Slow writes PERC H700 and Samsung 850 PRO SSD

  I saw several posts about this, but don't see a solution.

  I have a few servers R710 with H700 controllers. I have installed Samsung 1 TB SSD Pro 850 and see writing very slow compared to other performance SSD tested in one box, simply slow in general. SAMSUNG SSD are ~ 500 MB/s in both sense when connected to the stand-alone desktop computer, but ~ 500 MB/s read and ~ 45 MB/s, writes when it is connected to a server.

  1. 1 x SSD in RAID 0 - no redundancy, just to test the performance. I tried all combinations of memory cache writeback and read before possible setting (enabled, disabled) and has never had more than ~ 45 MB/s, writing.

  2. 6 x SSD in RAID 5 with all possible combinations of never getting write cache over ~ 110 MB/s, writing.

  3. 6 x SSD in RAID 10 yet one time all possible settings of the writeback cache - ~ 140 MB/s.

  Finally, I installed 120 GB Intel SSD and got more than 400 Mbps in both directions without worrying about the RAID or cached.

  What is the problem with the H700 and Samsung SSD controllers? No one knows how to fix?

  Any other R710 compatible controller, which do not have this problem? Might be able, JBOD? I also tried a stop the controller PCIe Board without a bit of luck (bottom of basket or any compatibility issues)

  Thanks in advance.

  P. S.

  The controller is a latest firmware and the BIOS and other components. Used Dell Repository Manager iDRAC discovery and the ISO bootable to update everything.

  SAMSUNG SSD are on a latest firmware as well.

  Hello.

  The Samsung 850 Pro SSD are not validated or certified to work with controllers from Dell, and as such, there is an incompatibility of communication between readers and the controller at the firmware level. Thus, you are required to achieve the unexpected poor read and write performance regardless of the controller cache settings. Consider using Dell certified readers.

  Thank you.

• Adding NFS Datastore for all guests

  I would add a NFS data store to all my guests who are spread across different data centers in vCenter. Is there anyway I can change this script to make it work... with a list of all hosts or maybe a list of data centers.
  We have more than 50 domain controllers and I Don t want to have 50 different Scriptures

  # The user variables: adjusted for the environment

  ###################################

  # Load VMWare supplements

  If (-not (Get-PSSnapin VMware.VimAutomation.Core - ErrorAction SilentlyContinue)) {}

  Add-PSSnapin VMware.VimAutomation.Core

  }

  # Define the location of the file of credentials

  $credsFile = "C:\Scripts\creds.crd".

  # import credential file

  $Creds = get-VICredentialStoreItem-file $credsFile

  # Datacenter name

  $datacenterName = "Ourdatacentername".

  # NFS host IP

  $nfsHost = 'xx.xx.xx.xx.

  # Name of the NFS share

  $nfsShare = "oursharename".

  # New data store name

  $nfsDatastore = "temp".

  ####################################################################################################################

  # Start of execution

  ####################################################################################################################

  #connect to vCenter using credentails provided in the credentials file

  SE connect-VIServer-Server $Creds.Host-$Creds.User username-password $Creds.Password - WarningAction SilentlyContinue | Out-Null

  echo "connected to vCenter.

  echo "adding to ESXi hosts to start NFS share.

  foreach ($esx in data center-get-name $datacenterName | get-VMhost | name to sort)

  {

  $esx | New datastore - Nfs - name $nfsDatastore - NFSHost $nfsHost - $nfsShare path

  echo 'Sharing NFS added to $esx'

  }

  echo "completed."

  Disconnect-VIServer-Server $Creds.Host - force - confirm: $False

  Try the attached script

• Slow performance NFS (16 MB/s) of the Service Console

  I'm trying to set up an environment with ghettoVCB but I have problems with the flow to the NFS share.

  My configuration:

  ESXi host:

  ESXi 4.0.0 164009

  Reference Dell R710 with 8x300GB 10 k SAS.

  NAS server:

  Nexenta 3.1.0

  Dell PE2950 with 6x2TB Sata

  Local write on the nas performance is around 230 Mbps. tested by running these two commands in two different sessions of PuTTY:

  zpool iostat tank 5
  
  dd if=/dev/zero of=sometestfile2 bs=1024000 count=5000
  
  

  Zpool iostat output:

                 capacity     operations    bandwidth
  pool        alloc   free   read  write   read  write
  
  ----------  -----  -----  -----  -----  -----  -----
  tank        7.59G  10.9T      0      0      0      0
  tank        6.74G  10.9T      0    709      0  64.2M
  tank        8.11G  10.9T      0  2.06K      0   235M
  tank        9.62G  10.9T      0  2.07K      0   254M
  tank        10.8G  10.9T      0  1.81K      0   219M
  
  
  

  Switch dedicated for storage area network:

  Cisco 3750 with mtu 9000 system

  Nexenta hosts both ESXi are connected to the switch with network cards with Jumbo enabled frames.

  I created an NFS share and mounted as a data store in ESXi. When I run the command "dd" even the service console, I get this:

  tank        6.94G  10.9T      0    178      0  16.9M
  tank        7.03G  10.9T      0    190      0  16.2M
  tank        7.11G  10.9T      0    180      0  17.0M
  
  
  

  To test another way, I created a vmnic in the vSwitch dedicated for nfs and then attached this nic to a guest vm under Debian. I mounted the same nfs share and run the same command "dd":

  splunk01:mount 192.168.XXX.XXX:/volumes/tank/vmbackup /mnt/nas0
  splunk01:/mnt/nas0# dd if=/dev/zero of=sometestfile4 bs=1024000 count=5000
  5000+0 records in
  5000+0 records out
  5120000000 bytes (5.1 GB) copied, 56.1965 s, 91.1 MB/s
  splunk01:/mnt/nas0#
  
  
  
  
  
  

  zpool iostat output:

  admin@nas0:~$ zpool iostat tank 5
               capacity     operations    bandwidth
  
  pool        alloc   free   read  write   read  write
  ----------  -----  -----  -----  -----  -----  -----
  tank        8.73G  10.9T      0      1  5.72K   143K
  tank        8.73G  10.9T      0      0      0      0
  tank        8.73G  10.9T      0    389      0  47.5M
  tank        9.49G  10.9T      0    684      0  80.7M
  tank        4.24G  10.9T      0    702      0  84.6M
  tank        4.63G  10.9T      0    780      0  92.7M
  tank        4.63G  10.9T      0    750      0  91.1M
  tank        5.74G  10.9T      0    820      0  98.4M
  tank        6.27G  10.9T      0    729      0  87.9M
  tank        6.27G  10.9T      0    756      0  91.1M
  tank        7.28G  10.9T      0    785      0  94.9M
  tank        7.80G  10.9T      0    694      0  83.6M
  tank        7.80G  10.9T      0    801      0  96.6M
  tank        8.74G  10.9T      0    595      0  69.2M
  tank        8.74G  10.9T      0      0      0      0
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  

  So there is clearly no problem on the side of Nexenta.

  What I am doing wrong?

  Is there a problem with this build exakt?

  Andreas

  Hello

  you try this optimalizations in the GUI:

  1 / disable ZIL
  Settings-> preferences-> system
  Sys_zfs_nocacheflush Yes (default: No.)
  WARNING: it is dangerous without UPS

  2 / disable Nagel algoritm
  Settings-> preferences-> network
  Net_tcp_naglim_def 1 (default: 4095)

  3 / adapt for HDD SATA

  Settings-> preferences-> system
  Sys_zfs_vdev_max_pending 1 (default: 10)

  4 / disabling synchronization

  Data-> actions-> folder xxx management

  Turning off synchronization (default: Standard)

• What is the largest file VMDK, that you can create on a NFS datastore?

  Hello

  I can't find the answer to this question anywhere. I know that the largest file on a VMFS datastore vmdk is 2 TB, but this applies to NFS?

  Thanks in advance.

  Scott

  for single VMDK, it should be the same that is 2 TB - 512 b

  Craig

  vExpert 2009

  Malaysia, VMware communities - http://www.malaysiavm.com

• Cannot remove NFS datastore

  I noticed that there is a duplicate nfs data store appear on one of my ESX hosts.  When I try to remove the client from dastore the VI throws an error stating that "the object has already been deleted or was not completely created" too, I can browse the data in double fine store.  Vcenter restart I tried and that did nothing.

  vCenter gets its information from the ESX host, I guess you tried to update the display of storage?

  You can try to refresh the information in ESX datastore and see if these information to propagate to vCenter, try to do both:

  [root@himalaya ~]# vmware-vim-cmd hostsvc/datastore/refresh 
  

  If it still does not work, a quick fix is to restart the agent on the ESX host management by:

  service mgmt-vmware restart
  

  This should force an update and storage information should be correct later.

  =========================================================================

  William Lam

  VMware vExpert 2009

  Scripts for VMware ESX/ESXi and resources at: http://engineering.ucsb.edu/~duonglt/vmware/

  repository scripts vGhetto

  VMware Code Central - Scripts/code samples for developers and administrators

  http://Twitter.com/lamw

  If you find this information useful, please give points to "correct" or "useful".