Is my VM running off the Snapshot Delta or Base disk?

I came across few instance recently where the the Virtual machine was either consolidating all the snapshots taken by backup software or adding/removing snapshots and at the same time Edit Settings of the virtual machine is not giving you any details as its grayed out.

User is finding so many delta disks on the respective datastore/s which is occupying the disk space on the Shared LUNs/local Datastores.

Now how to find if the VM is running off base disks or its running off snapshot files.

To recover the space consumed by the deltas which are not really having any information inside.

There are few ways you can find out if the VM indeed running on snapshot delta files or not.

1st method is to use the SSH/DCUI/DRAC/iLO/KVM/RSA whichever method you seems easy and you are comfortable with, using which access the console of ESXi.
Login with root account privileges and change to the virtual machine directory.

#cd /vmfs/volumes/datastore1/vmname

Once you are in the VM directory and then run the following command to see if the virtual machine snapshot file in use.

#less vmname.vmx | grep -i *.vmdk

The above command will list all the vmdk files in use for that virtual machine which includes all the snapshot delta file and base disks. The example will be scsi0:0 and the file name vmname-000001.vmdk or scsi1:0 and the file name vmname-000004.vmdk etc. etc. The number of scsi devices represents the number of VMDKs presented to the virtual machine.

Now there are many 3rd party applications available in the market which can do this job for you. You just need to run those tools and they will give you the report in various forms. You can use the PowerCLI script also to find out the same within the Datacenter or at the cluster level.


2nd method is if you can go under Edit Settings option then you can login through vSphere client and connect to the vCenter/ESXi host and then click on the Edit Setting of the Virtual Machine.

Click on HDD1 and go to the disk file option on top right corner and put the cursor and click "End" button on the key board which will take you to the end of the file name. If the datastore name and virtual name is shorter then you dont need to do that and you can see the file name right away and see if there is -000000.vmdk added to the file than you know that the VM is using the Snapshot file.


If the name does not have and if you see vmname.vmdk then you know its running off the base disk. For the subsequent disk you will see vmname_1.vmdk, vmname_2.vmdk and so on.


You need to repeat the step for each HDD which you are using on the virtual machine.


Hoping that the above will help you out finding unnecessary snapshot files for the VMs which are using critical space and you can recover that space by dealing with them accordingly.




Please share and care!!

Need 5 mins of your valuable Time

Hi All,


Back in 2012 the announcement of vExpert for 2011 was made and I was not selected based on the standards and criteria but I took that as a challenge and committed to myself to do blogging about all VMware related technologies and actively started blogging from April 2012 onwards and here I am with 70 Blog posts so far.

This blog is dedicated to all the VMware Community and to the people who are using VMware on a daily basis.

Spreading knowledge is the key for me when it comes to VMware. I am not in the race but just to get the support from you in one of the categories such as the following would be highly appreciated.

"Best New Blog"

"Best Networking Blog" 

"Best Storage Blog"



There are others blogs out there but if you think a single blog post helped you then Vote for my blog just for the support and extend the help to contribute more and more going further.

It will take few minutes of your time but will make more long time effect on me and will give enough fuel for the upcoming years. I expect nothing else as your invaluable Vote will have everything.

Use the following link to vote and urging not to do any Fake multiple voting.


http://www.surveygizmo.com/s3/1165270/Top-vBlog-2013

Thanks for your time and appreciate all the help you are doing to recognize the efforts




Share and care !!

Cheers !!

Vote for the "Best New Blog" of 2012

VOMA on ESXi 5.1 to find out Metadata Corruption on VMFS

I recently blogged about the how to verify the VMFS Heart beat corruption and in this blog I am going to show you how to use VOMA (vSphere On-disk Metadata Analyzer) to check if there is any inconsistencies after the events such as power outage or some others due to which everything went down at the same time.

voma can be used only on ESXi 5.1 and its not available in any prior version.

You can see the help of the voma command here.

Here are the screen shots of how voma will detect and identify the error found on the VMFS volume

As you can see in these example that there is file used against which the voma command was ran. e.g. .bin

Now before you run the command you may want to take a dump from each affected volume/s and then run the command against the dump file/s.




e.g.


dd if=/vmfs/devices/disks/naa.0000000000000000000000000000:1 of=/tmp/naa.0000000000000000000000000000p1.dmp bs=1M count=1500

voma -m vmfs -f check -d /vmfs/devices/disks/naa.0000000000000000000000000000p1.dmp

Now as you can see the highlighted parts  (with gray background) in the above screen shots which are errors and inconsistencies found on the volumes. The white spaces are names which are hidden due to confidential information. If you find the errors on the volume/s then contact VMware Support for further actions.

There are five phases of the disk analysis and @VMwareStorage (Cormac Hogan) has posted about voma here and here.

You can check out the capabilities of voma on your own on ESXi 5.1.

Help by sharing !!


Thanks for your time.

Traceroute and vMotion Traffic on vSphere

Recently I received a question about the behavior of the traceroute command.

Now here is the situation.

User has vMotion Network and Management Network on the separate subnets.

vSwitch0 has 2 x 1 GB nics (uplinks) for Management Network

esxcfg-vswitch -l
Switch Name      Num Ports   Used Ports  Configured Ports  MTU     Uplinks
vSwitch0         128         6           128               1500    vmnic1,vmnic0

  PortGroup Name        VLAN ID  Used Ports  Uplinks
  Management Network    56       1           vmnic0,vmnic1


vSwtich1 has 1 x 10 GB nic for Virtual Machine traffic and also for vMotion traffic.

Switch Name      Num Ports   Used Ports  Configured Ports  MTU     Uplinks
vSwitch1         128         41          128               1500    vmnic4

PortGroup Name        VLAN ID  Used Ports  Uplinks
  VLAN10                10       21          vmnic4
  VLAN11                11       7           vmnic4
  vMotion-01            80       1           vmnic4


Management vmkernel IP is 172.16.56  subnet and vMotion vmkernel IP is on 172.16.20 subnet


The default route is 172.16.56.1

 esxcfg-route -l
VMkernel Routes:
Network          Netmask          Gateway          Interface
172.16.83.32     255.255.255.224  Local Subnet     vmk2
172.16.20.0      255.255.255.0    Local Subnet     vmk5
172.16.37.0      255.255.255.0    Local Subnet     vmk3
172.16.56.0      255.255.255.0    Local Subnet     vmk0
default          0.0.0.0          172.16.56.1      vmk0



Output is truncated for other vmkernel ports and just focusing on Management and vMotion Network vmkernel interfaces.

esxcfg-vmknic -l
Interface  Port Group/DVPort   IP Family IP Address                              Netmask         Broadcast       MAC Address       MTU     TSO MSS   Enabled Type
vmk0       Management Network  IPv4      172.16.56.109                           255.255.255.0   172.16.56.255   2c:76:8a:4e:7c:8c 1500    65535     true    STATIC
vmk5       vMotion-01          IPv4      172.16.20.109                           255.255.255.0   172.16.20.255   00:50:56:64:de:cc 1500    65535     true    STATIC
Now if you see the result of traceroute command you will see that its going through the vmk0 interface which does not mean that the vMotion traffic will route through the vSwitch0 uplinks viz. vmnic0 and vmnic1

~ # traceroute 172.16.20.110traceroute: Warning: Multiple interfaces found; using 172.16.56.109 @ vmk0
traceroute to 172.16.20.110 (172.16.20.110), 30 hops max, 40 byte packets
 1  172.16.20.110 (172.16.20.110)  0.546 ms  0.265 ms  0.196 ms

So if you want to test the connectivity between the vMotion vmkernel interfaces then you can use the vmkping command

On the source host

#vmkping -i

Now lets say you want to see if the vMotion traffic is flowing through the expected interface or not so you can initiate the vMotion of a virtual machine from one host to another use following command:

'tcpdump-uw -i '


The above command will show you the real time traffic.

@FrankDenneman has recently blogged about the vMotion using the Management Network with two different scenarios.

There is a change in the ICMP behavior on ESXi 5.1 so please refer the KB 2042189 for more information as well.

Now this blog is just for information purpose only and do not include other scenarios with various types of configuration such as Jumbo Frames, Multi-nic vMotion etc. etc. so use the information accordingly.


Thanks for your time and please share !!

vSphere VSS & VDS - Cisco Nexus1000v Feature Comparison

I have been asked or requested, so many times before, while talking to the end user or with some one about the features offered by VMware VSS (Virtual Standard Switch), VDS (VMware Distributed Switch) and Cisco Nexus 1000v VDS and I was thinking to gather as much information as I could and blog about it. So here it is.

I believe that any advanced user, working with VMware environment for few years, is aware of the fact that Nexus 1000v is Cisco's product and it will appear as a VDS inside the vSphere GUI. So by looking at the GUI if the naming convention is not used properly for the VDS than you may interpret as a regular VMware VDS so to avoid the confusion one can check the "Summary" Tab of the VDS to verify the vendor of that VDS and also the version of that VDS.

IBM has released Nexus 5000 VDS about which I have not heard much discussion till date. So in this comparison I am not including IBM VDS.

Have a look at the following Table, which I have tried including possibly every feature offered by the VMware VSS, VMware VDS and Nexus VDS. If I miss something then do leave the comment or get in touch with me and I will update the article.

Feature	VMware vSphere 4.x VSS	VMware vSphere 5.5 VDS	Cisco Nexus 1000V 4.2 (1)-SV2(2.2)
Switching Features
Layer 2 Forwarding	YES	YES	YES
IEEE 802.1Q VLAN tagging	YES	YES	YES
Multicast Support (IGMP V2 and V3 support)	YES	YES	YES
IGMP V3 Snooping	-	-	YES
VMware vMotion	YES	YES	YES
Network VMware vMotion	-	YES	YES
Multi-Nic vMotion Support	-	YES	YES
Physical Switch Connectivity
Virtual Mac Pinning	YES	YES	YES
EtherChannel	YES	YES	YES
Virtual Port Channels	-	-	YES
Link Aggregation Control Protocol (LACP)	-	YES	YES
Static LAG	-	YES	YES
Dynamic LAG	-	YES	YES
Load – Balancing Algorithms
Virtual Port ID	YES	YES	YES
Source MAC Address	YES	YES	YES
Source and destination IP Address	YES	YES	YES
Source MAC Address	-	YES	YES
Additional hashing Options	-	YES	YES
Load Based Teaming	-	YES	-
Source and Destination port IP	-	YES	YES
Advanced Port channel	-	YES	YES
IP Hash	YES	YES	YES
Traffic Management Features
Transmit-rate (from Virtual Machine) limiting	YES	YES	YES
Receive-rate (to Virtual Machine) limiting	-	YES	YES
ISCSI Mutipathing	YES	YES	YES
Unicast Flooding Control	-	YES	YES
Quality-of-Service (QoS) marking
Differentiated Serviecs Code Point (DSCP)	-	YES	YES
Type of Service	-	-	YES
Class of Service	-	YES	YES
802.1Q	-	YES	YES
Network IO control (NIOC)	-	YES	YES
Transmit-rate (from Virtual Machine) limiting	YES	YES	YES
Receive-rate (to a Virtual Machine) limiting	-	YES	YES
802.1p	-	YES	YES

Security Features
Port Security	YES	YES	YES
VMware VMsafe Compatible	YES	YES	YES
Private VLANs (PVLANs) 512	-	YES (no limit)	YES(512)
Local PVLAN enforcement	-	YES	YES
PVLAN with Promiscuous Trunk	-	YES	YES
Access Control List (ACLs)	-	YES	YES
Virtual Service Domain	-	-	YES
DHCP Snooping	-	-	YES
IP source Guard	-	-	YES
Dynamic ARP Inspection	-	-	YES
MAC ACL	-	YES	YES
VXLAN	-	YES (no limit)	YES (2048)

Management Features
VMware vCenter Support	YES	YES	YES
VMware vCloud Director Support	YES	YES	YES
vCloud Director Automation Center support	YES	YES	YES
RESTful API	YES	YES	YES
Third-party-Accessible APIs	YES	YES	YES
Network Policy Groups	YES	YES	YES
Multitier Policy Groups	-	-	YES
Packet Capture and Analysis	-	YES	YES
RADIUS and TACACS+	-	-	YES
LLDP	-	YES	-
Network CLI	-	-	YES
Server CLI	YES	YES	-
Configuration and Management Console Interface	vSphere Client	vSphere Web Client/vSphere Client	vCenter and Cisco CLI
Graphical UI	YES	YES	-
Config Backup and Restore	-	YES	YES
Network Rollback and Recovery	-	YES	-
IPv6 for Management	YES	YES	YES
Monitoring and Troubleshooting
VMware Port Mirroring (promiscuous)	YES	YES	YES
Switched Port Analyzer (SPAN)	-	YES	YES
Encapsulated Remote SPAN (ERSPAN)	-	YES	YES
NetFlow ver. 9	-	-	YES
NetFlow ver. 10 (Ipv6, VXLAN flows)	-	YES	-
Network Health Check	-	YES	-

Simple Network Management Protocol (SNMP) V3 Read and Write (V1,V2C)	-	YES	YES
Cisco Discovery Protocol (CDP) v1 and v2	YES	YES	YES
Syslog *	YES	YES	YES
ACL Logging	-	YES	YES
SNMP ACLs	-	-	YES
Network Virtualization
VXLAN support with Multicast	-	YES	YES
VXLAN support without Multicast	-	-	YES
ARP suppression for VXLAN	-	-	YES
L3 Gateway for NV	-	YES	YES
Site-to-Site IPSec VPN	-	YES	YES
Remote Access SSL VPN	-	YES	-
Scalability
Hosts per Switch	500	500	128
Switches per management system (VC)	128	128	32
VXLAN segments	-	10000 (VCNS)	2048
VLAN (no vxlan)	4096	4096	2048
Port Groups/Profiles per Switch	4096	10000	2048
Virtual Ports per host	4096	4096	300
Virtual Ports per Switch	10000	60000	4096
Max Active Virtual ports per Host	1016	1016	300
Max MAC Addresses per Host	No limit	No limit	32000

(NOTE: Above Table is updated for VDS 5.5 so some features which are not shown as supported on VDS 5.5 are fully supported for VDS with NSX so please verify the same.)

* Syslog information is exported and included with VMware ESX/ESXi server events

So you can select the option depending on the features you need. Both VMware and Cisco VDS requires Enterprise + License. Now if you see that I have not included the column for VMware VDS 4.1. The reason behind it, as most of the features available with VDS 5.5 and there are some more features offered with the latest version of VDS so I encourage the reader to use the latest version anyway. Even the Cisco Nexus VDS version is 4.2 (1) - SV2(2.2) which is the latest one.

For configuration of VDS you can refer the online documentation available on www.vmware.com and for Cisco Nexus 1000v VDS you can refer the documentation page here.

I will update the Sheet with NSX 6.0.5 once I get some time so please be waited till then and DO NOT leave the comment for the requesting the same Update :-) but I appreciate any Feedback or comments regarding any features (if missed) or needs an update to the above Table.

Share and Care !!

Enjoy !!