Monitoring Your Clients VMWare Infrastructures.

Who manages your VMWare ESX Hardware Health?

If you are using ConnectWise Automate or LabTech RMM tool then this is a must have plugin for your environment. Plugins4Automate has a great plugin for managing ESXi Health monitoring in ConnectWise Automate and LabTech.

Visit them at www.plugins4automate.com

VMWare ESX Hardware Health monitor for Labtech

headerlogo

 

Plugins4LabTech has created a new plugin that will monitor the CIM data announced by the hardware running VMWare ESX software. We can report on any hardware that follows CIM standards and makes it’s hardware statuses known to VMWare. This simple to use plugin deploys in just minutes and can be setup by anyone with little or no  knowledge of LabTech functions and processes. If you want to see in action the easiest ESX Hardware Health Monitor available for LabTech then have a peek at our video on our YouTube Channel.

 

Never be in the dark again over hardware health!

 

Get alerts and tickets when hardware failures are detected. Using the monitor agent supplied with the plugin get quick responses to failures and warnings directly from VMWare. When alarms happen get emails, tickets and messaging alerting you to the failure that is taking place on the hardware.

Plugins4LabTech’s VMWare Health Monitor plugin for Labtech uses an agent to talk to VMware ESX hosts and retrieves the CIM data for the hardware the ESX host is running on. The plugin processes this data and stores it in LabTech’s database to be used in the views and alarms the plugin issues when failures are seen. You will see data on your RAID arrays and SCSI controllers, Power Supplies failures and system overheats. If the hardware is reporting it to VMWare we can see it.

 

Main ESX Health Monitor Console

The main view lets you see all the ESX hosts under management sorted by Client. In this view you are able to turn on and off the collection of data, set the interval of how often the data is collected and enable alerting for any seen failures.

ViewMenu

 

 

ESX Health Monitor Client Console Tab

The Client Console tab is where you would add and delete the ESX Hosts you want to monitor. You are able to force the update of ESX CIM data by selecting to rescan ESX hosts and you can view the full CIM data list of any ESX Host.

Clienttab

 

 

ESX Health Monitor CIM Data

The CIM Data view shows all the collected CIM data on a given ESX host. If any data is in a state not considered ok then those lines will be represented with warning and alarming icons. You will also get the hardware manufactures data back and in the case of Dell that is the Hardware type and Service Tag.

CIMData

 

 

ESX Health Monitor Internal Agent

Under the main view you can select to enable alerts. When you do this agent is created in the internal monitors of LabTech. You can use this monitor to email, ticket, alert or message anyone when failures are picked up.

monitor

A simple plugin all self contained and easy to deploy.  Just a few clicks to configure and you have data. So easy to use anyone can setup and have it working in just a few minutes.

 

The plugin is currently with the Squid Squad getting a final review before we release the new plugin. Follow our release notes at http://support.plugins4labtech.com

download

 

Labtech – How to access and manage VMWare ESX 5 Hosts remotely using Application Redirect

Labtech VMWare vSphere Client Redirector

Wouldn’t it be really cool if you could somehow safely access any VMWare vSphere ESX 5 host directly just using the local vSphere 5 Client installed on your workstation without porting and NATing traffic through your customers firewalls? With Labtech that is no problem,  by setting up a Application Redirector you can create a new proxy that will pass all your traffic to any ESX or vCenter host and allow you to fully manage that host using your own installed vClient on your workstation, using the Labtech server and an agent, wow is it fast.

 

Let me show you how to do it.

I want to thank the guys over at www.labtechgeek.com for creating the outline I am following here.

 

1.) You will need to create a new redirected app named vSphere Client.
Goto [System Dashboard] -> [Config] -> [Redirected Apps]

LT-RedirectedApplication

2.) In the program field, enter the location of your local vShpere client like ->  C:\Program Files (x86)\VMware\Infrastructure\Virtual Infrastructure Client\Launcher\VpxClient.exe

3.) Now we will redirect the following ports.

Local Port: 443 Local IP: 127.0.0.1 Remote Port: 443 Remote IP: %RemoteIP% Type: TCP Local Listen
Local Port: 902 Local IP: 127.0.0.1 Remote Port: 902 Remote IP: %RemoteIP% Type: TCP Local Listen
Local Port: 903 Local IP: 127.0.0.1 Remote Port: 903 Remote IP: %RemoteIP% Type: TCP Local Listen
Local Port: 8080 Local IP: 127.0.0.1 Remote Port: 8080 Remote IP: %RemoteIP% Type: TCP Local Listen
Local Port: 9443 Local IP: 127.0.0.1 Remote Port: 9443 Remote IP: %RemoteIP% Type: TCP Local Listen
Local Port: 10080 Local IP: 127.0.0.1 Remote Port: 10080 Remote IP: %RemoteIP% Type: TCP Local Listen
Local Port: 10443 Local IP: 127.0.0.1 Remote Port: 10443 Remote IP: %RemoteIP% Type: TCP Local Listen
Local Port: 902 Local IP: 127.0.0.1 Remote Port: 902 Remote IP: %RemoteIP% Type: UDP Local Listen

Redirector type should have the “Computer” box checked, this makes the redirector show up on computer consoles along side the other redirectors.

LT-redirectors

4.) Now we need to create an entry in your hosts file for the redirector to work. Add -> “127.0.0.1 vsphere-redir” to your local host file. If you just use 127.0.0.1 or localhost when the client pops up then the client may actually try to connect to the NetBIOS name of your PC, which will not work.

5.) Reload your systems cache and you should see the redirector show up under Computer consoles Redirectors menu.

 

To connect directly to an ESXi host,  while holding the Shift click on the vSphere Client redirector from meun. This will prompt you for an IP address for the remote ESX host you want to control – enter the IP of the ESXi host.

redirect-IP

Once you place your IP of the ESX host in the IP for Redirection box click OK. Give it a few seconds to get the proxy up and app to launch then you should see your VMware client pop up.  You now type in to IPaddress/Name area “vsphere-redir” as the host IP and then the user and pass needed to log in to the ESX host.

vclient

***Note – When you’re connecting directly to a host, or to vCenter, you must always enter vsphere-redir as the IP Address/Name  in the VMWare vSphere client.

***Note – If you’re connecting to a vCenter server, you won’t be able to view the console of any VMs (MKS) – this is because the vSphere Client makes a direct connection to the ESXi server on port 902. If you connect directly to the ESXi host, MKS works fine.

 

Enjoy,

Cubert 😎

 

[Solved] ESX VM shutdown stopped at 95% – VM fails to restart

We have several customers on sketchy hardware and on occasion the VM crash due to a SCSI card issue with the mother board used, that aside we have from time to time a need to force a hard reboot of the server running in a VM. Some times it works great and sometimes we have a lockup at 95% and have to force a kill of process that runs the VM to get it to restart.

So here is the process we take to get this to free up and reboot the VM on ESXi 5.0 and later VMware hosts.

 

  1.  Make sure if you do not already have it turned on, to turn on SSH on the ESXi Host. This can be done via the [Configuration -> [Security Profile] using the VMware client.
  2. Using your favorite SSH Client  (Putty), connect to your VMWare ESXi 5 Host.
  3. We now need to get and kill the process group for the VM that has failed. To do this we will look for the process group ID using this command.

    execute -> ps -g|grep “VMName”You should get a return that looks similar to this.

    3372 vmm0:MyVMSystem
    3374 3368 vmx-vthread-4:MyVMSystem 3368 3368 /bin/vmx
    3375 3368 vmx-mks:MyVMSystem 3368 3368 /bin/vmx
    3376 3368 vmx-vcpu-0:MyVMSystem  3368 3368 /bin/vmx

    We are looking for the common number across all processes and in this case that would be “3368” as seen near the end of each line.

  4. Now will need to kill the process. To do this we need to execute -> kill -9 3368   Replace “3368” with the ID number of your system.
  5. Now we need to do some clean up, We need to delete the swap file in the directory where the VM is stored. To get to where we store the swap file you will need to do the following.execute ->cd /vmfs/volumes/<YourDataStore>/<VMName>

    Next we need to make sure what our swap file name is so execute -> ls

    This will give you a directory listing find your swap file by looking for the file extension “.vswp”.  Now we will remove it with this command.

    execute -> rm –r <YourSwapFile.vswp>

     

  6. Now lets restart our VM services, This will not affect any running VM and is safe to run while VMs are active on host.execute -> /sbin/services.sh restart

     

  7. Reconnect your VMware client to the host and complete the process to power on the VM by first removing the VM from the inventory (Do not Delete from Disk) and re-adding it back in. This will reset the VM fully and allow you to restart it. After you remove your VM from inventory you can re-add it by browsing the datastore in your VMWare client finding your VM directory and right clicking on the “.vmx” file.  A menu will pop up and you can click “Add to Inventory” which will place VM back into the available VMs list. Now select the VM and click the Boot up arrow button to get started again.

 

Enjoy,

Cubert 😎

 

XYMON – ESX Hardware Health Monitor

Grab the Hardware Health of your VMware vSphere ESXi Host

Here is RC1 of my ESX Health Script for the XYmon monitoring server, not to be compared to the ESXi script going around for VM and Snapshot Health.  ESXHealth monitors the physical health of the hardware running the ESX hyper-visor  This script uses the xymoncgimsg.cgi to send status reports from a remote network to your XYmon Display Server thus allowing you to monitor ESX Host from anywhere really easily. Using this CGI allows you to run it from any windows box and send the notifications through port (http) friendly firewalls.

The only prereqs are, you must have installed the VMWare vSphere Perl SDK and have installed CURL or place the curl-nossl.exe provided in zip in the your PATH on Windows. Your XYmon server must have the xymoncgimsg.cgi moved from the xymon/server/bin folder to your xymon-cgi folder to allow web based status messages.

I have include the curl-nossl portable EXE with zip. Just drop it in the path for windows so you can call it by name. You will need to edit the script and update the URL to send the notifications to. I show an example in the script on how to use with a htpasswd protecting your web CGI for thoes who use that layer of security. If not then just place the standard URL for your xymoncgimsg.cgi and your good to go.

The script is very simple.

If all is good you get a “All’s OK” else if anything is bad it spits out any relevant info about that issue. The last error I got was for redundant Power lost to Supply 1 and was reported with plenty of detail to know what is wrong.

Just schedule windows to run every 15 minutes or so, see script for command line syntax for task scheduler

Download  -> ESXHealth-1.1  Monitor for XYmon

 

The monitor will look like the following examples: (Never mind my XYMon theme)

 

 

 

 

I hope this helps someone out there.

Cubert 😎

Quick and Easy Setup of Active Directory Authentication for VMware vSphere 5.1 SSO (Single Sign On)

VMware vSphere vCenter Server 5.1 now uses a new SSO (Single Sign On) service to authenticate with Microsoft Active Directory when deploying vCenter. If you do not install this services and configure it for AD then you will not be able to use your domain accounts with vCenter 5.1  During the initial install you may get errors when installing SSO.  KB 2034374 reports that a error of  ” Error 29155 Identity source discovery error”  is due to a failed attempt to automatically discover your Active Directory domain. Verify that the domain name and DNS are setup correctly.

Now lets setup an AD server in vCenter to allow our Domain Accounts. First we will login to vCenter Web Client (https://127.0.0.1:9443) if you used the default ports for the web client installs. The default login is admin@system-domain  and the password you set for SSO during the install process. Once you are logged in to the web client you can continue.

Now Select [Administration]

vCenter SSO-login

 

 

 

 

 

 

 

 

 

Now Select [Sign-On and Discovery] -> [Configuration]

SSO Configuration

 

 

 

 

 

 

 

 

Under the Identity Sources Tab in the right pane select the PLUS symbol to add a new AD source. This will pop up a “Add Identity Source” window, select the active directory radio button and fill out the requested information with you AD Domain name and the “OU” the holds your users and groups.

Here is the generic information you will need just replace the sesenviron.local with your domain and then place your AD credentials at the bottom.

Adding identity source

 

 

 

 

 

 

 

 

 

 

Now that we have a AD server assigned as a source we must now add this newly created connector to our “Default Domains ” list.

Add AD to Default  Domains list

 

 

 

 

 

 

 

 

Now that we have it in our Default Domains list lets move it up to be our primary source. To do this highlight the AD domain name and select the blue arrow head pointing up and move the domain name to the top of the list .

Set priority of the domains

 

 

 

 

 

 

 

 

 

 

Now lets select the small floppy disk icon to save the changes to the default domain list box. Once this is complete we should be able to open up the vSphere client application and log in with domain access. You should be using a domain level admin to access vCenter.

 

I hope this helps some people out there.

Cubert 😎

 

 

 

Error during the configuration of the host: Failed to update disk partition information

VMware ESX 4 -> Failed to update disk partition information.

After creating a new LUN on a Dell MD3000i, I wanted to add the new iSCSI LUN to my ESXi server. Everything went fine, the LUN was presented and the Add storage wizard has found the new LUN. But when i wanted to complete the wizard, the following error came up:

VMware ESX 4 -> Failed to update disk partition information.

After creating a new LUN on a Dell MD3000i, I wanted to add the new iSCSI LUN to my ESXi server. Everything went fine, the LUN was presented and the Add storage wizard has found the new LUN. But when i wanted to complete the wizard, the following error came up:

Now if you go on the Internet a search around you will find several references to this issue. Most if not all of them tell you to:

1. Use fdisk on the /dev/sd[a-z]* device. Create a partition with type 0xfb.
2. Format the VMFS datastore using
vmkfstools -C vmfs3 vmhbaI:T:L:P
In this example, I corresponds to the initiator of the VMware host bus adapter, T corresponds to the target number of the disk, L is the LUN number, and P is the partition number of the newly created partition.

3. Reboot ESX server

This is not always required, It may be just a hickup in your ISCSI registration on the ESX servers. Here is how I fixed mine and it worked with out any reboots or major adjustments.

Steps to take first before following the CLI instructions:

  1. Go create another 5Gb LUN on Power Vault.
  2. Set the access group assigned to new LUN to allow ESX servers
  3. On ESX Host GUI go back to configuration menu and select Storage Adapters menu
  4. Select your ISCSI Software Adapter from Storage Adapter list and select “Rescan” (even if you see your LUN)
  5. Go back to the Storage menu and select add storage, select Disk/LUN and follow through the menus to mount and format the 5gb LUN and it should go through now.
  6. Remove the 5gb LUN from the ESX Storage menu.
  7. Remove the 5gb LUN from Dell Vault.
  8. Go back to your ESX and rescan HBAs and make sure the 5gb LUN is gone and you see your original LUN you want to mount.
  9. Go back to the Storage menu and select add storage, select Disk/LUN and follow through the menus to mount and format the  LUN and it should go through now.

If this does not work then change the access to the LUN on the vault to a new access group. Rescan for LUN on ESX then go reset the LUN back to the original access group you want to use and then rescan again from ESX.  Try to remount LUN and format. This should now connect and no errors should appear. If you still are getting errors then you may have other issues and you can now try the CLI method or call in support.

Good Luck, I hope  this helps a few of you out there from dealing with production systems and not wanting to spend all day VMotioning, reboots and such.

Enjoy