Showing posts with label vRA. Show all posts
Showing posts with label vRA. Show all posts

Tuesday, January 3, 2017

vRA and SCCM Timeout Issue

Hi,

OK here I am again. Been away for few months working on various projects including vRA, vROPS, SRM, VIN, vRB etc.etc. And one of them is MSB. Yes its a new term I've learned and worked on with one of the Federal client along with VMware NSBU. 

But here today I'm going to discuss about the timeout issue with vRA and SCCM deployment.

Now before we begin let me give you the overview of what we are dealing with.

In my recent project Im working with VMware on a vRA Distributed architecture with a little bit of vRB and also integrating the SCCM on which the client is heaving depending on for VM/s deployment. 

Mainly these are windows 2008 R2 Std and Windows 2012 R2 Std VMs. They are leveraging SCCM to do all major tasks such as putting patches, updates, updating AV signatures, putting legacy apps, some monitoring apps such as WhatsupGold and Logrhythm etc. etc.



So once the VM is built in vRA its been handed over to SCCM for further provisioning and there is a default time out setting in vRA which is only 15 minutes.



So as you can see the default setting for SCCM machine registration timeout is 5 minutes. So now
lets assume you are doing few tasks using SCCM e.g. Installing OS, putting patches, doing AV updates, putting legacy or custom applications or softwares etc. etc. then you need time to finish all those tasks in the background on SCCM side. With having 5 minutes timeout vRA wont know what is the status of the SCCM VM as the whole process may not be finished and the gugent might not be updating vRA with any status at all so you may see the process/task in vRA as its still running and not completed and eventually the task will fail as no signal received by vRA in time (which is 5 minutes). So to avoid that you need to increase that time out setting by enough time that you can finish the required tasks on SCCM side before hitting the timeout window of vRA. We had to keep it at 45 minutes (screen shot below) on a safer side which accounts few reboots of the OS along with other tasks listed above.




After doing the above change we were started receiving finished tasks in vRA successfully which were handled by SCCM then gugent updates vRA server with appropriate status.

I will cover the SCCM and vRA in one of my upcoming blogs very soon.

Hope this will help integrating SCCM and vRA.

Please share and care.

Thanks for your time.


Thursday, October 22, 2015

EMC Networker v/s vCenter Templates

Hi,

Recently I was part of the discussion and found few interesting things when discussing the automation and backup of the critical environmental components.

Infrastructure contains vRA 6.2.x and vRO and all the virtual machines are using the Templates situated on vCenter Server.

Now the backup solution in use is EMC Networker which replaced  Veeam recently.

Due to left over snapshots and not regular clean up were the main reasons to change the backup solution.

After having discussion with the Backup Team they sent the list of the permission the backup user needed on vCenter Server as follows.

As per NetWorker-8.2-SP1-VMware-Integration-Guide (Refer Page 44)




Now the above permission were assigned the service account which was created on vCenter Server.

But when the Backup Team tried to register the service they were getting an error.

I have assigned the Full Administrator Role in vCenter to that Service Account then the registration was successful.

Now I am not sure what exact permissions are required just to register the extension/service with vCenter Server. So asked the same question back to the Backup Team and they in turn asked to EMC support the same question and the answer received was to go through the above permissions only.

Now my question is WHY the EMC Networker service account user needs Administrator Role on vCenter Server to do the Backup and Restore of the VMs ??

As from Security perspective this is clearly a risk to assign the Admin Role level access to vCenter Server as it has enough rights to do any kind of destruction in vCenter Server.

I have looked at the other permission for restore and from the documentation found out the following.

"permissions requirements
· The account used to log into the "EMC Data Protection Restore Client" login page must have Administrator rights in vCenter (or the user must be assigned to the role that contains the minimum vCenter user account permissions for EMC Backup and Recovery).
· The Account and role with the minimum vCenter account permissions must be assigned to the root of the vCenter and the option to propagate must be selected. (Add to root, not through a group)· 
· The user account being used must also be part of the Local Administrators group on the server that is performing the file level recovery - it cannot be an AD account, (explicitly add the account to the Local Administrator group on the recovery virtual machine)
· This is required for the File Level recovery process to access the local resources on the virtual machine performing the recovery.
· Ensure that the virtual machine performing the file level recovery is part of the same vCenter server as the source virtual machine.
· The File Level recovery also interacts with the virtual machine VMware Tools, so ensure the VMware tools are installed and up to date."

So even we need to create a local user on the vCenter Server which must have the Admin privilege.

Again a security concern there. Now to get the backup going for the templates, such compromised was needed as I have tried assigning different role to the vCenter Service Account (with Backup role, and also tried changing permission on specific components from the table) but none worked.

The biggest challenge with Networker we found is to backup Templates.

EMC Network CAN'T Backup Templates. (on page 82 of the same guide). BOOOOMMM .....


This was the biggest surprise to us as we have the vRA/vRO components backed up using the same solution but what about Templates. As the vRA blueprints relies on the vCenter Templates they needs to be backed up regularly in case of a disaster.

We got few options on our hand to backup the Templates

a) Convert the Templates into VMs a day before the scheduled Backup and after the backup successfully run then convert them back to Templates

Downside/s of doing that are

1) The whole automation platform is not available to the DevOps community internally during that time

2) If the backup fails for X reason we might have to attempt one mote time and it will be a down time for the end user and keeping track of such missed backup and then schedule them again and again is a simple Pain.

3) Put a change request each time when we convert the templates and one person has to stay late to make sure all the Templates are converted to VMs properly and none left out which is more man hours on each window.

b) 2nd option was to use the Linked clone/s for vRA which is/are nothing but powered off VM/s with a snapshot on it and that DO NOT have a .vmtx format so for EMC Networker, its easy to Backup the same.

Downside of doing this

1) Each VM has a snapshot for not reason and every change to the template will grow the snapshot and we are unnecessarily occupying the space on the SSD storage (yes Xtreme IO) on the back which is not advisable when the matter comes to SLA.

2) Management to differentiate between the actual Linked clones which we are using to serve other Team (such as Server Team and Database Team) to give them ready made VMs with necessary OS/Applications loaded based on the blue print selection and they will change the IP/DNS entries based on the type of VM and the person who requested (yes IPAM solution is Work-in-progress :-))

c) The last option was to not use any of the above but just dedicate a separate LUN where we can store the Templates and then replicate the LUN

Downside of doing this

1) We need an additional LUN which is not cost effective

2) More manageability from storage perspective

As option C was more effective apart from the cost involved, we had no other choice as it was easy for time being till EMC come up with an actual fix to resolve such issue in any future release.

Now not having such functionality is totally a down hill in the whole backup plan and such dilemma was resolved by putting the templates on a specific LUN and then replicate that LUN on to a DR Site using EMC Recover Point.

Eventually we took care of the Backup for Templates but could not understand the reason why such support is not provided. Is it Technical ly not possible which is hard to believe or some other gotchas involved (which I dont have any idea about).


Hopefully this will help people running in to the same situation and gives a better option to decide which solution to go with.


Please share and Care !!  

Sunday, August 16, 2015

REST while using CURL for vRA

DevOps needs REST :-)

In DevOps world users are mostly using the command line and APIs to do almost all of their work. So for the following example you’re going to need curl as the usage was done keeping in mind the universal tool and just not focusing other available large number of REST client or such, and as this is inside a Lab encryption is not needed bu you will need the version that supports SSL for end-to-end encryption !  

Use of jq, ( a new parsing tool for JSON) is used here, which does everything thro' command-line, and being portable (zero install) and open source, it runs on Windows and Linux both, so all of the developers out there can use it irrespective of the platform they’re using. I saved both utilities in C:\curl, and put my .bat file and .json files there too.  You can decide the best way to do that, Token is required.

So what is Token or a Bearer Token?  Putting it simply, its an authentication token.  First thing you’ll need is to get a bearer token and parse it.  You’ll need a .json file with your authentication details; the basic format is 

{“username”:”domain\\myusername“, “password”:”mypassword“, “tenant”:”vsphere.local”}.  

Fill in your username, password, and if you’re not using the default tenant, then save it to C:\curl\hellotoken.json and you’re good to go. 

Use the following curl command to submit your request:

curl –insecure -H “Accept: application/json” -H “Content-Type: 
application/json” –data @hellotoken.json https://vra-fqdn/identity/api/tokens > identity.json

Please note I used –insecure because my vRA environment is still using self-signed SSL certs.  If you have actual SSL certs that will validate, please drop the –insecure option.  This command will drop a .json file named identity.json in the current folder (c:\curl if you’ve been following along), the file looks a little like this:

{“expires”:”2015-07-08T18:47:58.963Z”,”id”:”long-token-goes-here“,”tenant”:”vsphere.local”}

So it has an expiry, an id, and a tenant. Bearer tokens are good for 24 hours, and what's not obvious is a user can only have one at a time; if you ask for a new bearer token, that invalidates all previous bearer token/s.  These two facts means, it’s very difficult to manually get and use a bearer token. What we want to do is extract that ID in such a way that we can automatically use it in subsequent curl calls.

For this I will be using jq:

jq .id -r < identity.json > bearer.txtset /p btoken=
)

Let’s look into what it consists of:

jq .id -r parse out the id field from the json.  Print it to stdout in raw mode (i.e. no surrounding quotes or symbols)

< identity.json use the previously generated identity.json file as input to jq

> bearer.txt output the parsed id field to a file bearer.txt

use bearer.txt as the input file for the following command

set /p btoken= set the environment variable btoken to the input.  The /p normally means to prompt for the environment variable, but in this case since we provided a stdin pipe, the contents of bearer.txt will be used instead of prompting the user

) Close the section using bearer.txt as input. Yes I know it is required to do it here. 

To use the bearer token add the following HTTP Header to all future curl calls:

-H “Authorization: Bearer %btoken%”

Machine Request

Now you can request the catalog item you want from the vRA console and note the request ID. After the request has completed, you can use curl to look at the request with the following command:

curl –insecure -H “Accept: application/json” -H “Authorization: Bearer %btoken%” https://vra-fqdn/catalog-service/api/consumer/resources/?$filter=request/requestNumber+eq+myReqNumber > myreqOutput.json

This will create a file myreqOutput.json with details of your request.  The item you care most about is the catalog ID which you will be able to find at .content.0.catalogItem.id  Note that ID, then request the entitled catalog item with the following:

curl –insecure -H “Accept: application/json” -H “Authorization: Bearer %btoken%”https://vra-fqdn/catalog-service/api/consumer/entitledCatalogItems/?$filter=id+eq+&#8217;catalogItemId‘ > catalogItem.json

This time, note the provider binding at .content.0.catalogItem.providerBinding.bindingId.  Also note the tenant ID specified here.

Now you have the information you need to format the machineRequest.json file:

{
“@type”: “CatalogItemRequest”,
“catalogItemRef”: {
“id”: “catalogItemId”
},
“organization”: {
“tenantRef”: “vsphere.local”,
“subtenantRef”: “TenantId”
},
“requestedFor”: “myUserName@domain“,
“state”: “SUBMITTED”,
“requestNumber”: 0,
“requestData”: {
“entries”: [{
“key”: “provider-blueprintId”,
“value”: {
“type”: “string”,
“value”: “Provider-binding”
}
},
{
“key”: “provider-VirtualMachine.Network0.NetworkProfileName”,
“value”: {
“type”: “string”,
“value”: “Network Profile Name”
}
},
{
“key”: “provider-Variable1″,
“value”: {
“type”: “string”,
“value”: “value”
}
}]
}
}

Note the key: provider-name section above; this is how you specify any custom properties for the blueprint, I provided a couple samples above but what you put here will really depend on the blueprint configuration.  Basically just add provider- to the start of any custom property (or even VM property, see provider-VirtualMachine.Network0.NetworkProfileName above) to submit it with your request… so the next step is to…

Submit a Request

The curl command to submit a request is:

curl -X POST –insecure -H “Content-Type: application/json” -H “Authorization: Bearer %btoken%” https://vra-fqdn/catalog-service/api/consumer/requests –data @machineRequest.json

And now you can see under Infrastructure --> Recent Events or under Requests that new VM getting deployed in vRA. You can also verify under vSphere client session that the VM is getting cloned and deployed.

Hope you find this useful.

Please share and care.




Saturday, August 1, 2015

vRA Blueprint and Ubuntu 14.04 with Open VM Tools customizaton

If you are using Ubuntu 64 bit 14.04 version with Open VM Tools installed then you can read further on how to use the Open VM Tools and then use it with vRA.

Ubuntu 14.04 VM w Open VMware Tools

The version of Open VM Tools its running is - 2:9.4.0-1280544-5ubuntu6.2. You can get the version details by using the following command

dpkg –l | grep open-vm-tools

You can install the open vm tools by giving the command:

apt-get install open-vm-tools

If the tools service is stopped or not running then you can run the following command:

service open-vm-tools start

If you encounter any issue then only try with the following command:

/etc/init.d/open-vm-tools start

Additionally when customizing the VM and deploy a VM from the template we required an additional package installed as per the http://kb.vmware.com/kb/2075048 .
To install this plug-in in Ubuntu: 
To obtain and import the VMware Packaging Public Keys:

Create a directory on the virtual machine to store the VMware Packaging Public Keys.  
Download all the VMware Public Packaging Public Key files from the http://packages.vmware.com/tools/keys directory.
Save the files to the directory you created.
For each key that you download, run this command to import the key:

sudo apt-key add /key_path/key_name

where:
key_path is the directory in which you saved the keys.
key_name is the file name of a key

Create the file /etc/apt/sources.list.d/vmware-tools.list with this content:
deb http://packages.vmware.com/packages/ubuntu precise main
or run this:

sudo echo "deb http://packages.vmware.com/packages/ubuntu precise main"
 >>  /etc/apt/sources.list.d/vmware-tools.list

Note
: Use either precise or trusty above.

To install the package, run this command:

sudo apt-get update
sudo apt-get install open-vm-tools-deploypkg

-----BEGIN PGP PUBLIC KEY BLOCK-----
Version: GnuPG v1.4.7 (GNU/Linux)
mI0ESAP+VwEEAMZylR8dOijUPNn3He3GdgM/kOXEhn3uQl+sRMNJUDm1qebi2D5b
Qa7GNBIlXm3DEMAS+ZlkiFQ4WnhUq5awEXU7MGcWCEGfums5FckV2tysSfn7HeWd
9mkEnsY2CUZF54lyKfY0f+vdFd6QdYo6b+YxGnLyiunEYHXSEo1TNj1vABEBAAG0
QlZNd2FyZSwgSW5jLiAtLSBMaW51eCBQYWNrYWdpbmcgS2V5IC0tIDxsaW51eC1w
YWNrYWdlc0B2bXdhcmUuY29tPoi8BBMBAgAmBQJIA/5XAhsDBQkRcu4ZBgsJCAcD
AgQVAggDBBYCAwECHgECF4AACgkQwLXgq2b9SUkw0AP/UlmWQIjMNcYfTKCOOyFx
Csl3bY5OZ6HZs4qCRvzESVTyKs0YN1gX5YDDRmE5EbaqSO7OLriA7p81CYhstYID
GjVTBqH/zJz/DGKQUv0A7qGvnX4MDt/cvvgEXjGpeRx42le/mkPsHdwbG/8jKveY
S/eu0g9IenS49i0hcOnjShGIRgQQEQIABgUCSAQWfAAKCRD1ZoIQEyn810LTAJ9k
IOziCqa/awfBvlLq4eRgN/NnkwCeJLOuL6eAueYjaODTcFEGKUXlgM4=
=bXtp
-----END PGP PUBLIC KEY BLOCK-----

Once installed you can reboot the VM, convert the VM back to Template by right clicking the VM and selecting an option "Convert to Template".  Now access the UI of vRA and go to Infrastructure Tab. Select the Compute Resources option and highlight the necessary Cluster which has the Ubuntu blueprint listed, and select Data Collection option at the bottom of the selection, and for the Inventory option, select "Request Now". Once you see "Succeeded" then the vRA blueprint got updated and now the customization should work for the Ubuntu Linux 14.04 version.

Please refer KB 2075048 for instructions on other Linux distros.

Please share and care !!

Thanks for reading.