Upgrading to vRealize Operations 8.3

vRealize Operations 8.3 was released on  4th Feb, 2021 . This release which came in after just four months from the last release of vRealize Operations 8.2, has something for everyone including FIPS mode and 20-Second metrics for more granularity.

Some of the key features available in vRealize Operations 8.3 are as listed below:

1. Introducing FIPS mode : It  allows to run vRealize Operations in compliance with the FIPS 140-2 standard and can be enabled upon deployment or through the Administration UI.
2. 20-second Peak Metrics – Ability to enable collection of metrics at up to 20-second granularity. Disabled by default and can be enabled through policies. With this new feature, vRealize operations captures the highest 20-second average in any given 5 minute and this is available only for specific metrics – detailed information on this new feature is available in Iwan Rahabok’s blog – http://virtual-red-dot.info/20-second-peak-metrics/
3. New inventory trees added for VMware Cloud on AWS.
4. New VMC Configuration maximums Dashboard.
5. Launch of Cloud management Assessment (CMA) – next logical step of the vSphere Optimization assessment (VOA) in vROPs. The CMA is out-of-the-box and easy to use, providing simple instructions, discussion points with links for more information, and a specific set of dashboards for each use case.
6. Ability to enable Mandatory Login Message and Consent Banner for vROps login page.
7. For new deployments, Improved experience upon initial login with the new Quick Start page and guided wizard to get started by creating first Cloud Account

Important Note : If you are currently using or plan to use Endpoint Operations Management Agents (“EPOps”) in vRealize Operations, please be aware that EPOps will no longer be supported by VMware vRealize Operations after the current release of vRealize Operations 8.3, Endpoint Operations Management Agents will be supported in vRealize Operations 8.3 but cannot be activated in FIPS mode. If you would like to use Endpoint Operations Management Agents in vRealize Operations 8.3, please download the binaries from the vRealize Operations 8.2 Download page. For more information, please see VMware Knowledge Base article KB 2137709.

The full list of new features and enhancements in vRealize Operations 8.3 is detailed in https://blogs.vmware.com/management/2021/02/whats-new-in-vrealize-operations-8-3.html

Please find below the recommended steps to follow for a successful upgrade to vROps 8.3 :

Down-time required	Yes
Time Required	Roughly 2 to 4 hours for environment of 2 x Large Nodes vROps ( This is a rough estimate and it can vary considerably with several environmental factors )
Impacts during upgrade	No Metrics will be collected during upgrade window.
Roll-back Plan	Snapshot need to be taken for all nodes ( with vROps Cluster Offline – very important) prior to initiating upgrade and this snapshot should be used as roll-back mechanism for any failures reported during upgrade steps.

Please make sure that all the steps below are followed in sequence.

Phase 1: Running Pre-upgrade Readiness Assessment tool – Time required: 15 minutes – Strongly Recommended to do this for any upgrades from vROps 7.0 or prior versions

1. Download upgrade assessment tool : vRealize Operations 8.3 – Pre-Upgrade Readiness Assessment Tool  ( Name of File: APUAT-8.3.0.17520080.pak, Size of File: 6.03 MB )
2. Log in to the master node vRealize Operations Manager Administrator interface of your cluster at https://master-node-FQDN-or-IP-address/admin.
3. Click Software Update in the left panel.
4. Click Install a Software Update in the main panel.
5. Follow the steps in the wizard to locate and install your PAK file. ( Check Install the PAK file even if it is already installed )
6. Install the Upgrade Assessment Tool.
7. Wait for the software update to complete.
8. Access to Pre-upgrade Readiness assessment report  ( Refer to KB 82346 ) – steps as below:

• Navigate to the Support > Support Bundles tab.
• Download the light support bundle that was generated from the installation of the Upgrade Assessment Tool.
•  In the downloaded support bundle, open the cluster_timestamp_nodeaddress/nodeaddress_timestamp_nodeaddress/apuat-data/report/index.html file.
• A list of all potentially impacted user content , if any , will be displayed in the “Removed / Discontinued Metrics “ tab
• In the tab of “System Validation Checks” , you will find the report of system validation checks for upgrade and recommendations , if any. Follow the recommendations as suggested.

9. If there are any remediations suggested for “System Validation Checks” in the above step , run the pre-upgrade readiness assessment tool again after the remediation tasks are completed. This is to make sure that all the upgrade pre-requisites are met before proceeding to the next phase of upgrade.
10. Proceed to the next phase ONLY after all the remediation activities, if required, and all the system validation checks are green.

Phase 2 : Upgrading to vROps 8.3  

Upgrading to vROps 8.3 from vROps 7.x  

1. Recommended to remove or update all impacted Management Packs, if any, prior to initiating the upgrade.
2. Review the vRops 8.3 sizing for the environment prior to upgrade using vRops sizing calculator OR follow the sizing guidelines from KB 82344. Make sure that the vROps nodes are sized based on the recommendations.
3. Validate network settings on all nodes – Gateway , DNS , NTP server ( This is very important when there are multiple nodes in vROps Cluster )
4. Download the following upgrade file required for upgrade: Use the download link from myvmware portal

File Name	Build#	File Size	File Type
vRealize_Operations_Manager-VA-8.3.0.17501349.pak	17501349	2.2 GB	.pak

4. Make sure the root logins to all the vROps nodes are available.
5. Make sure passwords for “root” account and “admin” account are reset and is not expired or not locked (On all nodes) – check by logging into SSH console of every node with root and admin accounts . If you have not logged into SSH earlier , the default password will be blank and just press enter at password prompt. After this it will ask for setting a new password.
6. Make sure vROps cluster’s  “admin” account’s  password is not expired or locked – you may change from vROps admin portal.
7. Make sure all the vROps nodes have sufficient disk space for upgrade (Minimum 5% free disk space required for each node): Check with command “df -h” from SSH session to each node.
8. Optional step: If you have a large vROps cluster with more than 4 nodes and deployed across firewalls, the upgrade process can be made faster by pre-copying upgrade files to all nodes. Pre-copy the upgrade files to all nodes except Master Node – follow steps as in KB 2127895
9. Take the vROps cluster offline from Admin UI.
10. Shut down the nodes in the sequence of Remote Collectors , Data nodes , Replica Node and Master Node.
11. Take the snapshot of all the nodes.
12. Power on the nodes in the sequence of Master Node , Replica Node , Data nodes and Remote collectors
13. Initiate vROps In-place upgrade from Admin UI. 
14. Once the upgrade is completed, verify functionality. If all functionality is reported correctly, take the cluster offline, shutdown nodes and delete the snapshots. Once snapshots are removed, power on the nodes in sequence of Master Node , Replica Node , Data Nodes and Remote Collectors
15. Once all nodes are powered on , bring the vROps cluster Online.
16. Since vROps 8.3 is included in vRealize Suite 2019 you should upgrade your vRealize Suite license from 2018 to 2019 as well.

Upgrading to vROps 8.3 from vRops 8.x 

1. Download the upgrade file from the link – Download link for Upgrade file

File Name	Build#	File Size	File Type
vRealize_Operations_Manager-VA-8.x-to-8.3.0.17501338.pak	17501338	2.08 GB	.pak

2. Make sure the root logins to all the vROps nodes are available.
3. Make sure all the vROps nodes have sufficient disk space for upgrade (Minimum 5% free disk space required for each node): Check with command “df -h” from SSH session to each node.
4. Take the vROps cluster offline from Admin UI.
5. Shut down the nodes in the sequence of Remote Collectors , Data nodes , Replica Node and Master Node.
6. Take the snapshot of all the nodes.
7. Power on the nodes in the sequence of Master Node , Replica Node , Data nodes and Remote collectors
8. Initiate vROps In-place upgrade from Admin UI. 
9. Once the upgrade is completed, verify functionality. If all functionality is reported correctly, take the cluster offline, shutdown nodes and delete the snapshots. Once snapshots are removed, power on the nodes in sequence of Master Node , Replica Node , Data Nodes and Remote Collectors
10. Once all nodes are powered on , bring the vROps cluster Online.

Additional resources :

KB 2001476 – How to reset the root password in vRealize Operations

KB 2078313 – How to reset the admin password in vRealize Operations Manager

Quick access to Commands available in vRops Nodes:

To reset vRops Cluster Admin password from root console :

$VMWARE_PYTHON_BIN $VCOPS_BASE/../vmware-vcopssuite/utilities/sliceConfiguration/bin/vcopsSetAdminPassword.py –reset

To reset root or admin account in OS – to be run on all nodes

passwd root

passwd admin

To unlock the admin and root accounts in vRops – to be run on all nodes

pam_tally2 –user admin –reset
pam_tally2 –user root –reset

To validate if the accounts are locked  – to be run on all nodes:

passwd -S root

passwd -S admin

( If the account is not locked, you will see PS next to the username.  If the account is locked, you will see LK next to the username )