Upgrading to vRealize Operations 8.5

vRealize Operations 8.5 was GA on 8^th July 2021 and is a minor release with some security updates and bugfixes.

For features and capabilities available in vRealize operations 8.5, please refer to vRealize operations 8.4 documentation or Upgrading to vRealize operations 8.4

Important Note before planning for vROps upgrade:

• vRealize Application Remote Collector virtual appliance is deprecated and is no longer available for download from the vRealize Operations user interface when you upgrade to vRealize Operations 8.4 or above. VMware recommends that you use cloud proxy to monitor your application services. 
• Support for Internet Explorer has been dropped from vRealize Operations 8.4 onwards. 
• vRealize Automation 7.x integration with vRealize Operations 8.3 and onwards is dropped.
• Support for vCenter Server 6.0 has been dropped from vRealize Operations 8.3 onwards.
• The End Point Operations Management Solution has been deprecated and will not be supported. However, it will work in FIPS disabled mode and will not work in FIPS enabled mode.  

Please find below the steps you can follow for a successful upgrade to vROps 8.5 :

Down-time required	Yes
Time Required	Roughly 2 to 4 hours for environment of 2 x Large Nodes vROps ( This is a rough estimate and it can vary considerably with several environmental factors )
Impacts during upgrade	No Metrics will be collected during upgrade window.
Roll-back Plan	Snapshots need to be taken ( with vROps Cluster Offline – very important) prior to initiating upgrade and this snapshot should be used as roll-back mechanism for any failures reported during upgrade steps.

We will be following below steps for the upgrade activity:

Phase 1: Running Pre-upgrade Assessment tool – Time required: 15 minutes – Strongly Recommended to do this for any upgrades.

1. Download upgrade assessment tool : vRealize Operations 8.5 – Pre-Upgrade Readiness Assessment Tool  ( Name of File: APUAT-8.5.0.18176777.pak, Size of File: 6.04 MB ). Only vRealize Operations 7.5 and later can directly upgrade to vRealize Operations 8.5.  However, this Pre-Upgrade Assessment Tool can be run on vRealize Operations 6.2.0 or later.
2. Log in to the master node vRealize Operations Manager Administrator interface of your cluster at https://master-node-FQDN-or-IP-address/admin.
3. Click Software Update in the left panel.
4. Click Install a Software Update in the main panel.
5. Follow the steps in the wizard to locate and install your PAK file. ( Check Install the PAK file even if it is already installed )
6. Install the Upgrade Assessment Tool.
7. Wait for the software update to complete.
8. Access to Pre-upgrade Readiness assessment report  ( Refer to KB 84286) – steps as below:

6. Navigate to the Support > Support Bundles tab.
7. Download the light support bundle that was generated from the installation of the Upgrade Assessment Tool.
8.  In the downloaded support bundle, open the cluster_timestamp_nodeaddress/nodeaddress_timestamp_nodeaddress/apuat-data/report/index.html file.
9. A list of all potentially impacted user content , if any , will be displayed in the “Removed / Discontinued Metrics “ tab
10. In the tab of “System Validation Checks” , you will find the report of system validation checks for upgrade and recommendations , if any. Follow the recommendations as suggested.

9. If there are any remediations suggested for “System Validation Checks” in the above step , run the pre-upgrade assessment tool again after remediation tasks are completed. This is to make sure that all the upgrade pre-requisites are met before proceeding to the next phase of upgrade.
10. Proceed to the next phase ONLY after all remediation activities, if required, and all the system validation checks are green. 

Phase 2 : Upgrading to vROps 8.5  

Upgrading to vROps 8.5 from vROps 7.5  

1. Recommended to remove or update all impacted Management Packs, if any, prior to initiating the upgrade.
2. Review the vRops 8.5 sizing for the environment prior to upgrade using vRops sizing calculator OR follow the sizing guidelines from KB 85062. Make sure that the vROps nodes are sized based on the recommendations.
3. The minimum hardware version required for vRealize Operations 8.x releases is version 11. If your vRealize Operations virtual appliance had a hardware version earlier than 11, you must first upgrade to hardware version 11 on vRealize Operations virtual appliance and then plan for upgrade to vRealize Operations 8.5.
4. Validate network settings on all nodes – Gateway , DNS , NTP server ( This is very important when there are multiple nodes in vROps Cluster )
5. Download the following upgrade file required for upgrade: Use the download link from myvmware portal

File Name	Build#	File Size	File Type
vRealize_Operations_Manager-VA-8.5.0.18255620.pak	18255620	2.29 GB	.pak

4. Make sure the root logins to all the vROps nodes are available.
5. Make sure passwords for “root” account and “admin” account are reset and is not expired or not locked (On all nodes) – check by logging into SSH console of every node with root and admin accounts . If you have not logged into SSH earlier , the default password will be blank and just press enter at password prompt. After this it will ask for setting a new password.
6. Make sure vROps cluster’s  “admin” account’s  password is not expired or locked – you may change from vROps admin portal.
7. Make sure all the vROps nodes have sufficient disk space for upgrade (Minimum 5% free disk space required for each node): Check with command “df -h” from SSH session to each node.
8. Optional step: If you have a large vROps cluster with more than 4 nodes and deployed across firewalls, the upgrade process can be made faster by pre-copying upgrade files to all nodes. Pre-copy the upgrade files to all nodes except Master Node – follow steps as in KB 2127895
9. Take the vROps cluster offline from Admin UI.
10. Shut down the nodes in the sequence of Remote Collectors , Data nodes , Replica Node and Master Node.
11. Take the snapshot of all the nodes.
12. Power on the nodes in the sequence of Master Node , Replica Node , Data nodes and Remote collectors
13. Initiate vROps In-place upgrade from Admin UI. 
14. Once the upgrade is completed, verify functionality. If all functionality is reported correctly, take the cluster offline, shutdown nodes and delete the snapshots. Once snapshots are removed, power on the nodes in sequence of Master Node , Replica Node , Data Nodes and Remote Collectors
15. Once all nodes are powered on , bring the vROps cluster Online.
16. Since vROps 8.5 is included in vRealize Suite 2019 you should upgrade your vRealize Suite license to 2019 as well.

Upgrading to vROps 8.5 from vROps 8.x 

1. Download the upgrade file from the link – Download link for Upgrade file

File Name	Build#	File Size	File Type
vRealize_Operations_Manager-VA-8.x-to-8.5.0.18255614.pak	18255614	2.17 GB	.pak

2. Make sure the root logins to all the vROps nodes are available.
3. Make sure all the vROps nodes have sufficient disk space for upgrade (Minimum 5% free disk space required for each node): Check with command “df -h” from SSH session to each node.
4. Take the vROps cluster offline from Admin UI.
5. Shut down the nodes in the sequence of Remote Collectors , Data nodes , Replica Node and Master Node.
6. Take the snapshot of all the nodes.
7. Power on the nodes in the sequence of Master Node , Replica Node , Data nodes and Remote collectors
8. Initiate vROps In-place upgrade from Admin UI. 
9. Once the upgrade is completed, verify functionality. If all functionality is reported correctly, take the cluster offline, shutdown nodes and delete the snapshots. Once snapshots are removed, power on the nodes in sequence of Master Node , Replica Node , Data Nodes and Remote Collectors
10. Once all nodes are powered on , bring the vROps cluster Online.

Important : The VMware vRealize Operations Certificate Renewal PAK file must be applied on vRealize Operations 8.5 if the internal certificate is going to expire soon. The internal certificate in vRealize Operations 8.5 is generated during initial deployment of the appliance. Currently, upgrading to later versions of vRealize Operations does not upgrade the internal certificate. See KB 71018. 

Additional resources :

KB 2001476 – How to reset the root password in vRealize Operations

KB 2078313 – How to reset the admin password in vRealize Operations Manager

Quick access to Commands available in vRops Nodes:

To reset vRops Cluster Admin password from root console :

$VMWARE_PYTHON_BIN $VCOPS_BASE/../vmware-vcopssuite/utilities/sliceConfiguration/bin/vcopsSetAdminPassword.py –reset

To reset root or admin account in OS – to be run on all nodes

passwd root

passwd admin

To unlock the admin and root accounts in vRops – to be run on all nodes

pam_tally2 –user admin –reset
pam_tally2 –user root –reset

To validate if the accounts are locked  – to be run on all nodes:

passwd -S root

passwd -S admin

( If the account is not locked, you will see PS next to the username.  If the account is locked, you will see LK next to the username )