Cisco Nexus team released the update to the Nexus 1000v product taking it to release 4.2(1) SV1(4). They did a great job of documenting the entire upgrade process both in docs and in a series of screencasts. Check it out it’s really worth your time.

However one obvious thing that they have missed so far is how to upgrade if your VSM is in standalone mode. Part of the problem is the Upgrade application (GUI) does not support standalone mode and the upgrade document does not address the manual upgrade (CLI) method for standalone VSM.

Here is a quick guide on how to upgrade your standalone VSM from 4.0(4) SV1(3, 3a, or 3b) to 4.2(1) SV1(4).

Before you start make sure you have run the pre-upgrade check script against your 1000v configuration to make sure you don’t run into the 7 configurations that are incompatible with Nexus 4.0.x to 4.2.x code.

Now the actual steps:

Step 0 Do a show mod and make sure you have already upgraded all the VEM modules. Remember this has changed from earlier releases. You HAVE to upgrade the VEM first and VSM second to upgrade to release 4.2(1) SV1(4).

show mod
Mod  Ports  Module-Type                       Model               Status
---  -----  --------------------------------  ------------------  ------------
1    0      Virtual Supervisor Module         Nexus1000V          active *
3    248    Virtual Ethernet Module           NA                  ok
4    248    Virtual Ethernet Module           NA                  ok
5    248    Virtual Ethernet Module           NA                  ok
6    248    Virtual Ethernet Module           NA                  ok

Mod  Sw                Hw
---  ----------------  ------------------------------------------------
1    4.0(4)SV1(3b)      0.0
3    4.2(1)SV1(4)      VMware ESXi 4.1.0 Releasebuild-320137 (2.0)
4    4.2(1)SV1(4)      VMware ESXi 4.1.0 Releasebuild-320137 (2.0)
5    4.2(1)SV1(4)      VMware ESXi 4.1.0 Releasebuild-320137 (2.0)
6    4.2(1)SV1(4)      VMware ESXi 4.1.0 Releasebuild-320137 (2.0)

Mod  MAC-Address(es)                         Serial-Num
---  --------------------------------------  ----------
1    00-19-07-6c-5a-a8 to 00-19-07-6c-62-a8  NA
3    02-00-0c-00-03-00 to 02-00-0c-00-03-80  NA
4    02-00-0c-00-04-00 to 02-00-0c-00-04-80  NA
5    02-00-0c-00-05-00 to 02-00-0c-00-05-80  NA
6    02-00-0c-00-06-00 to 02-00-0c-00-06-80  NA

Mod  Server-IP        Server-UUID                           Server-Name
---  ---------------  ------------------------------------  --------------------
1    172.16.165.55    NA                                    NA
3    172.16.165.91    dcedbaac-1dc0-11df-0000-00000000000e  172.16.165.91
4    172.16.165.92    dcedbaac-1dc0-11df-0000-00000000000d  172.16.165.92
5    172.16.165.93    dcedbaac-1dc0-11df-0000-00000000000c  172.16.165.93
6    172.16.165.94    dcedbaac-1dc0-11df-0000-00000000000b  172.16.165.94

Step 1 Copy the new software release system image and kickstart image files to the bootflash file system of the VSM.

switch# copy scp:[email protected]/srv/n1kv-images/nexus-1000v-mz.4.2.1.SV1.4.bin bootflash:
switch# copy scp:[email protected]/srv/n1kv-imagess/nexus-1000v-kickstart.mz.4.2.1.SV1.4.bin bootflash:

Step 2 Use the install all command to update the boot variables to release 4.2(1)SV1(4) images.

switch# install all system bootflash: nexus-1000v-mz.4.2.1.SV1.4.bin kickstart bootflash:nexus-1000v-kickstart-mz.4.2.1.SV1.4.bin
Boot variables are updated to running configuration.

Step 3 Save your configuration by copying the running configuration to the startup configuration.

switch# copy running-config startup-config
[########################################] 100%

Step 4 Reload VSM.

switch# reload
This command will reboot the system. (y/n)? [n] y
switch#show mod
Mod  Ports  Module-Type                       Model               Status
---  -----  --------------------------------  ------------------  ------------
1    0      Virtual Supervisor Module         Nexus1000V          active *
3    248    Virtual Ethernet Module           NA                  ok
4    248    Virtual Ethernet Module           NA                  ok
5    248    Virtual Ethernet Module           NA                  ok
6    248    Virtual Ethernet Module           NA                  ok

Mod  Sw                Hw
---  ----------------  ------------------------------------------------
1    4.2(1)SV1(4)      0.0
3    4.2(1)SV1(4)      VMware ESXi 4.1.0 Releasebuild-320137 (2.0)
4    4.2(1)SV1(4)      VMware ESXi 4.1.0 Releasebuild-320137 (2.0)
5    4.2(1)SV1(4)      VMware ESXi 4.1.0 Releasebuild-320137 (2.0)
6    4.2(1)SV1(4)      VMware ESXi 4.1.0 Releasebuild-320137 (2.0)

Mod  MAC-Address(es)                         Serial-Num
---  --------------------------------------  ----------
1    00-19-07-6c-5a-a8 to 00-19-07-6c-62-a8  NA
3    02-00-0c-00-03-00 to 02-00-0c-00-03-80  NA
4    02-00-0c-00-04-00 to 02-00-0c-00-04-80  NA
5    02-00-0c-00-05-00 to 02-00-0c-00-05-80  NA
6    02-00-0c-00-06-00 to 02-00-0c-00-06-80  NA

Mod  Server-IP        Server-UUID                           Server-Name
---  ---------------  ------------------------------------  --------------------
1    172.16.165.55    NA                                    NA
3    172.16.165.91    dcedbaac-1dc0-11df-0000-00000000000e  172.16.165.91
4    172.16.165.92    dcedbaac-1dc0-11df-0000-00000000000d  172.16.165.92
5    172.16.165.93    dcedbaac-1dc0-11df-0000-00000000000c  172.16.165.93
6    172.16.165.94    dcedbaac-1dc0-11df-0000-00000000000b  172.16.165.94

That’s it the VSM upgrade is complete to 4.2(1)SV1(4).

As an aside. These steps compare extremely well when contrasted with the HA based VSM manual upgrade which is 22+ steps. I actually had the VSMs in a HA pair and deleted the secondary, upgraded the primary as a standalone and deployed a new 4.2(1)SV1(4) VSM VM using the OVF files and powered it up and called it secondary and it happily synced with the primary and the upgrade is complete.

 

10 Responses to “Cisco Nexus 1000v Standalone VSM upgrade to Release 4.2(1) SV1(4)”

  1. [...] Cisco Nexus 1000v Standalone VSM upgrade to Release 4.2(1) SV1(4) [...]

    • marcel says:

      Hi Jay,

      Nice blog!

      A notice for people who are facing this upgrade, you could run into trouble. I did several upgrades before from earlier versions of the VSM without any problems and was not expecting any issues…

      After a succesful upgrade on the VEM’s on all our UCS blades it was time to upgrade the VSM. So it did and ran into trouble. After the reboot of the VSM, parts of the config were gone;
      - no ssh / telnet connection to the console, L3 management ip adres gone
      - no default gateway / route
      - L2 1000v VEM-VSM information erased:
      vsm-sw1# show svs domain
      SVS domain config:
      Domain id: 0
      Control vlan: 0
      Packet vlan: 0
      - no connection with vCenter:
      “Connect” string not there anymore
      Port-channel config “sub-group-id” commands on the physical interfaces from all blades were gone:
      interface Ethernet3/1
      sub-group-id 0
      interface Ethernet3/2
      sub-group-id 1

      We missed the last one (sub-group-id) when we restored the other settings and after we connected to vCenter, the ESX blades got the new settings and the port-channels went down isolating all guest. The storage and management connections of the ESX hosts were fine due to the use of system vlan configuration (misleading while troubleshooting the issue). Unfortunately we did not pick up the port-channel misconfiguration untill after a roll-back to version 1.3a so we have to upgrade again.

      After making a case at Cisco and sending them the logfiles from “show tech-support svs” the feedback we got was, quote: “Coming to the actual issue itself, the upgrade did not go through because of a single VSM environment. A synchronized HA pair is mandatory for the upgrade to be successful.”

      just my 2 cents, hoping others to quicker recognise the configuration issues after the reboot of the VSM and a heads-up for the possible downtime it can cause.

      Marcel

      • Jay says:

        Hi Marcel,

        Thanks for sharing the upgrade issue. Yes one of the things I generally do before making any major changes to the environment is to back up the running config and revert to it at the first sign for trouble :).

        Also am not buying the response from engineer on this issue. there is no way a partial config loss is attributable to running VSM in standalone mode. In fact running VSM in standalone takes away one of the most logical reason for a partial config loss which is the active and standby VSM were async when the reboot/reload occurred and the VSM with the partial config came online first hence becoming the active one erasing the working config to partial config on the now standby VSM.

        Also on a side note, do you have any specific reason to use “sub-group id” on the physical interfaces, instead of the recommended “mac-pinning” mode in the uplink-port profile for UCS blades.
        Also did you get a chance to run the pre-upgrade check utility against your running config to check for config that is known to cause issue with the nexus 4.2 code. This may explain a partial config loss, if there is something in your config that is incompatible with the latest code and may cause some process to crash or hang.

        In any case, please do follow up with Cisco on this. Also if you can unicast to me the TAC case number for this issue, I can see if we can look into this a little bit more.

        Jay

        • marcel says:

          Hi Jay,

          I had the same feeling about the response i got. I did run the pre-upgrade utility already before the VEM upgrade and after some modifications everything was fine. Right before the actual upgrade of the VSM i ran it again and no issues came up. Also the particular settings which were lost are all basic commands which did not change between version 1.3a and 1.4a. So no incompatibility there.

          Due to the nature of our support contract we had to open a case with our supplier and they open a case at Cisco. The supplier asked me if they could close the case because the issue was “clearly” with the single VSM environment and the solution is that i have to setup an secondary VSM and recommends upgrading using the “upgrade application”. I was really fed up with the answer and told our supplier to close the case. This will be a topic of discussion with our supplier in 2012, we are clearly not talking on the same level. I sent you the Cisco TAC details through LinkedIn.

          About the mac-pinning configuration, we were the first company in The Netherlands who purchased Cisco UCS back in 2009. UCS firmware was at 1.01 and the Nexus 1000v at version 1.1. At the time there was hardly any documentation, best practise whitepapers nor suppliers with extensive knowledge of UCS and the Nexus 1000v. I got from the Cisco UCS Engineer who helped us with the UCS implementation some internal powerpoint slides about the Nexus 1000v configuration options and did it myself.

          So now (2 jears later) i am in the middle of a project of upgrading to vSphere 5 (hence the 1000v upgrade and did already the UCS firmware upgrade to version 2.01s) and implementing the best practise whitepapers for Cisco UCS and the Nexus 1000v. I am creating project documentation of all the steps involved and this already included the mac-pinning best practise. But thanks for the advise, i am always eager to learn from others.

          I must say that we are very happy UCS customers and in 2012 we hope to have virtualized everything on UCS within the datacenter. I have worked in the past with HP blades before and UCS is a whole different world.

          On topic again: The upgrade of the VSM is scheduled again for next sunday (the 25th). Now with the knowledge of what went wrong the last time. I will let you know the result.

          • Jay says:

            Hi Marcel,

            Thanks for reaching out to me on LinkedIn, I did a search on all the cases opened on behalf of your organization with cisco in the last 6 months and found the SR you were talking about. I have already requested the TAC engineer to reach out your service provider to get some additional logs and information from about your environment. I will loop you into that thread shortly. Also I reached out the dev team and they agree that standalone mode, through officially unsupported, should not have caused the partial config loss. We are going look into this further and will let you know the results.

            Yes do let me know how the upgrade goes or if you run into issues feel free to give TAC a call with the case number and they will be able to make an exception and work with you directly as long as your provider is “online” with you.

            Jay

  2. Fred says:

    Nice job on this..question. When you deleted the secondary VSM, did you go back to Primary VSM and change its HA redundancy role to standalone, rebooted, then did the upgrade, then changed the redundancy role back to Primary?

    • Jay says:

      That step is not required for an manual upgrade.. it is safe to delete the secondary and deploy another VSM and add it to the HA pair. And that is all you are doing, if you are following this guide.

  3. marcel says:

    Hi Jay,

    I just wanted to let you know how the second try went on december the 25th. I waited (untill now) because i wanted to post the solution or at least the reason for the partial config loss, however i am still talking to Cisco TAC about this.

    As expected the same thing occured the 25th again, but the partial config loss was now only the following:
    - the management address (strictly it was there, but the line “interface mgmt0″ was missing, so the management address was “floating” as TAC called it)
    - the “sub-group id” command was gone again on all the phisical interfaces from the VEM’s, however i was prepared for it and close to zero downtime this time:-)

    I made extensive logfiles and backups of startup-config and running-config files during all steps and sent them to the TAC engineers. So we are now still waiting for the final conclusion but our upgrade succeeded and we can now upgrade our vSphere to version 5 and implement the Cisco UCS and 1000v best practises.

    Thank you for sharing your thoughts with me, sticking your neck out and linking me to a TAC support engineer!

    Marcel

    • marcel says:

      Hi Jay,

      Last week i was informed by Cisco TAC that after multiple attempts they were finally able to recreate the issue and is now filed as a software defect. Cisco engineering is contacted to debug the faulty setup and to investigate the root cause.

      I just wanted to let you know the current status.

      Marcel

      • Jay says:

        Hi Marcel,

        Thanks for the update. I did follow up with the team and they have already identified the root cause the the fix should be available in the next update release (4b) release.

Leave a Reply

Looking for something?

Use the form below to search the site:

Still not finding what you're looking for? Drop a comment on a post or contact us so we can take care of it!

Categories