Prepare Lenovo Flex Enterprise Chassis From the Scratch

This step by step guide will help you to configure IBM Flex System Manager (FSM) from the scratch. If anything about this tutorial becomes out of date we will do our best to update accordingly. If issues arise that this article cannot address please post your thoughts or question at the end of this article under comment box section and we will definitely try to address your issue.

STEP1- Flex System Manager Initial Setup

Please follow the below steps to setup your Flex Systems Manager for first time use. IBM has made this very basic but follow carefully as there are important pieces.

PREREQUISITES:

Ensure you have forward AND reverse lookup zones on your DNS server for the FSM hostname.

After you have setup to Chassis Management module then go to the IMM address of you Flex System Manager node.
From here click ‘remote control’ then launch the console in multi-user mode (this is important in case for some reason you get locked out of you session you will not be able to launch another)
Power on the FSM
The first step is to chose your chassis network configuration (best practice here is to use separate networks for both internal and external management of the chassis)

The next step is to choose your DNS configuration. The FSM should have already found your DNS servers but if it has not then enter them.

The next step is to configure the Chassis internal network. You will choose ONLY IPv6 for the internal management network (Eth0). Copy and paste the IPV6 address into the field and enter 64 for the prefix length. Make sure there is no space after the paste and then click add.

Next you will assign an IPv4 address to the external management port of ETH1.

Next choose your hostname (make sure this is in your DNS server) You will not need a IPv6 address unless you are using this in your core network.

Add your DNS servers if they have not already been discovered.

DO NOT CHECK PERFORM NETWORK VALIDATION.Verify your settings and click finish. Allow at least 1-2 minutes for the setup to complete.Then the FSM will startup for the first time but will need to reboot itself first. Allow this to run for up to 30 minutes.

Setup will run for upwards of 30 minutes. Next you will need to check for updated FSM code before you manage the chassis (see below)

STEP2 - Managing your Chassis

To manage your chassis within the FSM perform the following

Before we manage the chassis be sure there is a pingable IPv6 address between the CMM and Eth0 of the FSM. Easiest way to do this is to go the network configuration of the FSM and copy the first 4 octets from ETH0 and then assign a static IPv6 address to the CMM and use the last 4 octects of the CMMs self assigned IPv6 address. (you will find this under Chassis management module management ->network->network->IPv6) This will put the CMM and ETH0 of the FSM onto the same IPv6 subnet.

To verify this was done correctly ssh to the FSM and enter ‘ping6 ’

From the home tab click select chassis to manage

From the next page click discover new chassis

Once you see success click close and check the box next to the newly discovered chassis and click ‘Manage’ . The status column should show this chassis as ‘unmanaged’. Also be sure to check both check boxes, the second box to set IPv6 addresses on the chassis components.

You will then be asked for the credentials. These will be the credentials of the CMM. It will also ask you to set a recovery password for the CMM, it is CRITICAL that this is saved in the event that the FSM is lost and cannot authenticate the CMM using LDAP. Once managed you will want to run inventory.

To run inventory on your new chassis perform the following.

Right click on your newly discovered chassis and highlight ‘Inventory’ then click ‘Collect Inventory’.
Click ‘Run now’ when the job pops up.

Note: I like to display the properties of the job so I can monitor the progress and see if there were errors.

STEP3 - Updating the FSM

There are two ways to update your FSM. First I will outline updating via the FSM (Flex System Manager) GUI.

Make sure your CMM is at the latest firmware available BEFORE updating the FSM code (In some cases if the CMM is too far back leveled you could lose management of the chassis by upgrading the FSM too far ahead. The simplest way to check the CMM firmware status is make sure compliance is running on the chassis for ‘All systemx and blade-center updates’ and then run Inventory on the CMM. If an update is available it will show as a compliance warning and it is easily updated from the FSM GUI.

https://www-304.ibm.com/software/brandcatalog/puresystems/centre/update?uid=S_PUREFLEX

The above site will also give you information about the newest FSM updates and the compatible firmware for all of the available nodes INCLUDING the CMM.

Click here to download v1.30 Best Practices Doc

On the home tab click check and update Flex System Manager

Allow the search to find the FSM updates

If there are new updates check the box and click install.
walk through the wizard and at the end launch the job.

Now log back in and run inventory on ALL SYSTEMS. This could take quite a long time depending on how many MEP (managed end points) you have.
Lastly check for FSM updates as shown in Step #1 above. If it says you’re up to date then you have successfully upgraded your FSM!

Note: If the update fails the first time, reboot the FSM, and then try the update again from the GUI.

Note: If you'd like to check the progress of the upgrade by going to the console of the FSM. To do this go to the IMM of the FSM and launch the remote control in multi user mode to watch the upgrade run.

STEP4 - Setup FSM Backup and Restore

There are 3 different ways to backup your FSM. I will quickly outline all 3.

Backing up locally - this method is not recommended because the file-space gets locked after the backup and cannot be removed to an off site location. So if you lose the FSM appliance you also lose your backup.
Backup to External USB- This method will allow you to backup to an external hard drive that is connected to the front of the FSM via the external USB ports. Also please note that the supported Filesystem formats are ext3, ext4, and vfat.
Backup over SFTP - This is the IBM best practice method for backing up your FSM.

Note - The backup size can vary but is upwards of 30 GB. Ensure which ever backup method you use will have adequate space to hold the backup.

Go to the Home tab -> Administration
Scroll down and under Serviceability Tasks -> click ‘Backup and Restore’
Click ‘Backup now’ or you can schedule your backups
Pick your backup method and allow the job to run.

Restoring the FSM

VIA USB
1. Insert a USB device into the USB port on the management node. The USB device is mounted automatically.
2. Open a CLI prompt.
3. Use the restore -l usb command to restore the image from the USB drive.
VIA SFTP
1. Insert a USB device into the USB port on the management node. The USB device is mounted automatically.
2. Open a CLI prompt
3. Use the restore -l usb command to restore the image from the USB drive.
VIA LOCAL
1. Open a CLI prompt.
2. Use the restoreHDD file name command, where file name is the name of the backup file, to restore the image from the hard disk drive.

Note - If you completely lose access to the management node then you may need to reinstall from the recovery partition. Here are the steps to do so.

About this task

This procedure returns the management node and management software to factory defaults, and destroys data on the system (but not backups stored on the hard disk drive). After the recovery process is complete, you must configure the system with the Management Server Setup wizard or restore the system from a backup image.

Procedure

Restart the management node.
When the firmware splash screen is displayed, press F12. The setup menu is displayed. The screen displays confirmation that F12 has been pressed.
Select Recovery partition.
When the boot options screen is displayed, select Full system recovery. After approximately 30 minutes, the recovery process ends and the Management Server Setup wizard opens.
Either complete the Management Server Setup wizard or restore a previous configuration from a backup.

STEP5 - Updating Managed End Points using FSM

There are 2 simple steps to updating managed end points (MEPs)

Acquire updates
Show and install updates

Acquiring Updates -

First in the left pane go to ‘Release Management’ then ‘Updates’

Scroll down and click ‘Acquire Updates'
From the screen above you can pick which specific updates you want to search for.
Select and add your updates
Run the job and watch the logs to see new updates being pulled in.

The next step is Show and Install the updates that you found in the previous steps.

From the Updates menu scroll down and choose ‘Show and Install’ Updates

Browse to choose the system you wish to check for updates for as seen below
Check the update you wish to install and click ‘Install’
There will be a brief wizard making sure that the system you wish to install this is correct and possibly is you would like an auto restart of the system if one is needed.
Allow the job to run and check the log to make sure it ran properly

STEP6 - Compliance Policy Configuration

In order for this to work properly you will need to setup a job to run inventory on the system you wish to set a compliance policy before the compliance check runs (typically once a week will suffice). Use the following directions to set up compliance policies on critical systems.

First go to Release Management -> Upgrades
Next click Change compliance polices (as seen above) From here click browse to pick your system.
Navigate and find your system (easiest way is to pick the All Systems group and search for you system)

Next click Show Compliance Policies (this will allow you to add a compliance policies and also if there is another compliance policy in place)
Now click Add and choose the update that applies (if you are unsure choose ALL UPDATES)
Now click Save.

If you have an inventory job setup and the compliance was set proper you should see the following once compliance is configured.

STEP7 - Configuring Monitors and Thresholds

For this article I will be using a group that represents my ESXi Cluster.

From the FSM Explorer View go to Monitors -> Monitors. (if you are using FSM code 1.2.1 this will jump you to the old interface)

From the new window first choose ‘browse’ and pick the system you want to configure monitors and thresholds.

Next choose your monitor group and click ‘Show Monitors’

Next click the Monitor you wish to activate and configure.

Next you will configure your thresholds.

Now you should see your new Monitor activated and reading data.

Troubleshooting the Flex System Manager

STEP1 - Unlocking User Accounts

Sometimes the FSM administrator account could lock itself out after 20 failed login attempts. To unlock an account do the following

SSH in the FSM using the pe (product engineering) account. (This password should be the same as the USERID password)
Run smcli unlockuser -u USERID
Now try to login again.

Note: If the USERID account keeps getting locked out please open a ticket with IBM Support to correct this issue.

Reset User Password

NOTE: You CAN NOT use a previous password this will cause an error to be displayed.

SSH in to the FSM
Run the following command. smcli chuserpwd [-v] {-u user_name}{-o existing_password}{-p new_password}

Note: if you receive the following error please open a ticket with IBM support.

STEP2 - Restart FSM (Software)

If at an time the FSM GUI becomes buggy or you have lost the tab on the left side of the page to open the contents tree then the simplest way to troubleshoot this is by a software restart of the FSM server. To do this perform the following.

Log into the FSM via SSH
Enter the command ‘smstop‘
Once this returns enter ‘smstart‘
To watch the status of the software restart enter ‘smstatus -r‘ (the -r will refresh anytime a change in status is made).
When it says ‘Active’ this means the FSM is back up, however, it may still take at least 5 minutes for the full features to be operational.

STEP3 - Restart FSM (Hardware)

There may be times where support asks you to reboot the FSM after possibly making some changes to files after given root access. The easiest way without fumbling through the GUI is to do a hardware restart from the command line. Please follow below to accomplish this.

First log into the FSM.
Run ‘smshutdown -t now -r (Below are the flag definitions)

STEP4 - Recovering the Flex Systems Manager Node from Base Media

NOTE: this recovery would only be needed in SERIOUS situations, for example, after the replacement of the mother board or some other significant hardware failure.

NOTE 2: If you have any issues during this process getting the recovery to run, then try to reformat the RAID array by deleting and recreating the array in the LSI configuration utility.

Things you need:

Latest recovery media on a DVD
An external DVD drive (preferably an IBM supported as seen in the link below)

First download the base media packages from..https://www14.software.ibm.com/webapp/iwm/web/pick.do?source=swg-flrecov
Once downloaded open a console to your FSM from the IMM remote console shown below.

You may need to reactivate your RAID arrays, if you do here are the steps to do so, if not, move on to next step.

Press cntrl+c to enter the LSI configuration utility during the legacy initialization boot up sequence.
Verify your RAID arrays are online and optimal by going to RAID properties -> View Existing Volumes ->Manage Volumes->Activate Volume. This should bring your RAID array back online.

Next go to SAS topology and verify your ATA drive is the alternate boot device.
Next attached the first recovery ISO to the external DVD tray.
Reboot the FSM and during the UEFi splash screen press F12 to select the USB CD/DVD device.
Allow the IBM customized media to load and then press ’2′ to recover the management node and ’2′ to recover the complete system and press ‘y’ then wait for the recovery process to begin!

This process could take up to 3 hours.
Once completed reboot and at the UEFi splash screen presh ‘F12′ and select the recovery partition.
Select System Recovery and allow the recovery to run. Once complete refer to the FSM First Time Setup article above!

Please go through the link below if you need to configure IBM v7000 Storage.

IBM v7000 Storage Step-by-Step Configuration Guide