Download 86A193FF03-High performance clustering-2011-10

– fabric*.errors - Record the location of the problem and see “Diagnosing link errors” on page 210
– chassis*.errors - Record the location of the problem and see “Table of symptoms” on page 187.
– *.diff – indicates that there is a difference from the baseline to the latest health check run. See
“Interpreting health check .diff files” on page 172.
While the following information is intended to be comprehensive in describing how to interpret the
health check results, for the latest information about health check see the Fast Fabric Users Guide.
When any of the health check tools are run, the overall success or failure is indicated in the output of the
tool and its exit status. The tool indicates which areas had problems and which files must be reviewed.
The results from the latest run can be found in $FF_ANALYSIS_DIR/latest/. Many files can be found in
this directory which indicate both the latest configuration of the fabric and errors or differences found
during the health check. Should the health check fail, the following paragraphs decide an order for
reviewing these files.
If the -s option (save history) was used when running the health check, a directory would be crated with
the date and time of the failing run as the name. The directory would be created under FF_ANALYSIS_DIR,
in which case, that directory can be consulted instead of the latest directory shown in the following
examples.
First review the results for any esm (if using embedded subnet managers) or hostsm (if using host-based
subnet managers) health check failures. If the SM is misconfigured, or not running, it can cause other
health checks to fail. In which case the SM problems must be corrected first then the health check must
be rerun and other problems must then be reviewed and corrected as needed.
For a hostsm analysis, the files must be reviewed in the following order.
latest/hostsm.smstatus - ensure that this file indicates the SM is running. If no SMs are running on
the fabric, that problem must be corrected before proceeding further. After being corrected, the health
checks must be rerun to look for further errors.
latest/hostsm.smver.[changes|diff] - This file indicates the SM version has changed. If this was not
an expected change, the SM must be corrected before proceeding further. After being corrected, rerun
the health checks to look for further errors. If the change was expected and permanent, rerun a
baseline when all other health check errors have been corrected.
latest/hostsm.smconfig.diff - This file indicates that the SM configuration has changed. Review the
file and, as necessary compare thelatest/hostsm.smconfig file with baseline/hostsm.smconfig. As
necessary, correct the SM configuration. After being corrected, rerun the health checks to look for
further errors. If the change was expected and permanent, a rerun a baseline when all other health
check errors have been corrected.
For an esm analysis, the FF_ESM_CMDS configuration setting selects which ESM commands are used for
the analysis. When using the default setting for this parameter, the review the files in the following order.
latest/esm.smstatus - ensure that this file indicates the SM is running. If no SMs are running on the
fabric, correct the problem before proceeding further. After being corrected, rerun the health checks to
look for further errors.
latest/esm.smShowSMParms.[changes|diff] - This file indicates that the SM configuration has
changed. Review the file and, as necessary compare the latest/esm.smShowSMParms file with
baseline/esm.smShowSMParms. As necessary, correct the SM configuration. After being corrected, rerun
the health checks to look for further errors. If the change was expected and permanent, rerun a
baseline when all other health check errors have been corrected.
latest/esm.smShowDefBcGroup.[changes|diff] - This file indicates that the SM broadcast group for
IPoIB configuration has changed. Review the file and, as necessary, compare the latest/
esm.smShowDefBcGroup file with baseline/esm.smShowDefBcGroup. As necessary, correct the SM
configuration. After being corrected, rerun the health checks to look for further errors. If the change
was expected and permanent, rerun a baseline when all other health check errors have been corrected.
High-performance computing clusters using InfiniBand hardware
165