Monitor Server Health and Hardware Status

Last updated 27 July 2026

Monitor Server Health and Hardware Status

Every Brainsalt server continuously monitors its own hardware health and reports a single rollup status: OK, Warning, or Critical. This page explains where to find health information, what each status level means, and how to respond to faults.

[USER-ACTION-REQUIRED: Attach a screenshot of the System Management Overview page showing the Health section with its per-area rows here]

Where health appears

Health is visible in two places:

  • System Management → Overview — the landing page of the server’s web configuration interface. Shows the overall health rollup banner and one row per monitored area (server hardware, display outputs, storage, hardware errors, hardware baseline). Each row has a colored dot and links to its detail page. The Overview auto-refreshes every 10 seconds.

  • Brainsalt Hub — the desktop client shows a per-server health badge in the server list, polled roughly every 2 seconds. You can see the overall health status at a glance across all your servers without opening each one individually.

What is monitored

The server reads health data from its onboard management controller (BMC) and from the operating system. Health reads are event-driven — triggered at startup, when the BMC logs a new event, or when you open a detail page — not polled on a timer.

The overall rollup is the worst status across all of the following areas:

Area

What is checked

Detail page

Server hardware

BMC sensors (temperatures, fans, voltages, current, power) and power supply units (PSUs). Each sensor has its own OK / Warning / Critical threshold defined by the BMC. Any power fault or a critical sensor immediately makes the overall status Critical.

Maintain → Hardware

Display outputs

GPU display-port link status and DisplayPort bandwidth. A no-link or exceeded-bandwidth output raises a Warning or Critical.

Maintain → GPU Status

Storage

Free space on the media drive. Warning at 99% used; Critical at 99.9% used. Below 99% the drive is considered OK (media drives routinely run near full).

Maintain → Storage

Hardware errors (WHEA)

Windows hardware errors (machine-check exceptions, corrected/uncorrected memory and PCIe errors). Any WHEA event immediately latches the server to Critical. This status persists across restarts and must be cleared manually by an Administrator.

Maintain → Hardware

Hardware baseline

Compares the live hardware inventory against a locked reference snapshot. A component that was in the baseline but is now missing raises Critical (potential failure or tampering). A component that is present now but was not in the baseline raises Warning. Only appears on the Overview if a baseline has been captured.

Configure → Hardware Baseline

Understanding Warning vs Critical

Status

Meaning

Action

OK

All monitored areas are within normal parameters.

No action required.

WARNING

A sensor is outside its normal range, the media drive is nearly full, or an unexpected hardware component has been detected. The server is still operational but should be investigated.

Check the Overview for the row showing Warning and follow the link to its detail page.

CRITICAL

A fault needs immediate attention. Possible causes: a power supply failure, a critical sensor threshold exceeded, a WHEA hardware error, a missing hardware component, or the media drive is essentially full.

Identify the affected area on the Overview, open the detail page, and resolve or escalate the fault.

How to read the Hardware page

Navigate to Maintain → Hardware in System Management. This page has multiple tabs:

  • Summary — BMC identity, firmware version, and a link to the BMC’s own web interface.

  • Sensors — live readings for temperatures, fans, voltages, power supplies, and other BMC-reported values. Each sensor shows a colored health dot.

  • Components — installed components (processors, memory DIMMs, PCIe devices) with health and serial numbers.

  • Inventory — detailed FRU (Field Replaceable Unit) data from the BMC, exportable as CSV.

  • Event log — the BMC’s System Event Log (SEL), recording historical hardware events.

[USER-ACTION-REQUIRED: Attach a screenshot of the Hardware page Sensors tab showing sensor readings with health dots here]

The Hardware page sources its data from the BMC via the Redfish interface, overlaid with IPMI readings. IPMI is authoritative for faults: if the BMC’s Redfish interface reports an empty health status but IPMI detects a fault, the fault is surfaced. A genuine fault is never downgraded.

How to clear a latched WHEA error

When the operating system reports a hardware error (WHEA — Windows Hardware Error Architecture), the server immediately latches to Critical. This status is persisted to disk and survives service and server restarts. It will remain Critical until an Administrator explicitly clears it.

Steps

  1. Sign in to System Management as an Administrator.

  2. Navigate to Maintain → Hardware.

  3. The Hardware errors (WHEA) panel at the top of the page shows the error count and the most recent error message.

  4. Click Clear hardware-error state.

  5. The health status returns to OK. The WHEA log file is kept for later review.

Clearing the error state does not fix the underlying hardware issue. WHEA events typically indicate memory, PCIe, or CPU faults. If errors recur after clearing, investigate the hardware — check the BMC event log and consider scheduling maintenance. The next WHEA event will re-arm the Critical state automatically.

Only Administrators can clear the WHEA error state. The button is not visible to Operator or User roles.

How the hardware baseline works

An Administrator can lock the current hardware inventory as a reference snapshot (the hardware baseline). Once a baseline is captured, the server continuously compares the live inventory against it:

  • A component in the baseline that is now missing → CRITICAL (possible failure or removal)

  • A component present now that was not in the baseline → WARNING (unexpected addition)

The baseline tracks BMC sensors, power supplies, and hardware components (including memory DIMMs and PCIe devices). If a component is swapped (e.g. a DIMM with a different serial number), it appears as one removed item and one added item.

The baseline comparison is stateless — if a removed component reappears, the alert clears automatically on the next health read. To update the baseline after intentional hardware changes, re-capture it from Configure → Hardware Baseline.

If a baseline is locked but the BMC inventory is temporarily unavailable (e.g. during a BMC reboot), the comparison cannot run. This shows as a Warning ("cannot verify") rather than a false Critical.

The front panel

On B9-series servers, the physical front panel reflects the same health data. The System Management Service reads BMC health, maps it to OK / Warning / Critical, and drives the panel’s error indicator and buzzer. On B8-series servers, the front panel reads error state directly from motherboard GPIO pins independent of software.

Common Mistakes

  • Server stays Critical after a restart — WHEA errors are latched and persisted. Restarting the server or the service does not clear them. An Administrator must click Clear hardware-error state on the Hardware page.

  • Baseline shows Critical immediately after hardware maintenance — If you replaced or removed a component, the hardware baseline sees it as missing. Re-capture the baseline from Configure → Hardware Baseline after completing the maintenance.

  • All sensors show grey dots on the Hardware page — The BMC’s Redfish interface may return empty health for some sensor types. If IPMI is functional, the server overlays the IPMI-sourced status automatically. If all dots are grey, the BMC may be unresponsive — check that the in-band IPMI driver is functioning.

Tips

  • Use Brainsalt Hub to monitor all your servers at once. The health badge in the server list updates every ~2 seconds, so you can spot a fault without opening each server’s System Management interface.

  • The Overview page links each health row directly to the relevant detail page. Click the link on any row showing Warning or Critical to jump straight to the fault.

  • The BMC event log (Hardware → Event log tab) keeps a historical record of hardware events. Review it to identify intermittent faults, even if the current health is OK.

  • Capture a hardware baseline after commissioning a server and after any planned hardware change. This turns the health system into a tamper-detection and failure-detection tool.

  • The storage threshold is intentionally high (99% for Warning, 99.9% for Critical). Media drives on production servers normally run near full. Do not be alarmed by high usage percentages below 99%.

How to Control Server Power — How to safely shut down, restart, or power-cycle a server.

Set a Hardware Baseline — how to capture and manage the hardware reference snapshot (page to be published).

Check GPU Output Link Status — how to read the GPU Status page and interpret DisplayPort bandwidth verdicts (page to be published).

How to Access the BMC KVM Console from Brainsalt Hub — how to access the BMC’s own KVM remote console from Brainsalt Hub.