ELX-HealthMonitor
What a Linux server is doing, and what it did last week
A monitoring agent with a web interface, history in SQLite and e-mail alerts. One executable with no dependencies: the front end is inside the binary and the database creates itself. On a four-year-old Celeron it costs about four per cent of one core and eleven megabytes of memory.
Read from the kernel, not from a guess
Everything comes from /proc, /sys and smartctl directly. There is no collector to install, no time-series database to run and no agent-plus-server pair to keep in step.
Disks before they fail
SMART health, temperature, power-on hours, reallocated and pending sectors, CRC errors, SSD wear and total bytes written — with the full attribute table when you want to look closer.
RAID, including the honest part
Array state, level, members, failed disks and resync progress — plus the write-intent bitmap, which says whether the copies are actually identical right now, and the mismatch count from the last check.
Services with their own I/O
Every systemd unit with its processor, memory, disk reads and writes, task count and restarts — and buttons to start, stop and change whether it comes up at boot.
Alerts that do not cry wolf
Rules carry a hold-down: a ten-second spike to full load sends nothing, ten minutes at that level sends mail. That single setting is the difference between alerts you read and alerts you filter away.
History that does not grow without bound
Three days at full resolution, then five-minute buckets keeping the average, minimum and maximum, and nothing older than a month. Peaks survive the thinning; gigabytes do not accumulate.
What is measured
| Section | Readings |
|---|---|
| Processor | Total and per-core load, user/system/iowait/steal, frequency, load average, context switches, run queue |
| Memory | Used, available, cache, buffers, slab, dirty pages, swap and how hard it is being used |
| Disks | Read and write throughput, IOPS, latency, device utilisation, model and capacity |
| Filesystems | Used, free, percentage and inodes |
| SMART | Health, temperature, power-on hours, reallocated and pending sectors, CRC errors, SSD wear, lifetime writes, full attribute table |
| RAID | md array state, level, members, failed disks, resync progress and speed, current mirror consistency and the last check's mismatch count |
| Network | Bytes and packets in and out, errors and drops, link state, port speed, addresses, MTU |
| Temperature | Every hwmon sensor and ACPI thermal zone, plus drive temperatures from SMART |
| Services | systemd unit state and start-up setting, processor, memory, per-service disk I/O, task count, restarts |
| Processes | Top by processor and by memory, disk activity, owner, unit |
Each section can be switched off on its own, and a subsystem that is switched off is not polled at all.
The RAID question worth asking
A mirrored array holds the same data on both disks, but writes do not reach them at the same instant. The kernel marks the regions that might differ in a write-intent bitmap, and this agent shows that bitmap directly: while the dirty count is zero the copies are identical, and anything above zero means resynchronisation is still in progress.
Beside it is the mismatch count from the last integrity check. A non-zero value means a drive or controller has returned different data from the two halves of a mirror — which is exactly the failure a monitoring system exists to catch, and exactly the one a green “array is healthy” badge hides.
What it costs to run
Expensive readings are on their own schedule, separate from the main loop, because waking five disks every three seconds to ask their temperature would be worse than not monitoring them at all.
| Setting | Default | What it governs |
|---|---|---|
interval_sec | 3 s | The main polling loop |
services.interval_sec | 15 s | A full pass over systemd units |
disk.smart_interval_sec | 900 s | SMART polling — this one wakes the disks |
storage.persist_interval_sec | 30 s | Writing a snapshot into the history |
Measured on a four-core Celeron J1900 with 165 systemd units and five disks: about four per cent of one core and eleven megabytes of memory. On a Xeon E5-2690, under three per cent of a core.
The interface
- The browser remembers your choices: the period on each chart, curves you hid by clicking the legend, and the filters on the service and event tables. Reloading resets none of it.
- Every curve grows upward from zero, the unit on the axis is chosen from the range, and the number of decimals from the grid step — so two adjacent gridlines never carry the same label.
- After an upgrade an open tab notices the version has changed, says so and reloads, instead of leaving an old interface in the browser cache.
- Reboot and shutdown buttons are in the header; if you do not want them there, one configuration key removes them.
- Every irreversible action is behind a confirmation that explains the consequence, with the button labelled by the action rather than by “OK”.
At a glance
- Operating systems
- Debian and Ubuntu with systemd
- Architectures
- amd64, arm64, armhf
- Web interface
- Port 8555 by default
- History
- SQLite; 3 days raw, 31 days rolled up
- Optional dependency
- smartmontools, for the SMART section
- Typical cost
- ~4% of one core, ~11 MB of memory
- Alerts
- E-mail over SMTPS, STARTTLS or plain
- Configuration
- Web interface, or config.json