Jack Wallen, Contributing Writer for ZDnet explains he has several Linux systems connected to his home lab; some of them are desktops, and some of them are servers. Ninety-nine percent of the time, those machines work flawlessly. When that 1% happens, any machine that goes south needs help.
One way of helping is via a small software package called Watchdog. This piece of software runs various checks to see if the hardware has "locked up." If it detects that it has happened, it will reboot the machine.
There are two types of Watchdogs: software and hardware. The hardware Watchdog is much more reliable, but it requires specialty hardware for it to work. The software Watchdog isn't quite as reliable, but it works on most Linux systems.[1]
How Watchdog works:
- A kernel module (softdog), in conjunction with the Watchdog service, watches the system with a countdown timer.
- A virtual device is created (/dev/watchdog).
- If the virtual device is "kicked" by a process, the timer resets.
- If the virtual device isn't "kicked" by a process, Watchdog reboots the system.
- It's simple in theory, but the underpinnings are much more complicated. Fortunately, as a user, you don't have to dig too deeply to get the gist of Watchdog.
Although Watchdog can be essential for servers (especially those that don't have a monitor, keyboard, or mouse connected), it can also be useful for desktops. For example, say you need to log into a Linux machine on your home network from work. If that machine locks up, you won't be able to access it. If that machine has Watchdog keeping tabs on it, it'll reboot, and you'll be able to access it.
This can be very handy. You might think Watchdog is hard to set up, but you'll be surprised that it's not that much of a challenge, even if you're just starting out with Linux.
Let me show you how it's done. How to install Watchdog - What you'll need: Jack says going to demonstrate this is on a machine running Ubuntu 24.04. Watchdog is found in the standard Ubuntu repositories (as well as the Fedora standard repositories). For Arch users, you have to use yay to install this software. You'll also need a user with sudo privileges.
- Install Watchdog
The first thing to do is install Watchdog, which can be done with the command:
sudo apt-get install watchdog -y
If you're using a Fedora-based machine, the command is:
sudo dnf install watchdog -y
For Arch, it's:
yay -S watchdog
- Load the kernel module
With Watchdog installed, you then have to load the softdog kernel module, which is done with:
sudo modprobe softdog
Verify the module has loaded with:
lsmod | grep softdog
If you see softdog listed, then it's successfully loaded.
Also: The best Linux laptops in 2026: Expert tested for students, hobbyists, and pros
Check to make sure the device node exists with:
ls -la /dev/watchdog
- Load the module at boot
You'll also need to have the Watchdog kernel module loaded at boot. If you don't do this, the service won't be running after a reboot (so it won't be watching the system). This is done with:
Make sure to type this command correctly. You're now ready to configure Watchdog.
How to configure Watchdog - With Watchdog running, you'll want to make sure the configuration file is set up such that it'll actually do what it's supposed to do when it should. This is done by way of a configuration file. Open that file with the command:
sudo nano /etc/watchdog.conf
In that file, look for the following lines (they are not found consecutively in the file):
- # watchdog-device = /dev/watchdog
- # interval = 1
- # watchdog-timeout = 20 # Time in seconds before reboot
- # realtime = yes
- # priority = 1
- # max-load-1 = 24
- # max-load-5 = 18
- # max-load-15 = 12
- # min-memory = 1
What you need to do is remove the # and the space before each line. Note: If you don't see the watchdog-timeout = 20 line, manually add it.
Save and close the file.
You'll then need to start and enable the service with the command:
- sudo systemctl enable --now watchdog
- Watchdog is now running in the background and will do its thing, should something go awry.
If you want to test whether or not Watchdog is working, you can manually cause a kernel panic with the following three commands:
- sudo sysctl -w kernel.sysrq=1
- sudo su -
- echo c > /proc/sysrq-trigger
The system will become unresponsive, and Watchdog should reboot it.
The hardware method - If you happen to have a hardware watchdog, systemd can be configured to kick it and enact a reboot. Here's how you configure this.
Open the systemd config file with:
- sudo nano /etc/systemd/system.conf
Locate the following lines:
- #RuntimeWatchdogSec=0
- #RebootWatchdogSec=10min
- #WatchdogDevice=
Change those lines to:
- RuntimeWatchdogSec=30
- RebootWatchdogSec=10min
- WatchdogDevice=/dev/watchdog
Save and close the file.
Restart the systemd daemon with:
- sudo systemctl daemon-reload
And there you have it. You now have a service watching your system and will reboot it, should things go south.
This article is shared at no charge for educational and informational purposes only.
Red Sky Alliance is a Cyber Threat Analysis and Intelligence Service organization. We provide indicators of compromise information (CTI) via a notification/Tier I analysis service (RedXray) or an analysis service (CTAC). For questions, comments or assistance, please contact the office directly at 1-844-492-7225, or feedback@redskyalliance.com
Weekly Cyber Intelligence Briefings:
- Reporting: https://www.redskyalliance.org/
- Website: https://www.redskyalliance.com/
- LinkedIn: https://www.linkedin.com/company/64265941
Weekly Cyber Intelligence Briefings:
REDSHORTS - Weekly Cyber Intelligence Briefings
https://register.gotowebinar.com/register/5207428251321676122
[1] https://www.zdnet.com/article/how-linux-watchdog-reboots-my-frozen-system/
Comments