hunt systemd persistence hiding as a fake service

hunting systemd persistence hiding as a fake service

reboots used to kill malware. now they don't, because systemd is the thing keeping your linux box alive after every restart, and it's also the thing an attacker will happily abuse to keep their access alive too. if you can make systemd think your malicious script is just another boring service, it survives reboots, patches, even a lot of casual cleanup. that's exactly why it's one of the first places a defender should look during an incident or a routine health check.

why systemd is such a good hiding spot

systemd manages basically everything that starts on boot on most modern linux distros. it's normal to have dozens of enabled services. that's the problem for defenders and the opportunity for attackers, it's noisy by default. a new service named something like cleanup.service or network-helper.service just blends in with all the legit stuff unless you actually know what's supposed to be there.

step one, list everything that's enabled

systemctl list-unit-files --type=service --state=enabled

this pulls every service unit file on the system that's set to start automatically, not just what's currently running. that distinction matters, because a sneaky persistence mechanism might not be running right now but is absolutely sitting there ready to fire on next boot. you want the full enabled list, not just systemctl list-units, which only shows active stuff.

scroll through it with a simple question in mind, do i recognize this, and did a real package install it? legit services almost always map to something you installed on purpose, ssh, cron, docker, your web server. random generic names with no obvious vendor tie, especially ones that sound deliberately boring like "cleanup" or "update-checker," deserve a second look.

step two, read what it actually does

grep ExecStart /etc/systemd/system/cleanup.service

every service unit has an ExecStart line, it's literally the command systemd runs when the service starts. this is where the truth lives. a real service points to a real binary in a real path, something like /usr/bin/sshd. a malicious one might point to a base64-decoded one liner, a script sitting in /tmp, or a curl command piping straight into bash. if the ExecStart line looks like it's trying to hide what it's doing, that's your red flag right there.

also check the file's location. legit vendor services usually live under /usr/lib/systemd/system/. anything an attacker drops by hand tends to end up in /etc/systemd/system/, which is technically normal for local overrides, but it's also exactly where a persistence mechanism would get planted since it takes priority over the vendor version.

step three, let systemd grade its own risk

systemd-analyze security cleanup.service

this command is genuinely underused. it scores a unit file against a long list of hardening options, things like namespace isolation, capability restrictions, filesystem protections, and spits out a score from 0 to 10, with 10 being the least safe. a score like 9.6 paired with a big "UNSAFE" label doesn't automatically mean it's malicious, plenty of legit but lazily written services score badly too. but combine an unsafe score with a service you never installed, running with no sandboxing, no restricted permissions, full access to the filesystem, and you've got a very strong signal something's wrong.

putting it together

the workflow is simple and repeatable, list every enabled service, flag anything unfamiliar, read its ExecStart line to see what it's actually launching, then run the security analyzer to see how exposed it is. none of this requires special tooling, it's all built into systemd already. that's the beauty of it, the same feature that makes persistence so effective for an attacker also leaves a clean, auditable trail for you if you know where to look.

the takeaway

persistence isn't magic, it's just automation abused. systemd was built to make sure important things survive a reboot, and attackers know that, so they lean on it instead of reinventing the wheel. do a periodic pass on your enabled services, know your baseline so anything new stands out, and don't be afraid to actually read the unit files instead of just trusting the names. an afternoon spent auditing /etc/systemd/system/ is a lot cheaper than finding out six months later that "cleanup.service" was never cleaning anything.