Generate custom profile using btf2json

How to use btf2json to generate a kernel profile for Volatility 3, without using a virtual machine and entirely within WSL.

My previous guides (here and here) have used dwarf2json, fresh virtual machines, and full debugging kernels. This is a somewhat time intensive process and doesn't scale that well - you have to generate multiple fresh virtual machines (depending on flavour/variant), install multiple debugging profiles from slow Canonical debugging repositories, to generate multiple profiles. This assumes you have access to an environment you can (or are allowed to) generate new virtual machines, and doing this in an enterprise environments (with considerations like TLS/SSL inspection, EDR registration, host management, etc etc) becomes an incredibly long winded process.

A few days ago, I saw an announcement from Volatility regarding their plugin contest and btf2json caught my eye.

"Valentin Obst: btf2json

The btf2json project is a very promising effort to ease the burden of large-scale Linux memory analysis. By incorporating information in the readily available vmlinuz file, analysts can create Volatility 3 symbol tables without the need for a full debug kernel. Through acquisition techniques that incorporate filesystem data, it appears that the final version of this project will enable analysis with only information stored within the memory sample – a large shift from the currently difficult method to gather this information across Linux systems and distributions.

The tool is great and it works as expected, however the guide/instructions on GitHub are perhaps a little unclear, so I thought I'd write a guide here on how to practically generate a custom profile using btf2json and without access to a virtual machine.

At a high level, this is the process;

Obtain/generate memory sample (DumpIt for Linux, AVML, etc)
Identify full kernel banner (this is important, don't skip this step)
Identify & obtain kernel image and linux modules from relevant repository.
Extract kernel image from package
Decompress kernel image (decompress vmlinuz to vmlinux)
Extract system map from linux modules package
Bring it all together with btf2json
Modify volatility3 schema
Test profile

Let's get started.

Obtain/generate memory sample (DumpIt for Linux, AVML, etc)

For this step, we'll use a publicly available sample. 13Cubed recently released an analysis challenge on YouTube which involved examination of a memory sample from an Ubuntu machine. The sample is available to download here.

Identify full kernel banner (this is important, don't skip this step)

We can use the volatility3 plugin 'banners' to identify banner information. Be mindful that volatility sometimes outputs raw/escape characters in its output. Although the output below is the full value, make sure you remove any prepended/trailing values (like \b for backspace) are removed. Depending on how messy the output is, this may require some trial and error.

You'll see that this is important later as btf2json requires you to input this value so it can generate a compatible profile. The variable you provide is going to be converted/encoded in base64, and stored in the profile (as I covered here)

python3 vol.py -v -f /mnt/c/13cubed/memory/memory.vmem banners

Linux version 6.5.0-41-generic (buildd@lcy02-amd64-120) (x86_64-linux-gnu-gcc-12 (Ubuntu 12.3.0-1ubuntu1~22.04) 12.3.0, GNU ld (GNU Binutils for Ubuntu) 2.38) #41~22.04.2-Ubuntu SMP PREEMPT_DYNAMIC Mon Jun  3 11:32:55 UTC 2 (Ubuntu 6.5.0-41.41~22.04.2-generic 6.5.13)

Identify & obtain kernel image and Linux modules from relevant repository.

From the above, we can see we have an Ubuntu host with 6.5.0-41-generic.

A quick search gives us a download location; https://launchpad.net/ubuntu/mantic/amd64/linux-image-6.5.0-41-generic/6.5.0-41.41 and a direct link; http://launchpadlibrarian.net/731521331/linux-image-6.5.0-41-generic_6.5.0-41.41_amd64.deb

Extract kernel image from package

We have the kernel package, but it's inside a .deb file (which is an Ubuntu package). We don't want to install it, we want to extract a file from within it. We want to extract vmlinuz-6.5.0-41-generic, but how can we do that? Packages contain data streams and controls streams. If we were to use 7zip (as shown below), you'd be presented with 2 additional zst files (these are compressed with zstd and can't be uncompressed with 7zip)

$ 7z l linux-image-6.5.0-41-generic_6.5.0-41.41_amd64.deb

Scanning the drive for archives:
1 file, 14096990 bytes (14 MiB)

Listing archive: linux-image-6.5.0-41-generic_6.5.0-41.41_amd64.deb

--
Path = linux-image-6.5.0-41-generic_6.5.0-41.41_amd64.deb
Type = Ar
Physical Size = 14096990
SubType = deb

   Date      Time    Attr         Size   Compressed  Name
------------------- ----- ------------ ------------  ------------------------
2024-05-21 01:11:55 .....         2172         2172  control.tar.zst
2024-05-21 01:11:55 .....     14094625     14094625  data.tar.zst
------------------- ----- ------------ ------------  ------------------------
2024-05-21 01:11:55           14096797     14096797  2 files

We can use the dpkg -c command to identify which files we want.

$ dpkg -c linux-image-6.5.0-41-generic_6.5.0-41.41_amd64.deb
drwxr-xr-x root/root         0 2024-05-21 00:11 ./
drwxr-xr-x root/root         0 2024-05-21 00:11 ./boot/
-rw------- root/root  14327304 2024-05-21 00:11 ./boot/vmlinuz-6.5.0-41-generic
[snip]

We can then use dpkg with the fsystarfile option to inspect and ultimately extract the file.

This is the syntax;

dpkg --fsys-tarfile your-downloaded-package.deb | tar xOf - ./path/to/file/inside > dest.file

Using the package above, this would result in the following command (one liner, code block is wrapped)

dpkg --fsys-tarfile linux-image-6.5.0-41-generic_6.5.0-41.41_amd64.deb| tar xOf - ./boot/vmlinuz-6.5.0-41-generic > vmlinuz-6.5.0-41-generic

Decompress kernel image (decompress vmlinuz to vmlinux)

btf2json doesn't handle compressed linux kernels/images. vmlinuz is the compressed version of vmlinux. The bash script used to extract/decompress the compressed image should be included in your kernel tree (usually /usr/src/kernels/your-kernel-version/scripts/extract-vmlinux). If not, you can download it from GitHub.

Decompress vmlinuz into vmlinux

./extract-vmlinux.sh vmlinuz-6.5.0-41-generic > vmlinux

Extract system map from linux modules package

Download our corresponding kernel modules package

$ wget http://archive.ubuntu.com/ubuntu/pool/main/l/linux-hwe-6.5/linux-modules-6.5.0-41-generic_6.5.0-41.41~22.04.2_amd64.deb

Perform the same steps as 4, just targeting the System.Map file

$ dpkg -c linux-modules-6.5.0-41-generic_6.5.0-41.41~22.04.2_amd64.deb
drwxr-xr-x root/root         0 2024-06-03 19:39 ./
drwxr-xr-x root/root         0 2024-06-03 19:39 ./boot/
-rw------- root/root   8268513 2024-06-03 19:39 ./boot/System.map-6.5.0-41-generic
-rw-r--r-- root/root    280657 2024-06-03 19:39 ./boot/config-6.5.0-41-generic
drwxr-xr-x root/root         0 2024-06-03 19:39 ./lib/
[snip]

Extract System.map-6.5.0-41-generic (one liner, code block is wrapped)

dpkg --fsys-tarfile linux-modules-6.5.0-41-generic_6.5.0-41.41~22.04.2_amd64.deb| tar xOf - ./boot/System.map-6.5.0-41-generic > System.map-6.5.0-41-generic

Bring it all together with btf2json

In the same directory, download btf2json

$ wget https://github.com/vobst/btf2json/releases/download/v0.1.0/btf2json

Remember how we identified the banners in step 1? You'll need the full value here.

Linux version 6.5.0-41-generic (buildd@lcy02-amd64-120) (x86_64-linux-gnu-gcc-12 (Ubuntu 12.3.0-1ubuntu1~22.04) 12.3.0, GNU ld (GNU Binutils for Ubuntu) 2.38) #41~22.04.2-Ubuntu SMP PREEMPT_DYNAMIC Mon Jun  3 11:32:55 UTC 2 (Ubuntu 6.5.0-41.41~22.04.2-generic 6.5.13)

The final command;

./btf2json --btf /path/to/vmlinux --map /path/to/System.map-6.5.0-41-generic --banner "Linux version 6.5.0-41-generic (buildd@lcy02-amd64-120) (x86_64-linux-gnu-gcc-12 (Ubuntu 12.3.0-1ubuntu1~22.04) 12.3.0, GNU ld (GNU Binutils for Ubuntu) 2.38) #41~22.04.2-Ubuntu SMP PREEMPT_DYNAMIC Mon Jun  3 11:32:55 UTC 2 (Ubuntu 6.5.0-41.41~22.04.2-generic 6.5.13)" > 13cubed-btfs2json.json

You can see we end up with the profile "13cubed-btfs2json.json" which you'll need to place in volatility3 symbols folder.

Modify volatility3 schema

In \volatility3-2.5.21\volatility3\schemas you'll see a file named 'schema-6.2.0.json'. Open that in your favourite text editor, and follow the instructions here.

-          "pattern": "^(dwarf|symtab|system-map)$"
+          "pattern": "^(btf|symdb|dwarf|symtab|system-map)$"

Test Profile

Move your new profile (13cubed-btf2json.json) to your symbols folder.

Check it's registered.

$ mv 13cubed-btf2json.json /path/to/volatility3/symbols
$ python3 vol.py isfinfo

URI     Valid   Number of base_types    Number of types Number of symbols       Number of enums Identifying information

file:///mnt/c/volatility3-2.5.21/volatility3/symbols/13cubed-btf2json.json        True (cached)   19      13317   179428  2441    b'Linux version 6.5.0-41-generic (buildd@lcy02-amd64-120) (x86_64-linux-gnu-gcc-12 (Ubuntu 12.3.0-1ubuntu1~22.04) 12.3.0, GNU ld (GNU Binutils for Ubuntu) 2.38) #41~22.04.2-Ubuntu SMP PREEMPT_DYNAMIC Mon Jun  3 11:32:55 UTC 2 (Ubuntu 6.5.0-41.41~22.04.2-generic 6.5.13)'

Test the profile

$ python3 vol.py -v -f /mnt/c/13cubed/memory/memory.vmem linux.pslist

OFFSET (V)      PID     TID     PPID    COMM    File output

0x9ee0c027b300  1       1       0       systemd Disabled
0x9ee0c0279980  2       2       0       kthreadd        Disabled
0x9ee0c027e600  3       3       2       rcu_gp  Disabled
0x9ee0c0278000  4       4       2       rcu_par_gp      Disabled
0x9ee0c027cc80  5       5       2       slub_flushwq    Disabled
0x9ee0c028cc80  6       6       2       netns   Disabled
0x9ee0c0333300  11      11      2       mm_percpu_wq    Disabled
0x9ee0c0331980  12      12      2       rcu_tasks_kthre Disabled
0x9ee0c0336600  13      13      2       rcu_tasks_rude_ Disabled

[snip]

PreviousBuild Custom Linux Profile for Volatility NextBanners, isfinfo, and custom profiles

Last updated 3 months ago

Was this helpful?