Ivanti Connect Secure Auth Bypass and Remote Code Authentication CVE-2024-21887
This article provides guidance on how to inspect/analyse disk images/memory from a virtual Ivanti Connect Secure appliance, in response to CVE-2023-46085 and CVE-2024-21887.
On 10th January 2024, Volexity posted an article[1] advising they had identified active in-the-wild exploitation of two zero day vulnerabilities affecting Ivanti's Connect Secure (ICS) application.
On 12th January 2024, Mandiant also contributed to the public discussion and posted[2] their article.
Based on the above articles, we deployed a vulnerable Ivanti Connect Secure appliance (v22.3R1, build 1647) to test publicly available proof of concepts to understand where artefacts may reside in order to support forensic analysis.
This article does not reference indicators of compromise. Please check blog posts by Volexity, Mandiant, and Rapid7. Please also follow mitigation advice by Ivanti[7] and watchTowr[8]
This article assumes you have a virtual appliance, you have followed your incident response process, and have preserved system snapshots (including memory dumps), and disk images.
(If you're performing this on Hyper-V or ESXi, just create a snapshot and export resultant virtual disks and vmem and copy them to your analysis environment).
Inspecting Disk Image
A single VMDK was subject to analysis. Unaware of the filesystem, we simply initially inspected the disk using FTK Imager on a Windows host. This indicated (as you can see in the photo) there are a series of partitions of various sizes, with varying filesystems. I immediately felt there were similarities between the ICS application and the Citrix ADC application, that being it is likely based on BSD/FreeBSD, but this required further digging.
Expanding the 'unpartitioned space [LVM2]' node indicated physical volume (pv) groups. Each folder contains relevant metadata and LVM configuration data, such as volume group ID, format, underlying partitions (which physical volume/partition they relate to).
Upon inspecting groupA-runtme, groupZ-home, etc, we could see indications that LUKS was being used. This throws a spanner in the works as we don't know any of the protectors. (If you need a refresher or some test disk images for LVM and LUKS, head over here >ext4, LVM, and LUKS1/LUKS2
So we have a virtual disk, it contains multiple partitions, they're of an unknown filesystem, we need a key to decrypt the LUKS volume and then we can reassemble it.
A recent Blackhat[3] resource authored by Orange Tsai and Meh Chang came in rather handy. If Orange's name is familiar, have a look at the history behind ProxyShell. Anyway, that article suggests that if you append init=//bin/sh to the grub bootloader, you can spawn a shell during boot. Added that, F10 to boot with new options, and we have a new shell.
We know lvm/mdadm config is usually in /etc so this is the first place we look. We see /etc/lvmkey, so let's see what it contains. Executing $ cat lvmkey distorts the screen, throws characters everywhere, and makes the terminal unusable. There are limited command line tools so there's no real way to copy the key, other than using cat.
This Stackoverflow article[4] provides a decoding table for the above values. This converts to b99ecf89754ec76018ca0eda5d6ac7. Next step is to convert that to a key file so we can use it to decrypt our LUKS volume. If this were a simple password, you could attempt to mount the volume and enter the password[5]
Create a hex output of the key so we can use it as a keyfile;
... which turns out to be incorrect, and the resultant lvmkey is incorrect.
It was about this time that we were provided with Rapid7's technical analysis[6] which ultimately saved a lot of time, as we would've been going down a rabbit hole. The arguments originally used with cat (-v) wasn't sufficient, as there is a special character ($) at the beginning of the key. This is why there is a new line between the command and M-9. The correct key is 0ab99ecf89754ec76018ca0eda5d6ac7.
We need to activate each of the volume groups (groupA, S, Z) so we can map them (-a activates them, y is yes/confirm)
$ vgchange -ay groupA
2 logical volume(s) in volume group "groupA" now active
$ vgchange -ay groupS
1 logical volume(s) in volume group "groupS" now active
$ vgchange -ay groupZ
1 logical volume(s) in volume group "groupZ" now active
Now we can see that partitions within our block device (/dev/sdc) are identified as LVM parts
$ lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
sdc 8:32 0 40G 0 disk
├─sdc1 8:33 0 70.6M 0 part
├─sdc2 8:34 0 70.6M 0 part
├─sdc3 8:35 0 70.6M 0 part
├─sdc4 8:36 0 1K 0 part
├─sdc5 8:37 0 3G 0 part
│ └─groupZ-home 253:3 0 3G 0 lvm
├─sdc6 8:38 0 4G 0 part
│ ├─groupA-home 253:0 0 3.5G 0 lvm
│ └─groupA-runtime 253:1 0 8.8G 0 lvm
├─sdc7 8:39 0 8.3G 0 part
│ └─groupA-runtime 253:1 0 8.8G 0 lvm
├─sdc8 8:40 0 4G 0 part
├─sdc9 8:41 0 8.3G 0 part
├─sdc10 8:42 0 3.9G 0 part
│ └─groupS-swap 253:2 0 3.9G 0 lvm
└─sdc11 8:43 0 8.3G 0 part
We can also see our logical volumes are visible under /dev/mapper
We used Rapid7's public POC to simulate an attack on a new Ivanti appliance to generate artefacts, spawn a reverse shell, transfer files to a remote host, explore the file system, etc.
At the time of this article, we were only able to find memory resident artefacts relating to the shell itself and possible commands.
Possible Artefact Locations
Location
Contents
ivanti2/runtime/logs/log.admin.vc0
Console logins from foreign/remote IP addresses, user-agent data, username information, ICS appliance SSL information
ivanti2/runtime/logs/log.events.vc0
As above
ivanti2/runtime/system.j
Remote IP address entries relating to incoming connections
Relevant Memory Strings (if you don't have a volatility profile)
