## Windows Registry Analysis – Today’s Episode: Tasks

When it comes to persistence of common off-the-shelf malware, the most commonly observed persistence mechanisms are run keys, services, and scheduled tasks. For either of these, Windows or even the malware itself creates a set of registry keys to register the persistence mechanism with the operating system. Out of these mechanisms, this blog post specifically focuses on scheduled tasks. These are particularly interesting since they allow for much more versatile launch conditions and actions compared to services or run keys.All tasks currently registered to a Windows machine are represented by a set of registry keys and values in the HKLM\Software\Microsoft\Windows NT\CurrentVersion\Schedule\TaskCache tree. To the keen eye, most of the registry keys and their values are recognizable as they have descriptive names or contain strings. However, there are some values which seemingly contain binary data – and those are what this blog post is (mainly) about.

But before we dive into the depths of the Windows registry and the task scheduler, let me provide an important disclaimer: all this research was conducted using Windows 10 1909 with the occasional claim being verified on a Windows 7 SP1. Yet, there is certainly information presented in this blog post which is not valid for operating systems older than Windows 10. From my experience during this research, these differences are regularly just elements in structures which were not yet present in, for example, Windows 7. Vice versa, there are elements in the structures which are no longer used in current versions of the Windows operating system. Although these fields might not be used nowadays, they are still present in Windows 10. I assume this is to ensure backwards compatibility so that after, for example, an upgrade of the operating system, the tasks from the older version still work as expected with the newer one.

To make parsing the raw registry data easier, I created a set of struct definitions using kaitai struct. I published the kaitai definitions and some tooling based on these definitions on Github. For more information regarding how to use either of it, please refer to the README in the repository: https://github.com/GDATAAdvancedAnalytics/winreg-tasks

With all that being said, let’s get into it.

## The Happy Path – Creating a new Task and What Happens in the Registry

Assume we just created a new Task called “Simple Task” which starts calc.exe whenever a user logs on. The task scheduler then creates a set of registry keys which hold the information entered in the task creation wizard. These registry keys are split into roughly two groups which reference each other. The first picture shows the task in the Tree subkey, the second picture shows the values found in the respective Tasks subkey (please note my exceptional GIMP skills!). The arrows denote the references from one group to the other. I discuss the different subkeys and their values in the next section.

## Structural Overview

The TaskCache key has several subkeys which contain and organize all tasks registered in the system. All elements in the Tasks subkey reference a key in the Tree subkey and vice versa. Boot, Logon, Maintenance, and Plain tasks only have an ID value which references a key in the Tasks subkey.

The subkeys in the Tree key usually contain up to three values but only hold very limited information about a given task. If the subkey stands for a directory in the task scheduler snap-in and does not represent a task, there only is the SD value present which denotes the access rights to this specific task folder (if there is no SD value, the folder or task is hidden from the list of tasks for the user querying their task list). If a subkey references a task, there is at least the Id value set in addition to the SD value. Usually, the subkey then also contains an Index value but the presence of this specific value does not seem to be mandatory.

Each subkey in Tasks has several Values which define the task. Usually, only a subset of the values provided in the table below are set for a task. This is because many of these values are optional and are not saved in the registry if empty (e.g. Author, Data, Description, Documentation, Source). In fact, Actions and Triggers seem to be the only required values – for the MMC snap-in to show the task at least. But while the snap-in is much more restrictive with regards to how the state of the registry is, the task scheduler service is much less so. Thus, even a task without any Actions and Triggers might still be considered valid by the task scheduler service albeit the task not being of any use.

## Dynamic Information – The “When it Happened”

The DynamicInfo Value contains three timestamps denoting when the task was created, its last execution time, and the last time it finished successfully. The structure also holds any error code which occurred during the latest execution of the task. Prior to (at least) Windows 7, the structure also contained the current state of the task. But this value seems to be no longer used in this specific place but needs to be kept for compatibility reasons, presumably.A pseudo-C structure for the DynamicInfo Value could look like this:

struct DynamicInfo {
DWORD magic; // currently 0x3
FILETIME ftCreate;
FILETIME ftLastRun;
DWORD dwLastErrorCode;
FILETIME ftLastSuccessfulRun; // this field may not be present on older Windows versions (e.g. Windows 7)
};

Given our introductory example from above, the DynamicInfo Value may contain this byte sequence:

03 00 00 00
e9 79 2d f1 31 1c d8 01 # Mon 7 February 2022 14:49:43 UTC
5b 8b 6b 73 34 1c d8 01 # Mon 7 February 2022 16:19:59 UTC
00 00 00 00
00 00 00 00             # ERROR_SUCCESS
e4 70 d5 67 34 1c d8 01 # Mon 7 February 2022 15:07:21 UTC

If we changed the action of the task to execute a non-existent file, the data might look like this (note the non-zero error code):

03 00 00 00
e9 79 2d f1 31 1c d8 01 # Mon 7 February 2022 14:49:43 UTC
62 76 13 3b 33 1c d8 01 # Mon 7 February 2022 16:20:39 UTC
00 00 00 00
02 00 07 80             # 0x80070002 -> ERROR_FILE_NOT_FOUND
4c 30 75 3b 33 1c d8 01 # Mon 7 February 2022 16:20:39 UTC

## Actions – The “What Should Happen”

Whenever a Task is triggered by the scheduler, it may execute a set of actions. PowerShell does not allow to create more than 32 actions for a task, however, technically there is almost no limit to how many actions the scheduling service can handle. The only limitation seems to be the amount of elements an STL container can hold.Once again referring to the introductory example, the Actions value may contain a byte sequence similar to the following one. Since we only set the command to calc and did not add any arguments or passed a working directory, the structure is relatively small.

03 00                                            # version
0c 00 00 00 41 00 75 00 74 00 68 00 6f 00 72 00  # context ("Author")
66 66                                            # magic 0x6666 (-> execution action)
00 00 00 00                                      # id
08 00 00 00 63 00 61 00 6c 00 63 00              # command ("calc")
00 00 00 00                                      # arguments
00 00 00 00                                      # working directory
00 00                                            # flags

If one decides to add more details to the actions, the structure can grow in size rapidly. The next example shows the byte sequence for the Actions value when the action also passes arguments to the command and changes the working directory:

03 00                                            # version
0c 00 00 00 41 00 75 00 74 00 68 00 6f 00 72 00  # context ("Author")
66 66                                            # magic 0x6666 (-> execution action)
00 00 00 00                                      # id
08 00 00 00 63 00 61 00 6c 00 63 00              # command ("calc")
2c 00 00 00 61 00 72 00 67 00 31 00 20 00        # arguments ("arg1 arg2 verylongarg3")
61 00 72 00 67 00 32 00 20 00 76 00 65 00     #
72 00 79 00 6c 00 6f 00 6e 00 67 00 61 00     #
72 00 67 00 33 00                             #
56 00 00 00 43 00 3a 00 5c 00 74 00 68 00        # working directory ("C:\this\is\a\very\long\path\to\a\directory\")
69 00 73 00 5c 00 69 00 73 00 5c 00 61 00     #
5c 00 76 00 65 00 72 00 79 00 5c 00 6c 00     #
6f 00 6e 00 67 00 5c 00 70 00 61 00 74 00     #
68 00 5c 00 74 00 6f 00 5c 00 61 00 5c 00     #
64 00 69 00 72 00 65 00 63 00 74 00 6f 00     #
72 00 79 00 5c 00                             #
00 00                                            # flags

### The Actions Structure

A pseudo-C representation of the byte sequences from above cannot be given as easily as for the DynamicInfo . This is mainly because there is not only the execution action which we just looked at, but there is also email actions, COM handlers, and message box actions. We define these actions throughout the course of this section but for now we just assume they exist and “forward declare” them. The Actions structure can then be laid out like this:

struct Action;
struct ComHandlerAction : Action;
struct EmailAction : Action;
struct ExecutionAction : Action;
struct MessageBoxAction : Action;

struct Actions {
WORD version; // 0x16 on Win10 1909
BSTR context; // the id of the principal which is used to run the actions; must match the principal_id of the JobBucket (see below)
Action actions[]; // repeated until EOF
};

Technically, the Actions structure holds two properties: magic and id. But since the magic differs between all actions and the id is specific to each individual action, I include these two fields into the definitions of the sub structures and not into the generic Action structure. Either way, the magic denotes which type of action comes next in the data stream and the id may be used to assign to it an easily recognizable name. It seems that setting an id for an action is only supported when using the COM interface but not when using any other front-end tooling (Powershell, Task Scheduler Snap-In).

### ComHandler Action

The ComHandler action is the smallest one to find in the registry. Besides the common magic and id , it only contains a CLSID and a string for (optional) additional data:

struct ComHandlerAction : Action {
WORD magic = 0x7777;
BSTR id;
CLSID classId;
BSTR data;
}; 

Whenever one of these actions is triggered, the task scheduler service spawns a new taskhostw.exe process which then loads and executes the COM class configured by the classId property of the action. The COM class must implement the ITaskHandler interface and its Start method is passed the value from the data property of the action.When parsing the CLSID from the registry one must pay special attention to the byte ordering: since the data in this structure is memcpy’d into a buffer, the order order of bytes for data1 , data2 , and data3 is inverted. See the following listing for an example:

03 00 # version
14 00 00 00 4c 00 6f 00 63 00 61 00 6c 00 41 00 64 00 6d 00 69 00 6e 00 # context ("LocalAdmin")
77 77 # magic
00 00 00 00 # id
c2 d0 d1 89 cf a3 0c 49 ab e3 b8 6c de 34 b0 47 # clsid {89d1d0c2-a3cf-490c-abe3-b86cde34b047} (ReAgentTaskHandler)
16 00 00 00 56 00 65 00 72 00 69 00 66 00 79 00 57 00 69 00 6e 00 52 00 45 00 # data ("VerifyWinRE")

### Email Action

Although being obsolete and discontinued, tasks technically can still have Email actions as defined by the IEmailAction COM interface. However, the task scheduler is only able to parse the respective data from the registry but does not execute Email tasks anymore.

struct EmailAction : Action {
WORD magic = 0x8888;
BSTR id;
BSTR from;
BSTR to;
BSTR cc;
BSTR bcc;
BSTR server;
BSTR subject;
BSTR body;
DWORD numAttachments;
BSTR attachmentFilenames[numAttachments];
}

### Execution Action

The execution action is the only action which can be created using the task scheduler MMC snap-in (technically, you can also create email and message box actions – you are just not allowed to save the task then). An execution task has three customizable properties (not counting the id) which are defined in the IExecAction COM interface.

struct ExecutionAction : Action {
WORD magic = 0x6666;
BSTR id;
BSTR command;
BSTR arguments;
BSTR workingDirectory;
WORD flags; // only present if Actions.version >= 3
};

Example:

03 00 # version
0c 00 00 00 41 00 75 00 74 00 68 00 6f 00 72 00 # context ("Author")
66 66 # magic
00 00 00 00 # id
46 00 00 00 25 00 73 00 79 00 73 00 74 00 65 00 # command ("%systemroot%\system32\usoclient.exe")
6d 00 72 00 6f 00 6f 00 74 00 25 00 5c 00 73 00
79 00 73 00 74 00 65 00 6d 00 33 00 32 00 5c 00
75 00 73 00 6f 00 63 00 6c 00 69 00 65 00 6e 00
74 00 2e 00 65 00 78 00 65 00
18 00 00 00 53 00 74 00 61 00 72 00 74 00 49 00 # params ("StartInstall")
6e 00 73 00 74 00 61 00 6c 00 6c
00 00 00 00 00 # working directory
00 00 # flags

### Message Box Action

Similarly to Email actions, the MessageBox actions (or ShowMessage action as per Microsoft terms) have been discontinued and can no longer be used with the task scheduler. The structure of the data is simple, though, as a user only needed to define the caption and the content of the message box to show upon task activation.

struct MessageboxAction : Action {
WORD magic = 0x9999;
BSTR id;
BSTR caption;
BSTR content;
}

## Triggers – The “When Should it Happen”

The second, but not less important, building block of Windows tasks is their triggers. Simply put, triggers define when a given task shall be executed. Windows offers a range of different triggers (e.g. calendar-based triggers, boot triggers, logon triggers) which additionally have a variety of customization options (e.g. execution delay, repetition timings). All triggers share the same set of options but can be configured individually. I describe the top-level structure first and then descend into the different sub structures.The most significant difference between Triggers and Actions is that the data in the Triggers structure is aligned to 8-byte boundaries whereas the data in the Actions is not aligned at all. This makes parsing the Triggers more tedious since one must pay special attention to data alignments and cannot just read a number of bytes and interpret them. Unfortunately, this also makes the structure definitions look bloated. To counteract that, I define an enhanced set of types which have an ALIGNED_ prefix which hints that the encapsulated data is padded a multiple of 8 bytes. The following example applies to WORD, DWORD, and any other type analogously where necessary:

struct ALIGNED_BYTE {
BYTE value;
}

### The Triggers Structure

The triggers structure is defined by only three major members: the Header, the JobBucket, and the collection of Triggers.

struct Triggers {
JobBucket bucket;
Trigger triggers[]; // repeated until eof
}

The header only denotes the version of the struct and the time and date when the task shall be activated at and deactivated at, respectively. On Windows 10 1909, the struct version is 0x17; Windows 7 used 0x15.

struct Header {
ALIGNED_BYTE version;
TSTIME startBoundary; // the earliest startBoundary of all triggers
TSTIME endBoundary; // the latest endBoundary of all triggers
}

### The JobBucket

The JobBucket holds general information about the task, its triggers, and all of the options and settings which can be set for a task.

struct JobBucket {
// see Github ("job_bucket_flags") for a list of identified values
ALIGNED_DWORD flags;

// the crc32 checksum of the task XML
ALIGNED_DWORD crc32;

// the id of the principal which is used to execute the task; only if header.version >= 0x16
ALIGNED_BSTR principal_id;

// the DisplayName of the task principal; only if header.version >= 0x17
ALIGNED_BSTR display_name;

UserInfo user_info;

// if set, this structure contains additional settings which are not mandatory when creating a task
OptionalSettings optional_settings;
}

The task scheduler allows running tasks as a different user than the one who created the task initially. The UserInfo struct contains the information which are necessary to impersonate the task principal.

struct UserInfo {
// if non-zero, the rest of the struct is omitted in the registry
ALIGNED_BYTE skip_user;

// only if skip_user == 0
ALIGNED_BYTE skip_sid;

// any value of the SID_NAME_USE enum; only if skip_user == 0 and skip_sid == 0
ALIGNED_DWORD sid_type;

// SID in binary form; only if skip_user == 0 and skip_sid == 0
ALIGNED_BUFFER sid;

// only if skip_user == 0
}

The OptionalSettings contain the preferences and settings from the Conditions and Settings tabs in the Task Scheduler snap-in as well as additional settings which can (only) be set using the Task Scheduler COM interface (ITaskSettings, ITaskSettings2, ITaskSettings3).

struct OptionalSettings {
ALIGNED_DWORD len; // if len == 0, the rest of the structure is omitted in the registry
DWORD idle_duration_seconds;
DWORD idle_wait_timeout_seconds;
DWORD execution_time_limit_seconds;
DWORD priority;
DWORD restart_on_failure_delay;
DWORD restart_on_failure_retries;
GUID network_id;
BYTE padding0[4]; // probably there because the previous struct members are part of another struct which is inlined here
BYTE privileges; // only if len == 0x38 or len == 0x58
TSTIMEPERIOD periodicity; // only if len == 0x58
TSTIMEPERIOD deadline; // only if len == 0x58
BYTE exclusive; // only if len == 0x58
BYTE padding1[3]; // only if len == 0x58
}

Several of these settings have corresponding input fields in the task scheduler snap-in. To make these more easily recognizable, I copied over the labels to their respective setting in the struct where applicable:

• idle_duration_seconds: “start the task only if the computer is idle for” setting converted to seconds
• idle_wait_timeout_seconds: “wait for idle for” converted to seconds
• execution_time_limit_seconds: “stop the task if it’s running longer than” converted to seconds
• delete_expired_task_after: “if the task is not scheduled to run again, delete it after” converted to seconds
• priority: the process priority value the task scheduler assigns to the task process
• restart_on_failure_delay: “if the task fails, restart every” converted to seconds
• restart_on_failure_retries: “attempt to restart up to”
• network_id: “start only if the following network connection is available”
• privileges: a bitmap of Se* permissions (e.g., SeDebugPrivilege; see Github repository for full list) to grant to the task process when runningperiodicity: the amount of time a task needs when executed during automatic system maintenance (only applies to maintenance tasks)
• deadline: defines the amount of time which is allowed to pass before the task is executed during emergency maintenance if it failed to complete during normal maintenance (only applies to maintenance tasks)
• exclusive: defines whether the task shall be run as an exclusive task with no other maintenance task running at the same time (only applies to maintenance tasks)

### The Triggers

The list of trigger structures defines the triggers of a given task. There is (almost) no technical limit to the amount of triggers a task is allowed to have. Yet, there commonly are only 1 to 3 triggers with the occasional outlier in my testing system having up to 6 triggers.

Different to the Actions described above, the triggers only share the common field magic. However, I again include the magic into the different trigger definitions so that is is more easily recognizable which magic a given trigger has (similarly to how I have done it with the Actions above).

All of the triggers below contain a GenericData field (except for the TimeTrigger which uses a JobSchedule). I describe the structure below the trigger definitions.

The examples given for the different triggers may contain garbage or suspiciously-looking data in several fields. This mainly has two reasons. On the one hand, the task scheduler fills the buffer for the registry data with the character H before serializing any data. This is where all the 0x48 come from. On the other hand, the data structures being memcpy‘d into the buffer are usually not initialized. This has the side effect that there may be content from the stack (or heap, depending on where the source data structure resides) spoiled into the allocated buffer and eventually written into the registry. I have observed heap and stack pointers where only the least significant byte was overridden by a field of the structure, partial strings, and also “random” data which I could not identify.

#### WnfStateChangeTrigger

This trigger listens for notifications in the Windows Notification Framework (WNF). It appears that this trigger is intended to only be used internally in Windows, because the Microsoft documentation only provides the non-descriptive trigger type TASK_TRIGGER_CUSTOM_TRIGGER_01 for this trigger type. However, if one is well-versed with COM and how to work with “unknown” interfaces, it should be possible to create even these triggers.

struct WnfStateChangeTrigger {
ALIGNED_DWORD magic = 0x6666;
GenericData genericData;
BYTE state_name[8];
ALIGNED_DWORD cbData;
BYTE data[cbData];
}

Example:

17 00 00 00 00 00 00 00                         # header.version (0x17)
00 7c 10 22 98 7c 10 22 00 00 00 00 00 00 00 00 # header.start_boundary (localized: no, filetime: 0)
00 7c 10 22 98 7c 10 22 ff ff ff ff ff ff ff ff # header.end_boundary (localized: no, filetime: 0xffffffffffffffff)
00 90 c0 42 48 48 48 48                         # job_bucket.flags (0x42c09000)
27 82 bb 7f 48 48 48 48                         # job_bucket.crc32 (0x7fbb8227)
0c 00 00 00 48 48 48 48 55 00 73 00 65 00 72 00 73 00 00 00 48 48 48 48 # job_bucket.principal_id ("Users")
00 00 00 00 48 48 48 48                         # job_bucket.display_name ("")
00 48 48 48 48 48 48 48                         # job_bucket.user_info.skip_user (0x00 -> user struct is present)
00 48 48 48 48 48 48 48                         # job_bucket.user_info.skip_sid (0x00 -> sid is present)
05 00 00 00 48 48 48 48                         # job_bucket.user_info.sid_type (0x05 -> SidTypeWellKnownGroup)
0c 00 00 00 48 48 48 48 01 01 00 00 00 00 00 05 04 00 00 00 48 48 48 48 # SID (S-1-5-4 "INTERACTIVE")
00 00 00 00 48 48 48 48                         # job_bucket.user_info.username ("")
2c 00 00 00 48 48 48 48                         # job_bucket.optional_settings.len (0x2c)
00 00 00 00                                     # job_bucket.optional_settings.idle_duration_seconds (0)
ff ff ff ff                                     # job_bucket.optional_settings.idle_wait_timeout_seconds (-1)
58 02 00 00                                     # job_bucket.optional_settings.execution_time_limit_seconds (600)
ff ff ff ff                                     # job_bucket.optional_settings.delete_expired_task_after (-1)
06 00 00 00                                     # job_bucket.optional_settings.priority (6)
00 00 00 00                                     # job_bucket.optional_settings.restart_on_failure_retries (0)
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 # job_bucket.optional_settings.network_id
00 00 00 00 48 48 48 48                         # job_bucket.optional_settings.padding0
66 66 00 00 00 00 00 00                         # triggers[0].magic (0x6666)
00 7c 10 22 98 7c 10 22 00 00 00 00 00 00 00 00 # triggers[0].generic_data.start_boundary (localized: no, filetime: 0)
00 7c 10 22 98 7c 10 22 ff ff ff ff ff ff ff ff # triggers[0].generic_data.end_boundary (localized: no, filetime: 0xffffffffffffffff)
00 00 00 00                                     # triggers[0].generic_data.delay_seconds (0)
ff ff ff ff                                     # triggers[0].generic_data.timeout_seconds (-1)
00 00 00 00                                     # triggers[0].generic_data.repetition_interval_seconds (0)
00 00 00 00                                     # triggers[0].generic_data.repetition_duration_seconds (0)
00 00 00 00                                     # triggers[0].generic_data.repetition_duration_seconds_2 (0)
00                                              # triggers[0].generic_data.stop_at_duration_end (false)
01 7e e8 fa cd 31 1c be                         # triggers[0].generic_data.enabled (true)
6f 8a 99 8f 84 0b 3e 42                         # triggers[0].generic_data.unknown
00 00 00 00                                     # triggers[0].generic_data.trigger_id ("")
48 48 48 48                                     # triggers[0].generic_data.pad_to_block
75 78 bc a3 3a 07 80 08                         # triggers[0].state_name ("7578bca33a078008", WNF_WIFI_TASK_TRIGGER)
00 00 00 00 00 00 00 00                         # triggers[0].cbData (0)

#### SessionChangeTrigger

Triggers a task whenever the session state of the given user changes (see JobBucket for the definition of the UserInfo structure). Session state changes can be console connect and disconnect, remote connect and disconnect, and session lock and unlock. The specific values for the session states can be found in the Github repository (“session_state” enum).

struct SessionChangeTrigger {
ALIGNED_DWORD magic = 0x7777;
GenericData genericData;
ALIGNED_DWORD dwStateChange;
UserInfo user;
}


Example (only the trigger structure, full example at WnfStateChangeTrigger):

77 77 00 00 00 00 00 00                         # magic (0x7777)
00 67 10 22 80 67 10 22 00 00 00 00 00 00 00 00 # start_boundary (localized: no, filetime: 0)
00 67 10 22 80 67 10 22 ff ff ff ff ff ff ff ff # end_boundary (localized: no, filetime: 0xffffffffffffffff)
58 02 00 00                                     # delay_seconds (600)
ff ff ff ff                                     # timeout_seconds (0xffffffff)
00 00 00 00                                     # repetition_interval_seconds (0)
00 00 00 00                                     # repetition_duration_seconds (0)
00 00 00 00                                     # repetition_duration_seconds_2 (0)
00                                              # stop_at_duration_end (false)
00 00 49 00 6e 00 73 00                         # enabled (false)
74 00 61 00 6c 00 6c 00                         # unknown
34 00 00 00 4c 00 6f 00 63 00 61 00 6c 00 43 00 # trigger_id ("LocalConsoleConnectTrigger")
6f 00 6e 00 73 00 6f 00 6c 00 65 00 43 00 6f 00
6e 00 6e 00 65 00 63 00 74 00 54 00 72 00 69 00
67 00 67 00 65 00 72 00
01 00 00 00                                     # state_change ("ConsoleConnect")
65 00 00 00                                     # padding
01 48 48 48 48 48 48 48                         # user_info.skip_user (true -> no user struct)


#### RegistrationTrigger

struct RegistrationTrigger {
ALIGNED_DWORD magic = 0x8888;
GenericData genericData;
}


Example (only the trigger structure, full example at WnfStateChangeTrigger):

88 88 00 00 00 00 00 00                         # magic (0x8888)
00 59 10 22 70 59 10 22 00 00 00 00 00 00 00 00 # start_boundary (localized: no, filetime: 0)
00 59 10 22 70 59 10 22 ff ff ff ff ff ff ff ff # end_boundary (localized: no, filetime: 0xffffffffffffffff)
00 00 00 00                                     # delay_seconds (0)
ff ff ff ff                                     # timeout_seconds (0xffffffff)
00 00 00 00                                     # repetition_interval_seconds (0)
00 00 00 00                                     # repetition_duration_seconds (0)
00 00 00 00                                     # repetition_duration_seconds_2 (0)
00                                              # stop_at_duration_end (false)
01 00 00 00 00 00 00 00                         # enabled (true)
4c 4d 45 4d 48 00 00 00                         # unknown
00 00 00 00                                     # trigger_id
48 48 48 48                                     # block padding


#### LogonTrigger

Triggers a task whenever the given user logs on (see JobBucket for the definition of the UserInfo structure).

struct LogonTrigger {
ALIGNED_DWORD magic = 0xAAAA;
GenericData genericData;
UserInfo user;
}


Example (only the trigger structure, full example at WnfStateChangeTrigger):

aa aa 00 00 00 00 00 00                         # magic (0xaaaa)
00 59 10 22 70 59 10 22 00 00 00 00 00 00 00 00 # start_boundary (localized: no, filetime: 0)
00 59 10 22 70 59 10 22 ff ff ff ff ff ff ff ff # end_boundary (localized: no, filetime: 0xffffffffffffffff)
00 00 00 00                                     # delay_seconds (0)
ff ff ff ff                                     # timeout_seconds (0xffffffff)
80 70 00 00                                     # repetition_interval_seconds (28800 "PT8H")
00 00 00 00                                     # repetition_duration_seconds (0)
00 00 00 00                                     # repetition_duration_seconds_2 (0)
00                                              # stop_at_duration_end (false)
01 a9 a7 1c 94 a9 a7 1c                         # enabled (true)
00 00 00 00 00 00 00 00                         # unknown
00 00 00 00                                     # trigger_id
48 48 48 48                                     # block padding
01 48 48 48 48 48 48 48                         # user_info.skip_user (true -> no user struct)


#### EventTrigger

Triggers a task based on events that appeared in the Windows logs. The subscription field contains a list of XPATH queries which create the filters the task scheduler uses to check whether the conditions for the EventTrigger are fulfilled.

struct EventTrigger {
ALIGNED_DWORD magic = 0xCCCC;
GenericData genericData;
ALIGNED_BSTR_EXPANDSIZE subscription;
DWORD unknown0;
DWORD unknown1;
ALIGNED_BSTR_EXPANDSIZE unknown2;
ALIGNED_DWORD len_value_queries;
Pair<ALIGNED_BSTR_EXPANDSIZE, ALIGNED_BSTR_EXPANDSIZE> value_queries; // "ALIGNED_BSTR_EXPANDSIZE name; ALIGNED_BSTR_EXPANDSIZE query;" x len_value_queries
}

Example (only the trigger structure, full example at WnfStateChangeTrigger):

cc cc 00 00 00 00 00 00                         # magic (0xcccc)
00 59 10 22 70 59 10 22 00 00 00 00 00 00 00 00 # start_boundary (localized: no, filetime: 0)
00 59 10 22 70 59 10 22 ff ff ff ff ff ff ff ff # end_boundary (localized: no, filetime: 0xffffffffffffffff)
dc 05 00 00                                     # delay_seconds (1500 "PT25M")
08 07 00 00                                     # timeout_seconds (1800 "PT30M")
10 0e 00 00                                     # repetition_interval_seconds (3600 "PT1H")
40 38 00 00                                     # repetition_duration_seconds (14400 "PT4H")
40 38 00 00                                     # repetition_duration_seconds_2 (14400 "PT4H")
00                                              # stop_at_duration_end (false)
01 00 00 00 00 00 00 00                         # enabled (true)
0c 00 00 00 00 00 00 00                         # unknown
00 00 00 00                                     # trigger_id
48 48 48 48                                     # block padding
05 01 00 00 00 00 00 00 3c 00 51 00 75 00 65 00 # subscription: <Select Path="Microsoft-Windows-User Device Registration/Admin">*[System[Provider[@Name='Microsoft-Windows-User Device Registration'] and EventID=300]]</Select></Query></QueryList>
72 00 79 00 4c 00 69 00 73 00 74 00 3e 00 3c 00
51 00 75 00 65 00 72 00 79 00 20 00 49 00 64 00
3d 00 22 00 30 00 22 00 20 00 50 00 61 00 74 00
68 00 3d 00 22 00 4d 00 69 00 63 00 72 00 6f 00
73 00 6f 00 66 00 74 00 2d 00 57 00 69 00 6e 00
64 00 6f 00 77 00 73 00 2d 00 55 00 73 00 65 00
72 00 20 00 44 00 65 00 76 00 69 00 63 00 65 00
20 00 52 00 65 00 67 00 69 00 73 00 74 00 72 00
61 00 74 00 69 00 6f 00 6e 00 2f 00 41 00 64 00
6d 00 69 00 6e 00 22 00 3e 00 3c 00 53 00 65 00
6c 00 65 00 63 00 74 00 20 00 50 00 61 00 74 00
68 00 3d 00 22 00 4d 00 69 00 63 00 72 00 6f 00
73 00 6f 00 66 00 74 00 2d 00 57 00 69 00 6e 00
64 00 6f 00 77 00 73 00 2d 00 55 00 73 00 65 00
72 00 20 00 44 00 65 00 76 00 69 00 63 00 65 00
20 00 52 00 65 00 67 00 69 00 73 00 74 00 72 00
61 00 74 00 69 00 6f 00 6e 00 2f 00 41 00 64 00
6d 00 69 00 6e 00 22 00 3e 00 2a 00 5b 00 53 00
79 00 73 00 74 00 65 00 6d 00 5b 00 50 00 72 00
6f 00 76 00 69 00 64 00 65 00 72 00 5b 00 40 00
4e 00 61 00 6d 00 65 00 3d 00 27 00 4d 00 69 00
63 00 72 00 6f 00 73 00 6f 00 66 00 74 00 2d 00
57 00 69 00 6e 00 64 00 6f 00 77 00 73 00 2d 00
55 00 73 00 65 00 72 00 20 00 44 00 65 00 76 00
69 00 63 00 65 00 20 00 52 00 65 00 67 00 69 00
73 00 74 00 72 00 61 00 74 00 69 00 6f 00 6e 00
27 00 5d 00 20 00 61 00 6e 00 64 00 20 00 45 00
76 00 65 00 6e 00 74 00 49 00 44 00 3d 00 33 00
30 00 30 00 5d 00 5d 00 3c 00 2f 00 53 00 65 00
6c 00 65 00 63 00 74 00 3e 00 3c 00 2f 00 51 00
75 00 65 00 72 00 79 00 3e 00 3c 00 2f 00 51 00
75 00 65 00 72 00 79 00 4c 00 69 00 73 00 74 00
3e 00 00 00 48 48 48 48
00 00 00 00                                     # unknown0 (0)
00 00 00 00                                     # unknown1 (0)
00 00 00 00 00 00 00 00                         # unknown2 ("")
00 00 00 00 00 00 00 00                         # len_value_queries (0)

#### TimeTrigger

Unlike all other triggers, a TimeTrigger does not have a GenericData field. This is, because time-based triggers allow for more fine-grained options as to when a task shall be run. The GenericData does not have the required fields to hold all properties of these options and thus the JobSchedule replaces it. The only significant difference compared to other tasks is that a JobSchedule does not contain the trigger_id field so the TimeTrigger structure holds one instead. For all other structures, the trigger_id is part of the GenericData field (see below).

struct TimeTrigger {
ALIGNED_DWORD magic = 0xDDDD;
JobSchedule job_schedule;
BSTR trigger_id; // only if header.version >= 0x16
BYTE padding[8 - (trigger_id.cbData + 4)) % 8]; // pad to multiple of 8 bytes; only if header.version >= 0x16
}

Comparing the JobSchedule structure with the GenericData structure, there is a significant overlap of fields (start_boundary, end_boundary, repetition_*, execution_time_limit, stop_tasks_at_duration_end, is_enabled, max_delay_seconds). The most significant difference is the mode and its data1, data2, and data3 fields. Depending on the value of mode, the data fields contain different bitmaps of values which, for example, represent the different days of a month:

• mode 0 (ITimeTrigger): run at <start_boundary>
• mode 1 (IDailyTrigger): run at <start_boundary> and repeat every <data1> days
• mode 2 (IWeeklyTrigger): run on days of week <(data2 as day_of_week bitmap)> every <data1> weeks starting at <start_boundary>
• mode 3 (IMonthlyTrigger): run in months <(data3 as months bitmap)> on days <(data2:data1 as day in month bitmap)> starting at <start_boundary>
• mode 4 (IMonthlyDOWTrigger): run in months <(data3 as months bitmap)> in weeks <(data2 as week bitmap)> on days <(data1 as day_of_week bitmap)> starting at <start_boundary>
struct JobSchedule {
TSTIME start_boundary;
TSTIME end_boundary;
TSTIME unknown0;
DWORD repetition_interval_seconds;
DWORD repetition_duration_seconds;
DWORD execution_time_limit_seconds;
DWORD mode; // see above for possible values
WORD data1;
WORD data2;
WORD data3;
BYTE is_enabled;
DWORD unknown1;
DWORD max_delay_seconds;
}

Example (only the trigger structure, full example at WnfStateChangeTrigger):

dd dd 00 00 00 00 00 00                         # magic (0xdddd)
01 07 0b 00 00 00 09 00 00 78 18 25 ab 03 c7 01 # start_boundary (localized: yes, filetime: 0x01c703ab25187800; "Thu 9 November 2006 02:00:00 UTC")
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 # end_boundary (localized: no, filetime: 0)
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 # unknown0
00 00 00 00                                     # repetition_interval_seconds (0)
00 00 00 00                                     # repetition_duration_seconds (0)
ff ff ff ff                                     # execution_time_limit_seconds (0xffffffff)
01 00 00 00                                     # mode (1 -> daily)
01 00                                           # data1 (0x1 -> repeat every 1 day(s))
00 00                                           # data2 (0)
00 00                                           # data3 (0)
01                                              # is_enabled (true)
01 00 00 00                                     # unknown1
10 0e 00 00                                     # max_delay_seconds (3600 "PT1H")
6f a8 28 6a                                     # pad2
48 00 00 00 37 00 64 00 62 00 61 00 31 00 38 00 # trigger_id ("7dba1862-fdda-4030-83de-895375c111d4")
36 00 32 00 2d 00 66 00 64 00 64 00 61 00 2d 00
34 00 30 00 33 00 30 00 2d 00 38 00 33 00 64 00
65 00 2d 00 38 00 39 00 35 00 33 00 37 00 35 00
63 00 31 00 31 00 31 00 64 00 34 00
48 48 48 48                                     # block padding

#### IdleTrigger

The IdleTrigger causes a task to be run whenever the system goes into idle mode. This trigger structure does not have additional fields besides the GenericData.

struct IdleTrigger {
ALIGNED_DWORD magic = 0xEEEE;
GenericData genericData;
}

For an example see RegistrationTrigger.

#### BootTrigger

Tasks with BootTriggers are run by the scheduler when the system boots. These triggers structures do not have any additional fields besides the GenericData.

struct BootTrigger {
ALIGNED_DWORD magic = 0xFFFF;
GenericData genericData;
}

For an example see RegistrationTrigger.

#### GenericData

The GenericData is a set of options which can be modified individually for each trigger. It contains, for example, the range of time in which the trigger is active (start_boundary, end_boundary), whether there should be a delay between activating the trigger and running the task (delay_seconds), or for how long the task instance is allowed to run after being launched (timeout_seconds). The structure also contains the repetition pattern for the task (“repeat task every x hours for the duration of y days”, for example) and a boolean indicating whether all task instances shall be stopped once the repetition_duration has passed. Triggers can also be enabled and may have assigned a trigger_id to make them more recognizable.

struct GenericData {
TSTIME start_boundary;
TSTIME end_boundary;
DWORD delay_seconds;
DWORD timeout_seconds;
DWORD repetition_interval_seconds;
DWORD repetition_duration_seconds;
DWORD repetition_duration_seconds_2; // seems to be always the same value as repetition_duration_seconds; probably a remnant of something no longer implemented, the XML serializer skips this value as well
BYTE stop_at_duration_end;
ALIGNED_BYTE enabled;
BYTE unknown[8];
BSTR trigger_id; // only if header.version >= 0x16
BYTE pad_to_block[(8 - (trigger_id.len + 4)) % 8]; // only if header.version >= 0x16
}

For an example see WnfStateChangeTrigger.

## Auxiliary Structures

The struct definitions given above use of several other structs to define their content. This section provides the missing definitions in alphabetical order.

struct ALIGNED_BSTR {
ALIGNED_DWORD cbString;
WCHAR str[cbData/2];
}

struct ALIGNED_BSTR_EXPANDSIZE {
ALIGNED_DWORD nChars;
WCHAR str[nChars+1];
}

struct ALIGNED_BUFFER {
ALIGNED_DWORD cbData;
BYTE data[cbData];
}

struct BSTR {
DWORD cbData;
WCHAR str[cbData/2];
}

struct TSTIME {
ALIGNED_BYTE isLocalized;
FILETIME time;
}

struct TSTIMEPERIOD {
WORD year;
WORD month;
WORD week;
WORD day; // if used in conjunction with week this is "day of week"
WORD hour;
WORD minute;
WORD second;
}

# Closing Words

Although there are still a few unanswered question (read: unknown elements in structures), the analysis presented in this blog post covers most of the data located in the TaskCache in the Windows registry. However, this post mainly focuses on dissecting and describing the Actions, Triggers, and DynamicInfo blobs as these contain serialized data in binary form. Understanding these binary blobs may be crucial in scenarios where, for example, a malicious party tampered with the data in the registry to hide a certain task from a user.

Definitions for all structures I identified during the analysis can be found on Github together with some tooling based on the code generated by the Kaitai struct compiler. I chose Kaitai as the struct description language so that parsers implemented in different languages can be generated easily. Please refer to the documentation located on Github for more information on how to use the provided structure definitions.

## What the Pack(er)?

Lately, I broke one of the taboos of malware analysis: looking into the packer stub of a couple of malware samples. Fortunately, I must say. Because I discovered something I was really surprised by. But first, a little detour.

Historically, Emotet has been observed to assemble infected systems into three botnets dubbed Epoch 1, Epoch 2, and Epoch 3. After the takedown and the later resurrection, there seems to only be two botnets which have subsequently been dubbed Epoch 4 and Epoch 5. The differences between the old and the new core of the botnets are significant on the technical side – however, the old Epochs 1 through 3 shared the same core and so do the recent Epoch 4 and Epoch 5. The only noticeable difference between Epochs 1 through 3 was the config which was embedded into the Emotet core before a sample was rolled out to the victims. The same also applies to the more recent Epochs 4 and 5.

However, there is a significant difference in the operation carried out by the botnets between what happened before the disruption and what was observed since the rebirth. In the past, observations showed that Emotet bots used to drop whatever their operator’s customers paid them for. Brad Duncan alone already observed Emotet dropping QakBot/QBot, Trickbot, and Gootkit. Of these, the Trickbot group seemed to be their best and longest-running customer based on the numerous observations of Trickbot being dropped by Emotet. But after the resurrection, there were no longer observations of additional malware being dropped by Emotet. Instead, starting in December 2021, researchers observed a CobaltStrike beacon being dropped onto an infected machine without any evidence that there was another malware involved. Emotet has since been reportedly and repeatedly seen to deploy CobaltStrike beacons to infected machines, so this was definitely not a one-off drop and drew the attention of our researchers.

With the context of this analysis being setup properly, we can finally come back to the actual topic of this blog post: breaking taboos by analyzing packing stubs. Enjoy!

## Poking (in) Packing Stubs

For the first period of time after the resurrection, the Emotet core seems to have used XOR encryption to hide their bot from static analysis. It can easily be seen that the algorithms appear to be (almost) identical between Epoch 4 (left) and Epoch 5 (right) – disregarding a few compiler optimizations due to different key lengths:

At some point, the authors changed the encryption scheme to use RC4 instead of plain XOR. Although the code applying the RC4 algorithm looks different thanks to a substantial amount of superfluous API calls, there are obvious similarities between Epoch 4 on the left and Epoch 5 on the right:

The surprising discovery we made during the week preceeding the publication of this post is related to the CobaltStrike drops. Assuming from what was observed for Epochs 1 through 3, thoughts were that some other party paid the Emotet operators to drop CobaltStrike as their desired payload. Having a closer look at the samples reveals an interesting observation: all of the CobaltStrike drops used packing stubs which looked extremely familiar. The drops referred to in the following were received on March 11th, however, these specific packing stubs were already observed earlier for Emotet drops. Unfortunately, we did not see the connection until a couple of days ago. But have a look for yourself:

As it can be seen in both examples, Drop A used the packer which was observed in the early days after the rebirth while Drop B used the same packer as the Emotet core itself at the time of writing this post.

## Conclusion or (Educated) Guessing

Prior to the rebirth, drops were not bound to the operation of Emotet  – the botnet was known to drop whatever their operator’s customers paid them for; but since the resurrection, this seems to have shifted towards drops which are very tightly-bound to the Emotet core and thus the operation as well. Considering that Trickbot was used to revive the Emotet botnet back in november 2021 and the observation that Emotet since then only dropped CobaltStrike beacons to infected machines, one thought may arise: have the Trickbot operators perhaps invited their old friends from Emotet over to work for the Conti group as well? It has long been said that the Emotet operators are closely related to the Trickbot group because of their long-running partnership. The thought is also supported by information from the Conti playbook leak in 2021 where it can be seen that Conti makes heavy use of CobaltStrike as a reconnaissance tools before deploying their ransomware. AdvIntel also suspected that Emotet arose as part of the Conti group. The now-discovered use of identical packers for both the Emotet core and the CobaltStrike drops supports the claim in a fascinating way.

Alternatively, or additionally, the resurrection of Emotet may have been the final step in replacing Trickbot as the initial foothold of the Conti group in their victim’s networks by putting their remaining Trickbot bots to a last use. It cannot be denied that Emotet was a surprisingly efficient malware so the Conti operators may have gone for using both Emotet and BazarLoader to access their victim’s networks: with the Trickbot developers focusing solely on BazarLoader and the Emotet operators back into the business, this leaves the Conti group with two independent and powerful tools to access infected machines.

## Remarks

Of course, at the same time the author made the aforementioned discovery, researchers observed another drop being delivered by Emotet: SystemBC. It remains to be seen whether this was a one-time delivery in the sense of a test or if researcher will see this drop more often in the future.

## Reference Samples

c7574aac7583a5bdc446f813b8e347a768a9f4af858404371eae82ad2d136a01 – old Emotet Epoch 4 sample (2021-11-15)

1c9f611ce78ab0efd09337c06fd8c65b926ebe932bc91b272e97c6b268ab13a1 – old Emotet Epoch 5 sample (2021-11-18)

8494831bbfab5beb6a58d1370ac82a4b3caa1f655b78678c57ef93713c476f9c – recent Emotet Epoch 4 sample (2022-03-14)

31f7e5398c41d7eb8d033dbc7d3b90a2daf54995e20b5ab4a72956b41c8e1455 – recent Emotet Epoch 5 sample (2022-03-15)

cf7a53b0e07f4a1fabc40a5e711cf423d18db685ed4b3c6c87550fcbc5d1a036 – CobaltStrike Drop A (2022-03-11)

73aba991054b1dc419e35520c2ce41dc263ff402bcbbdcbe1d9f31e50937a88e – CobaltStrike Drop B (2022-03-11)

## A Chapter Closes

When we registered the domain cyber.wtf, G DATA Advanced Analytics (ADAN) was only Marion, Jan, and me. Our sole offer was malware analysis and we were sharing an office that had been vacated by G DATA’s security labs and was scheduled for a thorough workover later that year. That was almost exactly six years ago.

:~\$ whois cyber.wtfDomain Name: cyber.wtfCreation Date: 2016-02-25T10:00:20ZRegistry Expiry Date: 2022-02-25T10:00:20Z

A few months back, I had accepted the task to build a security service provider under the roof of an AV company. ADAN has grown quite a bit over the years in terms of people and portfolio, and G DATA has changed and grown as well. It’s been quite a ride and I’d like to thank everyone who supported our cause and who lent me and my colleagues their ears, heart, and brain over the years. At the end of February I’ll resign from my role as founding CEO of G DATA ADAN.

All the best,
Tilman

## Guess who’s back

tl;dr: Emotet

The (slighty) longer story:
On Sunday, November 14, at around 9:26pm UTC we observed on several of our Trickbot trackers that the bot tried to download a DLL to the system. According to internal processing, these DLLs have been identified as Emotet. However, since the botnet was taken down earlier this year, we were suspicious about the findings and conducted an initial manual verification. Please find first results and IOCs below. Currently, we have high confidence that the samples indeed seem to be a re-incarnation of the infamous Emotet.

We are still conducting more in-depth analyses to raise the confidence even further. New information will be provided as they become available.

## Initial Analysis

Sunday, November 14, 9:26pm: first occurence of the URLs being dropped; the URL we received was hxxp://141.94.176.124/Loader_90563_1.dll (SHA256 of the drop: c7574aac7583a5bdc446f813b8e347a768a9f4af858404371eae82ad2d136a01). Internal processing detected Emotet when executing the sample in our sandbox systems. Notably, the sample seems to have been compiled just before the deployment via several Trickbot botnets was observed: Timestamp : 6191769A (Sun Nov 14 20:50:34 2021)

The network traffic originating from the sample closely resembles what has been observed previously (e.g. as described by Kaspersky): the URL contains a random resource path and the bot transfers the request payload in a cookie (see image below). However, the encryption used to hide the data seems different from what has been observed in the past. Additionally, the sample now uses HTTPS with a self-signed server certificate to secure the network traffic.

A notable characteristic of the last Emotet samples was the heavy use of control-flow flattening to obfuscate the code. The current sample also contains flattened control flows. To illustrate the similarity in the style of the obfuscation, find two arbitrary code snippets below. Left side is a sample from 2020, on the right is a snippet from the current sample:

## Conclusion (so far)

As per the famous duck-typing, we conclude so far: smells like Emotet, looks like Emotet, behaves like Emotet – seems to be Emotet.

We are currently updating our internal tooling for the new sample to provide more indicators to strengthen the claim that Emotet seems to be back.

## IOCs

URLs:

Hashes:

Server List:
81.0.236.93:443
94.177.248.64:443
66.42.55.5:7080
103.8.26.103:8080
185.184.25.237:8080
45.76.176.10:8080
188.93.125.116:8080
103.8.26.102:8080
178.79.147.66:8080
58.227.42.236:80
45.118.135.203:7080
103.75.201.2:443
195.154.133.20:443
45.142.114.231:8080
212.237.5.209:443
207.38.84.195:8080
104.251.214.46:8080
138.185.72.26:8080
51.68.175.8:8080
210.57.217.132:8080

String List:
SOFTWARE\Microsoft\Windows\CurrentVersion\Run
POST
%s\rundll32.exe "%s",Control_RunDLL
Control_RunDLL
%s\%s
%s\%s
%s\%s%x
%s%s.exe
%s\%s
SHA256
HASH
AES
Microsoft Primitive Provider
ObjectLength
KeyDataBlob
%s\rundll32.exe "%s\%s",%s
Content-Type: multipart/form-data; boundary=%s

RNG
%s%s.dll
%s\rundll32.exe "%s",Control_RunDLL
%s%s.dll
%s\regsvr32.exe -s "%s"
%s\%s
%s%s.exe
SOFTWARE\Microsoft\Windows\CurrentVersion\Run
%s\rundll32.exe "%s\%s",%s
ECCPUBLICBLOB
ECDH_P256
Microsoft Primitive Provider
ECCPUBLICBLOB

%s\rundll32.exe "%s\%s",%s
%s:Zone.Identifier
%u.%u.%u.%u
%s\%s
%s\*
%s\%s
WinSta0\Default
%s\rundll32.exe "%s",Control_RunDLL %s
%s%s.dll
ECCPUBLICBLOB
ECDSA_P256
Microsoft Primitive Provider
%s\%s
SHA256
Microsoft Primitive Provider
ObjectLength

## Trickbot rdpscanDll – Transforming Candidate Credentials for Brute-Forcing RDP Servers

After some weeks of not seeing the RDP scanner module of Trickbot, I recently observed that the module was again distributed among the bots in our tracking lab. Since Bitdefender already published a report on the module in March 2020, I focused on checking whether or not the command-and-control (C2) communication of the module remained more or less the same or if there was anything groundbreakingly new. Short answer: there wasn’t. There may be some under-the-hood fixes or improvements but I (as of yet) did not stumble upon anything significant that wasn’t already found by Bitdefender: the module still receives its mode of action, target servers, usernames, and password candidates from the C2 server and then does what the mode tells it to do. But while I was checking that, I also had a look at the actual data that we received from the C2 server.

My intuition on the password list was that it is just a dictionary of words to try. This is also suggested by the URL which is used to retrieve the password list: hxxps://%c2%/%gtag%/%bot_id%/rdp/dict. Thus I did not have a closer look at the password list at that time, because everything looked the way Bitdefender described it and I had no reason to look at it in detail. But one or two days later, I re-requested the list of passwords to see whether the list changed in the meantime – and it did indeed. Because of that I had a quick look at what changed and then I noticed that I overlooked something right from the start (literally, duh!). On the left side of the picture you see what I had a quick look at after retrieving the password list from the C2 server with curl (and thus seeing only the last lines of the output). On the right side there is the very same password list, just seen from the start.

To the keen eye it seems that they may be using some kind of templating mechanism to adjust the list of passwords and use more specific credential candidates. With that thought in mind I spun up my analysis environment and started digging into the module to see what the Trickbot gang is actually doing there (spoiler: yes, they do some kind of templating – but not just the find-and-replace kind).

## Transforming them P@ssw0rds

As mentioned before, this is not a simple find-and-replace but instead they can change the credential candidates to better fit the attacked host. In that sense, I decided to call those things transforms instead of templates because they are not just templates that are filled out but a little bit more dynamic. Example:

• %Username%123 → Myuser123 (lowercase but first char uppercase)
• %UsErNaMe%123 → MyUsEr123 (alternating case, starting with uppercase char)
• %EMANRESU%123 → RESUYM123 (uppercase and reversed)

And that is essentially how the markers in the password list work. I was able to extract all 91 transformations that are currently available to the rdpscanDll (as of 2020-08-14). Please find the list with all transforms with an example and a description for each of them at the end of this blog post.

Some of the transforms can even be parameterized to a certain degree: %OriginalUsername%, %OriginalDomain%, and %domain% can be prepended or appended with an (N) to indicate whether the first N or last N characters of the element should be used (or everything if no parameters are present).

## Reconnaissance

After finding the list of transforms, I decided to ask my favorite internet search engine whether these names for the transforms are known related to RDP. And I indeed found a RDP brute force tool by a certain z668 which seemingly makes use of some of the transforms that are used in the rdpscanDll. Although this tool seems to be a standalone application, the names of the transforms and the context of their use could suggest a connection between z668 and the Trickbot gang – at least to a certain degree. Sure, the connection may not be really strong because the Trickbot module is written in C++ and the RDP tool seems to be written in C#. But given the fact that C# can load and use native DLLs and considering that z668 forked the FreeRDP project on Github, the actual scanner may indeed be written in C/C++ (and probably using FreeRDP). Thus it is possible that the Trickbot gang may have obtained the source code from z668 to integrate the RDP scanner into their module framework and to use their C2 communication protocol. But: this is just guessing based on some more or less loose facts – I could easily be completely wrong with that.

## Transform List

Transform Identifier Example Description
GetHost fills in the hostname of the currently attacked IP (ex: myhost)
IP the currently attacked IP address (ex: 234.234.234.234)
Port fills in the currently attacked port (ex: 3389)
IpReplaceDot 234.234.234.234 → 234234234234 remove the dots of the IP address
RemoveNumerics us3rn4me → usrnme removes all number from the username
RemoveLetters us3rn4m3 → 343 removes all letters from the username
RemoveOtherSymbols usern@m3 → usernm3 removes all non-alphanumeric characters from the username
OriginalUsernameLettersBeginInverse 123admin456 → 123654nimda keeps all non-letters (i.e. digits, special chars) at the beginning of the username and reverses the rest (“invert [from where] letters begin
OriginalUsernameLettersBeginSwap 123admin456 → admin456123 swaps all non-letters (i.e. digits, special chars) at the beginning of the username with the rest (“swap [where] letters begin”)
OnlyName Firstname Lastname → Firstname uses only the first name (everything left of the first space) of the username as password
OnlySurname Firstname Lastname → Lastname uses only the last name (everything right of the first space) of the username as password
ReplaceFirst_X-x administrator → @dministrator (ex: %ReplaceFirst_a-@%) replaces the first occurrence of X with x in the username (needle and replacement can be more than 1 char)
ReplaceFirstI_X-x Administrator → @dministrator (ex: %ReplaceFirstI_a-@%) case insensitively replaces the first occurrence of X with x in the username (needle and replacement can be more than 1 char)
ReplaceLast_X-x administrator → @dministrator (ex: %ReplaceLast_a-@%) replaces the last occurrence of X with x in the username (needle and replacement can be more than 1 char)
ReplaceLastI_X-x Administrator → @dministrator (ex: %ReplaceLastI_a-@%) case insensitively replaces the last occurrence of X with x in the username (needle and replacement can be more than 1 char)
ReplaceAll_X-x administrator → @dministrator (ex: %ReplaceAll_a-@%) replaces all occurrences of X with x in the username (needle and replacement can be more than 1 char)
ReplaceAllI_X-x Administrator → @dministrator (ex: %ReplaceAllI_a-@%) case insensitively replaces all occurrences of X with x in the username (needle and replacement can be more than 1 char)
DomainRemoveNumerics test-123.com → test-.com removes all digits from the domain
DomainRemoveLetters test-123.com → -123. removes all letters from the domain
DomainRemoveOtherSymbols test-123.com → test123com removes all non-alphanum chars from the domain
OriginaldomainInsert %OriginaldomainInsert%(N)SOMESTRING → SOMEdomainSTRING (ex: N = 4) insert domain after Nth character of SOMESTRING
OriginaldomainPart test-123.com → 123com (ex: %OriginaldomainPart%(6)) takes the last N chars of the domain name (ignoring any dots)
OriginaldomainNumsBeginInverse test-123.com → test-moc.321 keeps all non-digits at the beginning of the domain and reverses the rest (“invert [from where] nums begin
OriginaldomainNumsBeginSwap test-123.com → 123.comtest- swaps all non-digits at the beginning of the domain with the rest (“swap [where] nums begin”)
OriginaldomainNumsEndInverse 123-test.com → 123moc.tset- keeps all digits at the beginning of the domain and reverses the rest (“invert [where] nums end”)
OriginaldomainNumsEndSwap 123-test.com → -test.com123 swaps all digits at the beginning of the domain with the rest (“swap [where] nums end”)
OriginaldomainLettersBeginInverse test-123.com → test-moc.321 keeps all non-letters (i.e. digits, special chars) at the beginning of the domain and reverses the rest (“invert [from where] letters begin
OriginaldomainLettersBeginSwap 123-test.com → test.com123- swaps all non-letters (i.e. digits, special chars) at the beginning of the domain with the rest (“swap [where] letters begin”)
OriginaldomainLettersEndInverse test-123.com → testmoc.321- keeps all letters at the beginning of the domain and reverses the rest (“invert [where] letters end”)
OriginaldomainLettersEndSwap test-123.com → -123.comtest swaps all letters at the beginning of the domain with the rest (“swap [where] letters end”)
Originaldomainleft test-123.com → test-123 takes the left part of the domain (everything left of the first dot) and lowercases the first character
OriginalDomainleft test-123.com → Test-123 takes the left part of the domain (everything left of the first dot) and capitalizes the first character
Originaldomainright test-123.com → test-123 takes the right part of the domain (everything right of the first dot) and lowercases the first character
OriginalDomainright test-123.com → Test-123 takes the right part of the domain (everything right of the first dot) and capitalizes the first character
Originaldomain uses the plain domain name
OriginalDomain test-123.com → Test-123.com uses the domain name and capitalizes the first character
NiamodLowercase abc%NiamodLowercase%123 abc123
niamodLowercase test-123.com → Moc.321-tset reverses and lowercases the domain name, first character capitalized
niamodUppercase test-123.com → mOC.312-TSET reverses and capitalizes the domain name, first char lowercase
domainleftHyphen test-123.com → test takes everything left of the first hyphen
DOMAINLEFTHYPHEN test-123.com → TEST takes everything left of the first hyphen, capitalized
DomainleftHyphen test-123.com → Test takes everything left of the first hyphen, first char capitalized
domainrightHyphen test-123.com → 123.com takes everything right of the first hyphen
DOMAINRIGHTHYPHEN test-123.com → 123.COM takes everything right of the first hyphen, capitalized
DomainrightHyphen test-abc.com → Abc.com takes everything right of the first hyphen, first char capitalized
domainleftUnderscore test_123.com → test takes everything left of the first underscore
DOMAINLEFTUNDERSCORE test_123.com → TEST takes everything left of the first underscore, capitalized
DomainleftUnderscore test_123.com → Test takes everything left of the first underscore, first char capitalized
domainrightUnderscore test_abc.com → abc.com takes everything right of the first underscore
DOMAINRIGHTUNDERSCORE test_123.com → 123.COM takes everything right of the first underscore, capitalized
DomainrightUnderscore test_abc.com → Abc.com takes everything right of the first underscore, first char capitalized
DomainReplaceFirst_X-x EXAMPLE-attack.com → EXAMPLE-@ttack.com (ex: %DomainReplaceFirst_a-@%) replaces the first occurrence of X with x in the domain (needle and replacement can be more than 1 char)
DomainReplaceFirstI_X-x EXAMPLE-attack.com → EX@MPLE-attack.com (ex: %DomainReplaceFirstI_a-@%) case insensitively replaces the first occurrence of X with x in the domain (needle and replacement can be more than 1 char)
DomainReplaceLast_X-x EXAMPLE-attack.com → EXAMPLE-att@ck.com (ex: %DomainReplaceLast_a-@%) replaces the last occurrence of X with x in the domain (needle and replacement can be more than 1 char)
DomainReplaceLastI_X-x EXAMPLE-attack.com → EXAMPLE-att@ck.com (ex: %DomainReplaceLastI_a-@%) case insensitively replaces the last occurrence of X with x in the domain (needle and replacement can be more than 1 char)
DomainReplaceAll_X-x EXAMPLE-attack.com → EXAMPLE-@tt@ck.com (ex: %DomainReplaceAll_a-@%) replaces all occurrences of X with x in the domain (needle and replacement can be more than 1 char)
DomainReplaceAllI_X-x EXAMPLE-attack.com → EX@MPLE-@tt@ck.com (ex: %DomainReplaceAllI_a-@%) case insensitively replaces all occurrences of X with x in the domain (needle and replacement can be more than 1 char)
niamod test-123.com → moc.321-tset reverses the domain name
Niamod test-123.com → Moc.321-tset reverses the domain name, first char capitalized
domainleft TEST-123.com → test-123 everything left of the first dot, lowercased
DOMAINLEFT Test-123.com → TEST-123 everything left of the first dor, capitalized
Domainleft test-123.com → Test-123 everything left of the first dot, lowercased but first char capitalized
domainright TEST-123.com → com everything right of the first dot, lowercased
DOMAINRIGHT Test-123.com → COM everything right of the first dor, capitalized
Domainright test-123.com → Com everything right of the first dot, lowercased but first char capitalized
domain TEST-123.com → test-123.com domain name, lowercase
Domain TEST-123.com domain name lowercased, first char capitalized
DoMaIn test-123.com → TeSt-123.cOm domain name in alternating case, starting with uppercase
dOmAiN test-123.com → tEsT-123.CoM domain name in alternating case, starting with lowercase
DOMAIN test-123.com → TEST-123.COM domain name capitalized
NIAMOD test-123.com → MOC.321-TSET domain name reversed and capitalized

# Introduction

When analyzing malware, one often has to deal with lots of tricks and obfuscation techniques. In this post we will look at several obfuscation and anti-analysis techniques used by the malware Trickbot, based on the sample 8F590AC32A7C7C0DDFBFA7A70E33EC0EE6EB8D88846DEFBDA6144FADCC23663A from mid of December 2018.

After analyzing and understanding the obfuscation techniques, we will take care of deobfuscating the malware with IDA Python in order to make the code easier to analyze in Hexrays’ decompiler.

## Related Work

With a malware as wide spread and publicly known as Trickbot, there is already a lot of research. Some intersections with this article can be found in the work of Michał Praszmo at https://www.cert.pl/en/news/single/detricking-trickbot-loader/, where some of the obfuscation features are touched. A similar, but more in-depth analysis from Hasherezade can be found at https://blog.malwarebytes.com/threat-analysis/malware-threat-analysis/2018/11/whats-new-trickbot-deobfuscating-elements/ in conjunction with the tutorial on https://www.youtube.com/watch?v=KMcSAlS9zGE. Also Vitali Kremez explained the string obfuscation of a Trickbot sample at https://www.vkremez.com/2018/07/lets-learn-trickbot-new-tor-plugin.html

If you put the unpacked binary in IDA, you can see that Trickbot has several imported functions:

Yet, the first line of the decompiled wWinMain() shows lots of function calls relative to the address stored at dword_42A648.

Looking at the x-refs of this address, we can find out in which context it is written to:

Decompiling the function sub_402D30 () shows that dword_42A648 points to a buffer of 0x208 bytes (or 129 DWORDs). The buffer is modified in the same function with a call to sub_40C8C0().

Note that stru_42A058 holds a pointer to a structure which we will get to know in the following function, as it is an argument for the function call to sub_40C8C0(). This call is done 8 times in a loop as you can see in line 21 to 27.
Within sub_40C8C0() Hexrays’ decompiler shows the following picture:

We can see the following things:
The argument called “hModule” by IDA is a pointer to a structure. Its first DWORD contains a hint to a string used in LoadLibraryW() in line 12 and 13.

The second DWORD of the structure is used in line 14, 16, 18 and 21 and contains a list of hints to function names used in GetProcAddress().

The third DWORD of the structure is used to mark the end of the list of the second DWORD in the for-loop in line 16.

The fourth DWORD of the structure is used in line 15, 16 and 20 and points to a list of offsets, which is used to calculate the address to store the imported functions, with the base of our previously allocated 0x208 bytes array.

Putting it all together, our structure is defined as follows:

struct IATstruct
{
DWORD offsetForDecryptionDLLname;
DWORD offsetForDecryptionImportNamesArrayStart;
DWORD offsetForDecryptionImportNamesArrayEnd;
DWORD IAToffset;
};

Now our function looks much nicer:

It is now obvious that our mysterious buffer of 0x208 bytes is actually an IAT which is stored on the heap. The pointer to the IAT is located at dword_42A648 and the 383 x-refs to this address which we saw in the beginning are mostly calls to this IAT.

# Decrypt All Strings

Now the question remains, what the functions sub_407110() and sub_405210() are doing to yield library- and function names. When disassembling them, you can see that both call sub_40E970(). Only the first one, sub_407110(), has an additional call after that, but that is only used to transform a string to a wide string:

So the actual magic happens in sub_40E970():

We see a single call to sub_404080(). But most important is the first function argument, which adds a1 to the base address of the label which IDA called “Src”. Looking at Src, we can see it is a table with offsets to some scrambled strings:

So the argument a1 is simply an offset to the table pointed to by Src and decides which of the strings is provided to sub_404080().

When looking at sub_404080(), we can see a function which has over 100 lines of disassembled code. I just chose the most relevant part to display in a screenshot:

Without going too much into details, you can see that from line 44 to 63 a substitution takes place based on the first function argument (copied to “Dst”) and the string pointer named off_42A050 by IDA in line 44. The string looks like this:

From line 64 to 69 the previously substituted bytes are then mangled by some bit operations, where four input bytes are mapped to three output bytes. According to the blog of Vitali Kremez mentioned above, this was once a base64 algorithm with a custom alphabet. It still is similar to that, but is seems to be extended by the bit manipulation operations.

Putting it all together, we now know that each string of the IAT is scrambled by a substitution cipher and a bit manipulation algorithm. The function arguments provided in the functions sub_407110() and sub_405210() from the IAT algorithm described previously are offsets to string pointers to the scrambled strings stored at 0x00427C1C, called “Src” by IDA.

We also know that sub_407110() returns a wide string, while sub_405210() returns an ANSI string.
When cross referencing those two functions, we can see 159 and 52 calls to them:

Looking at the calls, we can see that the argument, which describes the string offset, is pushed on the stack as second function argument, in our case 73h. The pointer to the output string is the first argument:

Looking a bit further, we can find a third function sub_4019F0(), which calls sub_40E970() for decrypting strings. Again, the argument is provided via a push of a constant number.

So we can write a simple IDA python script to decrypt all strings and print them. The algorithm is quite simple:

1. Manually identify all three functions which call sub_40E970()
2. For each xref to one of those three functions:
1. Disassemble backwards until we find the first push which is a number
2. Add the base address of the crypted string table to find the referenced string
3. Decrypt the string based on the reversed algorithm

The output looks like this (note that line breaks are not encoded, but do actually break the lines):

We can also adapt our algorithm to print us the import address table, since we know the structure used in sub_40C8C0() to build the IAT:

1. Take the pointer at stru_42A058
2. Convert the values stored there into an array of the structure “IATstruct” (described previously) with eight array elements
3. For each of those eight elements:
1. Decrypt the first DWORD as DLL name
2. Iterate from second to third DWORD and decrypt them to get all imported function
3. Take the fourth DWORD as an offset where the function is placed on our IAT on the heap

Printing the IAT as a dictionary looks like this:

One thing that always bugged me is that it is trivial to add comments in the disassembly in IDA. But since I use the decompiler a lot, I wanted to add my decrypted strings as comments in the decompiler view.

After rather unsatisfying google searches, I spent hours in IDA’s API documentation, read a bunch of existing IDA plugins to look for hints and tried out a lot. Turns out my IDA 6.9 is very crashy when working with IDA Python and also the documentation is not always as helpful as one would like it to be.

But I finally succeeded with a lot of try and error and a little bit of brute forcing:
First you need to translate the address of the disassembly to the function line of the decompiled code. Then by using a ctree object, you can place a comment there. Unfortunately, the ctree object needs to have the correct “item preciser” (ITP). An ITP specifies if a comment is e.g. placed in a line of an “else”, “do”, “opening curly brace”, and so on.
If you set the incorrect ITP to your ctree object, your comment is “orphaned” and won’t be placed correctly.

I still do not understand how I know which ITP I should use, so I developed a little brute force algorithm:

1. Delete all orphaned comments from current function
2. For each possible ITP:
1. Set comment with current ITP
2. If no orphaned comment exists, break loop

This algorithm is rather stupid. But after spending too much time on this issue, I was finally happy to have something that works.

The result looks like this:

# Setting Function Information

Being able to decrypt all strings and setting them as comments in the decompiled code helps a lot when reversing the binary. What is missing is a properly useable IAT. We already know that the IAT is constructed during runtime on the heap.

Function calls to the IAT look like this:

The first two lines of the decompilation look as follows in the disassembly:

You can see in the first line that dword_42A648 is copied to eax, and eight lines later the offset 0xBC is added until a call to register ecx is executing the WinAPI call. The last five lines show a second WinAPI call in a simpler fashion, with only one function argument.
The mov instructions in line five to eight are irrelevant for the function call, but the compiler decided to put them between the three pushes for the function arguments of the first call anyways.

The idea of how to fix this is quite simple. Yet, the implementation of the idea turned out to be way more complex:
We write a light weight implementation of a taint tracker and track the usage of dword_42A648, which holds a pointer to the IAT, to find all WinAPI calls. For each call, our taint tracking provides us the offset within the IAT, so we know which WinAPI is called. In our previous example, we would start with eax, which gets a copy of dword_42A648. Then track eax until it is copied to ecx with the offst 0xBC. Then we track ecx until we see a call to ecx. Thus we know that the IAT offset 0xBC is used at this specific call.

In order to tell IDA what kind of return value and parameters each IAT call has, we need to do some more magic. First, we need to import all function definitions we need. E.g. for “SetCurrentDirectoryW” we need to define a function like this “typedef BOOL __stdcall SetCurrentDirectoryW(LPCWSTR lpPathName);”. We import those function definitions as local types in IDA.

In the second step, we create a local structure which reflects our IAT. So instead of only naming the pointer e.g. “CreateThread”, we also use the type CreateThread, which we imported as local type.

This IAT structure is then applied to the address dword_42A648, so we can see which function is called when dword_42A648 is referenced. The decompilation of e.g. sub_402B00 then looks like this:

We can see three calls to WinAPIs and their corresponding names in line 18, 31 and 34. Yet, neither the number of function arguments are correct, nor are their types identified properly. For example, in line 18 IDA shows five function arguments where there should be four and in line 31 there is one where there should be three.

Additionally, the structure PSECURITY_DESCRIPTOR is not set as the third argument in line 18, instead IDA set it to void*. And instead of LPSECURITY_ATTRIBUTES, IDA uses an int* in line 31.

In order to fix this, we can now leverage our taint tracking information and define each call with its corresponding function by using the IDA Python functions apply_callee_tinfo() and set_op_tinfo2() of the idaapi. This triggers IDA’s magic and populates the added information to the disassembly, so that even stack variables are redefined and renamed.

We can now see that the function calls have the correct number of arguments as well as the correct types of arguments. Also the stack variables got redefined and renamed correctly.

You always know you are going down a very dark trail when the IDA Python functions you are using have less than 10 hits on google and most of those hits are just copies of the same text.
Yet, I found the existing IDA Python script “apply_callee_type.py” from Jay Smith on https://github.com/fireeye/flare-ida/blob/master/python/flare/apply_callee_type.py extremely helpful in understanding how to do such magic in IDA.

The final pseudo algorithm looks like this:

1. Iterate over the decrypted IAT and for each imported function:
1. Look up function definition in IDAs database
2. Import function definition to local types for later use
2. Create IAT structure and import it as local type called “IAT”
3. Set dword_42A648 as type “IAT”
4. For each read-reference to dword_42A648
1. Get the register which holds dword_42A648
2. Disassemble forward until the register is copied with an offset to a new register, remember the used offset
3. Disassemble forward until the new register is called, remember this address
4. Depending on the used offset, look up the function definition of the IAT function
5. Apply function definition to current address

# Conclusion

In the first part we have learned how Trickbot obfuscates its strings and how we can leverage static code analysis in order to deobfuscate the strings and put them in a useable format in IDA.

In the second part we analyzed how the dynamically created import address table of Trickbot can be restored and how IDA can be instrumented to process the data types of the imported functions to get a nice and clean decompilation result.

Additionally, I would like to thank the Trickbot authors for the interesting and partially challenging malware.

You can find the IDA Python scripts on https://github.com/GDATAAdvancedAnalytics/IDA-Python

# Introduction

GandCrab is a ransomware that has been around for over a year and steadily altered (I explicitly do not say “improved”) its code. The author(s) version their builds, the version I analyzed in this blog post is GandCrab’s interal version 4.3 with the Sha256 c9941b3fd655d04763721f266185454bef94461359642eec724d0cf3f198c988.

GandCrab has been around for a while, but gained relevance for us, when we received incoming requests for incident response engagement, primarily from medium-sized companies. On the 24th of August 2018 GandCrab started to push some e-mail based campaigns against German speaking countries, as already described by our esteemed colleague Hauke here https://www.gdata.de/blog/2018/09/31078-professionelle-ransomware-kampagne-greift-personalabteilungen-mit-bewerbungen-an (G DATA’s corporate blog is typically obfuscated in German).

In the meantime Bitdefender released a tool to decrypt several variants of GandCrab, including the analyzed one https://labs.bitdefender.com/2018/10/gandcrab-ransomware-decryption-tool-available-for-free/

To the best of our knowledge, the tool does not use any flaw in the encryption of GandCrab, but it uses a copy of the master private key, which can be used to revert the whole encryption. Details on how the encryption is done by GandCrab can be found later on in this article.

# Starting the analysis

## Unpacking

The step of unpacking the sample will be skipped here, as it takes around 30 seconds if you have the correct setup and know what you will expect in the unpacked form. At our first encounter with the sample, we didn’t know what to expect, so it took us a few minutes.

## Removing the junk code

When putting the sample into IDA, you are first greeted by a scrambled main function, which trips IDA a bit up.

After rolling my eyes and being afraid I had not unpacked the sample properly, I looked at some random functions identified by IDA and noticed, that most of the code looked readable, but several functions also had the same anti-disassembling trick.

Hoping to see a cool VM packer or some advanced obfuscation tricks, I started to analyze the junk code, which starts at the first call in line number 3.

Obviously, the two conditional short jumps two instructions later point to a location which was not properly disassembled by IDA. After fixing the disassembly of the jump target, the code looks like this.

So, reading the disassembly, we have a call, which only pushes the return address on the stack. This return address, being the topmost stack element, is then increased by 0x11. In the next step, depending on the state of the ZF bit (or simply “zero flag”), either the JNZ or the JZ condition triggers and jumps to the pop eax, jmp eax instructions, which pop the altered return address from the stack and jump to it. Disassembling the jump target two bytes after the jump itself yields us the following result:

We can see that the jmp eax leads us to the call to address 0x40414B. Since afterwards the ExitProcess is called, we can assume that 0x40414B is the main function of GandCrab. Disassembling this function in IDA looks like this:

Well, we’ve seen this byte sequence at the function prologue somewhere before…

In case you’re only reading the text and not really looking at the pictures, you might have missed that the function prologue is not only looking the same for both functions we have seen so far, but it is the very same byte sequence.

Also, IDA did not notice that a new function starts at address 0x40414B, which is why it placed the “loc_40414B” label there.

After succeeding in decompiling the function when simply NOPing out the junk instructions by hand, I wrote a short IDA python script to patch all locations where the junk instructions where:

import idaapi
tmp = "E8 00 00 00 00 3E 83 04 24 11 75 05 74 03 E9 28 14 58 FF E0 00 E9"
patchbytes = "\x90\x90\x90\x90\x90\x90\x90\x90\x90\x90\x90\x90\x90\x90\x90\x90\x90\x90\x90\x90\x90\x90"

cur = 0
while  cur != 0xffffffffL:
cur = FindBinary(0, SEARCH_DOWN, tmp)
print hex(cur)
idaapi.patch_many_bytes(cur, patchbytes)

The Python script prints each location where it patched something, so I could then check if IDA detected this location as the start of a function and test if I could decompile it. Of course, defining a function could also be done by a script, but for 29 functions, this was still doable by hand, and the IDA API is also not the most intuitive API to use when you’re in a bit of a hurry.

So yep, patching was rather trivial, as Fortinet confirmed:https://www.fortinet.com/blog/threat-research/a-chronology-of-gandcrab-v4-x.html

# Following the execution flow

After a few small hurdles described before, we can start looking at GandCrab and analyze the execution flow step by step.

Before doing so, here is a reference of what we’re going to see and which functions calls which one. Since there will be a lot of function calls and returns, it is easy to get lost, so take this as a reference (maybe put it on a second screen, print it, open it in a second tab, …) while you’re reading the rest of this article:

main
----Eleveate Privileges
----closeRunningProcesses
----mainFunction
--------bIsSystemLocaleNotOk
--------bCheckMutex
--------decryptPubKey
--------0x00401C56
--------0x00405B7D
--------encRC4
------------0x004047BD
------------contactCnC
--------startEncryption
------------decryptFileEndings
------------createRSAkeypair
------------saveKeysToRegOrGetExisting
----------------getKeypairFromRegistry
----------------encrypPubKey
--------------------getRandomBytes
--------------------importRSAkeyAndEncryptBuffer
----------------writeRegistryKeys
------------createUserfileOutput
------------concatUserInfoToRansomNote
--------------------enumNetworksAndEncrypt
------------------------loopFoldersAndEncrypt_wrapper
----------------------------loopFoldersAndEncrypt
--------------------------------...
----------------prepareEncryption
--------------------loopFoldersAndEncrypt
------------------------0x0040512C
------------------------0x004053FD
------------------------0x00405525
------------------------encryptFile
----------------------------bIsFileEndingInBlacklist
----------------------------bIsFilenameOnBlacklist
----------------------------encryptionFunc
--------------------------------...
----AntiStealth
----deleteSelfWithTimeout

We’re beginning at what I call the main function at 0x0040414B. It starts very simple:

The first function call to a function named “nullsub_1” by IDA is, as the name already implies, nothing interesting:

Those kinds of functions are often generated by compilers if you remove the content of a function by preprocessor directions like “#ifdef DEBUG”. I suppose the author(s) of GandCrab either placed some debug string or breakpoint there when compiling the debug version. And since we are looking at the release compilation, the function is empty.

## Elevating privileges

The next part of the main function is a simple check if the sample is running on Windows Vista or newer. If so, GandCrab checks if the current process is running with integrity level low or even lower. If that is also the case, a very cheap but simple trick is used to gain normal user privileges:

By calling the WinApi ShellExecuteW with “%windir%\system32\wbem\wmic” as “lpFile” parameter, “runas” as “lpVerb” parameter and “process call create \”cmd /c start %s\”” as “lpParameters”, GandCrab starts a process that asks the user to execute a command line with normal user privileges, which in turn starts GandCrab from the command line. After the new process is started, the initial process ends itself by calling ExitProcess(0).

As you can see in the first lines of the screenshot, GandCrab obfuscates the strings by filling the wchar array during runtime with mov instructions. This is also a well-known trick for string obfuscation.

Given the distribution methods of GandCrab, where also Exploit Kits are used, this kind of functionality makes sense: Most exploit kits nowadays only deliver exploits to gain code execution via bugs in browsers or browser plugins. And all major browsers try to sandbox their worker processes in low or even zero privileged process containers. So, a successful exploit against a modern browser will initially run with low or zero integrity level, unless a second exploit is fired to elevate the processes privileges.

GandCrab goes the easy way and instead of firing a privilege escalation exploit, it simply asks the user for more privileges, but does this in a very sneaky way. Those user level (aka. medium integrity) rights are needed to later encrypt the user’s files.

In the above screenshot you can see what happens if you run GandCrab on a German Windows 8 with low integrity: The UAC dialogue pops up.

You might have noticed that the whole privilege escalation is wrapped in a loop with 100 tries, which makes it very dangerous for average users. You either have to click “No” 100 times, or you execute GandCrab with medium integrity.

## Ensuring File Access

So GandCrab either already has enough privileges, or it tries to start a new process with enough privileges with the user’s help. In any case the execution flow then goes back to the main function, where the next call to 0x00403F7D, a function which I named “closeRunningProcesses()”, takes place.

First, GandCrab fills an array, called lpString1 by IDA in the screenshot, with string pointers. Then, by using the CreateToolhelp32Snapshot API with the TH32CS_SNAPPROCESS flag, it iterates over all running processes and checks each process name against the list in the lpString1 array. Each matching process is being opened and terminated, if GandCrab gets the according process handle.

The full list of process names is:

msftesql.exe, sqlagent.exe, sqlbrowser.exe, sqlwriter.exe, oracle.exe, ocssd.exe, dbsnmp.exe, synctime.exe, agntsvc.exeisqlplussvc.exe, xfssvccon.exe, sqlservr.exe, mydesktopservice.exe, ocautoupds.exe, agntsvc.exeagntsvc.exe, agntsvc.exeencsvc.exe, firefoxconfig.exe, tbirdconfig.exe, mydesktopqos.exe, ocomm.exe, mysqld.exe, mysqld-nt.exe, mysqld-opt.exe, dbeng50.exe, sqbcoreservice.exe, excel.exe, infopath.exe, msaccess.exe, mspub.exe, onenote.exe, outlook.exe, powerpnt.exe, steam.exe, sqlservr.exe, thebat.exe, thebat64.exe, thunderbird.exe, visio.exe, winword.exe, wordpad.exe

Using my favorite open source intelligence tool, called Google, and searching for some of those process names, you can find a list from a Cerber sample which around two years ago did the very same thing. https://www.bleepingcomputer.com/news/security/cerber-ransomware-switches-to-a-random-extension-and-ends-database-processes/

The only difference is, that the list of Cerber has less entries. Yet, GandCrab seems to have copied the exact list. Even the order of items is nearly the same. GandCrab only added some entries at the end of the list.

The reason for this feature is as follows:
Those processes might have open handles on important files, which might block GandCrab in getting a writeable handle on those files when trying to encrypt them. So it kills those processes to ensure it can access the files which otherwise might be blocked.

## The MainFunction

Once GandCrab has ensured that a bunch of processes have been killed, the execution flow goes from the main function to a function which I called mainFunction at 0x0040398C. It might not have been my brightest idea to name the first function “main” (0x0040414B) and one of the following sub-functions “mainFunction” (0x0040398C), but let‘s stick with it for now.

In this function most of the GandCrab functionality takes place. Anti-Debugging/Emulator/Sandbox tricks, gathering and sending telemetry to the C&C, threading, encryption, as well as taunting IT security companies.

As this function is a bit bigger, I cut the screenshots in parts to explain the single steps.

### GandCrab does not like Emulators and Sandboxes

We start with a simple Anti-Emulator/Sandbox trick: By Calling OpenProcess() with invalid arguments, and a subsequent check for the error code, GandCrab ensures that no one is fiddling with the OpenProcess-API. Some simple emulators or sandboxes might always return “success” and will thus not set the expected error code, which is probably what this part is trying to detect.

### GandCrab is afraid of Russians and Cyrillic keyboards

In bIsSystemLocaleNotOk() at 0x00403528, GandCrab looks if a Russian keyboard is installed, or if the system’s default UI language is on the internal blacklist. In both cases GandCrab stops its execution and deletes itself from the system by calling deleteSelfWithTimeout() at 0x004032CE.

### There can be only one GandCrab

The next check in bCheckMutex() at 0x00403092 tries to avoid that several instances of GandCrab run at the same time. By creating a named mutex via CreateMutexW() and subsequently checking for the error codes ERROR_ACCESS_DENIED and ERROR_ALREADY_EXISTS, it ensures that the mutex is created and the function fails if the mutex already exists.

With the mutex name, GandCrab starts its first taunt against Ahnlab. According to https://www.fortinet.com/blog/threat-research/a-chronology-of-gandcrab-v4-x.html, the text in the picture behind the link in the string buffer translates to “I added you to my gay list. I used a pencil for the time being“. Since I don’t speak Russian, you have to take Fortinet’s word for the translation.

### Shipping its own public key

With the next call to decryptPubKey() at 0x004038DA, a public key stored in the .data section is decrypted. The decrypted key is put on heap memory and the pointer to the memory stored in a global variable for later use.

The public key is first XORed with 5 and afterwards decrypted with the Salsa20 stream cipher. The decryption key for the stream cipher is a greeting to the inventor of the Salsa20 algorithm, Daniel Bernstein, who is also addressed by his Twitter handle @hashbreaker.

### Im in ur machine, stealing ur infoz

In the next two subsequent calls to 0x00401C56 and 0x00405B7D (not shown in any screenshot), GandCrab initializes an internal structure and then fills it with information about the current system.

Most of the data in this structure is organized in groups of three. The first element is a boolean value set during initialization of the structure which controls if the next two elements are used. Those next two elements are a static name set during initialization and a value calculated during runtime (think of it like a key/value pair in JSON).
E.g.

DWORD bShouldFillDomainName; //set to 0/1 during initialization
DWORD pc_group; //static name
DWORD domainName; //calculated during runtime

By using this format, GandCrab can read the following information from the target computer, if configured to do so:

• User Name
• Computer Name
• Domain Name
• Installed AV Product
• Keyboard Locale
• Windows Product Name
• Processor Architecture
• Volume Serial Number
• CPU Name (as defined in HKLM\HARDWARE\DESCRIPTION\System\CentralProcessor\0)
• Type of each attached drive (as defined by GetDriveTypeW())
• Free disk space of each attached drive

Additionally, a “ransom_id” is calculated by getting the ntdll.RtlComputeCrc32() of the CPU name with the initial CRC 666 as seed, transforming this DWORD into a string and then concatenating the volume serial number of the volume on which Windows is installed.

The whole structure of stolen information is then serialized into a string in the form of  “key1=value1&key2=value2[…]” and then two IDs are added, as well as the version information.

Afterwards the whole string is encrypted with RC4 and the static key “jopochlen” in 0x00404B66, which I called encRC4.

In between those string concatenation functions, you can see another mock of Ahnlab: GandCrab claims to have a possible write-what-where kernel exploit with a privelege[sic] escalation for their security suite Ahnlab V3 Lite. You can read about the analysis of this exploit later in this article.

### GandCrab home phone

Once the information about the infected system has been gathered, a thread is started which pushes this information on the C&C server, starting at 0x004048D7, called internetThread().

This part is rather weird, but very effective in regards to network based IDS/IPS as well as sinkholing attacks against the C&Cs of GandCrab.

It starts with GandCrab decrypting a huge char array with the previously seen XOR algorithm.

As this blob is very huge, I’m not showing it here. It contains 960 different domains and IPs separated by a semicolon. For each of those domains/IPs the function 0x004047BD is called. In this function, several randomized strings are generated, which form a random path for the C&C URI.

The first random string is one of those seven. The seed of the randomness is based on GetTickCount().

The second random string is chosen from one of the eight strings shown above.

The third string is built a little bit more complex. From a pool of 16 two-char strings, one is chosen randomly. Then, depending on further random numbers, between zero and five more times a random string from the same pool is concatenated. The result is later used as file name in the URL’s path.

The fourth and last random string is one of the four file name extensions shown above (since the char* array from the first random string is re-used for the fourth random string, the offset starts at 3 instead of 0, which looks odd in the screenshot).

Then, with the call to wsprintfW(), the URL is built and the function contactCnC() at 0x00404682 is called, which ultimately sends the gathered system information to the C&C server.

In contactCnC() there is not too much interesting to show. The already serialized and RC4 encrypted system information is accessed via a global variable (which is why you can’t see it as an argument in the above screenshot) and is then getting base64 encoded before being transmitted.

Before sending the information to the C&C server as multipart/form-data in a POST-request, GandCrab first contacts the domain with a GET request to decide on the HTTP status code (30x), whether the server should be contacted with HTTP or HTTPS.

What GandCrab does is actually easy to describe, but it poses a few problems for defenders and analysts. Most of the domains/IPs contacted by GandCrab are benign websites from real companies or organizations. So, I assume that GandCrab either sneaked at least one of their C&C domains/IPs in there, or GandCrab compromised one of those legit websites to receive C&C traffic. We didn’t follow up on that aspect so far.

By sending the stolen information to several hundred of domains/IPs, it is hard to block the C&C communication based on domains/IPs, because you would block a lot of benign websites, too.

If you use a network-based IDS/IPS, it is also not trivial to detect or block GandCrab traffic based on the URL, since there are a lot of randomizations in there and it is not easy to tell those URLs apart from legit URLs.

### Encrypt ALL the things!

After starting the thread that calls the C&C server, the mainFunction() initializes three critical sections, of which only one is used at all.

Then, with a call to the function startEncryption() at 0x00402E60 the actual encryption of files on the system starts.

In the first call to decryptFileEndings() at 0x00402E14 a list of file endings is decrypted with the already known XOR loop. This list is later used to exclude files from encryption based on their file ending.

The excluded file endings are:

.ani .cab .cpl .cur .diagcab .diagpkg .dll .drv .lock .hlp .ldf .icl .icns .ico .ics .lnk .key .idx .mod .mpa .msc .msp .msstyles .msu .nomedia .ocx .prf .rom .rtp .scr .shs .spl .sys .theme .themepack .exe .bat .cmd .gandcrab .KRAB .CRAB .zerophage_i_like_your_pictures

As a second step in createRSAkeypair () at 0x00404BF6, a 2048 bit RSA keypair is created. This keypair is then put into to the function I called saveKeysToRegOrGetExisting() at 0x00402B85.

Here are two branches: If the registry path “HKCU\SOFTWARE\keys_data\data” exists, the previously generated keypair is thrown away – WTF? Why generate it in the first place? –  and the registry keys “private” and “public” are read from said path via getKeypairFromRegistry() at 0x0040298D and used further on. Please note that the registry name “private” is actually not only the private key, but a slightly more complex buffer, as you can see when looking at the second branch, in case the registry path does not exist.

The second branch is executed if the registry path does not exist. A call to encrypPubKey() at 0x00402263 is executed.

First a random IV and a random key are generated – getRandomBytes() uses advapi32.CryptGenRandom(), so it is probably really random and not some pseudo random rand() function.

Those two random values are then used to encrypt the private key with Salsa20.

The function importRSAkeyAndEncryptBuffer() imports a public RSA key and uses it to encrypt the provided buffer. Note that not the previously generated public key is used here, but the g_Mem_pubkey, which was decrypted in the beginning of the main function.

In order to understand what is actually encrypted by importRSAkeyAndEncryptBuffer() it is important to know how the structure behind the outBuf pointer looks like, so here is my IDA Local Type definition:

#pragma pack(1)
struct keypairBuffer
{
DWORD privkeySize;
char salsaKey[0x100];
char IV[0x100];
char privKey[0x100];
}

You can now see that all 0x100 bytes starting at keypairBuffer->salsaKey and all 0x100 bytes of keypairBuffer->IV are encrypted.Although the key is only 32 bytes and the IV only 8 bytes long, if you look at the first argument of getRandomBytes(). GandCrab still encrypts the whole buffer, including lots of unused null bytes. ¯\_(ツ)_/¯

Yet, this means, that without the private key of the embedded g_Mem_pubkey, you cannot decrypt the Salsa20 key and IV. And without this Salsa20 key and the IV, you cannot decrypt the locally generated private RSA key.

Unfortunately, this looks like solid use of cryptography to me.

Of course with a call to writeRegistryKeys() at 0x00402AAD the public key of the previously generated RSA keypair is written to the registry key “public” and the whole encrypted keypairBuffer structure is written to the registry key called “private” in the above mentioned registry path.

Back in the startEncryption() function, as a next step the memory of the generated private RSA key is freed in order to avoid having it in clear text in memory during runtime.

Then, with a call to createUserfileOutput() at 0x004023CF, a part of the GandCrab ransom note is generated. The encrypted keypairBuffer is base64 encoded and by using a global variable the RC4 encrypted and base64 encoded system data previously generated in the mainFunction() are concatenated with the following strings:

---BEGIN GANDCRAB KEY---
<base64(RSAencrypted(keypairBuffer))>
---END GANDCRAB KEY---
---BEGIN PC DATA---
<base64(RC4(systemData))>
---END PC DATA---

With the subsequent call to concatUserInfoToRansomNote() at 0x00402C36 the rest of the ransom note is decrypted with the same XOR loop as before, but this time 0x10 is used as XOR key.

Within this text the placeholder {USERID} is searched and substituted with the previously mentioned “ransom_id” (CRC32 over CPU name + Windows volume serial number). The {USERID} string is part of the path parameter of the URL of GandCrab’s hidden service: “http://gandcrabmfe6mnef.onion/{USERID} “

Thus, each machine infected with GandCrab gets a unique ransom note, where the link includes the identifier of the infected machine. Additionally, the ransom note holds all information needed to decrypt a file, if you have the private key belonging to the public key that is stored within GandCrab’s .data section.

It is funny to note that in concatUserInfoToRansomNote() not the already calculated and known ransom_id is used, but the whole previously mentioned internal structure containing information about the current system is built again. Only this time all but one of the bShouldFillDomainName bits are not set. So, the needed values are read and calculated a second time.

By calling startEncryptionsInThreads() at 0x0040211E, GandCrab starts several threads which take care of the encryption:

The first thread starts at the function encryptNetworkThread() at 0x00402097, which will be described in the next subsection.

Then, by calling prepareEncryption() at 0x00401D84, the driveInfos structure gets filled, containing the number of processors minus one (minimum one), the number of drives to encrypt and a list of drives to encrypt.

The list of drives to encrypt is filled by iteratprovides
ing over the alphabet (from A to Z), calling GetDriveTypeA() for each letter and checking if the drive type is DRIVE_REMOVABLE, DRIVE_FIXED, DRIVE_CDROM or DRIVE_RAMDISK. This specifically excludes all drives of type DRIVE_REMOTE, which should be already handled by the thread running encryptNetworkThread().

Back in startEncryptionsInThreads(), after prepareEncryption() has been executed, you can see in the for-loop for each drive, addressed by its drive letter, number of processors minus one threads are spawned, which call the encryptLocalDriveThread() function at 0x00401D1C, which will be described in one of the following subsections.

The main thread then waits for all threads running on the current drive to finish by calling WaitForMultipleObjects(). As soon as one drive is finished and all according threads end, the next drive is encrypted with the same number of threads, and so on.

At the end of the function, the main thread waits until the encryptNetworkThread()-thread has finished by calling WaitForSingleObject().

#### Network encryption – Im in ur network, encrypting ur sharez

The encryptNetworkThread() function at 0x00402097 does nothing more than resolving the computer’s name and providing this information to the function I called enumNetworksAndEncrypt() at 0x00401EA2 together with the crypto keys which were provided in the threadArgs structure.

It is weird to see that the computer name is not actually used in the enumNetworksAndEncrypt() function. So maybe it was once used and the authors forgot to remove it, or it is part of an upcoming feature, which is still in development. Nonetheless, from the control flow point of view it makes no sense to query the computer name here. ¯\_(ツ)_/¯

So the actual beef we are looking for is in the function enumNetworksAndEncrypt().The main part of this function looks like this:

The function has two parts, the upper half and the lower half, each marked by a call to WNetOpenEnumW().

In the first half, a maximum of 128 previously known network disks are enumerated by calling WNetOpenEnumW() with the RESOURCE_REMEMBERED and RESOURCETYPE_DISK arguments.

Then, for each found network resource of type DISK, the function I called loopFoldersAndEncrypt_wrapper() at 0x00401E47 is executed. For each network resource of type RESOURCEUSAGE_CONTAINER, the currently executed function enumNetworksAndEncrypt() is executed recursively to further enumerate the network resources in the found container at the second half of the function.

The second half of the function does pretty much the same as the first half, the only two differences are that for the enumeration the argument RESOURCE_GLOBALNET is used, in order to enumerate the whole network, and not only the previously used resources, and that the argument argument_NetResource is used in WNetOpenEnumW(), which makes the recursive calls possible.

Note that in the first call to enumNetworksAndEncrypt() the argument_NetResource is zero, which starts the enumeration at the root of the network.

To sum it up:
GandCrab first enumerates up to 128 network disks and encrypts them based on all “remembered (persistent) connections”, according to MSDN. Additionally, GandCrab enumerates and encrypts up to 128 network disks starting at the root of the local network. For each resource container a recursion is made.

#### Encrypting local drives – Im in ur machine, encrypting ur local drivez

The encryptLocalDriveThread() function at 0x00401D1C is nothing more than a wrapper around the loopFoldersAndEncrypt() function at 0x00405653, and forwards the crypto keys and the root of where the encryption should start. The function loopFoldersAndEncrypt() is not very pretty to look at, so there is no screenshot to describe everything in one picture, but rather several smaller screenshots. The function takes three arguments: The keys need for encryption, the current path where the files and folders are to be iterated and a boolean value, which is used to avoid iterating and encrypting everything in paths containing the string “Program Files” or “Program Files (x86)”, unless the path additionally contains the string “SQL”.

Before starting to recursively iterate over files and folders, GandCrab does some checks on the current path by calling the function at 0x0040512C.

The function has two ways of returning a value. An output parameter and the classical ret-instruction with eax. If the current folder contains the string “Program Files” or “Program Files (x86)”, the pointer bProgramFiles_1 is set to “true”, and thus returns the information via an output parameter.

If one of the other folders listed above is found, the function’s return value stored in bRet is set to “true” and returned via eax and a ret-instruction.

Note that GandCrab tries to ensure that the system can still be booted by excluding the folders “Boot” and “Windows”, and tries to ensure you can still pay your ransom by not encrypting the Tor Browser files. It also spares all files installed in “Program Files” or “Program Files (x86)”, unless they contain the string “SQL”, as you can see in the encryption loop later on.

Further on in loopFoldersAndEncrypt() it is then checked if the current path is in one of the special folders. If this is the case and the folder is not in “Program Files” or “Program Files (x86)”, the function returns, thus breaking the recursion and not encrypting the files in the current folder.

In the next step, GandCrab creates the ransom note for the current folder with the hard-coded string KRAB-DECRYPT.txt and the text content which was previously calculated by calling 0x004053FD. In case the ransom note already exists, GandCrab also breaks the recursion by returning.

After that, by calling the function at 0x00405525, a lock file is created by the following algorithm:

By mangling the serial number of the drive where Windows is installed with the current day, month and week, as well as some constants, a string is created, which is unique for the current computer at the current day. This string is then used to create a file with the flag FILE_FLAG_DELETE_ON_CLOSE, which keeps the file alive as long as the current file handle is open. The handle is closed after the iteration step in the current folder is finished, thus deleting the file once the current folder and all its sub-folder have been encrypted. In case the file already exists, the recursion loop is broken by returning.

This mechanism is used to synchronize the different threads running in parallel, so that only one thread encrypts the same folder. This means that the file is a marker that a thread is currently recursively running through the current folder.

Note that this only works if GandCrab is not running over midnight, because a change in wDay and wDayOfWeek will change the file name. ¯\_(ツ)_/¯

But, the creation of the ransom note already provides a synchronization token, since it breaks the recursion in case a file with the name of the ransom note is already in the current folder. Additionally, there is another mechanism to avoid encrypting the same file twice later on. 🙂

The actual recursive loop for iterating over files and folders in loopFoldersAndEncrypt() is as simple as it can be:

By using the WinAPI FindFirstFileW() (not in screenshot) and FindNextFileW(), GandCrab iterates over the content of the current folder. If a sub-folder is found, the current function calls itself recursively to iterate the sub-folder. For each file that is found, the function encryptFile() at 0x004054B8 is called.

Note that the function behaves differently, if the bSQLfoldersOnly variable is set. This is the case, if the current folder is in “Program Files” or “Program Files (x86)”. If the folder then contains the string “SQL”, the recursion is executed with the third argument set to “true”, which then implicitly always sets the bSQLfoldersOnly to true. This ensures GandCrab does not encrypt anything in “Program Files” or “Program Files (x86)”, unless it has something to do with SQL.

#### Double check to encrypt only the targeted files

The function I called encryptFile() first does several checks on the current file before it actually encrypts it.

First, the current file’s name is copied and extended with the ending “.KRAB”. Then a check on the original file ending of the current file is done. Here the file ending blacklist mentioned before is used to avoid encrypting files with a certain file ending.

Note that “.KRAB” is on that blacklist, so it avoids encrypting a file twice. Additionally, to keep the system running and bootable, no executables, DLLs, drivers, etc. are encrypted.

Then, by calling bIsFilenameOnBlacklist, GandCrab checks if the current file is one of a hard-coded list of filenames.

This once again ensures that the system stays bootable – you should be able to pay your ransom after all. But since GandCrab does not want to blacklist those files by their extension, because there could be user files with those extensions, GandCrab decided to exclude only a few specific files from encryption.

If the file name is ok and the file has at least two Bytes, the encryption is started by calling encryptionFunc() at 0x00401AA7. After the encryption, the file is renamed into the file name with the .KRAB ending by calling MoveFileW().

#### The actual encryption

The actual encryption of each file takes place in encryptionFunc(). There are two function arguments. The first is the path to the file to be encrypted and the second one is a pointer to a structure I called cryptKeys and is defined as follows:

struct cryptKeys
{
void* pubkey;
void* privkey;
void* privkeySize;
void* pubkeySize;
keypairBuffer *keypairBuffer;
};

Although the struct has several different members, only the pubkey and the pubkeySize are used here (remember that the private key buffer has already been freed at this point).

For each file to be encrypted, a function call to 0x004019F8 creates a new random IV of 8 bytes and a symmetric key for the Salsa20 algorithm of 32 bytes (not in screenshot). Those two random values are then encrypted with the previously created RSA public key and stored in the structure I called encryptionInfoBlob in the following screenshot.

GandCrab is reading the file in chunks of 1 MB, then adds the number of read bytes to the encryptionInfoBlob structure, encrypts the 1 MB blob, moves the file pointer back by 1 MB and writes 1 MB of encrypted data. In case less than 1 MB was read, the sizes are adapted accordingly and the loop finishes.

Once the whole file is encrypted, GandCrab adds the encryptionInfoBlob structure to the end of the file.

The structure looks like this:

struct encryptionInfoBlob
{
byte encryptedSymkey[0x100];
byte encryptedIV[0x100];
LARGE_INTEGER encryptedBytes;
}

So, each file is fully encrypted in chunks of 1 MB with Salsa20, no matter how big the file is. For each file a new Salsa20 key and IV are randomly created and then stored at the end of the file after they are encrypted with the RSA public key, which was newly created during the run of GandCrab. Additionally, the number of encrypted bytes is also added at the end of each file.

Once all encryption threads are finished, the control flow goes way back to the mainFunction(). Here GandCrab deletes the shadow copies of the system to ensure a victim cannot simply restore his/her files.

On Windows Vista or later GandCrab executes “wmic.exe” with the parameter “shadowcopy delete”. On Windows XP it calls “cmd.exe” with the parameter “vssadmin delete shadows /all /quiet” via ShellExecuteW().

Before returning from the mainFunction() to the main(), GandCrab waits for the previously spawned network thread, which is trying to contact the C&C to finish by calling WaitForSingleObject().

## Ransomware with a kernel driver “exploit”

Back in main(), GandCrab calls the function I called AntiStealth() at 0x00401270. The backstory of this function seems to be a somewhat personal feud between the author(s) of GandCrab and the security vendor Ahnlab, who released a “vaccine” against GandCrab. The details can be read here https://www.bleepingcomputer.com/news/security/gandcrab-ransomware-author-bitter-after-security-vendor-releases-vaccine-app/

Previously, when analyzing the gathering of the system information during the MainFunction(), you could see an unused string taunting Ahnlab once again, saying there was a “full write-what-where condition with privelege escalation”, even providing a download link for an exploit proof of concept.

In the blog post mentioned above, the alleged GandCrab author states that the “exploit will be an reputation hole for ahnlab for years”. Well see about that. 🙂

First, the AntiStealth() function parses the time stamp in the PE header of “%windir%\\system32\\ntoskrnl.exe” and saves it for later.
Second, the device with the path “\\.\AntiStealth_V3LITE30F” is opened by calling CreateFileW(). Note that this is the first bug in the driver: It does not set its access rights correctly, if a random process without admin privileges can open a private kernel device.

Then three heap buffers with R/W access rights are allocated by calling VirtualAlloc(), the first two of size 0x200, the third of size 0x10.

After that, GandCrab checks if it is running as Wow64 process and if so, it uses the Heavens Gate technique to call x64 functions of ntdll. On x64 it uses NtDeviceIoControlFile() on x86 simply DeviceIoControl() to communicate with the kernel device.

The actual “exploit” can fit into a single screenshot:

First the IOCTL 0x82000010 is sent with the input buffer as seen above. Then a second IOCTL 0x8200001E is sent, which makes your system bluescreen if everything goes according to plan.

The exploit is also mentioned by https://www.fortinet.com/blog/threat-research/a-chronology-of-gandcrab-v4-x.html. In this blogpost Fortinet states that they “were able to confirm this on Ahnlab V3 Lite 3.3.46.1 with TSFltDrv.sys file version 9.6.0.5“. However, that is not correct:

The two IOCTL are not handled in TSFltDrv.sys, but in a driver called TfFRegNt.sys, of which I analyzed version 4.6.0.1 with the Sha256 2B07F2CA6FC566EF260D12B316249EEEBA45E6C853E5A9724149DCBEEF136839.

In its x86 variant the driver has 275 functions, as identified by IDA and exposes at least a file system minifilter driver functionality.

The function which handles the IOCTLs is placed at 0x0040921C, and it handles the IOCTL used in the exploit like this:

Parsing the user buffer is done like this:

First, a check on the value I called bExInitializeResourceLiteSucceeded is executed. It marks that during initialization of the driver a call to the WinAPI ExInitializeResourceLite() was successful.

Then, by accessing the global variable I called kernelBase, which stores a copy of ntoskrnl.exe, which is read during initialization, and by executing the function getNtoskrnTimeStamplDword() at 0x0040318E, the time stamp in the PE header of ntoskrnl.exe is extracted.

If the second DWORD of the buffer from user space is the correct time stamp, the global variable I called userBufferFirstDWORD is set to the first DWORD in the user buffer.

Comparing this functionality with GandCrab’s code, the variable userBufferFirstDWORD is now set to 0xDEADBEEF.

The second IOCTL is handled like this:

At first, the user buffer is interpreted as a structure, which I called userObj. It has, besides other unimportant members, a length and a “buf”. With this information, several size and sanity checks are executed to ensure that the userObj does not exceed 0xff bytes in size. With the following function call to checkBufferRanges() at 0x0040912E, the driver ensures that userObj is within the user provided buffer.

Afterwards, the driver calls handleUserBuffer() at 0x00408F76 in order to process the user input.

The first two function calls map the MDL address of the IRP to a virtual address and then deserialize a string from the user buffer into a custom object which I called memObj. The memObj, as well as the mapped memory of the MDL and the second element of the userObj are then passed to another function which I called thisWayGoesToCrash() at 0x004052EA.

The first function call to findObjectInList() at 0x00406996 looks up a file handle by iterating over a driver internal linked list, comparing the list objects based on the user input. If some object is found, the function ExAcquireResourceSharedLite_onUserBuffer() at 0x0040606A is called:

You might notice that the first argument of ExAcquireResourceSharedLite() is userBufferFirstDWORD, the very same buffer which was set with the first IOCTL.

When the function call ExAcquireResourceSharedLite() is execute, Windows bluescreens. The crash dump analysis of Windbg looks like this:

Note that the first argument is 0xDEADBF23, which is similar to the address 0xDEADBEEF, which was the first argument of ExAcquireResourceSharedLite().

Looking at the function in Ntoskrnl to see where the crash actually happened, we can see this code:

The red marked area is where the crash happened. You can see a few lines above, that ecx got loaded by the instruction “lea ecx, [edi+34h]”, while edi was holding the first function argument. So 0xDEADBEEF + 0x34 = 0xDEADBF23, which is the memory referenced, which caused the crash.

So, what is happening in Ahnlab’s driver?

With the first IOCTL, you can give the kernel driver a pointer to an object which the kernel driver expects to be a ERESOURCE pointer. With the second IOCTL, the driver tries to acquire the resource object, and thus crashes.

Is this really a “full write-what-where condition with privelege escalation” as the GandCrab authors state? In my humble opinion, no. There is no fully controllable write primitive and the exploit does not show any privilege escalation.

You can specify an arbitrary memory location on which the WinAPI ExAcquireResourceSharedLite() gets executed. So, whatever the API does with the ERESOURCE object, you can do to an arbitrary memory location.

In theory, a very skilled attacker might be able to use this to manipulate a memory address as one gadget. But without any further gadgets, it is very hard to create some kind of real working exploit out of this.

So, I would say this is rather a denial of service bug, than a full write-what-where privilege escalation security issue.

## Covering its tracks

In case the driver bug did not bluescreen the system, GandCrab tries to delete itself by calling the function deleteSelfWithTimeout() at 0x004032CE.

It opens a new command line process, which first calls “timeout -c 5” and then deletes the current file from which GandCrab was started. After the command line process has been started, GandCrab ends its process by calling ExitProcess().

The intention of the timeout is most probably to give the current process enough time to end itself, before the newly started command line tries to delete it. It is funny to note that the command “timeout” has no switch “-c”. I could not make the timeout work with “-c” on Windows XP, 7, 8 or 10. ¯\_(ツ)_/¯

Nonetheless, in all my tests the start of the new process took a few milliseconds longer than exiting the GandCrab process, which is why the deletion always worked, although it is very racy.

# Conclusion

When analyzing GandCrab, I was fascinated by the simplicity of the malware in comparison to its efficacy. This malware does on point what it aims to do: Encrypt as much files as possible as fast as possible with a strong encryption algorithm.
There is not too much unnecessary code, e.g there is no persistency to survive reboots.

One oddity that sticks out is the kernel driver exploit, which is probably intended to show off the GandCrab author(s) skills in order to gain a big media echo, which is important to support GandCrab‘s affiliate model.

## One framework to build them all, one framework to name them, and in their IDBs to bind them

Authors: Luca Ebach, Tilman Frosch

Rejoice everyone, today we pushed bindifflib to our Github! Bindifflib is a framework to build a set of libraries with a set of different compilers, currently the compilers of Visual Studio 2010, 2013, 2015, and 2017 – both 32 bit and 64 bit. After compilation, bindifflib will import all DLLs into IDA Pro and will use the Program Database (PDB) files to properly name (almost) all functions in the IDB.

We have created bindifflib out of the need speed up our understanding of binary code dropped at our doorstep, mostly malware. Occasionally, we encounter larger binaries that smell of statically linked libraries. Figuring out which part of the binary is a library, which library specifically, and which part is actually relevant, often takes some time that could be spent better and is also not very exciting. Fortunately there is BinDiff! Having some functions already identified allows us to spend the time to understand the purpose of the code and the authors‘ intentions instead of first digging through the basic capabilities as provided by knowing that a certain library was used in general. So let‘s just create some targets to diff a unknown binary against, so that we can focus our reversing efforts on the non-generic parts of the binary. Having just one library compiled by one compiler didn’t really cut it, so it was time to automate things to have a selection of likely or frequently used libraries compiled with a set of popular compilers. Naturally, one could also replace „likely“ and „frequently“ by „vulnerable“ and head into a completely different direction that bindifflib can help with.

We consider the code more a proof of concept than a software product. Use at your own risk, we love feedback!

# Introduction

After a destructive cyber attack had hit this year’s olympics, the malware was quickly dubbed Olympic Destroyer.  Talos were fast to provide initial coverage. A malware explicitly designed to sabotage the computer systems of the Olympic opening ceremony sounded very interesting, but other duties were more pressing at that time, so analysis for pure curiosity had to wait.  A few weeks later I had some free evenings on my hands and decided to combine a few interests of mine: Listening to music, consuming high quality whisky and analyzing malware – regrettably one of those things is frowned upon at work, and it’s not malware analysis. 😉

I had most of the binaries reversed and already written up a few pages, when Kaspersky released an article with some more details than previously publicly known. Having finished my work and focusing on the technical aspects of Olympic Destroyer, I think I can add several technical details about the malware. In the following expect plain and straight-forward binary analysis and reverse engineering in the form of a walk-through.

Olympic Destroyer comes in two types. The first one is a little bit simpler. It was discovered by Talos, who published it in their comprehensive blog post. One example of this type has the Sha256 of edb1ff2521fb4bf748111f92786d260d40407a2e8463dcd24bb09f908ee13eb9.

The second type of the binary has, to the best of my knowledge, not yet been explicitly named, but it was implicitly analyzed by Kaspersky in their also very comprehensive blog post. One example has the Sha256 sum of e8349cfcc422310c259688b0226cb14f5196a6daad77b622405282aeac89ab06.

In the following blog post I will mainly describe the first type of Olympic Destroyer. At the end I will discuss the main differences between the two types, which revolve around the usage or non-usage of the well-known tool PsExec.

# The Orchestrator

In this part we will cover the innermost functionality of the Olympic Destroyer. As orientation point we will use the main() function, from where on we will cover the single function calls step by step. Luckily Olympic Destroyer runs single threaded – except for the spreading functionality – which makes it easier to follow the execution one call after another.

The analyzed orchestrator has a Sha256 of edb1ff2521fb4bf748111f92786d260d40407a2e8463dcd24bb09f908ee13eb9 and is 0x1C6800 (~1.7MB) in size. A lot of this size is made up of five resources, whose role will be explained later on. IDA detects 756 functions of which not even ten were automatically identified by IDA FLIRT in version 6.9, which made the analysis more time consuming. IDA version 6.95 seems to have a newer FLIRT database and a lot of functions are identified automatically, as I realized way too late.

## Configuration – or not

The main() function is located at 0x004071E0. It creates a structure on the local stack, which I called “config” when starting to reverse the binary. Over the time I discovered that it is merely a state or a singleton data container – nonetheless I kept the name “config” for reasons of consistency. This structure is carried throughout many of the subsequent function calls, most of the time in the form of thiscalls in the ecx register. You can find the whole structure in the appendix section below.

It contains different type of data, like simple integers, which for example describe the bitness of the OS with either the value 32 or 64 by dynamically resolving and calling IsWow64Process. Yes, the author(s) actually use full integers instead of encoding this information in a simple bit ¯\_(ツ)_/¯.

More interesting are probably the different paths of files dropped to the filesystem during runtime, which are also stored in this structure. I will describe them when writing about the resources.

Additionally, we find some security related variables, like the security token information of the current user, which is gathered by calling GetTokenInformation(TokenUser) and comparing the result against “S-1-5-18” (Local System), “S-1-5-19” (NT Authority Local Service), and “S-1-5-20” (NT Authority Network Service).

Most important is probably the vector of objects, which contains domain names and domain credentials, that are used to spread laterally through the network. More about this later on when I will cover the lateral movement.

After the config structure is initialized in 0x00406390 by nulling its members, it is dynamically filled with its respective values in the subsequent call to 0x00406500, where most of the previously mentioned values and information is generated. From then on, the config is ready for use and most values are only read instead of written – except for the file paths, which are generated more or less randomly on the fly when used and of course the credential vector, which gets expanded a few calls later.

## Magical Code Injections

With a call to 0x004066C0 Olympic Destroyer checks for the existence of two files, which it uses to mark and avoid multiple runs of itself:

1. C:\<MD5(Computer Name)>
2. %SystemDrive%\Users\Public\<MD5(Computer Name\User Name)>

If one of those files is found, the function which I called “selfDeleteInjectBinary” at 0x00405DD0 is executed. Most of this function is already described in Endgame’s blogpost at https://www.endgame.com/blog/technical-blog/stopping-olympic-destroyer-new-process-injection-insights, but somehow they either misinterpreted the feature, or missed the main point of the shellcode. I’m not sure what their intention was, but their blog post somehow does not say what the shellcode actually does ¯\_(ツ)_/¯.

Olympic Destroyer starts an invisible “notepad.exe” by using the flags CREATE_NO_WINDOW in dwCreationFlags as well as STARTF_USESHOWWINDOW in StartupInfo.dwFlags and SW_HIDE in StartupInfo.wShowWindow before calling CreateProcessW.

Then it injects two blocks of data/code into the running notepad by calling VirtualAllocEx and WriteProcessMemory. The first block contains addresses of APIs and the path to the current executable, the second one is a shellcode which uses the addresses of the first block. By calling CreateRemoteThread the execution of the shellcode within notepad is started.

After this injection, the main process exits by calling ExitProcess while the execution of the injected thread runs in the process space of notepad. But all the shellcode does, is a simple delayed self-deletion mechanism:
First it sleeps a configurable number of seconds. In our case it is five seconds. After that, the shellcode looks for the original path of Olympic Destroyer, which was passed by the first injected memory block by checking GetFileAttributesW != INVALID_FILE_ATTRIBUTES. Then it tries to open the file with CreateFileW, gets the file size by calling GetFileSize and then loops over the file size and calls WriteFile with always one zero byte until the whole file is overwritten with zeros. After closing the file handle, the file is finally deleted with DeleteFile and the shellcode calls ExitProcess to end its execution.

To sum it up: The code injection is simply a nulling and deletion mechanism to hide the traces of the main binary.

## Setting the markers for self-deletion

The two files, which mark multiple runs of Olympic Destroyer, mentioned in the previous paragraph, are then created:

Depending on the rights under which the binary is running, the markers in C:\ and %COMMON_DOCUMENTS% are created. From then on, a second run of Olympic Destroyer will wipe the original executable.

## Stealing credentials

In the following call to 0x004065A0, we will dive into a very important feature of Olympic Destroyer: The stealing of credentials from the current system, which are later used for lateral movement.

Olympic Destroyer contains five resources of type “BIN”. All of those resources are encrypted with AES. The calculation of the key is hard coded in the binary and can be described as a trivial MD5 hash of the string “123”. This hash is then concatenated twice in order to reach a key length of 256 bits for the AES algorithm. The evaluation of whether those shenanigans of symmetric cryptography with a hard coded key makes sense is left as an exercise for the reader. 🙂

When stealing the credentials, at first the resource 101 is decrypted, written to a more or less randomly generated filename in %tmp%. “More or less” because the algorithm is based on calls to GetTickCount() with Sleeps in between the calls.

After writing the decrypted resource to disk, a proper random string is generated by calling CoCreateGuid. The GUID is then used as the name for a named pipe in the form \\.\pipe\, which is created by calling CreateNamedPipeW and then used as inter process communication mechanism with the process, which is then started from the file written to %tmp%.

### Resource 101

When resource 101 is started, it also gets the name of the pipe to communicate with its parent process as well as the password “123” as arguments. The main task of resource 101 is to use the password to decrypt and execute another resource of type “BMP” embedded in the file of resource 101 and send a buffer with stolen credentials to its parent process. So, it’s a simple loader which transfers a buffer via IPC.

The BMP resource is a DLL called “BrowserPwd.dll”. This DLL is not written to disk but parsed and loaded in memory. It seems that its only purpose is to steal credentials from the browsers Internet Explorer, Firefox and Chrome. In order to work with Firefox and Chrome, an SQLite library is compiled into the DLL, which makes up most of the DLL’s code.

• For Internet Explorer it uses COM to iterate over the browsers history and then reads all autocomplete passwords from the registry in Software\Microsoft\Internet Explorer\IntelliForms\Storage2 and decrypts them using the WinAPI CryptUnprotectData.
• For Firefox, the credentials are stolen from sqlite and logins.json. The nss3.dll from Firefox is used to decrypt the protected passwords.
• For Chrome, the user’s database in […]\Application Data\Google\Chrome\User Data\Default\Login Data is copied temporarily and then the credentials are read and decrypted by calling the WinAPI CryptUnprotectData.

All stolen credentials are returned in a buffer which uses a special style of separating the single items. This buffer is constructed within the DLL and returned to its original loader, which is resource 101:

This buffer is then sent from the loader via the named pipe to its parent process:

### Resource 102 and 103

After resource 101 was executed, a second attempt to steal credentials is started, in case the current process could acquire debug privileges during initialization of the config object. In case it has those right, depending on the architecture of the operating system, either resource 102 (x86) or 103 (x64) is started. Both executables have the same logic as resource 101 – decrypt and load a DLL in memory, execute the DLL and return its buffer via IPC –  only the payload in form of their internal DLL, the resource of type “BMP”, is different. Everything else stays the same.

So, the question is, what are the DLLs in the resources of resource 102 and 103? For 103, the x64 version, I did not look into it in order to save some time, but I assume it’s the very same payload as in 102, only for x64 systems. For 102, which is an x86 binary, the loaded internal DLL seems to be a custom version of the well-known penetration testing tool Mimikatz, which, besides other nifty features, can dump credentials from a Windows system. I did not spend too much time in the analysis of this DLL, but a swift look (as in “1-2 hours”), compared with several matching functions, structures and strings from the original code of Mimikatz are strong indicators that this DLL has actually Mimikatz’ credential dumping capability. This assumption was also verified by dynamic analysis, where the binary was actually stealing the credentials of my analysis machine. Additionally, the author(s) of Olympic Destroyer named this DLL “kiwi86.dll”, which is a reference to the nickname “gentilkiwi”, who is the author of Mimikatz.

After receiving the stolen credentials via the named pipes, Olympic Destroyer parses the received buffers and saves the credentials in its config structure. Then it returns its control flow to the main function.

## Saving the Credentials – Or how to build a network worm

Back in the main function, right after stealing credentials from browsers and by the power of Mimikatz, Olympic Destroyer creates a copy of itself in the %tmp% folder in 0x00404040. If this copy succeeds, the copied file is modified in the next function call to  0x00401FB0. Here the whole file is read into a buffer in the process’ memory. Then this buffer is searched for the byte marker 9E EC 87 D4 89 16 42 09 55 E2 74 E4 79 0B 42 4C. Those bytes mark the beginning of the serialized credentials vector as an array:

I tried to mark the single elements of the array in different colors to describe them, but it turns out my MS Paint skills are really bad. So, you’ll just get a two pseudo structs defining what you can see around the red marked bytes:

struct credentials
{
byte marker[16];
WORD numberOfElements;
CREDENTIAL credentialArray[numberOfElements];
};
struct CREDENTIAL
{
}

In our case there are 0x2C stolen credentials. The first block of credentials has a username\domain string of 0x1B bytes length and has a password of 0x0C bytes length. Then the second block of credentials follows, and so on.

Once Olympic Destroyer has located the array in its buffer, the array is written over with the serialized version of the current credentials vector of the config object. Then the executable modified in memory is written back to disk in the %tmp% directory.

In other words: The list of credentials, which was present when Olympic Destroyer was executed first, is now updated with all credentials stolen during runtime.

## Resource 104 and 105 – Preparing the next steps

After updating a copy of itself with all stolen credentials, the execution flow returns to the main function where two consecutive calls to 0x00403F30 prepare the network spreading algorithm and the destructive parts. Both calls take a resource name as a first parameter for input and return a string with a path to a file. In this function Olympic Destroyer takes the same decryption algorithm as previously described and decrypts the resources 104 and 105. Both files are not yet executed but written to disk with a random filename in the %tmp% folder.

Resource 104 is a simple copy of the well-known tool “PsExec” which can be used to execute commands and files on remote computers. It will come into play when I describe the lateral movement.

Resource 105 though is the actual “Destroyer” of Olympic Destroyer.

## Starting the Destroyer – Fulfilling the real purpose

After writing resource 104 and 105 to %tmp%, the function at 0x00404220 is called with the path to resource 105 as an argument. Here nothing magical happens. The file from the resource is simply executed without a visible window/console and the function returns:

From here on the destroyer from resource 105 is running. It has its own chapter later on.

## Lateral Movement

Once the destroyer part of Olympic Destroyer has been started in its own process, the main function calls 0x00406ED0 to start the network spreading routine.

At first two sanity checks are made by calling GetFileAttributesA in order to ensure that PsExec from resource 104 and the copy of Olympic Destroyer with the updated credentials list in the %tmp% folder exist. If both checks pass, a list of potential targets within the local network is built:
With a call to 0x00406DD0 Olympic Destroyer utilizes the GetIpNetTable API to enumerate all IPv4 addresses of the current ARP cache, thus getting all IP addresses the local machine had access to – considering ARP cache timeouts which can remove older entries, of course.

The list of IPv4 addresses is then passed to the function at 0x004054E0, along with a pointer to the config object as well as the path to PsExec and the updated copy of Olympic Destroyer in the %tmp% folder. I think it is noteworthy that passing both paths to the files in %tmp% is completely superfluous, since they are already a part of the config object, which is also passed as argument.

The function at 0x004054E0 is the heart of the spreading algorithm:
First, it reads the updated copy of Olympic Destroyer into memory. Then it initializes a new structure with all information passed as arguments as well as some additional information, which is somehow not really used later on. After that it calls 0x00407680, where the spreading in the network begins:
For each IP address, a new thread is spawned, which starts at 0x00407D40. This thread then loops over all credentials of the config object, trying to use WMI via COM objects in order to infect remote computers:

The first important function for that is at 0x004045D0 (called executeRemoteCmdline), which gets one IP and one pair of credentials as input, as well as one command line to execute on the target machine – outPtr is used to transport the return value. The whole function is a mess of COM calls, but I’ll try to explain their meaning anyways. Words in italic are quotes from the binary:
This function creates a COM object of CLSID {4590F811-1D3A-11D0-891F-00AA004B2E24} and IID {dc12a687-737f-11cf-884d-00aa004b2e24} in order to remotely execute WMI commands. Then a connection to \\\root\CIMV2 is created and the credentials are applied by calling CoSetProxyBlanket. With the class Win32_Process and the function Create a Commandline is executed on the remote computer. With “Select * From Win32_ProcessStopTrace” the event for the termination of the remote process is registered in order to read its ExitStatus code afterwards.
The executed command line is rather simple:

“cmd.exe /c (ping 0.0.0.0 > nul) && if exist %programdata%\\evtchk.txt (exit 5) else ( type nul > %programdata%\\evtchk.txt)”

With the execution of ping a short delay is introduced, since the execution waits for ping to fail four times to ping the address 0.0.0.0. Then, in case the file %programdata%\evtchk.txt exists on the target system, the execution returns the exit code five. Otherwise said file is created and the execution finishes with its standard error code of zero.

The return value of the remote command line is then read and is returned via outPtr as a function argument from 0x004045D0. Interestingly the outPtr is only written to in case of a successful remote execution. All error cases leave the outPtr untouched. As the memory address of the target of outPtr is initialized with zeros, the caller of 0x004045D0 is unable to distinguish between an error during the remote code execution (e.g. because of false credentials or an unavailable IP) and the successful write of %programdata%\evtchk.txt file on the remote machine. ¯\_(ツ)_/¯

At 0x00404C30 the second interesting function (called writeFileToRemoteRegistryAndExecuteCommandlineVbs) is located. It takes the target IP address as well as the credentials as input. It is very similar to the function 0x004045D0 described previously. The main difference is that by using the StdRegProv class and the function SetBinaryValue a registry key in HKEY_CURRENT_USER\Environment with the name Data is created on the remote computer. The value of the registry key an executable file, but interestingly it is not the copy of Olympic Destroyer with the updated credential list in %tmp%, as I would have expected, but it is the binary which is currently executed and thus does not contain any of the current system’s credentials:

After the binary is written to the remote registry, the function at 0x00404C30 calls the function at 0x004044B0. Here the function Create of the COM class Win32_Process is used to remotely execute another command line. This command line is already known from the Talos blog post. For readability I pretty-printed the commands:

cmd.exe /c
(
echo strPath = Wscript.ScriptFullName
& echo.Set FSO = CreateObject^(\”Scripting.FileSystemObject\”^)
& echo.FSO.DeleteFile strPath, 1
& echo.Set oReg = GetObject^(\”winmgmts:{impersonationLevel=impersonate}!\\\\.\\root\\default:StdRegProv\”^)
& echo.oReg.GetBinaryValue ^&H80000001, \”Environment\”, \”Data\”, arrBytes
& echo.Set writer = FSO.OpenTextFile^(\”%ProgramData%\\%COMPUTERNAME%.exe\”, 2, True^)
& echo.For i = LBound^(arrBytes^) to UBound^(arrBytes^)
& echo.s = s ^& Chr^(arrBytes^(i^)^)
& echo.Next
& echo.writer.write s
& echo.writer.close
) > %ProgramData%\\_wfrcmd.vbs && cscript.exe %ProgramData%\\_wfrcmd.vbs && %ProgramData%\\%COMPUTERNAME%.exe

The first set of echos outputs parts of a VB script, which are then written to %ProgramData%\_wfrcmd.vbs by using the redirect operator “>”. Afterwards this file is executed via the cscript interpreter before the executable %ProgramData%\%COMPUTERNAME%.exe is executed. This executable is created during the runtime of the newly created VB script, which basically just reads the executable stored in HKEY_CURRENT_USER\Environment\Data and writes it to %ProgramData%\%COMPUTERNAME%.exe.

Back in 0x00405170, the function at 0x004045D0 (executeRemoteCmdline) is called a second time. This time it removes the file %programdata%\evtchk.txt, which was previously checked or created on the remote computer by executing the command line “del %programdata%\evtchk.txt”.

To state the obvious, in case it got lost in all the text: %programdata%\evtchk.txt is intended as a mutex object on the remote computer, which marks that a remote infection is currently ongoing. This avoids that two computers running Olympic Destroyer’s infection routine infect the same target at the very same time. Yet, as this file is deleted right after the infection, it does not avoid multiple infections of the same target in general, but only in parallel.

While all previously mentioned remote infection threads are running, the main thread waits for their termination by calling WaitForMultipleObjects, where it waits for all spawned threads to finish.

Once all threads are finished and back in the function 0x00406ED0, the control flow enters a loop, which iterates over all credentials and passes them to the function at 0x00406780. This function also has the purpose of enumerating network targets. Once again COM objects are involved:
One main part of this function is the call to NetGetDCName, which gets the name of the primary domain controller. This name is formatted into the string “%s\\root\\directory\\LDAP” in order to use it with the same COM objects as before during the remote code execution (CLSID {4590F811-1D3A-11D0-891F-00AA004B2E24} and IID {dc12a687-737f-11cf-884d-00aa004b2e24}) by using the credentials, which are passed as function arguments. If everything works so far, the statement “SELECT ds_cn FROM ds_computer” is executed in order to get all computer names from the current domain. Then, for each computer, by calling GetAddrInfoW and ntohl the domain names are resolved to IPs. A vector of IPs is returned from 0x00406780. The IPs are then passed to the already known function at 0x004054E0 in order to infect those computers remotely.

When this IP enumeration and remote infection loop is finished, some objects and memory is cleaned up before the control flow returns back to the main function.

## Self-Deletion – Or how to hide your traces, well, at least one of the many…

The last step in the main function, before freeing the remaining objects and memory, is the call to the already described function “selfDeleteInjectBinary” at 0x00405DD0. This time the sleep interval is only three instead of five seconds. So the spawned process tries to wipe the binary of the parent process every three seconds until it succeeds. The control flow of Olympic Destroyer then leaves the main function and the process exits, which will make the wiping of the binary possible.

I think it is noteworthy that none of the other dropped files are deleted. Everything in %tmp% remains and also all infection markers described previously are still there.

# The Destroyer

A big part of this component’s functionality can be described in one picture by looking at the main function:

After giving itself the SeShutdownPrivilege and bluntly ignoring all potential erroneous API calls, the Destroyer calls the function at 0x00401000 (“execProcAndWaitForTerminate”) five times in a row in order to:

1. Delete all shadow copies without prompt to avoid restoring the system
2. Silently delete all backups created by the tool wbadmin
3. Ignore all failures during boot and avoid starting the recovery mode
4. Clear system logs
5. Clear security logs

Then the function at 0x004012E8 (“deactivateAllActiveServices”) is called. The name in the screenshot is already a spoiler of the actual functionality: All services of the local computer are disabled. This is done by iterating over all possible types of services by calling EnumServicesStatusW with the dwServiceType parameter set to 0x13F and dwServiceState to SERVICE_STATE_ALL, and then calling ChangeServiceConfigW(SERVICE_DISABLED) for each service.
In combination with the previously disabled recovery mode and deleted backups, this bricks the local system on the next boot.

Back in the main function a thread is spawned which executes the function 0x004016BF (“wiperThread”). The main thread then sleeps for a fixed single hour before shutting down the system – no matter what the wiper thread did or didn’t do in the meantime. Note that this might also interrupt the spreading routine of Olympic Destroyer, which might still run.

The first thing the wiper thread does is setting its own thread priority to THREAD_PRIORITY_TIME_CRITICAL in order to get as much CPU cycles as possible. Then it recursively iterates over all available network resources with the APIs WNetOpenEnumW and WNetEnumResourceW. Each available resource is temporarily mounted by calling WNetAddConnection2W(CONNECT_TEMPORARY), yet the parameters for the username and password are set to zero, thus the current user’s credentials are used. It is important to note that the stolen credentials are not used here. This decouples the Destroyer logically from its parent process.
For each successfully mounted resource the function at 0x00401441 is called.
This function is also best described with a screenshot:

This function simply iterates recursively over all folders, starting at the mountpoint which is provided as an argument, and then destroys each single file that it finds:

1. Files equal or smaller to 1MB in size are completely written over with zeros
2. For files bigger than 1MB only the first 4096 bytes are nulled. Yet for most files this should be enough to render them useless

The wiper thread does not communicate with the main thread and there is no synchronization in any way. No matter if the wiping already finished or not, the system is shut down after one hour.
It might be a simple mistake to shut the system down after a fixed time: The wiping may not have wiped everything it can reach, or it could have already finished and the local computer is still useable until the shutdown. Additionally the remote spreading could still be ongoing.

Yet, I think it is more likely that this feature is a well-planned and sophisticated time bomb: Imagine Olympic Destroyer spreading through a network, wiping all it could wipe for one hour, when suddenly one system after another shuts down and is unable to boot.

# Different types of Olympic Destroyer

As mentioned in the introduction, I found two different types of Olympic Destroyer. The simpler type was described previously. The second type has the very same functionality, it only adds a few more functions. Those additional functions have the purpose of extending the spreading functionality of Olympic Destroyer by leveraging PsExec, which was written to %tmp% but then ignored by the simpler version.

## Using PsExec

The additional function call is placed right after writing/checking the file %programdata%\evtchk.txt and before the spreading function which uses COM objects and spreads the version of Olympic Destroyer which was not updated with the stolen credentials. This bugged behavior of spreading the wrong binary over COM exists in both versions.

The additional call to PsExec is done in the following way:

PsExec is started with several parameters:

• The first three parameter identify the target computer and the credentials which are applied
• Then the dialogue to confirm the EULA of PsExec is skipped with “accepteula”
• “-d” runs PsExec in a non-interactive way, which means that the caller does not wait for PsExec to terminate
• With “-s” the remote process is started with System rights (in case the credentials allow that)
• “-c” and “-f” specify that the actually executed file is copied to the target computer and overwritten in case it already exists
• The last parameter is the remotely executed file, which is obviously Olympic Destroyer

This time the remotely executed binary is the copy of Olympic Destroyer in %tmp%, which was updated with the credentials stolen during the current run.

The output buffer returned from PsExec is parsed for the string “started”, which indicates to Olympic Destroyer that its call was successful. A successful remote infection using PsExec breaks the loop which iterates over the credentials for a fixed target computer. Thus the target is only infected once and the bugged COM infection is avoided.

## A crippled worm and a capable worm

The simple version of Olympic Destroyer has some spreading functionality, although it is broken in the sense that the wrong binary is spread through the network. By not spreading the updated version of Olympic Destroyer, which contains the credentials stolen during the run, it loses a crucial part of its spreading capability:

Assume we have a computer “A” with a logged in user who has the rights which allow remote spreading of Olympic Destroyer. And a computer “B”, which is in reach of A, but where no user is logged in. A third computer “C” is only reachable over B but not over A.
If the simple version of Olympic Destroyer is executed on computer A, it will use the stolen credentials to infect computer B. But on computer B there are no credentials to steal, so it won’t be able to infect computer C.

In other words: The simple version of Olympic Destroyer can only spread to computers which are “one hop” in distance.
Yet, in most cases this should still be enough to infect a whole network, since a central Domain Controller is usually connected to most computers in the network.

Spreading the more advanced version including the stolen credentials gives Olympic Destroyer even better worming capabilities, since it gathers more and more credentials as it spreads further and further through a network.
In the previous example computer C could be infected from computer B by using the credentials stolen on computer A.

## Crunching some numbers

In order to verify my findings with the two versions of Olympic Destroyer, I grabbed 36 different samples which are identified as Olympic Destroyer and compared their sets of stolen credentials. One sample had and empty list of credentials, so I discarded it.

It turns out that 23 of those samples are from the simple version type. All of them contained the same set of credentials, which were already described by Talos. They are for the domains g18.internal and Pyeongchang2018.com. All of the samples contained additional credentials stolen from various sandbox systems and virtual machines of researchers, who probably uploaded the files from the %tmp% folder to Virus Total during their analysis.

I could not find a single sample which contained only a subset of the credentials stolen from the g18.internal and Pyeongchang2018.com domains. If you strip the credentials from sandboxes and researchers, all 23 samples contain the same set of crendetials. This supports the findings that the simple version of Olympic Destroyer has a broken spreading algorithm.

In contrast to that, 12 samples of the total 36 are from the ATOS network with the domain ww930, as partially described by Kaspersky. Apparently the more capable version of Olympic Destroyer was spreading here, thus the differences in the list of credentials is bigger. The first pair of credentials in this set can be found in all 12 samples. But the rest of the credentials is a mix stolen from different computers in the same network. We can see that the worm took different paths when spreading though the network, acquiring the credentials of at least five different computers.

After removing the credentials from researchers and sandboxes, we are left with five unique sets of credentials. If one subset of credentials is one letter, the sets can be described as A, AB, AC, AD and ADE. This shows that the more capable version of Olympic Destroyer actually inherits its list of stolen credentials to the infected systems.
The samples in question are:

1942f14326f8ffa3afc83946ba9ec06abe983a211939f0e58362f85dd2a6b96a
25089ec24167f3caa413a9e1965c7dfc661219f45305187070a1e360b03f869c
6d7d35b4ce45fae4a048f7e371f23d1edc4c3b6998ab49febfd7d33f13b030a5
9085926d0beacc97f65c86c207fa31183c5373e9a26fb0678fbcd26ab65d6e64
90c956e8983116359662f8b82ae156b378d3fae02c07a18827b4c65f0b5fe9ef

It is likely that there are more samples out there which give a better picture of the way Olympic Destroyer wormed itself through the ATOS network.

## PE timestamps

As the blog article of Kasperky has already shown, the author(s) of Olympic Destroyer had quite the fun in planting false flags. So, the compilation time stamps of the PE files should be taken with a grain of salt, as they can be easily forged.
Nonetheless they provide an interesting picture.

Simple version of Olympic Destroyer, PE time stamps ordered ascending:

 Name Compilation Time Stamp Description Resource 104 2016-06-28 18:43:09 Copy of PsExec Resource 105 2017-12-27 09:03:48 Destroyer DLL in Resource 101 2017-12-27 11:44:17 Browser Password Stealer DLL in Resource 102 2017-12-27 11:44:21 Windows Account Password Stealer Resource 101 2017-12-27 11:44:30 Loader for internal DLL Resource 103 2017-12-27 11:44:35 Loader for internal x64 DLL Resource 102 2017-12-27 11:44:40 Loader for internal DLL Main binary 2017-12-27 11:44:47 Olympic Destroyer

(Note that I did not extract the time stamp for the DLL in the resource of resource 103)

The PE time stamps of the more complex version in ascending order:

 Name Compilation Time Stamp Description Resource 104 2016-06-28 18:43:09 Copy of PsExec Resource 105 2017-12-27 09:03:48 Destroyer DLL in Resource 101 2017-12-27 11:38:53 Browser Password Stealer DLL in Resource 102 2017-12-27 11:38:58 Windows Account Password Stealer Resource 101 2017-12-27 11:39:06 Loader for internal DLL Resource 103 2017-12-27 11:39:11 Loader for internal x64 DLL Resource 102 2017-12-27 11:39:17 Loader for internal DLL Main binary 2017-12-27 11:39:22 Olympic Destroyer

Some of those values actually make sense, although they might have been crafted in order to do so. The DLLs which are resources of resource 101 and 102 have to be compiled before they can be embedded as resources, so their time stamps come first. The same goes for all resource which are embedded in them main binary of Olympic Destroyer.
PsExec in resource 104 is the original copy of PsExec, thus has the original time stamp.
A time difference of four to nine seconds for each binary sounds realistic, given only a few dependencies on external libraries. Unfortunately, the compilation with the biggest external dependencies, the DLL in resource 101 where SQLite is used, seems to be the first binary in the build chain. This is where I would have expected to see the biggest gaps in between the time stamps. But as it is the start of the build chain, we cannot compare it to any binary built before it.
By looking at the gaps, we can also see that everything except the destroyer part seem to be compiled in one block.
Also the more complex version of Olympic Destroyer seems to be compiled five minutes before the simpler version. Most probably the attacker(s) just compiled the first set of Olympic Destroyer, before commenting out the one function using PsExec (implicitly removing all the used sub-functions), and then recompiled the whole set.

It is noteworthy to point out that both versions of Olympic Destroyer use the very same copy of the destroyer component. Not only the compilation time stamps are the same, but also their hash sums.

# Summary

This article has shown the innermost working of the malware called Olympic Destroyer. We have seen that by pure reverse engineering of the malware samples, a plethora of information can be obtained and deduced.

The analysis indicates that Olympic Destroyer consists of two completely independent parts: The first one is a framework for network spreading using resource 101 to 104 in order to spread as fast and as far as possible in the local network. The second one is the destructive component. Both parts work completely independent from each other. Resource 101 to 103 have a strong logical dependency on the main binary by receiving the decryption key as well as the name of the named pipe as arguments. And the main binary depends on the information returned from the resources 101 to 103 formatted in a certain way. In contrast to that, the destroyer in resource 105 is only dropped and executed in a fire and forget manor. No arguments, return values or status codes are exchanged.
So I think it is correct to state that everything except the destroyer is merely a vehicle in form of a spreading framework to deliver a payload. And the delivered payload is the destroyer. In theory every other payload could be delivered by simply exchanging the resource 105.

We have also seen that Olympic Destroyer comes in two different versions, which have been spread in two different networks. The spreading algorithm differs in the way that credentials stolen on one system are not carried on to the next infected system in one version.
It is unknown to me why the differences exist. Reading the Kasperky article indicates that the attackers already had a strong foothold in the g18.internal and Pyeongchang2018.com network. So it might have been enough to spread only one hop from the initial infected machine.  This decision could also be influenced by the defensive mechanisms employed at the targeted network. A proper network monitoring tool should mark the execution of PsExec as red flag, which might have been the reason to remove this part of the spreading algorithm.
The analysis of stolen credentials in the network of ATOS indicates that the attackers had a weaker foothold in the network, since, juding by the samples I looked at, only two sets of credentials were stolen on the initial infection (compared to 44 in the simpler variant). All other credentials were added during the spreading in the network. This weak foothold might have been the cause to go with a more aggressive spreading algorithm.

# Appendix

Config structure as used in Olympic Destroyer:

struct config
{
DWORD credentialsVectorStart;
DWORD credentialsVectorEnd;
DWORD credentialsVectorMaxSize;
CRITICAL_SECTION critSect;
char ressourceHpath[1024];
char randomTempPath[1024];
char ressourceIpath[1024];
char selfModulePath[1024];
char domainName[256];
char accountName[256];
char domainAndAccountName[256];
char v13[256];
DWORD bitness;
DWORD bVersionGreaterEqualVista;
DWORD bVersionSmallerEqualXP;
DWORD bHasSelfDebugPrivs;
DWORD bIsUserAccount;
};

Hashes used for analysis:

edb1ff2521fb4bf748111f92786d260d40407a2e8463dcd24bb09f908ee13eb9
01e640a91d32230cd3f45e1594177393415585dbeba9ddbd31be2139935058d3
1942f14326f8ffa3afc83946ba9ec06abe983a211939f0e58362f85dd2a6b96a
25089ec24167f3caa413a9e1965c7dfc661219f45305187070a1e360b03f869c
254fbeb13f8d2dc36de3a3ffca653608d1b3420a20a20248d330500785b3945c
2bcbb1c165a6e31e085306224de3410249df50742ca3af069d58c7fd75d2d8c4
2bf9f3703b48bf1578a43479444107b33ff6ecea108b364fc73913a639c511d4
2c28f3b297a990b9d7a7163bac57ab68228c66109bb7a593702e556cdd455cce
3131a8208dc7441bf26592d7fed2ba5d9f9994e21d9b8396b4d2cda76a8a44d7
36a65a47cc464aac45a5d27372ea3b3584726d354f0792b9a77bfbe0cd0558bd
5181fe760f456719b0ec505370df0b38055a5a3b202e1d50948fc92383a61c18
5d85fba3ff021b35bfba30d5d56b957ef084d818778ff77550bcf65755aa7849
5f37829988e827f05b42774db94e8a15e87e9de12e61b89c91bf5fddee90650c
6d7d35b4ce45fae4a048f7e371f23d1edc4c3b6998ab49febfd7d33f13b030a5
725efe161b8d0024cd330e3a3da194b46d16be14d57392fbfdf1ea71415d67b1
75fa1309be8fdca4a6df345a009b47938503d5227149838334581b08d40b7e2f
9085926d0beacc97f65c86c207fa31183c5373e9a26fb0678fbcd26ab65d6e64
90c956e8983116359662f8b82ae156b378d3fae02c07a18827b4c65f0b5fe9ef
98d4f0e8f91d7f4f1a3058b1a30220e3460cc821be704acfcb7fa2eb0c88818c
99ca9d41c2ea6a18436fbc173ce8f3e94b5a3d592d9e4fa978120d140d96aefa
a9f66d9dd3fd0f977381e83c1379fe664f22ebdd5695258fc388465cd3749562
b30b4acf05898c8a6338f5df6c3df7d7f06df8e67ccd773ffd83b5b8acff4cb8
bdfb1a9f59be657b5375689b357ef8e70e1e7332f52c2e79ab3be796e06858d1
ca8be57bbd2f3169d0c1c4b5145e8f955ea69ddde701f94a2b29c661389b3aa2
d17d32048aae06ec60b693cd83e1cf184e8c2e4d1f0299a28423fdc624f56bb8
d2e43c41acd40324813d51df99fa127b86d8e384671dcc77f748d86afc3993a5
e2153c73ec9fd15dc8389523515a96c3477fce5503be78ff82ab3cc7e9386e83
e4dd30d5d85c4aaf05e01d8f40fb0e01e4e8ba99e82ec58946c045ce53783bde
f99610f8e36eb65e75979ef3ea4b7382bfb0bf2b72191cefccaaa19283d23606

## In debt to Retpoline

Appendix was added on the 14th of Febuary 2018, in response to comments made to me on twitter. In this connection “retpoline pause lfence” and “retpoline ud2” was added to the table. Other than that only typos where fixed since original post.

Abstract

In this blog post I explore the Retpoline mitigation that Paul Turner of Google suggested for the Spectre indirect branch variant issue [1]. A short differential side channel analysis is made along with a performance analysis. The impact of the use of a pause instruction in retpoline is discussed. Finally I consider the technical debt leveraged on CPU developers of a widely deployed retpoline.

How does retpoline work

This section follows the google presentation of retpoline closely [1]. I included it because it provides the context for the remainder of this blog post. A jmp rax is turned into the following code:

1. call set_up_target;
2. capture_spec:
3. pause;
4. jmp capture_spec
5. set_up_target:
6. mov [rsp],rax
7. ret;

call set_up_target (1) pushes the address of capture_spec(2) to the return stack buffer (RSB, cpu internal buffer used to predict returns). It also pushes the address of capture_spec(2) to the program stack and transfers control to set_up_target (5).  The mov [rsp],rax overwrites the return stack address on the program stack, so that any subsequent returns will pop the stack and return to the original target in its architectural path. The speculative path of the return remains the value from the RSB and thus the CPU will speculatively execute the code starting at (2). The pause instruction is meant to relinquish pipeline resources to co-located hyperthreads and save power if no co-located hyperthreads are present. The jmp in line 4 continues to repeat the pause until the speculative execution is rolled back when the ret in line 7 finished executing.

In simple terms this means the speculative path will resolve to a spinlock thus not leak any information through a side channel. The architectural path will eventually resolve correctly and the program will run as it is supposed to.

Side channel analysis of retpoline

Retpoline’s architectural side channels consist of flushing an entry in the RSB caused by the call in line (1). This information is unlikely to bring an attacker much advantage and would require an attack to have a sufficient amount of control of the RSB. The 6th line makes a store access on the stack. This is visible through a cache side channel to an attacker — provided the attack has sufficient control. However, it is unlikely to provide much information given that the stack is usually used by the functions themselves. It is worth noting that any information this side channel can provide is also provided by the jmp rax (6). Presumably retpoline’s stack access provides slightly less information to an attacker than the original jmp rax instruction – i.e. BTB indexing bits vs. cache set indexing bits. The 7th line is more tricky: if the CPU updates the BTB after a misprediction, the BTB side channel will be similar useful to the original jmp rax. I think it is likely that the BTB is not updated until retirement, so that this side channel isn’t present. Since the pause instruction (line 3) is a CPU hint, the CPU may choose to take or ignore the hint at its discretion – more on this below. Thus, pause may or may not provide a side channel.  If external power is plugged in and the hint is taken one can see a co-located hyperthread speed up (see [2]), which is the purpose of having the pause instruction here, but certainly a side channel as well. If the CPU is running in power saving mode (e.g. unplugged laptop), pausing provides a side channel since executing a pause instruction on two co-located hardware threads causes a delay for both, presumably through C-State interaction. See[3]. In sum, I think the side channels provided by retpoline is less valuable than the side channel provided by the original indirect branch (but is still present).

Performance analysis of retpoline

A jmp rax instruction takes about 4 clock cycles to execute and predicts correctly very often. It is replaced in retpoline with a ret instruction which will always mispredict and thus executes far slower. However, unlike doing hard serialization once the ret instruction has been executed out-of-order, the CPU seems to be able to continue without a pipeline flush and thus allowing out-of-order execution to continue. This creates two corner cases: firstly, where dependencies block out-of-order execution across the indirect branch, and secondly, when there is no dependence across the indirect branch. I managed to create a sequence of instructions where retpoline was just as fast as the original unpredicted branch – this is the case when the instructions after the branch depend on the results of the instructions before the branch. Obviously, any co-located hyperthread will be more affected by the retpoline than a predicted indirect branch, but the thread itself does not lose any cycles. At first this seems weird, but imagine completely dependant ALU integer instructions on both sides of the indirect branch – with the branch unit being completely free, both retpoline and the indirect branch will execute concurrently with the integer ALU instructions before the branch. Since the integer ALU instructions after the branch are dependant of those before the branch, they only get scheduled for execution once all prior instructions have been executed. Thus, retpoline and an indirect branch perform equally. In the case of no dependencies across the indirect branch, retpoline is slower. Retpoline will not allow the out-of-order execution to continue until the ret instruction is executed and consequently adding a penalty compared to a predicted indirect branch. In general, the longer it takes for the indirect branch to resolve, the higher the penalty of retpoline is. There is also an indirect performance cost of retpoline which – in my opinion – is likely to be somewhat smaller. The call in line 1 will push a return address onto the RSB (and consequently may evict the oldest entry in the RSB), and thus potentially causing an RSB underflow once a previous call returns. A RSB underflow will manifest itself as a negative performance impact if the evicted RSB entry causes mispredictions or stalls in unrelated code later on. For this to happen a call stack is required to be deeper than the the size of the RSB. The stall penalty of the underflow was big enough to cause Intel to add prediction to the microarchitecture (for Broadwell and Skylake). If this prediction was as efficient as using the RSB, the RSB would not exist.

The following table presents the results of my micro benchmarking:

 Unit Clock cycles Mean Std.dev Median jmp rsi 350.33 95.48 346 retpoline pause 410.64 65,94 410 retpoline lfence 403.76 25,96 404 retpoline clean 402.13 19,99 402 retpoline pause lfence 406.74 64,76 404 retpoline ud2 404,29 28,70 402

The “retpoline clean” is without the pause instruction, “retpoline lfence” is where pause has been replaced with a lfence instruction. The results are generally not very stable over multiple runs but with all three versions of retpoline often ending up with an average of around 400-410 cycles and the indirect branch being around 50 clks faster on average. Thus, some additional care should be taken before concluding

that “retpoline pause” is slower than the others. I ran the tests with 100k observations and removed the slowest 10% observations in a primitive noise reduction approach. The microbenchmark is for a bad case with no dependency across the indirect branch for integer instructions (add rsi,1, add rbx, 1 respectively) and is run on a Intel  i7-6700k. While microbenchmarking is important for the arguments I will put forth in this text, it is important to note that they are not reflective on the system’s performance as the incidence of indirect branches is relatively small and this benchmark is manufactured to portrait cases which are worse than in normal scenarios. Also, the microbenchmarking completely ignores any indirect effects.

The weird case of pause

Discussion of technical debt of retpoline

As clever as retpoline seems, I think it is fundamentally broken. Not because it does not work but instead because it builds a large amount of technical debt. Adding retpoline to a piece of software would require CPU designers to make sure that legacy software is compatible with new CPUs. If a company like Google applies retpoline in their instructure it is fairly unproblematic, Google has a nice inventory of software running on their systems and they can make sure that software applied to new CPUs is recompiled and consequently this poses no constraints on a CPU designer. However, if we add retpoline to a compiler we can be sure it will be added to all kinds of software including virtual machines, containers, specialized software etc. These pieces of software often do not remain supported, they are poorly catalogued and consequently, if the behavior of retpoline changes significantly, these systems may perform suboptimal, be unsafe or even completely break. This effectively ties the hands of CPU designers as they strive to improve the CPU in the future.

The first concrete problem I see is the use of the pause hint: regardless of whether the pause hint is taken in retpoline or not, its non-intended use in retpoline ties the behavior of this instruction down for CPU designers in future generations of CPUs. It is worth noting here that we have at least 3 different implementations of the pause hint in different CPUs already (treated as nop, stall 10 clks, stall 100 clks)[5]. Adding to that the complexity of the instruction I outlined above, it would be fair to assume that this instruction might need changes in future generations of CPUs. Thus, having the pause hint in retpoline in scattered over software everywhere might turn out to be a bad idea thinking long term. The good news here is that there probably is a good solution for this. Replacing the pause instruction with lfence will serialize the speculative path and probably even stop it from looping; effectively stopping the execution of the spinlock may free up resources for co-located hyperthread as well as branch execution units for the main thread that otherwise would have been tied up by the jmp instruction in line 4. I ran some test on Skylake and found very near identical performance results for pause and lfence, suggesting that this is viable solution. It is important to note that the lfence instruction was previously documented to only serialize loads. But instead it silently serialized all instructions — which is now the documented behavior since Intel published their errata documents for the Spectre/Meltdown patches [7]. So, the mortgage on lfence is small and has already been signed.

The second concrete problem I see is the construct which repoline uses to direct speculative execution into the spinlock is a much bigger problem. On Skylake, return instructions predict by using the indirect branch predictor, requiring Skylake to be handled differently than other CPUs. The problem is twofold: on the one hand, the very common ret instruction needs to be replaced with retpoline (or other non-speculative branching), and on the other hand, the hardware interrupt raised in line 6 may underflow the RSB in the interrupt handler. Thus, repoline is already potentially unsafe on some CPUs (in my opinion much less than without retpolinie though). The technical debt here is that retpoline may be completely unsafe if future CPUs stop relying on the RSB for return prediction. There may be many reasons for a CPU designer to change this. For example a completely unified system for indirect branches or prediction of monotonic returns (returns with only one return address). The latter will keep the RSB save from non-monotonic returns if the RSB underflows and thus may perform better when there are deep call stacks. I do not know whether these things are good ideas, but retpoline might effectively rule them out. Also, one could imagine conflicts with future CPU-based control-flow integrity systems, etc.

One could argue that performance optimization rely on microarchitecture details all the time, but there is an important difference between breaking a performance optimization and a security patch. Already we see problems with updating broken libraries amongst software vendors, it’s not difficult to imagine what would happen if a secure software becomes insecure because of CPU evolution.

Instead of taking on technical debt potentially forever, I suggest we use either a less secure option (i.e. lfence ahead of indirect branches) or a more expensive option such as IPBP [7] or replacing indirect calls with iret (which is documented to be serializing). That constitutes a high price now, but avoids paying rent on technical debts in perpetuity.

Conclusion

Retpoline is an effective mitigation for Spectre variants which rely on causing misprediction on indirect branches on some CPUs. From a pure side channel perspective retpoline adds a different side channel but it is an improvement over the side channel of a traditional indirect branch nonetheless. Retpoline’s performance penalty is complex, but likely smaller than the penalty of the serializing alternatives. However, as Retpoline relies on assumptions about the underlying microarchitecture, it adds technical debt if used widely. If Google, Microsoft or whoever with software-deployment management wants to use it, they have my blessing but there is reasons for scepticism if it is a good idea to have it in general purpose compilers. CPU Vendor’s short term marketing deficits should not lead us to trade small short term performance gains causing technical debt to be paid back with interest in the future for a more complex microarchitecture.

Appendix added 14th of Febuary 2018

Stephen Checkoway of University of Illinois at Chicago commented that it might be worth testing an ud2 instruction in the speculative executed spinlock. I find this idea promising because ud2 essentially just throws an “invalid opcode” exception and thus abusing it for retpoline is likely to produce equivalent performance to that of lfence, perhaps even better, Further, it’s unlikely that using this instruction is associated with any technical debt. The simple test result here shows that the direct performance impact is approximately similar (see table above) that of the other versions of the spinlock.

Some implemented versions of retpoline uses a pause followed by an lfence instruction. I added this to the performance table as well. Thanks to Khun Selom & @ed_maste (twitter account).

Literature