Tuesday, May 3, 2022

Windows Sysinternals Suite: "Process Monitor" tool

Process Monitor 

(I created these notes while learning about Process Monitor tool.) 



Process Monitor (or "ProcMon") is a combination of two older tools: Filemon and Regmon

Mark Russinovich wrote these old tools by doing something called as syscall hooking. Filemon and Regmon were one of the first tools written by him in the Sysinternals toolset.

ProcMon still uses some undocumented APIs in Windows and that's why it's not open source.

Introduction

Procmon helps to get to the root cause of often misleading error messages. And thus, Procmon has been proven as an indispensable troubleshooting tool.

There is a saying from David Solomon: "When in doubt, run Process Monitor".

  • Process Monitor monitors syscall activity for files and registry.
  • File and registry issues can be in the form of misleading error messages, application crashes, application hangs, silent process exits etc.
  • Procmon can help determine the root cause for missing or corrupt files, missing or corrupt registry data, permission problems and wrong DLL versions.
  • Other uses of procmon are: tuning I/O activity, understanding hard drive activity and understand file and registry usage of apps.
  • Procmon captures a ton of live event data related to files and registry. It's like wireshark but for syscall activity related to files and registry!
  • We can export the captured data and analyze it.

[The below explanation might be outdated but it applied to Filemon. Now it can be considered as the explanation to the file monitoring part of Procmon.]

How Filemon works?

  • Filemon installs a filesystem "filter driver".
  • First run of filemon requires "Load Driver" user right.
  • After the filter driver is loaded, filemon offers a level of security to prevent low-privilege user from manipulating the filemon driver.
  • Next, Filemon requires "Debug Programs" user right. This right allows the processes to debug any other process even if they are running in different accounts. Even though Filemon doesn't really require this right as it already has a driver to intercept all syscalls, Filemon checks if the user has the "Debug programs" right because the user will be able to see all the filesystem I/O events in Filemon even for the files they may not have access to. So Filemon checks for this right to make sure the user is privileged enough to get all that critical info. As administrators already have "debug programs" right by default, running filemon as administrator might be necessary.

Usage

We can pause/resume logging process by using Ctrl+E or using the capture icon in the toolbar.

We can clear all the logs and start from scratch using Ctrl+X or the icon in the toolbar.

We can search for any text thoughout all the columns using the search icon in the toolbar.

If we find anything interesting in the file activity and want to jump to that location in the explorer, we can right click on that entry and select Jump To to go to that location in the explorer.

History Depth option in (Options -> History Depth) can be useful if we're running ProcMon for a long period of time. ProcMon stores all the event data in the virtual memory. So running it for a long period of time can consume a lot of virtual memory or even cause ProcMon to raise an exception and crash because it ran out of memory. That's where History Depth option can be helpful. The default is 0 which means unlimited. Setting it to a fixed value will tell the Procmon to use only that much amount of memory and you can keep running it for however long you want.

We can add filters using the filter icon and even reset if we mess up anything and none of the events show up.

We can also set filters using the Filter options in the menu and set Highlighting feature as well. We also have an option to save the filter, import the previously exported filters etc.

Basic vs Advanced Mode

There is an option in Filters menu called Enable Advanced Output.

Things not seen in Basic mode:

  • Raw I/O request names (basic mode displays a user friendly name for a few parameters).
  • Internal filesystem ops
  • Activity in System process (including the ops performed by NTFS itself)
  • Procmon's own activity

Basically, Advanced mode has less Filters applied. You can see the change in the Filter list when you switch on the Advanced mode.

Using Regmon (or Registry monitoring part of Procmon)

Regmon is very similar to Filemon and it also creates a device driver to intercept Registry syscalls and checks for Debug Programs right of the user just like Filemon.

Note!!! The syscall hooking was done earlier before Windows XP. On Windows XP, Microsoft actually provides Registry IO interception hooks and Regmon uses that instead of syscall hooking. Mark discourages syscall hooking as it was being exploited by rootkits and various other types of malware. And Regmon was the first software to actually demonstrate syscall hooking in Windows and that led to all the bad guys taking advantage of this technique.

Starting Filemon/Regmon before Logon

If we want to capture file and registry logs even before logging in, we need to run Filemon and Regmon under the SYSTEM user. And because Filemon and Regmon don't belong to us, i.e., the current user, it keeps running before and after we login and logout respectively.

We can run Filemon and Regmon under the SYSTEM user using psexec (which is another Sysinternals tools).

The psexec command to do that is:

psexec -i -s <local path to filemon/regmon>

-i is to give filemon/regmon access to interactive desktop (for a GUI application and for us to see a window). -s is to run filemon/regmon under the SYSTEM account.

Running Filemon/Regmon as a Service to get traces from the Boot of the system

We need to run Filemon/Regmon as a service if we need to get traces right from the boot stage of the system.

We can actually do this by going to Options -> Enable Boot Logging. This will get us all the traces right from the beginning of the OS boot.

Enabling Boot logging will start procmon as a Boot Critical driver and captures boot traces.

Notes made from the video on Procmon by Sami Laiho

Source: https://www.youtube.com/watch?v=gVYZrUJdXqU&list=PL96F5PDvO1HHRbaGiDmcI0v92nb4yIcm3&index=5

We can view the Filter Driver installed by Procmon using the following command:

C:\Windows\system32>fltmc

Filter Name                     Num Instances    Altitude    Frame
------------------------------  -------------  ------------  -----
bindflt                                 1       409800         0
PROCMON24                               4       385200         0
WdFilter                                4       328010         0
storqosflt                              0       244000         0
wcifs                                   0       189900         0
CldFlt                                  1       180451         0
FileCrypt                               0       141100         0
luafv                                   1       135000         0
npsvctrig                               1        46000         0
Wof                                     3        40700         0
FileInfo                                4        40500         0

There might be cases where the traces might not reach the altitude of procmon filter driver. The syscall "packets" go from the highest altitude to the lowest altitude.

We can modify the altitude of the procmon driver in the Registry.

Sometimes, procmon can miss some traces from Defender or some security solution and in that case, we can higher or lower the value of the altitude in the Registry.

More on Filter Driver altitudes: https://docs.microsoft.com/en-us/windows-hardware/drivers/ifs/allocated-altitudes

Normally, filtering the logs will only filter them in GUI but all of the event logs will still be written to the page file and it keeps filling the memory.

To avoid filling up the memory and run procmon for a long time (like, for weeks), then we can choose Filter -> Drop Filtered Events option to truly drop events and only put the events that we are interested into the memory.