Timezones

When NTP Kills Your Sandbox

If it’s common to say that “Everything is a Freaking DNS problem“, other protocols can also be the source of problems… NTP (“Network Time Protocol”) is also a good candidate! A best practice is to synchronize all your devices via NTP but also to set up the same timezone! We learn by doing mistakes and I wanted to share this one with you.

After spending time to debug, I finally found why many of the automated submissions to my malware analysis Sandbox failed. It’s was due to the timezone and… NTP!

When you prepare a Sandbox system, it must be based on guest images. These images have to reflect your “environment”: Your classic tools must be installed (Microsoft Office, a PDF reader, a browser, etc). To achieve this, I usually start from a standard Windows image that I clone then fine-tune to match my requirements. The problem is that, by default, the Windows operating system synchronizes itself automatically with the Microsoft NTP servers:

How the guest image is used by the Sandbox system? When your environment is ready, you take a snapshot. Later, to analyze a malicious file, the sandbox system will restore the snapshot, copy the file to it and executed it. The snapshot being some “picture” of the system,, the date & time are also frozen and, when you restore it, it continues to run at the time the snapshot was taken. That’s why, the Sandbox must update the time:

2020-06-04 11:48:17,428 [root] INFO: Date set to: 20200615T00:31:18, timeout set to: 300

(Note that it’s a classic feature. Some malware must be analyzed at a specific time or date to ensure that it will execute properly!)

My last snapshot was created on 2020/06/04 11:48:17 and the analyzis started on 2020/06/15 00:31:18:

2020-06-15 00:31:18,734 [root] DEBUG: Starting analyzer from: C:\tmpv556hytw
2020-06-15 00:31:18,734 [root] DEBUG: Storing results at: C:\FMiZIvH
2020-06-15 00:31:18,734 [root] DEBUG: Pipe server name: \\.\PIPE\nRwJWEHQaG
2020-06-15 00:31:18,734 [root] DEBUG: Python path: C:\Users\user01\AppData\Local\Programs\Python\Python38-32
2020-06-15 00:31:18,734 [root] DEBUG: No analysis package specified, trying to detect it automagically.
2020-06-15 00:31:18,734 [root] INFO: Automatically selected analysis package "exe"

But suddenly, I saw this in the log:

2020-06-15 00:31:31,359 [root] DEBUG: DoProcessDump: Dumping Imagebase at 0x00860000.
2020-06-15 02:32:18,074 [root] INFO: Analysis timeout hit, terminating analysis.
2020-06-15 02:32:18,074 [lib.api.process] ERROR: Failed to open terminate event for pid 3392
2020-06-15 02:32:18,074 [root] INFO: Terminate event set for process 3392.
2020-06-15 02:32:18,074 [root] INFO: Created shutdown mutex.
2020-06-15 02:32:19,073 [root] INFO: Shutting down package.
2020-06-15 02:32:19,073 [root] INFO: Stopping auxiliary modules.

You can see that suddenly, the system time was 2h ahead (00:31 to 02:32) and the Sandbox triggered a timeout and stopped the analysis. Why?

The Sandbox is running in UTC (tip: it’s always good to use UTC as a standard timezone to avoid issues when correlating events) but my original Windows guest was running in the CET timezone (UTC+2 with the summertime) and NTP synchronization was left configured by default. When the snapshot is restored, the operating system runs as usual and, at regular intervals, synchronized its internal clock via NTP…

Conclusion: do NOT configured NTP in your Sandbox guest images to save you some headaches with broken analysis!

One comment

Leave a Reply

Your email address will not be published.

This site uses Akismet to reduce spam. Learn how your comment data is processed.