DEF CON 26 - Alexei Bulazel - Reverse Engineering Windows Defenders Emulator

2NawGCUOYT4/default.jpg

Windows Defender Antivirus's mpengine.dll implements the core of Defender's functionality in an enormous ~11 MB, 30,000+ function DLL.

In this presentation, we'll look at Defender's emulator for analysis of potentially malicious Windows binaries on the endpoint. To the best of my knowledge, there has never been a conference talk or publication on reverse engineering any antivirus binary emulator before.

We'll cover a range of topics including emulator internals—machine code to intermediate language translation and execution; memory management; Windows API emulation; NT kernel emulation; file system and registry emulation; integration with Defender's antivirus features; the virtual environment; etc.—building custom tooling for instrumenting the emulator; tricks that binaries can use to evade or subvert analysis; and attack surface within the emulator.

Attendees will leave with an understanding of how modern antivirus software conducts emulation-based dynamic analysis on the endpoint, and how attackers might go about subverting or attacking these systems. I'll publish code for a binary for exploring the emulator from within, patches that I developed for instrumenting Defender built on top of Tavis Ormandy's loadlibrary project, and IDA scripts to help with analyzing mpengine.dll and Defender's "VDLLs"

2NawGCUOYT4/default.jpg
DEF CON 26 - Alexei Bulazel - Reverse Engineering Windows Defenders Emulator DEF CON 26  - Alexei Bulazel - Reverse Engineering Windows Defenders Emulator Reviewed by Dump3R H3id3gg3R on October 22, 2018 Rating: 5