Malware Development Essentials
Portable Executable (PE) File Format
The Portable Executable (PE) format is a file format used for executables, object code, DLLs, FON font files, and other file types in 32-bit and 64-bit versions of Windows operating systems. The PE format is a data structure that encapsulates the information required by the Windows OS loader to manage the executable code. This includes dynamic library references for linking, API export and import tables, resource management data, and thread-local storage (TLS) data. In NT-based operating systems, the PE format is used for EXE, DLL, SYS (device driver), and other file types.

Key PE Sections
.text: Contains executable code (useful for malware developers)..rdata: Contains read-only data..data: Contains the application’s global variables (useful for malware developers)..pdata: Contains exception handling information..rsrc: Contains resources such as objects, pictures, icons, manifest files, embedded EXEs, and DLLs (useful for malware developers)..reloc: Contains relocation information, allowing Windows loaders to properly load DLLs or EXEs into memory.
PE Bear
A Portable Executable reversing tool with a user-friendly GUI.

EXE vs. DLL
EXE (Executable)
Programs that can be loaded into memory as independent processes.
Require a
mainfunction.
DLL (Dynamic Link Library)
PE modules loaded into an existing process.
Provide functionalities required by the process.
Require a
DllMainfunction.
The only difference between an EXE and a DLL is how they are invoked in a program.
Payload Storage
Payload Storage in PE Files
1. Dropper
A dropper is a specialized program or file (e.g., PDF, DOCX) used to deliver a payload to a target machine.
Its primary function is to bypass detection and execute the payload on the victim's system.
2. Payload
A sophisticated program designed to perform malicious actions, such as:
Deploying malware.
Establishing a reverse shell.
Disabling antivirus (AV).
Downloading/executing additional programs.
Escalating privileges.
3. Storing Payload in PE Sections
Payloads can be embedded in different sections of a Portable Executable (PE) file:
Section
Storage Method
Use Case
.text
Stored as local variables inside functions.
Executable code region; useful for hiding small payloads.
.data
Stored as global variables.
Stores writable data; payload remains accessible throughout execution.
.rsrc
Embedded as resources (e.g., icons, binaries).
Evades detection by storing payload as non-executable data.
Generate payload using MSFVENOM
Store Payload .text section
File: compile.bat
Payload is defined as a local variable inside main function.
Store Payload .data section
File: compile.bat
Payload is defined as a global variable.
Store Payload .rsrc section
Payload is embedded as a resource (e.g., fake icon file).
File: resources.rc
File: resources.h
File: compile.bat
Payload Encoding and Encryption
The purpose is to evade AV/EDR and make it difficult to detect
Difference between encoding and encryption
Encode: The purpose is to transform data using a public schema to and make it easy to process by software. (To decode needs to know the decode type)
Base64
Encryption: The purpose is to transform data using an encryption algorithm in order to keep it secret from others (To decrypt needs Encryption type and key)
XOR
AES
Base64 encoding
Base64 encoding is a weak and detectable, it is not recommended to use it for malware.
File: compile.bat
XOR encryption
File: xor_encrypt.py
File: compile.bat
AES Encryption
File: aes_encrypt.py
File: compile.bat
Function Call Obfuscation
Function call obfuscation is a technique used in malware development to hide calls to external functions (usually from Windows DLLs) in order to evade detection by antivirus (AV) software.
How does a normal executable work?
EXE or DLL files use external functions from system DLLs like
kernel32.dll,user32.dll, etc.These external functions are imported normally, and their names appear clearly in the Import Address Table (IAT).
AV engines analyze these imports to detect suspicious or malicious behavior.
How does AV detect malware?
AV software inspects imported DLLs and their functions. If the executable uses functions often seen in malware—such as:
VirtualAllocWriteProcessMemoryCreateRemoteThread
...then the AV might flag the binary as suspicious or malicious.
How Does Function Call Obfuscation Help?
To evade AV detection, malware developers avoid importing sensitive functions directly. Instead, they:
Resolve function addresses at runtime using Windows API functions:
GetModuleHandle("DLL_Name"): Returns a handle to a loaded DLL.GetProcAddress(handle, "FunctionName"): Returns the address of a function in the specified DLL.
Listing imports using dumpbin
This tells AVs that VirtualProtect is used—which can be suspicious.
How to search for function (E.x. VirtualProtect)
Example:
msdn virtualprotectCopy the function structure
E.x. VirtualProtect Structure
First, define a global function pointer:
Then resolve it at runtime:
The problem here is the VirtualProtect into the code and it can be detectable, to fix it use XOR encryption.
Obfuscating Strings using XOR encryption and CyberChef
Site: CyberChef
Key: RANDOMKEY
Input: FUNCTION_NAME

OLD CODE WITHOUT CALL OBFUSCATION
This code calls VirtualProtect directly, exposing its name in the binary/
NEW CODE USING FUNCTION CALL OBFUSCATION
This version hides the call to VirtualProtect using XOR and dynamic resolution.
Using Function Call Obfuscation:
Prevents AV from easily detecting sensitive API calls.
Allows dynamic resolution of Windows API functions at runtime.
Can be extended to any function:
CreateRemoteThread,LoadLibrary, etc.Increases stealth of malware or offensive tools.
Code Injection
Payload injection is a technique used in malware development where a malicious code payload is inserted into a legitimate process or system.

Most Common Combination
VirtualAllocEx: Allocates memory in the remote process.WriteProcessMemory: Writes the shellcode into the allocated memory.CreateRemoteThread: Starts a new thread in the remote process to execute the shellcode.
DLL Injection
LoadLibrary: The address of this function is obtained indropper.exeusingGetProcAddress.VirtualAllocEx: Called in the target process (e.g.,outlook.exe) to allocate space for the DLL path.WriteProcessMemory: Writes the DLL path into the allocated memory.CreateRemoteThread: ExecutesLoadLibrarywith the path to the DLL as its argument, effectively loading the malicious DLL into the target process.
File: implantDLL.cpp
File: injectDLL.cpp
File: compileDLL.bat
File: CompileInjector.bat
Hiding console
Using FreeConsole(); but the problem is the console will pop up a milliseconds.
Use a WinMain instead of main method and compile using
✅ Right Way to hide console
⛔ Will pop up the console even using FreeConsole
And should compile the program as GUI program using /SUBSYSTEM:WINDOWS
Final project
Create a dropper where the shellcode is loading from
.rsrcEncrypt/Decrypt using XOR encryption
Inject the shellcode into explorer.exe or notepad.exe
Get rid of console windows (pop up)
Payload:
File: compile.bat
File: resources.rc
File: resources.h
File: xorencrypt.py
File: implant.cpp
Bypassing AV (Detectable)
Implement function call obfuscation to VirtualAllocEx, WriteProcessMemory, CreateRemoteThread .
Binary Analysis
Last updated