DLLs & Dynamic Loading
Windows API - GUI & System Programming · Windows Programming track
9 min read
In this module
Dynamic Link Libraries and the Component Object Model
DLLs are how Windows shares code. Every Win32 call you've ever made — MessageBox, CreateFile, the lot — punches through a DLL on its way to the kernel. Loaders, exports, imports, the search order that's caused a thousand CVEs: this lesson is the user-mode side of how a Windows process is actually wired together.
DLL Fundamentals
A DLL (Dynamic Link Library) is a PE — Portable Executable — file holding code and data that multiple programs can share. Static libraries get welded into each binary at link time and bloat every executable that uses them. DLLs sit on disk once, get loaded into memory once per file, and have their pages shared (read-only) across every process using them.
DllMain: The Entry Point
Every DLL can have an optional entry point called DllMain:
BOOL WINAPI DllMain(HINSTANCE hinstDLL, DWORD fdwReason, LPVOID lpReserved) { switch (fdwReason) { case DLL_PROCESS_ATTACH: // DLL loaded into a process // Called once per process // hinstDLL = base address of the DLL break;
case DLL_THREAD_ATTACH: // A new thread was created in the process break;
case DLL_THREAD_DETACH: // A thread is exiting cleanly break;
case DLL_PROCESS_DETACH: // DLL is being unloaded // lpReserved != NULL means process is terminating // lpReserved == NULL means FreeLibrary was called break; } return TRUE; // Return FALSE to fail DLL loading}Critical rules for DllMain:
- Do NOT call LoadLibrary or LoadLibraryEx (can cause deadlock due to loader lock)
- Do NOT call functions from other DLLs (they may not be loaded yet)
- Do NOT create threads (they will block waiting for DllMain to return)
- Do NOT use synchronisation primitives that wait on other threads
- Keep it minimal: initialise per-process or per-thread state only
DllMain is called while the Windows loader holds a process-wide lock called the loader lock. Any operation that requires the loader (loading other DLLs, resolving imports) will deadlock. This is one of the most common pitfalls in Windows DLL development. Complex initialisation should be deferred to a separate initialisation function that the application calls after the DLL is loaded.
Exporting Functions
// Method 1: __declspec(dllexport) in source code__declspec(dllexport) int Add(int a, int b) { return a + b;}
// Method 2: .def file (more control over names and ordinals)// mylib.def:// LIBRARY mylib// EXPORTS// Add @1// Subtract @2// _InternalFunc @3 NONAME ; Export by ordinal onlyThe PE export table maps function names — and ordinal numbers, an older mechanism still used by Windows itself — to addresses inside the DLL. Run dumpbin /EXPORTS mylib.dll to see what a DLL is offering the world.
Implicit vs Explicit Linking
Implicit Linking (Load-Time)
The default. Your linker records which DLLs you need; the Windows loader resolves the addresses at process startup and patches up your import table before main() runs:
// In your code, just call the function#include "mylib.h"int result = Add(3, 4);
// Compile: cl app.c /link mylib.lib// mylib.lib is an import library (not the DLL itself)// It contains stubs that redirect to the DLL at runtimeExplicit Linking (Run-Time / Dynamic Loading)
When you don't know at compile time which DLL you need — plugins, optional features, undocumented APIs that may or may not exist on this Windows version — LoadLibrary and GetProcAddress let you wire it up by hand at runtime:
// Load the DLL at runtimeHMODULE hMod = LoadLibrary(TEXT("mylib.dll"));if (!hMod) { DWORD err = GetLastError(); // DLL not found or failed to load return;}
// Get function address by nametypedef int (*AddFunc)(int, int);AddFunc pAdd = (AddFunc)GetProcAddress(hMod, "Add");if (pAdd) { int result = pAdd(3, 4); // Call the function}
// Get function by ordinaltypedef int (*SubFunc)(int, int);SubFunc pSub = (SubFunc)GetProcAddress(hMod, MAKEINTRESOURCEA(2));
// Unload when doneFreeLibrary(hMod);Plugin systems live or die on this pattern. So does any code that has to gracefully handle "this function might not exist on Windows 7."
DLL Search Order
Hand LoadLibrary a bare filename — no path — and Windows starts hunting through directories in a fixed order:
1. Directory containing the application executable2. System directory (C:\Windows\System32)3. 16-bit system directory (C:\Windows\System)4. Windows directory (C:\Windows)5. Current directory6. Directories in the PATH environment variableDLL Hijacking
Bare-filename loads are an attack surface. Drop a malicious DLL into a directory that gets searched earlier than the legitimate one and the application happily loads your code instead:
Scenario: app.exe loads "helper.dll" (expected in System32)
Attack:1. Attacker places malicious "helper.dll" in app's directory2. app.exe calls LoadLibrary("helper.dll")3. Search order: app directory is checked FIRST4. Malicious DLL is loaded instead of the real one5. DllMain runs attacker's code with app's privilegesMitigations:
- Use full paths:
LoadLibrary(L"C:\\Windows\\System32\\helper.dll") - Use
SetDefaultDllDirectories(LOAD_LIBRARY_SEARCH_SYSTEM32) - Use
SetDllDirectory(L"")to remove current directory from search - The KnownDLLs registry key protects common system DLLs
DLL hijacking exploits the predictable order in which Windows searches for DLLs. By placing a malicious DLL in a higher-priority directory (typically the application's own directory), an attacker can execute code in the context of the loading application. This is used for privilege escalation (hijacking a DLL loaded by a privileged service), persistence (placing DLLs in autostart application directories), and lateral movement.
COM: The Component Object Model
COM is the binary component-sharing framework that runs underneath half of Windows. An object written in C++ can be called from C# or VB or PowerShell; an object running in another process (or another machine, with DCOM) looks the same to your code as one in your own address space. The abstraction is the point.
The Three Pillars of COM
- Interfaces: COM objects expose functionality through interfaces (pure virtual method tables). Every interface inherits from IUnknown.
- Reference Counting: Objects are kept alive by AddRef/Release reference counting.
- Registration: COM classes are registered in the registry (HKCR\CLSID) with their implementing DLL or EXE.
IUnknown: The Root Interface
// Every COM interface inherits from IUnknowninterface IUnknown { HRESULT QueryInterface(REFIID riid, void **ppvObject); ULONG AddRef(void); ULONG Release(void);};- QueryInterface: Ask the object if it supports a specific interface
- AddRef: Increment reference count
- Release: Decrement reference count (destroy when it reaches 0)
Creating COM Objects
// Initialize the COM libraryHRESULT hr = CoInitializeEx(NULL, COINIT_APARTMENTTHREADED);
// Create an instance of a COM classIShellLink *psl = NULL;hr = CoCreateInstance( &CLSID_ShellLink, // Which class to create NULL, // No aggregation CLSCTX_INPROC_SERVER, // In-process DLL server &IID_IShellLink, // Which interface we want (void **)&psl // Output pointer);
if (SUCCEEDED(hr)) { // Use the interface psl->lpVtbl->SetPath(psl, L"C:\\target.exe"); psl->lpVtbl->SetDescription(psl, L"My Shortcut");
// Query for another interface on the same object IPersistFile *ppf = NULL; hr = psl->lpVtbl->QueryInterface(psl, &IID_IPersistFile, (void **)&ppf); if (SUCCEEDED(hr)) { ppf->lpVtbl->Save(ppf, L"C:\\Users\\Public\\Desktop\\shortcut.lnk", TRUE); ppf->lpVtbl->Release(ppf); }
psl->lpVtbl->Release(psl);}
CoUninitialize();COM Registration
COM classes are registered in the registry:
HKCR\CLSID\{00021401-0000-0000-C000-000000000046}\ (Default) = "Shortcut" InprocServer32\ (Default) = "C:\Windows\System32\shell32.dll" ThreadingModel = "Apartment"Call CoCreateInstance and the runtime walks the registry, finds the DLL path keyed by your CLSID, loads it, and asks it for the requested object via DllGetClassObject. That registry lookup is also another reason COM has historically been a happy hunting ground for attackers — registry hijacks, type-library tricks, and surrogate processes all start by abusing this lookup.
COM is everywhere in Windows. The Shell uses it. So does DirectX, WMI, Active Directory, the Task Scheduler, and most of the system services that talk to each other. Get comfortable with IUnknown, QueryInterface, and reference counting, and a huge chunk of Windows internals stops looking like dark magic.