File & I/O APIs
The Windows API (Win32) Fundamentals · Windows Programming track
11 min read
In this module
File Operations and High-Performance I/O
CreateFile sits at the centre of the Win32 file world. Everything else — synchronous reads, overlapped writes, named pipes, the lot — hangs off that one entry point. For servers that need to juggle thousands of connections without melting the scheduler, Windows provides I/O Completion Ports. We'll work through CreateFile, then ReadFile/WriteFile, then overlapped I/O, and finish with the IOCP machinery that powers IIS and SQL Server.
CreateFile: The Universal Opener
The name lies. CreateFile opens files, directories, pipes, mailslots, serial ports, and raw disks — anything that can hide behind a kernel handle. What it actually creates is a file object, and the function got named after the object rather than the operation:
HANDLE CreateFile( LPCSTR lpFileName, // Path or device name DWORD dwDesiredAccess, // What you want to do DWORD dwShareMode, // What others can do simultaneously LPSECURITY_ATTRIBUTES lpSA, // Security (NULL = default) DWORD dwCreationDisposition,// How to handle existing/missing files DWORD dwFlagsAndAttributes, // Flags and file attributes HANDLE hTemplateFile // Template for attributes (rarely used));dwDesiredAccess: What You Want to Do
GENERIC_READ // 0x80000000 - Read data, attributesGENERIC_WRITE // 0x40000000 - Write data, attributesGENERIC_EXECUTE // 0x20000000 - Execute (for EXEs)GENERIC_ALL // 0x10000000 - Full access
// Fine-grained access rights:FILE_READ_DATA // Read file contentsFILE_WRITE_DATA // Write file contentsFILE_APPEND_DATA // Append to fileFILE_READ_ATTRIBUTES// Read file attributesFILE_WRITE_ATTRIBUTES// Write file attributesFILE_EXECUTE // Execute fileDELETE // Delete the fileREAD_CONTROL // Read the security descriptorWRITE_DAC // Modify the DACLWRITE_OWNER // Change ownershipdwShareMode: Coexistence Rules
0 // Exclusive access (no sharing)FILE_SHARE_READ // Others can open for readingFILE_SHARE_WRITE // Others can open for writingFILE_SHARE_DELETE // Others can mark for deletion
// Common combinations:FILE_SHARE_READ // Read-sharing (log files)FILE_SHARE_READ | FILE_SHARE_WRITE // Full sharing (temp files)0 // Exclusive (data processing)If somebody else has the file open with a share mode that doesn't match what you're asking for, CreateFile bails with ERROR_SHARING_VIOLATION. This is the classic "the process cannot access the file because it is being used by another process" message every Windows user has cursed at.
dwCreationDisposition: Handling Existing Files
CREATE_NEW // Create new, FAIL if existsCREATE_ALWAYS // Create new, OVERWRITE if existsOPEN_EXISTING // Open existing, FAIL if not existsOPEN_ALWAYS // Open existing, CREATE if not existsTRUNCATE_EXISTING // Open and truncate, FAIL if not existsCreateFile can open more than files. Use \\.\PhysicalDrive0 to open a raw disk, \\.\C: for a volume, \\.\COM1 for a serial port, \\.\pipe\PipeName for a named pipe, or \\.\mailslot\SlotName for a mailslot. The device namespace is accessible through the \\.\ prefix (which maps to \??\ in the object namespace).
Flags and Attributes
// File attributes:FILE_ATTRIBUTE_NORMAL // No special attributesFILE_ATTRIBUTE_HIDDEN // Hidden fileFILE_ATTRIBUTE_READONLY // Read-onlyFILE_ATTRIBUTE_TEMPORARY // Optimize for temp files (keep in cache)
// Behavioural flags:FILE_FLAG_OVERLAPPED // Enable asynchronous I/OFILE_FLAG_NO_BUFFERING // Bypass file system cache (direct I/O)FILE_FLAG_WRITE_THROUGH // Write through to disk (no lazy write)FILE_FLAG_SEQUENTIAL_SCAN // Optimize for sequential accessFILE_FLAG_RANDOM_ACCESS // Optimize for random accessFILE_FLAG_DELETE_ON_CLOSE // Delete file when last handle closesFILE_FLAG_BACKUP_SEMANTICS// Required to open directoriesReadFile and WriteFile
// Synchronous readDWORD bytesRead;BOOL success = ReadFile( hFile, // File handle buffer, // Destination buffer bufferSize, // Bytes to read &bytesRead, // Bytes actually read NULL // NULL = synchronous);
// Synchronous writeDWORD bytesWritten;WriteFile(hFile, data, dataSize, &bytesWritten, NULL);
// Set file positionLARGE_INTEGER pos;pos.QuadPart = 1024 * 1024; // 1 MB offsetSetFilePointerEx(hFile, pos, NULL, FILE_BEGIN);Overlapped (Asynchronous) I/O
When you don't want ReadFile to block the calling thread for fifty milliseconds while the disk wakes up, open the handle with FILE_FLAG_OVERLAPPED and hand the API an OVERLAPPED structure:
// Open file for async I/OHANDLE hFile = CreateFile("large.dat", GENERIC_READ, FILE_SHARE_READ, NULL, OPEN_EXISTING, FILE_FLAG_OVERLAPPED, // Enable async NULL);
// Start async readOVERLAPPED ov = {0};ov.Offset = 0; // File offset (low 32 bits)ov.OffsetHigh = 0; // File offset (high 32 bits)ov.hEvent = CreateEvent(NULL, TRUE, FALSE, NULL);
char buffer[4096];BOOL result = ReadFile(hFile, buffer, sizeof(buffer), NULL, &ov);
if (!result && GetLastError() == ERROR_IO_PENDING) { // I/O is in progress... // Do other work here while waiting...
// Wait for completion DWORD bytesRead; GetOverlappedResult(hFile, &ov, &bytesRead, TRUE); // TRUE = wait for completion, FALSE = check without waiting}
CloseHandle(ov.hEvent);I/O Completion Ports: Scalable Async I/O
When you have thousands of sockets in flight and a small worker pool, IOCP is the abstraction Windows wants you to reach for. IIS uses it. So does SQL Server, and so does the .NET ThreadPool under the hood:
Each packet that comes out of the queue carries the completion key you passed when associating the handle and the OVERLAPPED pointer you passed with the call. That pair is how a worker that has never seen the socket before works out which connection and which operation just finished.
Setting Up an IOCP Server
// Step 1: Create the completion portHANDLE hIOCP = CreateIoCompletionPort( INVALID_HANDLE_VALUE, // No handle association yet NULL, // No existing IOCP 0, // Completion key 0 // Concurrent threads (0 = CPU count));
// Step 2: Associate a socket/file handle with the IOCPtypedef struct { OVERLAPPED overlapped; WSABUF wsaBuf; char buffer[4096]; int operationType; // OP_READ, OP_WRITE, etc.} PerIoData;
SOCKET clientSocket = accept(listenSocket, ...);CreateIoCompletionPort( (HANDLE)clientSocket, // Associate this socket hIOCP, // With this IOCP (ULONG_PTR)clientSocket, // Completion key (identifies the socket) 0);
// Step 3: Start an async operationPerIoData *ioData = calloc(1, sizeof(PerIoData));ioData->wsaBuf.buf = ioData->buffer;ioData->wsaBuf.len = sizeof(ioData->buffer);ioData->operationType = OP_READ;
DWORD flags = 0;WSARecv(clientSocket, &ioData->wsaBuf, 1, NULL, &flags, &ioData->overlapped, NULL);// Returns immediately - I/O happens in background
// Step 4: Worker thread loopDWORD WINAPI WorkerThread(LPVOID param) { HANDLE hIOCP = (HANDLE)param;
while (TRUE) { DWORD bytesTransferred; ULONG_PTR completionKey; PerIoData *ioData;
// Block until a completion is available BOOL success = GetQueuedCompletionStatus( hIOCP, &bytesTransferred, &completionKey, (LPOVERLAPPED *)&ioData, INFINITE );
SOCKET socket = (SOCKET)completionKey;
if (!success || bytesTransferred == 0) { // Connection closed or error closesocket(socket); free(ioData); continue; }
// Process the completed I/O if (ioData->operationType == OP_READ) { // Data received - process it ProcessData(ioData->buffer, bytesTransferred);
// Start another read ioData->operationType = OP_READ; DWORD flags = 0; WSARecv(socket, &ioData->wsaBuf, 1, NULL, &flags, &ioData->overlapped, NULL); } } return 0;}The IOCP kernel object limits the number of threads concurrently running completion callbacks to the number specified at creation (typically the CPU count). If a completion thread blocks on I/O, the IOCP allows another waiting thread to run, maintaining optimal CPU utilisation. This prevents the common problem of thread pool oversubscription where too many threads compete for CPU time.
Directory Management and File Notifications
Enumerating Directory Contents
WIN32_FIND_DATA findData;HANDLE hFind = FindFirstFile("C:\\MyDir\\*.txt", &findData);
if (hFind != INVALID_HANDLE_VALUE) { do { printf("%s %llu bytes %s\n", findData.cFileName, ((ULONGLONG)findData.nFileSizeHigh << 32) | findData.nFileSizeLow, (findData.dwFileAttributes & FILE_ATTRIBUTE_DIRECTORY) ? "[DIR]" : ""); } while (FindNextFile(hFind, &findData)); FindClose(hFind);}Monitoring File Changes
// Watch for changes in a directoryHANDLE hDir = CreateFile("C:\\WatchDir", FILE_LIST_DIRECTORY, FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE, NULL, OPEN_EXISTING, FILE_FLAG_BACKUP_SEMANTICS | FILE_FLAG_OVERLAPPED, NULL);
BYTE buffer[4096];OVERLAPPED ov = {0};ov.hEvent = CreateEvent(NULL, TRUE, FALSE, NULL);
ReadDirectoryChangesW( hDir, buffer, sizeof(buffer), TRUE, // Watch subtree FILE_NOTIFY_CHANGE_FILE_NAME | FILE_NOTIFY_CHANGE_LAST_WRITE, NULL, // Bytes returned (async) &ov, // Overlapped NULL // Completion routine);
// Wait for changesWaitForSingleObject(ov.hEvent, INFINITE);
// Parse notifications from bufferFILE_NOTIFY_INFORMATION *info = (FILE_NOTIFY_INFORMATION *)buffer;// info->Action: FILE_ACTION_ADDED, FILE_ACTION_REMOVED, etc.// info->FileName: name of changed file (wide string)That's the toolkit: CreateFile to get a handle, ReadFile/WriteFile for the bytes, OVERLAPPED to keep the thread free, and IOCP when one thread isn't enough. IOCP in particular is the bit worth getting fluent in — if you're writing anything that needs to scale on Windows, sooner or later the path leads through GetQueuedCompletionStatus.