Python - Write to File | Application Architect

Key Insights

Python offers multiple file writing modes: w (overwrite), a (append), x (exclusive creation), and r+ (read/write), each serving distinct use cases in production applications
Context managers (with statement) automatically handle file closing and exception scenarios, preventing resource leaks that plague manual file handling
Binary mode, buffering controls, and atomic writes using temporary files are essential techniques for robust file operations in production systems

Basic File Writing Operations

Python’s built-in open() function provides straightforward file writing capabilities. The most common approach uses the w mode, which creates a new file or truncates an existing one:

# Simple write operation
with open('output.txt', 'w') as f:
    f.write('Hello, World!\n')
    f.write('Second line\n')

The with statement ensures the file closes properly, even if exceptions occur. Without it, you risk resource leaks:

# Manual file handling (not recommended)
f = open('output.txt', 'w')
try:
    f.write('Data\n')
finally:
    f.close()  # Must explicitly close

For appending to existing files without losing content:

with open('log.txt', 'a') as f:
    f.write('New log entry\n')

The x mode provides exclusive creation, raising FileExistsError if the file already exists:

try:
    with open('config.ini', 'x') as f:
        f.write('default_config=true\n')
except FileExistsError:
    print("Configuration file already exists")

Writing Multiple Lines Efficiently

The writelines() method writes a list of strings without adding line separators:

lines = ['First line\n', 'Second line\n', 'Third line\n']
with open('output.txt', 'w') as f:
    f.writelines(lines)

For cleaner code with automatic newlines, use a loop with print():

data = ['apple', 'banana', 'cherry']
with open('fruits.txt', 'w') as f:
    for item in data:
        print(item, file=f)

When working with large datasets, write in chunks to manage memory:

def write_large_dataset(filename, data_generator):
    with open(filename, 'w') as f:
        for chunk in data_generator:
            f.writelines(f'{line}\n' for line in chunk)

Binary File Writing

Binary mode (wb, ab) is essential for non-text files like images, executables, or serialized data:

# Writing binary data
data = bytes([0x48, 0x65, 0x6C, 0x6C, 0x6F])  # "Hello" in ASCII
with open('data.bin', 'wb') as f:
    f.write(data)

# Writing image data
import requests
response = requests.get('https://example.com/image.jpg')
with open('downloaded.jpg', 'wb') as f:
    f.write(response.content)

For structured binary data, use struct:

import struct

# Write binary records: ID (int), value (float), active (bool)
records = [
    (1, 99.5, True),
    (2, 87.3, False),
    (3, 105.2, True)
]

with open('records.dat', 'wb') as f:
    for record_id, value, active in records:
        packed = struct.pack('If?', record_id, value, active)
        f.write(packed)

Encoding and Character Sets

Explicitly specify encoding for non-ASCII text to avoid platform-dependent behavior:

# UTF-8 encoding (recommended default)
with open('unicode.txt', 'w', encoding='utf-8') as f:
    f.write('Hello 世界 🌍\n')

# Reading back with matching encoding
with open('unicode.txt', 'r', encoding='utf-8') as f:
    content = f.read()

Handle encoding errors gracefully:

# Replace invalid characters instead of crashing
with open('output.txt', 'w', encoding='ascii', errors='replace') as f:
    f.write('Café')  # é becomes ?

# Ignore invalid characters
with open('output.txt', 'w', encoding='ascii', errors='ignore') as f:
    f.write('Café')  # é is omitted

Buffering Control

Python buffers file writes by default. Control buffering for performance or data integrity:

# Unbuffered binary write (immediate disk writes)
with open('critical.log', 'wb', buffering=0) as f:
    f.write(b'Critical error occurred\n')

# Line-buffered text mode (flush on newlines)
with open('app.log', 'w', buffering=1) as f:
    f.write('Line 1\n')  # Flushed immediately
    f.write('Line 2')    # Buffered until newline

# Custom buffer size (8KB)
with open('data.txt', 'w', buffering=8192) as f:
    for i in range(10000):
        f.write(f'Record {i}\n')

Manual flushing ensures data reaches disk:

with open('transaction.log', 'w') as f:
    f.write('Transaction started\n')
    f.flush()  # Force write to disk
    # Perform critical operation
    f.write('Transaction completed\n')

Atomic Writes with Temporary Files

Prevent data corruption during writes by using temporary files and atomic renames:

import os
import tempfile

def atomic_write(filename, content):
    # Create temp file in same directory (same filesystem)
    dir_name = os.path.dirname(filename)
    fd, temp_path = tempfile.mkstemp(dir=dir_name, text=True)
    
    try:
        with os.fdopen(fd, 'w') as f:
            f.write(content)
            f.flush()
            os.fsync(f.fileno())  # Ensure data on disk
        
        # Atomic rename (POSIX systems)
        os.replace(temp_path, filename)
    except:
        os.unlink(temp_path)  # Clean up on failure
        raise

# Usage
atomic_write('config.json', '{"version": "2.0"}')

For Windows compatibility and additional features, use atomicwrites library:

from atomicwrites import atomic_write

with atomic_write('important.txt', overwrite=True) as f:
    f.write('Critical configuration data\n')

Writing Structured Data

JSON files are common for configuration and data exchange:

import json

data = {
    'users': [
        {'id': 1, 'name': 'Alice', 'active': True},
        {'id': 2, 'name': 'Bob', 'active': False}
    ],
    'timestamp': '2024-01-15T10:30:00Z'
}

with open('data.json', 'w') as f:
    json.dump(data, f, indent=2)

CSV writing for tabular data:

import csv

rows = [
    ['Name', 'Age', 'City'],
    ['Alice', 30, 'New York'],
    ['Bob', 25, 'San Francisco']
]

with open('users.csv', 'w', newline='') as f:
    writer = csv.writer(f)
    writer.writerows(rows)

# Dictionary-based CSV writing
with open('users.csv', 'w', newline='') as f:
    fieldnames = ['Name', 'Age', 'City']
    writer = csv.DictWriter(f, fieldnames=fieldnames)
    writer.writeheader()
    writer.writerow({'Name': 'Alice', 'Age': 30, 'City': 'New York'})

Error Handling and Path Management

Use pathlib for robust path handling:

from pathlib import Path

output_path = Path('output') / 'data' / 'results.txt'
output_path.parent.mkdir(parents=True, exist_ok=True)

with output_path.open('w') as f:
    f.write('Results data\n')

Comprehensive error handling:

def safe_write(filename, content):
    try:
        with open(filename, 'w') as f:
            f.write(content)
    except PermissionError:
        print(f"Permission denied: {filename}")
    except OSError as e:
        print(f"OS error writing {filename}: {e}")
    except Exception as e:
        print(f"Unexpected error: {e}")
        raise

These patterns cover the essential file writing operations you’ll encounter in production Python applications. Choose the appropriate mode, encoding, and error handling strategy based on your specific requirements for data integrity, performance, and reliability.