Image by Author | Canva
Error handling is an important aspect of writing reliable Python applications. While basic try-except blocks are useful, they’re often not enough for complex applications that need to handle resources efficiently, provide meaningful error information, and maintain system stability.
This article explores five advanced error handling techniques that are particularly useful in production environments:
- Context managers for reliable resource management
- Custom exception hierarchies for domain-specific error handling
- Exception chaining for maintaining error context
- Error handling decorators for reusable error management
- Cleanup actions for guaranteed resource cleanup
We’ll go over each technique with practical examples and real-world use cases. So you’ll understand not just how to implement them, but when and why to use them.
▶️ You can find all the code on GitHub.
1. Context Managers
Context managers are perfect for handling resource management like file operations, network connections, or database transactions. They ensure proper cleanup even if errors occur.
Why Use Context Managers?
- Automatically handle setup and cleanup
- Guarantee resource release even if exceptions occur
- Make code cleaner and more maintainable
- Reduce chances of resource leaks
Example: Database Connection Handler
Here we create a DatabaseConnection class that manages database connections using Python’s context manager protocol:
class DatabaseConnection:
def __init__(self, connection_string):
self.connection_string = connection_string
self.conn = None
def __enter__(self):
try:
print(f"Connecting to database: self.connection_string")
# you'd use something like psycopg2 or SQLAlchemy
self.conn = "database_connection"
return self
except Exception as e:
raise ConnectionError(f"Failed to connect: str(e)")
def __exit__(self, exc_type, exc_val, exc_tb):
print("Closing database connection")
if self.conn:
# Close connection here
self.conn = None
return False # Don't suppress exceptions
The __enter__ method establishes the connection when entering the with block and returns the connection object. The __exit__ method automatically closes the connection when leaving the block, regardless of whether an error occurred.
This specific implementation is a simplified example – in real code, you’d replace the string “database_connection” with actual database connection logic using libraries like psycopg2 or SQLAlchemy.
with DatabaseConnection("postgresql://localhost:5432/mydb") as db:
# Do database operations
# Connection is automatically closed after this block
pass
This is particularly useful when you’re working with:
- Database connections
- File operations
- Network sockets
- Lock management in concurrent programming
▶️ To learn more read 3 Interesting Uses of Python’s Context Managers and How To Create Custom Context Managers in Python.
2. Custom Exception Hierarchies
Custom exceptions help you create more meaningful error handling specific to your application domain. They make error handling more precise and maintainable.
Why Create Custom Exceptions?
- Better error classification
- More specific error handling
- Improved debugging
- Clearer handling of possible failure modes
Example: E-commerce Order System
This code creates a hierarchy of custom exceptions for an order processing system.
class OrderError(Exception):
"""Base exception for order-related errors"""
pass
class PaymentError(OrderError):
"""Raised when payment processing fails"""
def __init__(self, message, transaction_id=None):
self.transaction_id = transaction_id
super().__init__(f"Payment failed: message")
class InventoryError(OrderError):
"""Raised when inventory is insufficient"""
def __init__(self, product_id, requested, available):
self.product_id = product_id
self.requested = requested
self.available = available
super().__init__(
f"Insufficient inventory for product product_id: "
f"requested requested, available available"
)
The base OrderError class serves as a parent for more specific exceptions. PaymentError includes a transaction_id to help track failed payments, while InventoryError carries information about the requested and available quantities.
def process_order(order):
try:
check_inventory(order)
process_payment(order)
except InventoryError as e:
# Handle inventory issues
notify_inventory_team(e.product_id)
raise
except PaymentError as e:
# Handle payment issues
if e.transaction_id:
reverse_transaction(e.transaction_id)
raise
The process_order function shows how to catch these specific exceptions and handle them differently – inventory issues trigger notifications to the stock team, while payment issues attempt to reverse the transaction.
3. Exception Chaining (raise from)
Exception chaining helps preserve the full error context when converting between exception types. It’s useful for debugging and maintaining error traceability.
Why Use Exception Chaining?
- Preserves error context
- Makes debugging easier
- Maintains error history
- Provides better error reporting
Example: Configuration System
The following snippet implements a configuration loader that attempts to read and parse a YAML configuration file. You can use PyYAML to work with YAML files in Python.
class ConfigError(Exception):
"""Configuration-related errors"""
pass
def load_database_config():
try:
with open('config/database.yaml') as f:
# Imagine we're using PyYAML here
return yaml.safe_load(f)
except FileNotFoundError as e:
raise ConfigError(
"Database configuration file not found"
) from e
except yaml.YAMLError as e:
raise ConfigError(
"Invalid database configuration format"
) from e
If the file isn’t found, it raises a ConfigError while preserving the original FileNotFoundError using the raise from syntax.
Similarly, if the YAML is invalid, it raises a ConfigError while maintaining a reference to the original YAMLError.
try:
config = load_database_config()
except ConfigError as e:
print(f"Configuration error: e")
print(f"Original error: e.__cause__")
When caught, both the high-level ConfigError message and the underlying cause can be accessed, providing complete error context.
4. Error Handling Decorators
Decorators allow you to separate error handling logic from business logic, making your code more modular and reusable.
Why Use Error Handling Decorators?
- Centralize error handling logic
- Make code more DRY (Don’t Repeat Yourself)
- Easy to apply consistent error handling
- Simplify testing
Example: API Request Handler
This code snippet creates a decorator that adds automatic retry logic to any function.
from functools import wraps
import logging
def handle_api_errors(retries=3, fallback_value=None):
def decorator(func):
@wraps(func)
def wrapper(*args, **kwargs):
for attempt in range(retries):
try:
return func(*args, **kwargs)
except (ConnectionError, TimeoutError) as e:
logging.error(
f"API call failed (attempt attempt + 1/retries): str(e)"
)
if attempt == retries - 1:
if fallback_value is not None:
return fallback_value
raise
except Exception as e:
logging.error(f"Unexpected error: str(e)")
raise
return wrapper
return decorator
The decorator takes two parameters: the number of retries and a fallback value. When applied to a function, it wraps the original function in retry logic that will:
- Attempt to execute the function
- Catch connection/timeout errors and retry up to the specified number of times
- Return a fallback value if all retries fail
- Log each failure attempt
- Let other unexpected errors propagate immediately
@handle_api_errors(retries=3, fallback_value=[])
def fetch_user_data(user_id):
# Make API call here
pass
The wrapper preserves the original function’s metadata using @wraps from functools. You may use it as shown above.
5. Cleanup Actions with try-finally
While similar to context managers, try-finally blocks give you more fine-grained control over cleanup actions and are useful in complex scenarios.
Why Use try-finally?
- Guarantee cleanup code execution
- Handle multiple resources
- More flexible than context managers
- Custom cleanup logic
Example: Image Processing Pipeline
Here we implement an image processing class that manages temporary files during image manipulation:
class ImageProcessor:
def __init__(self):
self.temp_files = []
def process_image(self, image_path):
temp_output = f"temp_image_path"
self.temp_files.append(temp_output)
try:
# Process the image
raw_data = self.load_image(image_path)
processed = self.apply_filters(raw_data)
self.save_image(processed, temp_output)
return self.upload_to_cloud(temp_output)
finally:
# Clean up temporary files
for temp_file in self.temp_files:
try:
os.remove(temp_file)
except OSError:
logging.error(f"Failed to remove temp file: temp_file")
self.temp_files = []
The process_image method creates temporary files for intermediate processing steps and tracks them in self.temp_files. The finally block ensures these temporary files are deleted even if an error occurs during processing.
The cleanup code itself is also wrapped in a try/except block to handle cases where file deletion fails, ensuring the cleanup attempt doesn’t raise new exceptions that could mask the original error.
Wrap-Up and Next Steps
That’s a wrap! Consider implementing these techniques where they provide the most value in your current projects:
- Use context managers for resource management (files, connections, locks)
- Create custom exceptions for domain-specific error cases
- Use decorators to standardize error handling across similar operations
- Implement proper cleanup actions for critical resources
Start with one error handling technique that makes the most sense for your application. Happy error handling!
Bala Priya C is a developer and technical writer from India. She likes working at the intersection of math, programming, data science, and content creation. Her areas of interest and expertise include DevOps, data science, and natural language processing. She enjoys reading, writing, coding, and coffee! Currently, she’s working on learning and sharing her knowledge with the developer community by authoring tutorials, how-to guides, opinion pieces, and more. Bala also creates engaging resource overviews and coding tutorials.