Pandas - Display All Rows and Columns (set_option)

Key Insights

Use pd.set_option() to configure Pandas display settings globally, controlling how many rows and columns are shown in DataFrame output
The display.max_rows, display.max_columns, display.width, and display.max_colwidth options give you fine-grained control over DataFrame rendering
Context managers and option_context() allow temporary display changes without affecting global settings, essential for production code

Understanding Pandas Display Limitations

By default, Pandas truncates large DataFrames to prevent overwhelming your console with output. When you have a DataFrame with more than 60 rows or more than 20 columns, Pandas displays only a subset with ellipses (...) indicating hidden data.

import pandas as pd
import numpy as np

# Create a large DataFrame
df = pd.DataFrame(
    np.random.randn(100, 30),
    columns=[f'col_{i}' for i in range(30)]
)

print(df)
# Output shows only first/last 5 rows and first/last 10 columns

This default behavior is sensible for exploratory data analysis, but when you need to inspect specific data or debug issues, you need full visibility.

Setting Maximum Rows Display

The display.max_rows option controls how many rows Pandas will show before truncating. Set it to None to display all rows, or specify an integer for a custom limit.

import pandas as pd

# Create sample DataFrame
df = pd.DataFrame({
    'A': range(100),
    'B': range(100, 200),
    'C': range(200, 300)
})

# Default behavior - truncated
print(df)

# Display all rows
pd.set_option('display.max_rows', None)
print(df)

# Display maximum 20 rows
pd.set_option('display.max_rows', 20)
print(df)

# Reset to default
pd.reset_option('display.max_rows')

For DataFrames with thousands of rows, displaying everything is rarely practical. Instead, use a higher threshold that suits your typical analysis:

# Set a reasonable maximum for large datasets
pd.set_option('display.max_rows', 500)

Setting Maximum Columns Display

The display.max_columns option works identically for columns. This is particularly useful when working with wide datasets common in machine learning feature engineering.

import pandas as pd
import numpy as np

# Create wide DataFrame
df = pd.DataFrame(
    np.random.randn(10, 50),
    columns=[f'feature_{i}' for i in range(50)]
)

# Display all columns
pd.set_option('display.max_columns', None)
print(df)

# Display specific number of columns
pd.set_option('display.max_columns', 30)
print(df)

Controlling Column Width and Display Width

Two additional options control how data is rendered within columns:

import pandas as pd

df = pd.DataFrame({
    'short': ['a', 'b', 'c'],
    'long_text': [
        'This is a very long string that will be truncated by default',
        'Another extremely long string with lots of text content',
        'Yet another long string that exceeds normal display limits'
    ]
})

# Default - truncates long strings
print(df)

# Show full column content
pd.set_option('display.max_colwidth', None)
print(df)

# Set specific width
pd.set_option('display.max_colwidth', 100)
print(df)

# Control total display width in characters
pd.set_option('display.width', None)  # Auto-detect terminal width
pd.set_option('display.width', 200)   # Set specific width

The display.width option is particularly important when working in different terminal environments or when generating reports.

Using Context Managers for Temporary Changes

Modifying global settings can have unintended side effects in larger applications. Use option_context() to temporarily change display settings:

import pandas as pd
import numpy as np

df = pd.DataFrame(np.random.randn(100, 25))

# Global settings remain unchanged
print("Default display:")
print(df)

# Temporary change within context
with pd.option_context('display.max_rows', None, 
                       'display.max_columns', None):
    print("\nFull display:")
    print(df)

# Back to default
print("\nDefault again:")
print(df)

This approach is essential in production code, Jupyter notebooks shared with teams, and automated reporting scripts where you don’t want to alter the global state.

Practical Configuration for Data Analysis

Here’s a comprehensive setup that works well for most data analysis workflows:

import pandas as pd

def configure_pandas_display():
    """Configure Pandas display settings for optimal data analysis."""
    pd.set_option('display.max_rows', 100)
    pd.set_option('display.max_columns', None)
    pd.set_option('display.width', None)
    pd.set_option('display.max_colwidth', 50)
    pd.set_option('display.precision', 3)
    pd.set_option('display.float_format', '{:.2f}'.format)

configure_pandas_display()

# Test with sample data
df = pd.DataFrame({
    'id': range(150),
    'value': np.random.randn(150) * 1000,
    'category': np.random.choice(['A', 'B', 'C'], 150),
    'description': ['Long text ' * 10 for _ in range(150)]
})

print(df)

Viewing All Options and Resetting

To see all available display options and their current values:

import pandas as pd

# Show all display-related options
pd.describe_option('display')

# Show specific option
pd.describe_option('display.max_rows')

# Get current value
current_max_rows = pd.get_option('display.max_rows')
print(f"Current max_rows: {current_max_rows}")

# Reset specific option
pd.reset_option('display.max_rows')

# Reset all options to defaults
pd.reset_option('all')

Performance Considerations

Displaying large DataFrames has performance implications. Rendering thousands of rows to the console takes time and memory:

import pandas as pd
import numpy as np
import time

# Create large DataFrame
large_df = pd.DataFrame(np.random.randn(10000, 100))

# Time full display
pd.set_option('display.max_rows', None)
start = time.time()
print(large_df)
full_time = time.time() - start

# Time truncated display
pd.set_option('display.max_rows', 60)
start = time.time()
print(large_df)
truncated_time = time.time() - start

print(f"\nFull display: {full_time:.3f}s")
print(f"Truncated display: {truncated_time:.3f}s")

For large datasets, consider alternative approaches:

# View specific slices
print(df.head(50))
print(df.tail(50))
print(df.iloc[1000:1050])

# Use sampling
print(df.sample(100))

# Export to file for external viewing
df.to_csv('full_data.csv', index=False)
df.to_html('full_data.html')

Integration with Jupyter Notebooks

In Jupyter environments, these settings integrate with the notebook’s display system:

# Optimal Jupyter configuration
pd.set_option('display.max_rows', 200)
pd.set_option('display.max_columns', None)
pd.set_option('display.notebook_repr_html', True)
pd.set_option('display.max_colwidth', None)

# For specific cells, use option_context
with pd.option_context('display.max_rows', None):
    display(df)  # Jupyter's display function

The display.notebook_repr_html option enables rich HTML rendering in Jupyter, providing better formatting and interactivity compared to plain text output.

Common Patterns and Best Practices

Create reusable configuration functions for different scenarios:

import pandas as pd
from contextlib import contextmanager

@contextmanager
def full_display():
    """Context manager for full DataFrame display."""
    with pd.option_context(
        'display.max_rows', None,
        'display.max_columns', None,
        'display.max_colwidth', None,
        'display.width', None
    ):
        yield

@contextmanager
def compact_display():
    """Context manager for compact DataFrame display."""
    with pd.option_context(
        'display.max_rows', 20,
        'display.max_columns', 10,
        'display.max_colwidth', 20
    ):
        yield

# Usage
df = pd.DataFrame(np.random.randn(100, 30))

with full_display():
    print(df)

with compact_display():
    print(df)

Understanding and controlling Pandas display options is fundamental for efficient data analysis. These settings don’t modify your data—they only affect how it’s rendered. Use global settings for interactive sessions, context managers for production code, and always consider performance implications when working with large datasets.