How to Use String Operations in Polars

Key Insights

Polars string operations live under the .str namespace and execute significantly faster than pandas equivalents due to Rust’s underlying implementation and lazy evaluation support.
Pattern matching with contains(), extract(), and regex support handles most real-world text filtering and parsing needs without resorting to slow apply() functions.
String splitting returns list columns that integrate naturally with Polars’ expression system, enabling clean data transformations without awkward workarounds.

Introduction to String Operations in Polars

Polars handles string operations through a dedicated .str namespace accessible on any string column expression. If you’re coming from pandas, the mental model is similar—you chain methods off a special accessor—but the execution model differs fundamentally.

Every string operation in Polars is an expression. This means you can compose complex transformations, benefit from query optimization in lazy mode, and avoid the row-by-row Python overhead that plagues pandas string operations. Polars processes strings in Rust, operating on entire columns at once.

Here’s the basic pattern:

import polars as pl

df = pl.DataFrame({
    "name": ["  Alice  ", "BOB", "charlie"],
    "email": ["alice@example.com", "bob@test.org", "charlie@example.com"]
})

# Access string methods through .str namespace
result = df.with_columns(
    pl.col("name").str.strip_chars().str.to_lowercase().alias("clean_name")
)

print(result)

shape: (3, 3)
┌───────────┬─────────────────────┬────────────┐
│ name      ┆ email               ┆ clean_name │
│ ---       ┆ ---                 ┆ ---        │
│ str       ┆ str                 ┆ str        │
╞═══════════╪═════════════════════╪════════════╡
│   Alice   ┆ alice@example.com   ┆ alice      │
│ BOB       ┆ bob@test.org        ┆ bob        │
│ charlie   ┆ charlie@example.com ┆ charlie    │
└───────────┴─────────────────────┴────────────┘

Notice how operations chain naturally. Each .str method returns an expression, so you can stack transformations without intermediate variables.

Essential String Transformations

Most string work involves cleaning messy data. Polars provides the core transformations you need for normalization.

Case conversion uses to_lowercase() and to_uppercase(). There’s also to_titlecase() for proper noun formatting.

Whitespace handling relies on strip_chars() for both ends, strip_chars_start() for leading whitespace, and strip_chars_end() for trailing. These methods accept an optional argument to strip specific characters.

Text replacement comes in two forms: replace() for the first occurrence and replace_all() for global replacement. Both support literal strings and regex patterns.

df = pl.DataFrame({
    "product_code": ["  ABC-123  ", "def-456", "GHI-789  "],
    "description": ["Widget (NEW)", "Gadget (SALE)", "Tool (discontinued)"],
    "price_text": ["$19.99", "$24.50", "$9.99"]
})

cleaned = df.with_columns(
    # Normalize product codes: strip, uppercase, replace hyphens
    pl.col("product_code")
        .str.strip_chars()
        .str.to_uppercase()
        .str.replace("-", "_")
        .alias("normalized_code"),
    
    # Remove parenthetical notes from descriptions
    pl.col("description")
        .str.replace_all(r"\s*\([^)]+\)", "")
        .alias("clean_description"),
    
    # Strip currency symbol for numeric conversion
    pl.col("price_text")
        .str.strip_chars("$")
        .cast(pl.Float64)
        .alias("price")
)

print(cleaned)

shape: (3, 6)
┌──────────────┬─────────────────────┬────────────┬─────────────────┬───────────────────┬───────┐
│ product_code ┆ description         ┆ price_text ┆ normalized_code ┆ clean_description ┆ price │
│ ---          ┆ ---                 ┆ ---        ┆ ---             ┆ ---               ┆ ---   │
│ str          ┆ str                 ┆ str        ┆ str             ┆ str               ┆ f64   │
╞══════════════╪═════════════════════╪════════════╪═════════════════╪═══════════════════╪═══════╡
│   ABC-123    ┆ Widget (NEW)        ┆ $19.99     ┆ ABC_123         ┆ Widget            ┆ 19.99 │
│ def-456      ┆ Gadget (SALE)       ┆ $24.50     ┆ DEF_456         ┆ Gadget            ┆ 24.5  │
│ GHI-789      ┆ Tool (discontinued) ┆ $9.99      ┆ GHI_789         ┆ Tool              ┆ 9.99  │
└──────────────┴─────────────────────┴────────────┴─────────────────┴───────────────────┴───────┘

The regex in replace_all() removes any parenthetical content. This runs at native speed—no Python regex engine involved.

Pattern Matching and Extraction

Filtering and extracting substrings based on patterns covers a huge portion of text processing needs.

Boolean matching uses contains(), starts_with(), and ends_with(). These return boolean columns perfect for filtering.

Extraction uses extract() for a single capture group and extract_all() for multiple matches. Both require regex patterns with capture groups.

df = pl.DataFrame({
    "email": [
        "alice@example.com",
        "bob.smith@test.org",
        "support@company.co.uk",
        "invalid-email"
    ],
    "phone": [
        "Call: (555) 123-4567",
        "Phone: 555.987.6543",
        "N/A",
        "Contact: 555-111-2222"
    ],
    "log_entry": [
        "ERROR: Connection failed at 2024-01-15",
        "INFO: User logged in",
        "WARNING: Disk space low",
        "ERROR: Timeout after 30s"
    ]
})

# Filter and extract patterns
result = df.with_columns(
    # Extract domain from email
    pl.col("email")
        .str.extract(r"@([a-zA-Z0-9.-]+)", group_index=1)
        .alias("domain"),
    
    # Extract phone digits only
    pl.col("phone")
        .str.extract_all(r"\d+")
        .list.join("")
        .alias("phone_digits"),
    
    # Extract log level
    pl.col("log_entry")
        .str.extract(r"^(ERROR|WARNING|INFO)", group_index=1)
        .alias("log_level")
).filter(
    # Keep only error entries from example.com domain
    pl.col("email").str.contains("example.com") |
    pl.col("log_entry").str.starts_with("ERROR")
)

print(result)

shape: (3, 6)
┌───────────────────┬──────────────────────┬──────────────────────────────────┬─────────────┬──────────────┬───────────┐
│ email             ┆ phone                ┆ log_entry                        ┆ domain      ┆ phone_digits ┆ log_level │
│ ---               ┆ ---                  ┆ ---                              ┆ ---         ┆ ---          ┆ ---       │
│ str               ┆ str                  ┆ str                              ┆ str         ┆ str          ┆ str       │
╞═══════════════════╪══════════════════════╪══════════════════════════════════╪═════════════╪══════════════╪═══════════╡
│ alice@example.com ┆ Call: (555) 123-4567 ┆ ERROR: Connection failed at 20…  ┆ example.com ┆ 5551234567   ┆ ERROR     │
│ bob.smith@test.o… ┆ Phone: 555.987.6543  ┆ INFO: User logged in             ┆ test.org    ┆ 5559876543   ┆ INFO      │
│ invalid-email     ┆ Contact: 555-111-22… ┆ ERROR: Timeout after 30s         ┆ null        ┆ 5551112222   ┆ ERROR     │
└───────────────────┴──────────────────────┴──────────────────────────────────┴─────────────┴──────────────┴───────────┘

Notice that extract_all() returns a list column. We chain .list.join("") to concatenate the matches into a single string. Failed extractions return null, which Polars handles gracefully.

Splitting and Joining Strings

Splitting strings creates list columns. Polars embraces this rather than fighting it—you work with the list type directly or explode it into rows.

split() divides on a delimiter and returns a variable-length list. split_exact() expects a specific number of parts and returns a struct with named fields.

concat_str() joins multiple columns with an optional separator.

df = pl.DataFrame({
    "full_name": ["Alice Johnson", "Bob Smith Jr", "Charlie"],
    "street": ["123 Main St", "456 Oak Ave", "789 Pine Rd"],
    "city": ["Boston", "Chicago", "Denver"],
    "state": ["MA", "IL", "CO"],
    "zip": ["02101", "60601", "80201"]
})

result = df.with_columns(
    # Split name into parts (variable length)
    pl.col("full_name").str.split(" ").alias("name_parts"),
    
    # Extract first and last name using split_exact
    pl.col("full_name")
        .str.split_exact(" ", n=1)
        .struct.rename_fields(["first_name", "rest"])
        .alias("name_split")
).with_columns(
    # Get first name from struct
    pl.col("name_split").struct.field("first_name"),
    
    # Get last element as last name
    pl.col("name_parts").list.last().alias("last_name"),
    
    # Combine address fields
    pl.concat_str(
        pl.col("street"),
        pl.col("city"),
        pl.col("state"),
        pl.col("zip"),
        separator=", "
    ).alias("full_address")
)

print(result.select(["full_name", "first_name", "last_name", "full_address"]))

shape: (3, 4)
┌───────────────┬────────────┬───────────┬─────────────────────────────────┐
│ full_name     ┆ first_name ┆ last_name ┆ full_address                    │
│ ---           ┆ ---        ┆ ---       ┆ ---                             │
│ str           ┆ str        ┆ str       ┆ str                             │
╞═══════════════╪════════════╪═══════════╪═════════════════════════════════╡
│ Alice Johnson ┆ Alice      ┆ Johnson   ┆ 123 Main St, Boston, MA, 02101  │
│ Bob Smith Jr  ┆ Bob        ┆ Jr        ┆ 456 Oak Ave, Chicago, IL, 60601 │
│ Charlie       ┆ Charlie    ┆ null      ┆ 789 Pine Rd, Denver, CO, 80201  │
└───────────────┴────────────┴───────────┴─────────────────────────────────┘

The split_exact() approach works well when you know the structure. For variable-length data, use split() and access elements via list methods.

String Length and Slicing

Length and substring operations help with validation and truncation.

len_chars() counts Unicode characters. len_bytes() counts bytes—important for storage limits or binary protocols.

slice() extracts substrings by offset and length. head() and tail() grab characters from the start or end.

df = pl.DataFrame({
    "description": [
        "This is a very long product description that exceeds our limit",
        "Short desc",
        "Medium length description here"
    ],
    "sku": ["ABC123XYZ", "DEF456", "GHIJ789012"]
})

MAX_DESC_LENGTH = 30
SKU_PREFIX_LENGTH = 3

result = df.with_columns(
    # Character and byte lengths
    pl.col("description").str.len_chars().alias("char_count"),
    pl.col("description").str.len_bytes().alias("byte_count"),
    
    # Truncate long descriptions
    pl.when(pl.col("description").str.len_chars() > MAX_DESC_LENGTH)
        .then(
            pl.col("description").str.head(MAX_DESC_LENGTH - 3) + "..."
        )
        .otherwise(pl.col("description"))
        .alias("truncated"),
    
    # Extract SKU prefix and suffix
    pl.col("sku").str.head(SKU_PREFIX_LENGTH).alias("sku_prefix"),
    pl.col("sku").str.tail(3).alias("sku_suffix"),
    
    # Validate SKU length (should be 6-10 chars)
    pl.col("sku").str.len_chars().is_between(6, 10).alias("valid_sku")
)

print(result)

shape: (3, 8)
┌─────────────────────────────────┬────────────┬────────────┬────────────┬─────────────────────────────────┬────────────┬────────────┬───────────┐
│ description                     ┆ sku        ┆ char_count ┆ byte_count ┆ truncated                       ┆ sku_prefix ┆ sku_suffix ┆ valid_sku │
│ ---                             ┆ ---        ┆ ---        ┆ ---        ┆ ---                             ┆ ---        ┆ ---        ┆ ---       │
│ str                             ┆ str        ┆ u32        ┆ u32        ┆ str                             ┆ str        ┆ str        ┆ bool      │
╞═════════════════════════════════╪════════════╪════════════╪════════════╪═════════════════════════════════╪════════════╪════════════╪═══════════╡
│ This is a very long product d… ┆ ABC123XYZ  ┆ 62         ┆ 62         ┆ This is a very long produc...   ┆ ABC        ┆ XYZ        ┆ true      │
│ Short desc                      ┆ DEF456     ┆ 10         ┆ 10         ┆ Short desc                      ┆ DEF        ┆ 456        ┆ true      │
│ Medium length description here  ┆ GHIJ789012 ┆ 30         ┆ 30         ┆ Medium length description here  ┆ GHI        ┆ 012        ┆ true      │
└─────────────────────────────────┴────────────┴────────────┴────────────┴─────────────────────────────────┴────────────┴────────────┴───────────┘

Performance Considerations

Polars string operations outperform pandas significantly, especially on larger datasets. The difference comes from three factors: Rust execution, columnar processing, and lazy evaluation optimization.

import polars as pl
import pandas as pd
import time

# Generate test data
n_rows = 1_000_000
data = {
    "text": [f"  Sample Text {i} with MIXED case  " for i in range(n_rows)],
    "code": [f"PREFIX-{i:06d}-SUFFIX" for i in range(n_rows)]
}

# Pandas benchmark
pdf = pd.DataFrame(data)
start = time.perf_counter()
pdf["clean"] = pdf["text"].str.strip().str.lower()
pdf["extracted"] = pdf["code"].str.extract(r"-(\d+)-")[0]
pandas_time = time.perf_counter() - start

# Polars eager benchmark
pldf = pl.DataFrame(data)
start = time.perf_counter()
result = pldf.with_columns(
    pl.col("text").str.strip_chars().str.to_lowercase().alias("clean"),
    pl.col("code").str.extract(r"-(\d+)-", group_index=1).alias("extracted")
)
polars_eager_time = time.perf_counter() - start

# Polars lazy benchmark
start = time.perf_counter()
result = (
    pl.LazyFrame(data)
    .with_columns(
        pl.col("text").str.strip_chars().str.to_lowercase().alias("clean"),
        pl.col("code").str.extract(r"-(\d+)-", group_index=1).alias("extracted")
    )
    .collect()
)
polars_lazy_time = time.perf_counter() - start

print(f"Pandas:       {pandas_time:.3f}s")
print(f"Polars eager: {polars_eager_time:.3f}s")
print(f"Polars lazy:  {polars_lazy_time:.3f}s")
print(f"Speedup:      {pandas_time / polars_lazy_time:.1f}x")

Typical results show Polars running 3-10x faster depending on the operations. The lazy API often edges out eager mode because the query optimizer can fuse operations.

Avoid map_elements() for string work. It drops you back into Python and kills performance. If you need custom logic, check if you can express it with existing string methods or regex patterns first.

Conclusion

Polars string operations cover the essentials: transformation with to_lowercase(), strip_chars(), and replace(); pattern matching with contains() and extract(); splitting with split() and joining with concat_str(); and measurement with len_chars() and slice().

The key advantage over pandas isn’t just speed—it’s composability. String methods chain into expressions that optimize together. You write readable transformations and get fast execution without manual optimization.

For advanced operations like fuzzy matching or natural language processing, you’ll need external libraries. But for data cleaning, validation, and extraction, the built-in .str namespace handles the job efficiently.