Unlock pandas performance with zero code changes.

unlockedpd is a drop-in performance booster for pandas that achieves 5-15x speedups on rolling, expanding, EWM, and cumulative operations. Just import unlockedpd after pandas and your existing code runs faster.

import pandas as pd
import unlockedpd  # That's it. Your pandas code is now faster.

df = pd.DataFrame(...)
df.rolling(20).mean()  # 5x faster!
df.expanding().max()   # 15x faster!
df.ewm(span=20).mean() # 4.8x faster!

Key advantages:

• Zero code changes: Works with your existing pandas code
• No infrastructure: No Ray, no Dask, no distributed setup
• No new API to learn: It's still pandas
• Automatic fallback: Falls back to pandas for unsupported cases

Tested on a 64-core machine with a 0.8GB DataFrame (10,000 rows x 10,000 columns):

pip install unlockedpd

Requirements:

• Python 3.9+
• pandas >= 1.5
• numba >= 0.56
• numpy >= 1.21

import pandas as pd
import unlockedpd  # Import after pandas

# Your existing code works unchanged
df = pd.DataFrame(np.random.randn(10000, 1000))
result = df.rolling(20).mean()  # Automatically optimized!

import unlockedpd

# Disable optimizations temporarily
unlockedpd.config.enabled = False

# Set thread count (default: min(cpu_count, 32))
unlockedpd.config.num_threads = 16

# Enable warnings when falling back to pandas
unlockedpd.config.warn_on_fallback = True

# Set minimum elements for parallel execution
unlockedpd.config.parallel_threshold = 500_000

export UNLOCKEDPD_ENABLED=false
export UNLOCKEDPD_NUM_THREADS=16
export UNLOCKEDPD_WARN_ON_FALLBACK=true
export UNLOCKEDPD_PARALLEL_THRESHOLD=500000

from unlockedpd import _PatchRegistry

with _PatchRegistry.temporarily_unpatched():
    # Uses original pandas here
    result = df.rolling(20).mean()

unlockedpd achieves its speedups through:

1. Numba JIT compilation: Operations are compiled to optimized machine code
2. nogil=True: Releases Python's GIL during computation
3. ThreadPoolExecutor: Achieves true parallelism across CPU cores
4. Column-wise chunking: Distributes work efficiently across threads

The key insight: @njit(nogil=True) + ThreadPoolExecutor combines Numba's fast compiled loops with true multi-threaded parallelism.

┌─────────────────────────────────────────────────────────────┐
│                    ThreadPoolExecutor                        │
│  ┌─────────┐  ┌─────────┐  ┌─────────┐       ┌─────────┐   │
│  │ Thread 1│  │ Thread 2│  │ Thread 3│  ...  │Thread 32│   │
│  │ Cols 0-k│  │Cols k-2k│  │Cols 2k..│       │Cols ..N │   │
│  │ (nogil) │  │ (nogil) │  │ (nogil) │       │ (nogil) │   │
│  └─────────┘  └─────────┘  └─────────┘       └─────────┘   │
└─────────────────────────────────────────────────────────────┘

Fully optimized (5-15x faster):

• rolling().mean(), sum(), std(), var(), min(), max(), count(), skew(), kurt(), median(), quantile()
• expanding().mean(), sum(), std(), var(), min(), max(), count(), skew(), kurt()
• ewm().mean(), std(), var()
• cumsum(), cumprod(), cummin(), cummax()
• rank() (both axis=0 and axis=1)
• pct_change(), diff(), shift()
• rolling().corr(), rolling().cov() (pairwise)

Passes through to pandas (unchanged):

• rolling().apply() (custom functions)
• Series operations (optimizations target DataFrames)
• Non-numeric columns (auto-fallback)

unlockedpd is designed for 100% pandas compatibility:

• Drop-in replacement: No code changes required
• Automatic fallback: If optimization fails, falls back to pandas
• Type preservation: Returns same types as pandas
• Index preservation: Maintains DataFrame/Series indices
• NaN handling: Correctly handles missing values

# Clone the repo
git clone https://github.com/Yeachan-Heo/unlockedpd
cd unlockedpd

# Install with dev dependencies
pip install -e ".[dev]"

# Run benchmarks
pytest benchmarks/ -v

Contributions are welcome! Areas of interest:

• Additional operation optimizations
• Performance improvements
• Documentation and examples
• Bug reports and fixes

100% Pandas Compatibility Fixes:

• Fixed pct_change() division by zero - Now correctly returns inf/-inf when dividing by zero (previously returned NaN)
• Fixed rolling skew() and kurt() - Corrected bias correction formulas to match pandas exactly
• Fixed expanding skew() and kurt() - Corrected bias correction formulas to match pandas exactly
• Fixed zero variance handling - Rolling/expanding skew returns 0.0, kurt returns -3.0 for constant data (matching pandas)
• Fixed EWM mean() formula - Corrected weight decay calculation to match pandas exactly
• Fixed EWM std() and var() - Corrected bias correction formula and first value handling (returns NaN for first observation)

All operations now pass strict pandas compatibility tests (rtol=1e-10) for edge cases including:

• All-NaN columns
• Zero variance (constant data)
• Near-zero variance (numerical precision edge cases)
• Division by zero in pct_change

Critical Bug Fix:

• Fixed pct_change() NaN handling to match pandas default behavior
  • Previous versions treated fill_method=None as default, causing 5x more NaN values
  • Now correctly defaults to fill_method='pad' (forward fill before computing), matching pandas
  • This fix resolves "Weights are all zero" errors in downstream applications using unlockedpd

API:

• pct_change(fill_method='pad') - Default, matches pandas behavior (forward fills NaN before computing)
• pct_change(fill_method=None) - No fill, fastest option (4.8x vs pandas), use when data has no NaN

• Major performance improvements across all operations
• Added EWM, expanding, cumulative, and pairwise operations
• Improved parallel dispatch and memory layout optimization

• Initial release with rolling, rank, and transform operations

MIT License - see LICENSE for details.

Built with:

• Numba - JIT compilation for Python
• pandas - Data analysis library
• NumPy - Numerical computing

This entire project was built using oh-my-claude-sisyphus, an advanced Claude Code harness that enables autonomous, iterative development with specialized AI agents. The codebase, benchmarks, documentation, and optimizations were all generated through the sisyphus workflow orchestration system.

Key oh-my-claude-sisyphus features used:

• Ralph-Plan: Iterative planning with Prometheus (planner), Oracle (advisor), and Momus (reviewer) agents
• Ultrawork Mode: Parallel agent execution for maximum throughput
• Sisyphus-Junior: Focused task execution for implementation work

unlockedpd - Because your pandas code deserves to be fast.