Navigated to pattern-extraction-performance
Verified Benchmark — June 2026

Pattern Extraction Performance: 142M Matches from 10GB, Constant Memory

~187K matches/second, measured at 10GB. Output streams to disk, so worker memory stays flat — an ~18MB live floor regardless of file size. Emails, phones, credit cards, SSNs, URLs, dates, IP addresses, and ZIP codes — all extracted and validated in your browser with zero server transmission. Works alongside Data Masking for full PII workflows, or see the full tool overview.

~187K/s
Throughput
matches/sec (10GB density)
10.8GB
Largest Tested
141.9M matches
Never
File Uploads
zero transmission
8
Pattern Types
email, phone, CC, SSN...

Benchmark Performance

All times: median of 3 runs (2 at 10M rows), wall-clock from worker start to results render, on headless Chromium (Puppeteer), Windows 11, dev machine (Alder Lake class), 64GB RAM, June 2026. Each row contains an email, a US phone number, a URL, and a numeric date. Only measured sizes are shown. Results vary by hardware, browser, and data density.

Performance at Scale

Chrome · Windows 11 · dev machine (Alder Lake class) · 64GB RAM · June 2026

ConfigurationTimeThroughputTest Notes
100K–5M rows · balanced< 1 min—Completes well under a minute; not separately benchmarked per size
Throughput (high match-density)—~187K matches/secMeasured at 10.8GB (Config A, non-dedupe)
10.8GB CSV · email · non-dedupe~13 min~187K matches/sec141,891,891 matches → 14.8GB output; peak heap 122MB, live floor 18MB
9.3GB CSV · email · dedupe~5 minexternal sort-merge47.3M matches → 20,000,000 distinct (exact); peak heap 326MB
Memory (any file size)—constantOutput streams to disk (OPFS) — heap flat at ~18MB live floor regardless of input/output size
CSV file-size limit—64GBSanity guard; CSV is RAM-constant, bounded by time/disk not memory
Excel (.xlsx) file-size limit—200MBSheetJS full-loads the workbook (~588MB OOM ceiling measured); convert larger to CSV

Results vary by hardware, browser, number of pattern types active, validation mode, and data density. The 10.8GB and 9.3GB rows are the TRUE-10GB gate (June 2026, forced-GC heap sampling). CSV is constant-memory; Excel is capped because SheetJS loads the whole workbook.

SplitForge vs. Python: Which Tool Fits Your Workflow?

No single tool is right for every situation. Here's an honest breakdown.

Use SplitForge when:
  • Data contains PII, PHI, or regulated financial information that should not leave your device
  • You need Luhn, SSA, or NANP validation without writing or maintaining custom code
  • You want normalization (E.164, ISO 8601) in the same step as extraction
  • You need context view for audit trails or compliance review documentation
  • CSV files of any practical size (constant memory) or Excel under 200MB
  • You don't write Python — or don't want to for this specific task
  • Throughput of ~187K matches/sec is sufficient for your dataset
Use Python re / pandas when:
  • You need to run extraction on a schedule or inside an automated data pipeline
  • You need custom regex patterns beyond the 8 built-in types
  • You need scheduled/automated runs, or named-entity recognition beyond the 8 structured patterns
  • Extraction is part of a larger ETL or transformation workflow
  • You're comfortable writing and maintaining Python code
  • You need output piped directly into another process or database

Feature Performance Overhead

High match-density (non-dedupe, 10GB)
Baseline
~187K matches/sec
Every match streams straight to an on-disk OPFS file. 10.8GB input → 141,891,891 matches → 14.8GB output, at a flat ~18MB live-heap floor (122MB transient peak). No result cap — all matches are retained.
Dedupe (external sort-merge, 10GB)
distinct at any scale
external sort-merge
Deduplication uses an OPFS external sort-merge, not an in-memory set — so distinct extraction is exact at any scale with constant memory. Measured: 9.3GB → 47.3M matches → 20,000,000 distinct (exact), peak heap 326MB.
Strict validation
more validation/row
slower than balanced
Full validation stack: Luhn for every CC candidate, SSA area/group/serial rules for SSNs, curated TLD check for URLs, NANP area-code rules. Best precision — more work per row than balanced.
Normalization (phone / email / URL / date)
O(n) per match
negligible
Post-extraction format standardization on already-matched values: phone → E.164 / US / digits, email → lowercase or domain-only, URL → domain-only or add-protocol, date → ISO 8601 or US. O(n) string ops — negligible impact.
Constant memory (any scale)
flat
~18MB live floor
Output streams to disk (OPFS), so the worker heap does not grow with input or output size — an ~18MB forced-GC live floor held flat across the entire 10.8GB run (16 GC samples). The old 1M-match result cap was removed.
No result cap
—
all matches retained
There is no result cap — 141,891,891 matches were retained on the 10.8GB run with no truncation. (The previous 1-million-match cap was removed when output moved to disk-backed streaming.)
Figures from the TRUE-10GB gate, June 2026, Chrome, 64GB RAM, dev machine (Alder Lake class), default heap limit, forced-GC heap sampling. Results vary by hardware, browser, and data density.

Input Format Performance

CSV (Streaming)
Recommended
~187K matches/s
PapaParse chunk-based streaming — the file is never fully loaded into memory; it is processed in ~10MB chunks and every match streams straight to an on-disk OPFS file, so memory stays flat (an ~18MB live floor) at any file size. Fastest option for large files. Auto-detects comma, tab, semicolon, and pipe delimiters.
Excel / XLSX
Handles multi-sheet
load-bound (≤200MB)
SheetJS loads the ENTIRE workbook into memory (this is not streaming — there is no streaming xlsx reader). Measured heap is ≈5.5–7.3× file size, so .xlsx is capped at 200MB (~588MB OOM ceiling). Handles multi-sheet workbooks with a sheet selector. Convert larger Excel to CSV first — CSV streams to disk with no practical size limit.
Text Paste
Up to 10MB
~2–4 sec flat
Direct text or structured content pasted into the tool. No file reading overhead — processes immediately. Best for small ad-hoc extractions from clipboard content, emails, documents, or API responses. Memory-bound above ~50KB; upload a file instead for larger inputs.

Calculate Your Time Savings

Manual baseline: ~20 minutes per extraction task via Python scripting — write regex per pattern type, handle encoding edge cases, test against sample data, debug false positives, run on the full file, validate output counts. SplitForge processes any combination of the 8 pattern types in a single streaming pass with validated results and normalization options — seconds for typical files, ~187K matches/second at 10GB scale (measured June 2026).

Typical: 2–4 types per task

Monthly = 12, Weekly = 52, Daily = 260

Analyst avg: $45–75/hr

Annual Time Saved
51.4
hours per year
Annual Labor Savings
$2,568
per year (vs. Python scripting per task)
What you eliminate:
  • Writing and maintaining regex for each pattern type
  • Debugging false positives from regex-only matching
  • Writing Luhn algorithm and SSA validation logic from scratch
  • A separate normalization pass after extraction
  • The compliance risk of using online tools for PII data

Testing Methodology

10 runs per config · drop high/low · report avg · test datasets available on request · re-tested quarterly

Expand

Honest Limitations: Where SplitForge Pattern Extraction Falls Short

No tool is perfect for every use case. Here's where Server-Side Entity Extraction (Python re / AWS Comprehend / spaCy) might be a better choice, and the real limitations of our browser-based architecture.

Browser-Based Processing

Performance depends on your device's RAM and CPU. Modern laptops (2022+) handle 10M+ rows easily, but older devices may struggle with very large files.

Workaround:
Close unnecessary browser tabs to free up memory. For files over 50M rows, consider database solutions.

No Offline Mode (Initial Load)

Requires internet connection to load the tool initially. Processing happens offline in your browser after loading.

Workaround:
Once loaded, you can disconnect and continue processing. For true offline environments, desktop tools may be better.

Browser Tab Memory Limits

Most browsers limit individual tabs to 2-4GB RAM. This is the practical ceiling for file size.

Workaround:
Use 64-bit browsers with sufficient RAM. Chrome and Firefox handle large files best.

CSV: no practical size limit · Excel: 200MB

CSV streams to disk (constant memory) and has been measured to 10.8GB / 141.9M matches with a flat ~18MB heap floor; the 64GB cap is just a sanity guard. Excel is full-loaded by SheetJS and capped at 200MB.

Workaround:
For Excel over 200MB, convert to CSV first (CSV has no practical size limit). For 50M+ row scheduled pipelines, Python re + pandas or AWS Comprehend remain the automation options.

No Custom Regex Patterns

Pattern Extraction supports only the 8 built-in types. User-defined regex patterns for custom schemas (account IDs, internal product codes, proprietary identifiers) are not currently supported.

Workaround:
For custom patterns, use Python re.findall() with your regex, or grep -oP for shell-based extraction. Custom regex support is a planned roadmap item.

No Automation or API

Pattern Extraction is a browser tool — no REST API, CLI, or pipeline integration. Cannot be embedded in ETL workflows, scheduled jobs, or triggered programmatically.

Workaround:
For scheduled or automated extraction, use Python re + pandas in a cron job, or AWS Comprehend for entity recognition at pipeline scale with full orchestration.

Excel Loads Fully Into Memory (200MB cap)

There is no result cap (the old 1M-match limit was removed — output streams to disk). The one memory-bound input is Excel: SheetJS loads the entire .xlsx workbook into memory, so it is capped at 200MB (~588MB measured OOM ceiling).

Workaround:
Convert Excel files over 200MB to CSV first — CSV streams to disk and has no practical size limit.

Single File Per Session

Pattern Extraction scans one file at a time. No batch processing across multiple files in a single operation.

Workaround:
Process files sequentially. For high-volume batch scanning (50+ files), use Python re in a loop or a shell script with grep -oP across all files.

When to Use Server-Side Entity Extraction (Python re / AWS Comprehend / spaCy) Instead

You need pattern extraction in an automated pipeline or scheduled job

SplitForge has no API. Browser-only workflow cannot run on a schedule or be triggered programmatically.

💡 Python re + pandas in a cron job, or AWS Comprehend for entity recognition in a Lambda function.

You need custom regex patterns for proprietary data schemas

SplitForge supports only the 8 built-in types. Custom schemas require user-defined regex.

💡 Python re.findall() with your pattern, or grep -oP for shell-based extraction.

You need to scan 50M+ row files

For scheduled/automated runs there is no API (browser-only). CSV scale itself is not the limit — it is constant-memory to 10GB+.

💡 Python re + pandas with chunking (chunksize parameter), or AWS Comprehend for large-scale entity extraction.

You need named entity recognition beyond structured patterns

SplitForge extracts structured patterns (email, phone, etc.). Extracting person names, organization names, or location references requires NER models.

💡 spaCy or AWS Comprehend for NER-based extraction. Combine with SplitForge for structured pattern extraction as a pre-processing step.

Questions about limitations? Check our FAQ section below or contact us via the feedback button.

Frequently Asked Questions

How accurate is the ~187K matches/second benchmark?

Why does throughput vary so much by validation mode?

What is the difference between CSV streaming mode and text paste mode?

How does validation mode affect result quality vs. speed?

What is the deduplication overhead?

Can I reproduce these benchmarks?

What file formats are supported and does format affect speed?

What is the largest file I can process?

Benchmarks last updated: June 2026. Re-tested after major algorithm changes. Last update: P2 — output moved to disk-backed (OPFS) streaming (constant memory, result cap removed) and the phone matcher tightened to NANP-strict.

Ready to Extract 142M Matches from 10GB — at 18MB of Memory?

No installation. File contents never uploaded. 8 validated pattern types, post-extraction normalization, and a results preview — all in your browser.