Benchmark Performance
Performance at Scale
Chrome (stable) · Windows 11 · dev machine (Alder Lake class) · 64GB RAM · June 2026
| Dataset Size | Format-only | Full Schema (5 rules) | Uniqueness (1 col) | Test Notes |
|---|---|---|---|---|
| 1K rows | <1s | <1s | <1s | Setup-bound; completes instantly |
| 10K rows | <1s | <1s | <1s | Setup-bound; completes instantly |
| 100K rows | <1s | <1s | <1s | Typical CRM export batch — completes in well under a second |
| 1M rows | ~982K rows/sec | ~534K rows/sec | ~730K rows/sec | Salesforce Contacts import scale |
| 5M rows | ~1.04M rows/sec | ~552K rows/sec | ~710K rows/sec | Per-row rate plateaus at scale |
| 10M rows | ~1.06M rows/sec | ~567K rows/sec | ~685K rows/sec | Measured June 2026 |
| 10 GB (228M rows) | ~270K rows/sec | — | ~265K rows/sec | TRUE-10GB gate; merge-sort multipass, constant memory |
Results vary by hardware, browser, rule count, and file complexity. Format-only = one email-format check. Full schema = 5 stateless rules. Uniqueness = single-column collision-free merge-sort. Rows under 100K complete in well under a second (setup-bound, not a meaningful rows/sec). 10 GB row is the TRUE-10GB gate (format + uniqueness; full-schema not separately run at 10 GB).
Feature Performance Overhead
When Data Validator Is Slower Than Expected
Transparency on conditions that reduce throughput below published benchmarks
Uniqueness streams to the Origin Private File System (OPFS). It is required — there is no in-memory fallback. In a context without OPFS (some private-browsing modes, older browsers), large-scale uniqueness validation cannot run. Memory itself is constant, so RAM is never the constraint.
RegExp.test() cost scales with pattern complexity. Simple patterns like /^\d{10}$/ are fast. Complex lookaheads, alternation, or backtracking patterns can be 5–10x slower per row than basic data type checks.
Each uniqueness column is sorted and scanned in its own sequential pass — constant memory, not N simultaneous structures. Three uniqueness columns at 10 GB ran ~110K rows/sec overall.
Excel's binary XLSX format requires an additional parsing step via SheetJS before validation can begin. A 10M row Excel file adds 6–8 seconds of XLSX parsing on top of validation time. CSV files with UTF-8 encoding and comma delimiters give the best performance.
The NPI Luhn checksum is the most expensive per-row check — ~263K rows/sec vs ~810K for format-only at 10M. ICD-10/CPT/taxonomy are cheap pattern checks.
Uniqueness is ~685K rows/sec at 10M and settles to ~265K rows/sec at 10 GB / 228M as the external merge-sort runs more passes. This is expected; memory stays constant. A 10 GB uniqueness validation takes ~14 min.
Calculate Your Time Savings
Files that needed 2+ import attempts
Active months doing CRM imports
Data analyst avg: $50–75/hr
- 15–25 minute upload wait times per failed attempt
- Hunting for validation errors one at a time (Salesforce only reports one error per upload)
- Manual Excel cleanup with no regex support and a 256-rule limit
- Re-uploading the same file 3–5 times before it imports cleanly
- Discovering errors for the first time in production data
SplitForge Browser Validation Standard (SVBP-2026)
June 2026 refresh: 3-run median, session warmup discarded, default browser heap limit · test datasets available on request · v1.2
Validation Engine Changelog
- OPFS external merge-sort uniqueness — constant memory at any scale, exact (no 10M sampling cap), collision-free; validated at 10 GB / 228M rows
- Errors, duplicates, and failed-row export stream to OPFS (heap stays flat)
- Uniqueness on multiple columns, each checked in its own sequential pass
- Short-circuit at 100 blocking errors for corrupt file performance
- SVBP-2026 benchmark protocol established
- Compound-key uniqueness (uniqueness across a combined multi-column key — distinct from the shipped per-column uniqueness)
- Regex precompilation for repeated pattern checks
- Code-set membership validation (ICD-10-CM / CPT) — currently format checks only
- Saved validation schemas (cross-session)
Honest Limitations: Where SplitForge Data Validator Falls Short
No tool is perfect for every use case. Here's where Server-Side Validation Tools (Great Expectations / dbt tests / AWS Glue) might be a better choice, and the real limitations of our browser-based architecture.
Browser-Based Processing
Performance depends on your device's RAM and CPU. Modern laptops (2022+) handle 10M+ rows easily, but older devices may struggle with very large files.
No Offline Mode (Initial Load)
Requires internet connection to load the tool initially. Processing happens offline in your browser after loading.
Browser Tab Memory Limits
Most browsers limit individual tabs to 2-4GB RAM. This is the practical ceiling for file size.
Very Large Files Are Time- and Disk-Bound, Not RAM-Bound
Memory is constant (OPFS external merge-sort) — validated to 10 GB / 228M rows. The practical limit is time and free disk (a large uniqueness run uses temporary on-disk space; allow several GB of headroom for multi-gigabyte inputs), not RAM. For 50M+ rows on a schedule, a server-side pipeline is the better automation fit.
Short-Circuit After 100 Blocking Errors
Validation stops after finding 100 blocking errors to prevent overwhelming the UI on severely corrupt files. If your file has thousands of blocking errors, you'll need multiple validation passes.
No API or Automation Support
Data Validator is a browser tool — no REST API, CLI, or pipeline integration. Cannot be embedded in ETL workflows, CI/CD pipelines, or scheduled quality checks.
Single File Per Session
Data Validator processes one file at a time. No batch validation across multiple files in a single operation.
When to Use Server-Side Validation Tools (Great Expectations / dbt tests / AWS Glue) Instead
You need to validate data in an automated CI/CD or ETL pipeline
Data Validator has no API. Browser-only workflow cannot run on a schedule or be triggered programmatically.
You need to validate 50M+ row files regularly
Browser memory limits practical ceiling to ~10–15M rows depending on hardware. Server-side tools scale horizontally.
You need team-shared validation schemas with version control
Data Validator schemas exist only in browser sessions — no sharing, no versioning, no team collaboration features.
You need statistical anomaly detection (outlier detection, distribution checks)
Data Validator handles rule-based validation only — required, format, range, regex, enum, uniqueness. No statistical profiling.
Questions about limitations? Check our FAQ section below or contact us via the feedback button.
Frequently Asked Questions
How fast is validation, and at what scale?
What's the difference between Simple Validation and Full Schema mode?
Why is uniqueness checking slower than other validation rules?
How does Excel Data Validation compare at scale?
What browser and hardware give the best validation performance?
Does CSV vs Excel format affect validation speed?
What happens when validation hits the 100 blocking-error short-circuit?
Can I reproduce these benchmarks?
Benchmarks last updated: June 2026 · Re-tested quarterly and after major algorithm changes · Validation Engine v1.2 · SVBP-2026 Protocol