Before your CSV or Excel file reaches ChatGPT, Claude, or an AI API, it passes through two independent checks — an upload-size test and a token-count test — and six recurring structural problems cause silent failures that look like correct answers. This guide covers the complete local-first preparation workflow: platform limits, encoding, PII masking, format selection, size reduction, and the 10-item pre-flight checklist that runs before any file leaves your device.
TL;DR: The two most important facts before any AI upload: (1) there are two separate limits — a file-size cap before the AI reads your data and a token-count cap during processing — and a file can pass one while failing the other silently; (2) consumer AI plans train on uploaded data by default, so PII should be masked before upload, not after. Every preparation step in this guide runs in your browser — the raw file stays on your device.
Clean and prepare your data locally →
An analyst uploads a 40,000-row customer export to ChatGPT and asks for a monthly revenue breakdown by region. ChatGPT returns a confident table with totals that almost add up — but when the analyst cross-checks against a known total, the numbers are off by 18%. The CSV encoding was Windows-1252, not UTF-8: currency symbols in the Revenue column corrupted during tokenization. The date column mixed two formats. Eleven percent of rows had null Region values that ChatGPT filled with a plausible-sounding guess. None of these problems generated an error; they generated a confident-looking wrong answer. Every problem was detectable — and fixable — before the upload.
Methodology: ChatGPT, Claude, and Gemini upload limits verified against official OpenAI, Anthropic, and Google documentation, May 2026. Data Masking streaming validated to 10GB (2026-05-19). Workflow validated across all 10 AI-prep cluster spokes — see individual posts for per-topic verification details.
Table of Contents
- Quick Start: Where Do I Begin?
- Why AI Rejects, Truncates, or Misreads Your File
- The Master Limits Reference
- The Local-Prep Workflow
- Per-Platform: ChatGPT, Claude, and Gemini
- Privacy-First Prep: What Stays Local vs What Doesn't
- FAQ
Quick Start: Where Do I Begin?
If you landed here with a specific problem, use the table below to jump directly to the relevant step. If you are preparing a file for the first time, start at The Local-Prep Workflow and follow the steps in order — each catches a different failure class. For a one-page reference to upload caps and context windows, see The Master Limits Reference.
| Your situation | Recommended action | Jump to |
|---|---|---|
| File is too large to upload to ChatGPT or Claude | Export .xlsx to CSV first; if still over the cap, split by rows | Upload Rejection |
| AI results seem incomplete or the row count is lower than expected | Ask "how many rows are in this file?" immediately after upload and compare to your known total | Silent Truncation |
| File contains customer names, email addresses, or account numbers | Pseudonymize PII-bearing columns locally before upload using Data Masking | Mask PII Before Any Upload |
| Currency symbols or accented characters are garbled in AI output | Re-save the file as UTF-8; verify encoding with Format & Encoding Checker | Fix Encoding and Headers |
| Building a fine-tuning dataset or batch API input | Export as JSONL and reshape flat rows to the messages-array format OpenAI fine-tuning requires | Reduce Size and Convert Format |
| Not sure if the file is ready — want a pre-upload checklist | Run the 10-item pre-flight check before any upload leaves the device | 10-Item Pre-Flight Check |
| Choosing between ChatGPT, Claude, or Gemini for this dataset | Compare upload caps, context windows, and training defaults in the limits matrix | The Master Limits Reference |
Why AI Rejects, Truncates, or Misreads Your File
AI tools fail on bad input in four distinct ways — and only one of them produces an obvious error. The other three return confident-looking output with wrong answers, dropped rows, or hallucinated values and no visible indication anything went wrong.
Upload Rejection: The Byte Test Before AI Reads Your Data
The upload-size cap is a byte test the platform runs before the AI model sees any data — the only failure mode that produces a clear error. ChatGPT's Data Analysis tool caps CSV and spreadsheet uploads at ~50MB. The common cause is file format, not data volume: .xlsx files carry formatting metadata that can triple the size of equivalent CSV data. Exporting to CSV before upload strips that overhead; see Excel File Too Big for AI?.
Silent Truncation: When Success Looks Like Failure
The context-window limit is a token-count test that runs during processing, after the upload succeeds. A file under the 50MB upload cap can still overflow GPT-4o's 128,000-token context window — and it will if the data is wide or value-dense. The result is silent truncation: ChatGPT processes only the rows that fit and responds as if it analyzed the full file, with no error or warning. After any large-file upload, ask "how many rows are in this file?" and compare to your known total before running analysis. See How Many Rows Can ChatGPT Handle?.
Parsing Failures: Encoding, Dates, and Format Mismatch
Three structural problems corrupt data silently during tokenization:
Encoding corruption: Files in Windows-1252 or Latin-1 silently corrupt characters outside basic ASCII — accented letters, currency symbols, non-Latin scripts — during tokenization. A €1,250.00 Revenue value may arrive as garbled text and be treated as a string field.
Date ambiguity: Columns mixing MM/DD/YYYY and DD/MM/YYYY formats force the model to assume one throughout; a fraction of rows are then silently misinterpreted. Standardize to ISO 8601 (YYYY-MM-DD) before upload.
Format mismatch: Sending CSV to a JSONL-only endpoint (OpenAI fine-tuning API) or the wrong JSON shape to an API produces rejection or silent downstream errors. Check the API reference for the required format before submission.
PII Exposure: The Invisible Failure
PII exposure is a policy failure at the moment of upload. Consumer AI platforms — ChatGPT Free and Plus, Claude Free, Pro, and Max on claude.ai, Gemini consumer plans — train on uploaded files by default. A file uploaded without masking means those records potentially enter training datasets with no recovery path. Mask PII-bearing columns locally before upload; commercial API tiers (OpenAI API, Anthropic API, Google Cloud AI API) are exempt from training by default. See How to Remove PII From a CSV Before Using AI.
The Master Limits Reference
Every "too big for AI" error traces back to one of two limits: the upload-size cap (bytes) or the context-window limit (tokens). They are independent constraints — a file can pass one while failing the other — and they apply at different stages of processing. The training-default behavior is a third constraint that operates at the policy level, before any technical limit is reached. The table below consolidates all three across the three major AI platforms.
| ChatGPT (Plus / Free) | Claude (claude.ai) | Gemini (consumer) | |
|---|---|---|---|
| Upload cap | ~50MB for spreadsheets/CSV; 512MB hard cap (all file types) | 500MB per file; up to 20 files per chat | Not Tier-1 verifiable [1] — see ChatGPT vs Claude vs Gemini |
| Context limit | 128,000 tokens (GPT-4o); ~850–2,500 rows depending on data width | 200,000 tokens (Claude Sonnet/Opus, per Anthropic model documentation) | Not Tier-1 verifiable [1] — see Gemini's model documentation |
| Consumer training default | On by default; opt out in Settings → Data Controls | On by default; opt out in Account Settings | On by default; opt out in Google Account → Data & Privacy |
| Format accepts natively | CSV, XLSX (Data Analysis tool); PDF, plain text | CSV (drag-and-drop); XLSX requires code execution and file creation to be enabled | CSV; XLSX availability varies by Workspace plan |
| API / commercial tier | OpenAI API: exempt from training by default; data not used to train base models | Anthropic API: exempt from training by default; commercial terms apply | Google Cloud AI API: exempt from training by default |
[1] Gemini's upload cap and context-window limits vary by Workspace plan and are not published at the granularity required for a verified cross-platform comparison. Values for the ChatGPT and Claude rows are sourced from official API documentation; the Gemini rows reflect the best available public information as of May 2026.
On the Gemini rows: The upload cap and context limit cells are marked unverified because Gemini's file-upload behavior varies by Workspace plan and is not consistently published at the granularity required for a Tier-1 citation. The training default and API tier behavior are established cluster facts. For the full Gemini platform comparison, see ChatGPT vs Claude vs Gemini: File Upload Limits Compared.
On the Claude context limit: The 200,000-token figure is from Anthropic's published model documentation for Claude 3 Sonnet and Opus. The practical implication differs from ChatGPT's 128K limit: Claude can hold approximately 1,300–4,000 rows of typical structured CSV data in a single turn without truncation, compared to GPT-4o's 850–2,500 rows. For files too large even for Claude's 200K window, the aggregate-first workflow — collapsing rows to a summary before upload — resolves the limit regardless of platform. See Summarize a Huge CSV Before Sending It to AI.
On the consumer training default: All three consumer plans train on uploaded data by default. This is not a bug or an oversight — it is an explicit product feature in each platform's terms of service. The mitigation is local masking before upload (remove or pseudonymize PII-bearing columns), not post-upload deletion (which does not reliably prevent training inclusion). Commercial API tiers across all three platforms are explicitly exempt from training-by-default under their commercial terms.
On the upload cap vs context-window relationship: These limits fail independently. A 45MB spreadsheet passes ChatGPT's upload cap but may tokenize to 400,000 tokens — more than 3× the 128K context window — resulting in silent truncation with no error message. A 100MB spreadsheet fails the upload cap and never reaches the context-window check. Always verify both before uploading: check the file size against the upload cap, then estimate the token count against the context-window limit using the token estimate table.
The Local-Prep Workflow
The preparation workflow runs six steps in order — profile, encode, clean, mask, reduce, and verify — and every step runs locally before the file reaches any remote system. The output of each step is a file closer to what the AI platform or API actually needs. The final step embeds the 10-item pre-flight check that confirms readiness before anything leaves your device.
Step 1: Profile the File Before Touching Anything
Profiling answers three questions before any cleaning begins: how many rows are there, what is the null rate by column, and what encoding is the file saved in? None of these can be determined from the file name or size — they require reading the data. A file with 12% null values in a key column will produce different AI errors than a file with a Windows-1252 encoding problem; knowing which problem exists determines which fix to apply first.
Run a column-level null-rate scan on any file before cleaning it. For encoding, open the file in a text editor that reports encoding (VS Code, Notepad++) — or run it through the Format & Encoding Checker — and confirm UTF-8 before any other step. Encoding corruption that enters the pipeline early propagates through every downstream transformation.
Step 2: Fix Encoding and Headers
AI tokenizers are built around Unicode. Files saved in Windows-1252, Latin-1, or any non-UTF-8 encoding corrupt characters outside the basic ASCII range — accented letters, currency symbols, non-Latin scripts — silently during tokenization. Re-save the file as UTF-8 without BOM before any other transformation; most spreadsheet and database export dialogs expose this option explicitly.
Header rows are the AI's schema for the dataset. Clean headers are single-word or underscore-joined with no special characters: revenue_usd rather than Revenue ($). Trailing empty columns and rows from Excel's auto-extend range add token overhead without data value; strip them before upload.
The Format & Encoding Checker scans for encoding, BOM presence, mixed delimiters, and structural anomalies in a single local pass. For header normalization and trailing-row removal, the Data Cleaner handles both without uploading the source file.
Step 3: Handle Missing Values Explicitly
Missing values produce three silent failure modes: the model skips the row, fills with a plausible-sounding value, or treats the empty cell as zero in numeric columns. Make the treatment explicit before upload: fill with N/A for "not applicable," UNKNOWN for genuinely missing data, or drop the row if the column is critical to the analysis.
Date columns mixing 2024-01-15 and 01/15/2024 formats cannot be resolved by filling. Standardize to ISO 8601 (YYYY-MM-DD) throughout; see Fix CSV Date Format Errors.
Step 4: Mask PII Before Any Upload
Consumer AI platforms train on uploaded data by default — ChatGPT Free and Plus, Claude Free, Pro, and Max on claude.ai, and Gemini consumer plans all include uploaded files in training data under their published terms of service. Masking is not optional if the file contains customer names, email addresses, account numbers, government IDs, health identifiers, or any field that identifies a natural person. The masking step must run before upload — there is no recovery path once a file enters a platform's training pipeline.
Pseudonymization replaces names and IDs with consistent placeholder tokens, preserving the relational structure the AI needs for analysis (the same customer's transactions still group under the same token) while eliminating the identifiable data. For the full masking workflow and field-by-field classification, see How to Remove PII From a CSV Before Using AI.
Step 5: Reduce Size and Convert Format
After cleaning, check two thresholds: the upload-size cap for the target platform and the estimated token count against the context-window limit. These are independent checks — passing one does not guarantee passing the other. A 45MB cleaned CSV passes ChatGPT's ~50MB upload cap but may tokenize to 300,000+ tokens, well over the 128K GPT-4o limit, resulting in silent truncation with no error.
For files that exceed the context-window limit after passing the upload cap, the aggregate-first approach resolves the issue without losing analytical value for trend and distribution questions: collapse rows to a summary — monthly totals, regional breakdowns, cohort counts — and send the summary rather than the raw rows. See Summarize a Huge CSV Before Sending It to AI for the decision table that maps question type to preparation path. For record-lookup and anomaly-detection questions, split the file and query each chunk separately — see Split a CSV for AI Analysis.
Step 6: Run the 10-Item Pre-Flight Check
Before any file leaves your device, run through the following checklist. Each item addresses a recoverable failure class; items that fail should trigger the linked prep step before upload.
- UTF-8 encoding, no BOM — Re-save from the source application or run the Format & Encoding Checker. Windows-1252 and Latin-1 are the most common sources of silent currency and accent corruption.
- Clean, descriptive headers — Single-word or underscore-joined, no special characters, no trailing empty columns. Ambiguous headers ("Column1", "Field_A") produce ambiguous AI outputs.
- No empty critical cells — Profile null rates by column. Fill missing values explicitly (
N/A,UNKNOWN, or drop) before upload; do not leave the AI to infer intent. - PII masked — Customer names, emails, IDs, health identifiers, and any field identifying a natural person removed or pseudonymized. Required for any consumer-plan upload; mandatory regardless of platform for GDPR-, HIPAA-, and PCI-scoped datasets.
- Consistent date format — ISO 8601 (
YYYY-MM-DD) throughout. Mixed date formats are the second most common source of silent misinterpretation after encoding errors. - Format matches use case — CSV for direct prompt and RAG ingestion; JSON array for structured API context blocks; JSONL for fine-tuning and batch API calls. See What Format Should I Send to AI? and Excel to JSON for AI.
- File size within the upload cap — Verify against the target platform's byte limit before upload. Convert from .xlsx to .csv to eliminate formatting overhead if needed; see the limits matrix above.
- Token count within the context window — Estimate using the row-count-to-token table in How Many Rows Can ChatGPT Handle?. Aggregate or split if the estimate exceeds the platform limit.
- 10-row sample test passed — Upload a 10-row sample and ask the AI to describe the schema and row count. Verify the description against known facts before uploading the full file.
- Prep history noted — Record the encoding fix applied, masking method used, and any rows dropped. If the AI's analysis is questioned later, this log is the audit trail.
For the full checklist with extended per-item rationale, cross-links, and the "when you can skip steps" guidance, see AI-Ready Data Checklist.
Per-Platform: ChatGPT, Claude, and Gemini
The three major consumer AI platforms accept similar file types but differ in format-handling behavior, context-window size, and the degree to which account settings affect what you can send in a single turn. This section covers the platform-specific preparation decisions not captured in the cross-platform checklist above. For the full limits comparison, see the matrix in The Master Limits Reference.
ChatGPT (Data Analysis Tool)
ChatGPT's Data Analysis tool accepts CSV and XLSX uploads via the paperclip icon in Plus and Team plans. The upload cap is approximately 50MB for spreadsheet and CSV files; the hard cap across all file types is 512MB. GPT-4o's context window is 128,000 tokens. A structured CSV row consumes approximately 50–150 tokens depending on column count and value length — roughly 850–2,500 rows before the context window fills, with the wide range reflecting dataset-specific variation in value length and column count.
The most important ChatGPT-specific preparation decision is format: always send CSV rather than XLSX unless the workbook contains multiple sheets that must be analyzed together. XLSX files carry formatting metadata, shared-string tables, and revision history that increase byte size without adding analytical value. A workbook that is 35MB as .xlsx may be 6MB as a CSV of identical data rows. The size reduction also reduces tokenization overhead; the structured-data tokenizer handles raw CSV values more efficiently than the rich-format XML that underlies an .xlsx file. See Excel File Too Big for AI? for the conversion workflow.
Silent truncation is the primary risk at scale. After any upload larger than 10,000 rows, verify the row count by asking ChatGPT "how many rows are in this file?" and comparing the answer to the known total. A discrepancy indicates the context window was exhausted before the file was fully read.
Claude (claude.ai and Anthropic API)
Claude's upload limit on claude.ai is 500MB per file and up to 20 files per conversation. Claude Sonnet and Opus operate with a 200,000-token context window — larger than GPT-4o's 128K limit — which translates to approximately 1,300–4,000 rows of typical structured CSV data before truncation. For datasets within this range, Claude can hold more context in a single turn than ChatGPT, making it possible to send larger cleaned files without splitting.
The critical Claude-specific format consideration involves XLSX files. Claude on claude.ai reads CSV files via drag-and-drop natively. XLSX reading requires the "code execution" and "file creation" capabilities to be enabled in the conversation — a toggle in conversation settings that is not active by default across all plan types. If an XLSX upload appears to succeed but the response describes the file in generic terms or fails to reference specific column names, the code execution toggle is likely off. The fix is to enable it in settings or to convert the file to CSV before upload.
On claude.ai Free, Pro, and Max plans, uploaded files are included in Claude's training data by default. To opt out, navigate to Account Settings and disable the training-data option. This is separate from the Anthropic API context: data submitted via the Anthropic API is not used to train base models under Anthropic's commercial terms. Consumer claude.ai plans and the commercial API are two distinct data environments with different privacy defaults — the same file uploaded through each has a different data destiny.
Gemini (Consumer and Workspace)
Gemini's file upload behavior varies by Workspace plan and is not consistently published at the granularity required for a verified comparison with ChatGPT and Claude. CSV upload is available across consumer Gemini plans; XLSX availability varies by Workspace tier. For a current comparison of Gemini's upload limits relative to ChatGPT and Claude, see ChatGPT vs Claude vs Gemini: File Upload Limits Compared.
Two preparation decisions apply regardless of Gemini plan. First, the training default: Gemini consumer plans train on uploaded data and conversations by default. To opt out, navigate to Google Account → Data & Privacy and disable the relevant setting before uploading any file containing business or personal data. Second, size reduction: converting XLSX to CSV before upload removes format overhead and reduces the risk of hitting plan-specific file caps — a safe default across all Gemini upload workflows regardless of the specific cap in effect.
The Google Cloud AI API operates under Google Cloud's data processing terms and is not subject to consumer training defaults. Data submitted via the Google Cloud AI API is not used to train base models under its commercial terms — the same exemption structure that applies to the OpenAI API and the Anthropic API.
Privacy-First Prep: What Stays Local vs What Doesn't
Every step in this guide runs in your browser. The file you prepare — whether you are profiling null rates, splitting large CSVs, pseudonymizing customer columns, or converting formats — stays on your device through the entire preparation sequence; the only network event is the final AI prompt itself. This distinction is not a product claim: it is the direct consequence of offloading every computation to a local Web Worker rather than routing the source file through a preparation service first.
What Stays Local
Six preparation operations run entirely in-browser with no file upload:
Profiling reads the source file in a background thread to calculate null rates by column and detect encoding. No data leaves the device during this step.
Splitting breaks a large file into row-count chunks (see Split a CSV for AI Analysis); the worker processes the source file and writes output chunks directly to the browser's download queue without transmitting the source rows.
Aggregating collapses rows to summary format (see Summarize a Huge CSV Before Sending It to AI); the source rows are read once in the worker and the aggregate output is the only file that leaves the device.
Cleaning — header normalization, null-fill, date standardization — runs against the local file, modifying a copy in memory without transmitting the source.
Masking pseudonymizes PII-bearing columns using Data Masking (see How to Remove PII From a CSV Before Using AI); the worker generates consistent token-to-value mappings in memory with no file upload, and the masked output is the only artifact that leaves the device. Validated to 10GB.
Converting transforms CSV to JSON array or JSONL format (see Excel to JSON for AI); conversion runs in the worker layer and outputs the converted file locally.
For HR and payroll datasets — workforce exports containing salary, benefits, and performance fields — the full local-first preparation sequence applies. For field-by-field preparation guidance specific to HR data, see the upcoming D14 (Prepare HR and Payroll Data for AI) — coming as part of the Week 3 AI-prep buildout.
What About the AI Itself?
The AI vendor is the final exposure surface regardless of how the file was prepared. A perfectly masked, locally prepped file still reaches an AI model's inference infrastructure at upload or API call time; local preparation does not eliminate that exposure. What it eliminates is every other exposure layer before it, reducing the preparation pipeline from a multi-surface audit to a single decision point.
The user's remaining decision is which AI tier to use and whether to opt out of consumer training defaults. Masking before upload addresses the consumer training default directly: a pseudonymized file that enters a consumer AI platform's training pipeline contains token labels, not identifiable values. For broader privacy-first analytics workflow guidance, see Privacy-First Data Processing Guide.
When Local Prep Is Overkill
Preparation rigor scales with data sensitivity. For a public dataset — government statistics, open research data, anonymized benchmark files with no PII — the upload-first workflow raises no meaningful privacy risk; there is nothing to expose. For small demo files built from synthetic or fictional records, the same applies.
The checklist in Step 6 above identifies which items are mandatory and which are context-dependent. Match the preparation depth to what is actually in the file: a 500-row synthetic product catalog used to test a prompt does not need a masking pass; a 200,000-row customer export with email addresses and account numbers does.
Operational Guidance, Not Legal Advice
The preparation workflow in this guide — local processing, pseudonymization, consumer training opt-out, format conversion before upload — supports the data-minimization principle in GDPR Article 5, addresses HIPAA's Safe Harbor 18-identifier list for de-identification, and provides operational support for PCI scope reduction by removing cardholder data fields before they reach any AI platform. These are observations about what the workflow does operationally, not certifications of regulatory compliance. Organizations operating under GDPR, HIPAA, PCI DSS, or any equivalent regulatory framework should consult their data protection officer or legal counsel to determine whether any preparation workflow meets their specific compliance obligations.
FAQ
Additional Resources
Methodology: ChatGPT, Claude, and Gemini upload limits verified against official OpenAI, Anthropic, and Google documentation, May 2026. Data Masking streaming validated to 10GB on 2026-05-19. AI-prep workflow validated across all 10 cluster spokes (A1, A2, A3, B4, B5, B6, C8, C9, D12, C10), May 2026.
Related Reading
AI-prep cluster spokes:
- Excel File Too Big for AI? How to Fix It
- ChatGPT vs Claude vs Gemini: File Upload Limits Compared
- How Many Rows Can ChatGPT Handle?
- Split a CSV for AI Analysis
- Spreadsheet Token Limit Explained
- Summarize a Huge CSV Before Sending It to AI
- What Format Should I Send to AI?
- Excel to JSON for AI APIs and LLM Pipelines
- How to Remove PII From a CSV Before Using AI
- AI-Ready Data Checklist
Cross-pillar:
Disambiguation:
External References
Official Standards:
- RFC 4180 — Common Format and MIME Type for Comma-Separated Values (CSV)
- RFC 8259 — The JavaScript Object Notation (JSON) Data Interchange Format
- JSON Lines Specification
- The Unicode Standard
Vendor Documentation:
Browser / Technical: