Methodology

Every score and warning should be explainable.

Reference

How the workbench reaches its results

The same input and settings produce the same findings. The methods favor understandable evidence over opaque predictions, and every automated conclusion has a known boundary.

Delimiter detection

The detector tests comma, tab, semicolon, and pipe candidates against up to 120,000 characters and 40 logical rows. Candidates are scored primarily by consistent row width, with smaller weights for useful column density and sample size. Quoted separators and quoted line breaks are parsed as field content.

Quality profile and readiness score

The profile measures missing cells, exact duplicate rows, mixed observed types, empty rows, duplicate JSON keys, formula-like spreadsheet cells, and columns with likely PII. The score begins at 100 and applies capped penalties: missingness up to 25 points, duplicate rows up to 20, mixed types up to 20, PII up to 20, structural warnings up to 14, and formula risk by 8. Labels are Strong (90–100), Workable (75–89), Needs review (50–74), or High risk (below 50).

The score is a triage aid. It is not a statistical quality measure and does not evaluate truth, bias, provenance, representativeness, or suitability for a particular decision.

Likely PII detection

The scanner combines column-name hints with deterministic value patterns for common emails, US-style SSNs, IPv4 addresses, Luhn-valid payment-card numbers, and phone-like values. Name, address, date-of-birth, credential, and passport warnings rely on field-name hints. Values are sampled per column, so rare identifiers can be missed and ordinary values can be flagged.

Schema inference and validation

Schema generation describes types, nullability, required fields, and common formats observed in the loaded records. It cannot predict unseen values. JSON Schema validation targets Draft 2020-12 and supports local fragment references; external references are not fetched.

Verification and change control

Parser, transformation, browser-layout, build, and Worker behavior are covered by automated regression checks before deployment. Method changes are recorded in the public changelog and source history.

Does Parse The Data use AI to classify my data?

No. Current format detection, profiling, PII warnings, schema inference, and scoring use deterministic code running in the browser.

Is the readiness score a guarantee of data quality?

No. It is a capped heuristic based on observable structural risks. It does not measure accuracy, bias, provenance, representativeness, or fitness for a particular use.

Can the PII scanner miss sensitive data?

Yes. It samples values and recognizes only documented patterns and field-name hints. It can miss uncommon identifiers and produce false positives.