Evidence

What ATS Resume Kit has actually tested against its own parsing and scoring pipeline — not general advice, a report of specific, reproducible comparisons and what they showed.

This page indexes the controlled experiments described in full on the methodology page, which remains the canonical explanation of how the scoring rules, skill taxonomy and parser work. This page exists to make the evidence itself easy to find and compare, in one consistent format per item, separate from the mechanics of how the product works.

Every item below follows the same structure: what was tested, what was held constant, what changed, what was measured, what was observed, what that may indicate, and — explicitly — what it does not establish. Nothing here generalizes past this codebase’s own pipeline to any external ATS product.

Completed research

Completed · 2026-09-19

How did a one-column and a two-column resume differ when processed by ATS Resume Kit’s own extraction and parsing pipeline?

Test setup: Two PDFs built from the same fixture resume, differing only in column arrangement, both run through this codebase’s pdf.js-based text extraction and its heading/section parser.
Control: The real production exporter, one column.
Variant: An equivalent two-column layout. This product’s own exporter cannot produce this layout by design (see /methodology), so the variant was generated separately for this test only, using the same fonts and text-wrapping code as the real exporter.
Held constant: All resume content — name, contact details, summary, skills, experience, projects, education, certifications, achievements — plus fonts, font size, page size and margins.
Changed: Column arrangement only.
Measurements taken: Sections the parser could map to a known heading; Lines the parser could not assign to any section; Distinct content words present in the extracted text
Observed result: The two-column variant’s extraction merged the “Professional Summary” heading onto the same line as an unrelated contact detail from the neighboring column. The parser then failed to recognize that heading at all: sections detected dropped from 7 to 6, and unassigned lines rose from 2 to 6. Separately, a full word-level comparison found zero content words present in one variant and missing from the other — the two-column layout did not lose or add wording, only reordered and regrouped it.
What this may indicate: In this controlled test, a two-column layout produced a different extraction order and reduced section-mapping accuracy in this codebase’s own parser, in this specific layout instance.
What this does NOT establish: This does not establish behavior across Workday, Greenhouse, Lever, Taleo, iCIMS, or any other commercial ATS product, and it tested one two-column arrangement, not every possible one.

Completed · 2026-09-19

Does enabling the builder’s contact-icon setting change the exported PDF?

Test setup: Two PDFs built from the same fixture resume with the icon setting off and on, both extracted through the same pdf.js pipeline used elsewhere, plus the ATS Resume Checker’s score run against both.
Control: settings.showIcons = false.
Variant: Identical resume with settings.showIcons = true.
Held constant: Every field of the resume; only the one setting differs.
Changed: The icon setting only.
Measurements taken: Extracted PDF text (not raw file bytes); ATS Resume Checker total score; Per-category score; Which rules fired
Observed result: The text extracted from the two PDFs was identical — this setting does not currently change what a parser reads from the exported file. (The raw files are not byte-identical: pdf-lib stamps a generation-time metadata field into every export regardless of content, which sits outside the text a parser reads — extracted text, not raw bytes, is the comparison that answers this question.) The ATS Resume Checker’s own score did change: the formatting category dropped from 20 to 17 points, and one additional rule was flagged.
What this may indicate: This is checker scoring behavior, not PDF parser behavior — the observed change is this site’s own scoring tool reacting to a setting, not a difference any PDF parser (this codebase’s own or any external one) would see in the file itself.
What this does NOT establish: This does not establish that icons are read or ignored by any external ATS — no PDF containing an actual icon glyph was tested, because the current exporter never draws one into the file regardless of this setting.

Completed · 2026-09-19

What happens when standard section headings are replaced with unconventional heading vocabulary?

Test setup: Two PDFs built from the same fixture resume, differing only in the text of four section headings, both run through the same extraction and parsing pipeline as the column experiment.
Control: The fixture’s own default headings: Professional Summary, Skills, Work Experience, Education.
Variant: Identical content with all four of those headings replaced simultaneously by phrases this codebase already recognizes as needing a suggested correction (“Why Me,” “Toolbox,” “My Journey,” “School Days”).
Held constant: All resume content, fonts, layout — only the four heading strings changed.
Changed: Four section headings at once — this is a heading-vocabulary substitution, not a test of any single heading in isolation.
Measurements taken: Sections the parser could map; Lines the parser could not assign to any section
Observed result: Sections mapped dropped from 7 to 3, and unassigned lines rose from 2 to 27 — every line under one of the four renamed headings, including its bullets, went unassigned rather than merely mis-labeled.
What this may indicate: In this test, replacing multiple section headings with creative alternatives sharply reduced this parser’s ability to map content to sections.
What this does NOT establish: Because four headings changed together, this result cannot be attributed to any single renamed heading on its own — that would require testing each heading individually, which has not been done. It also does not establish behavior in any external ATS.

Reproducibility

Every experiment above is generated from source-controlled fixtures in the project repository and run through the real production parser and scorer — not a separate, hand-maintained dataset. They can be reproduced with the project’s experiment runner (npm run experiments), which regenerates each PDF, re-extracts it and re-computes every measurement above from scratch on every run.

Research status

Three experiments are completed and reported above. The following are planned only — none of them has been run, and no result exists for any of them yet:

Individual heading substitutions tested one at a time, rather than four at once
Additional two-column arrangements (different split ratios, different section-to-column assignment)
Typography variations (font family, font size) in isolation
Text extraction ordering under other layout patterns
Tables and multi-column content blocks within a single section
Decorative or graphical elements other than icons
Multi-page layouts

A planned experiment will move to the “Completed research” section above, with a real measured result, once it has actually been run — never before.

Limitations that apply to everything on this page

Every measurement here comes from this codebase’s own pdf.js-based extraction pipeline and its own parser and scorer, described in full on the methodology page. None of it establishes how Workday, Greenhouse, Lever, Taleo, iCIMS or any other commercial ATS product behaves.
Each experiment tested one specific instance of its variable — one two-column arrangement, one icon setting, four particular heading substitutions — not an exhaustive survey of every possible variation.
See each item above for the specific reasoning behind its own conclusion; this page does not draw any broader conclusion than what is stated per item.

To see the rules, skill taxonomy and scoring weights these experiments were run against, read the methodology page. To try the tool the checker experiment above measured, open the ATS Resume Checker.

Last reviewed: 2026-09-19.