Problem
AI resume tools optimize for polished similarity. That creates two trust failures: they can invent or inflate experience, and they can make every candidate sound interchangeable. Product managers need tailoring that reads the role intelligently without turning adjacent experience into expertise they do not have. I also needed the product to work repeatedly for real applications, not just produce one impressive demo.
The Hard Call
The first tailor could produce a strong resume in one model call, but it asked the model to select evidence, rewrite bullets, frame the candidate, and fit the page at once. During dogfooding, one thin but relevant experience could pull the entire resume toward a manufactured specialist identity. I could keep adding prompt rules, or change the system so the failure was harder to create. I chose the architectural reset: separate selection from writing, anchor both to a stable candidate identity, and move mechanical constraints into code.
What I Did
Product strategy
- Narrowed the private-beta wedge to reliable resume tailoring and onboarding; kept Jobs, Pipeline, and Interview Prep hidden rather than treating built code as launch-ready product.
- Turned founder dogfooding issues into a lane-based roadmap so urgent applications did not erase launch-critical work.
- Defined honesty as the product differentiator: every claim should be traceable, reviewable, and interview-defensible.
- Designed onboarding around an early reviewable artifact instead of a long AI interview, so a user who pauses still leaves with something useful.
Agent architecture
- Split the system into construction-time candidate modeling and a repeated per-job pipeline: Coach → evidence libraries and identity prior; Analyzer → real must-haves and fit read; Tailor → evidence selection, deterministic assembly, and constrained writing.
- Replaced free-form bullet rewriting with select-by-ID from a verified library. Selected bullets remain verbatim; the generative surface is primarily title and summary.
- Built deterministic A/B/C/F fit scoring from model classifications, distinguishing transferable competencies from non-transferable thresholds.
- Added provenance and approval states so extracted or inferred material stays staged until it is reviewed or explicitly promoted.
- Used code for metric grounding, structural checks, deduplication, page-fit budgets, and other rules that do not require language judgment.
Evaluation and economics
- Built a 13-job-description regression set with founder-labeled directional fit expectations and explicit bad-role traps.
- Requested a 12-model analyzer bake-off, compared 11 completed model responses, and selected by failure severity, cost, and threshold judgment rather than generic benchmark ranking.
- Rejected an image-context optimization after an N=5 test showed that the honesty-critical trap passed only 1/5 times with image delivery versus 5/5 with text.
- Split the tailor between a low-cost selector and a higher-quality constrained writer, reducing the measured path from about $0.042 for the preceding all-Sonnet rank-select call to about $0.019 cold / $0.013 warm per tailoring.
- Ran four synthetic candidate personas through onboarding and tailoring. No invented roles or experience were detected; the tests exposed unsupported metric transformations, duplicate role grouping, pseudo-STAR stories, and stale-profile handling, which became concrete fixes.
See the system flow
Coach
reviewed evidence + identity
→
Analyzer
must-haves + fit read
→
Selector
evidence by ID
→
Assembly
deterministic fit
→
Writer
title + summary only
See an experiment that changed the product
Image delivery compressed the analyzer context dramatically, but the saving damaged the exact distinction the product needed to protect. On the deShaw threshold trap, image delivery produced the correct F only 1/5 times; identical text delivery produced 5/5. The image path was removed. The decision traded a possible token saving for a more reliable honesty-critical read.
Outcome
Private beta — hosted product and core tailoring path
13 JDs — repeatable regression set
11 models — completed analyzer responses
~2.2–3.1× lower tailoring cost vs. prior path
4 personas — synthetic onboarding-to-tailoring E2E
Current evidence is strongest on architecture, regression testing, and founder dogfooding. The first external PM pilot is secured but has not yet begun; broader activation, retention, and outcome validation remain the next proof point.
Skills:
AI Product Strategy · Agent Architecture · Evaluation Design · Model Selection · Trust & Safety · Unit Economics · Product Scoping · Founder Dogfooding
