Hiring-agent scores identical resume 66-99 across 100 runs

The GitHub repository interviewstreet/hiring-agent parses PDFs, calls an LLM six times for structured extraction, appends GitHub repo data, then produces a 100-point score plus 20 bonus points. Tests removed debug statements and disabled DEVELOPMENT_MODE, exposing category-level instability: projects fluctuated due to subjective judgments on complexity and deployment while technical skills remained consistent at 8/10 in 98 runs. Experience scoring hit 25/25 uniformly because its prompt contained only two lines with no rubric or anchors.

Non-determinism persists at temperature 0, matching a prior GitHub issue reporting scores 27-34 across six runs at temperature 0.2. Gemini produced tighter clusters (48-64) yet still crossed common cutoffs 28 percent of the time. The 65 percent weight on open-source and projects favors checklist artifacts over production impact, inverting preferences of engineers who built core infrastructure.

Operational effect is direct: any ATS cutoff at 85 rejects the same candidate 65 percent of runs. Structured extraction works; judgment calls on project merit or experience value do not, because LLMs lack consistent anchors regardless of prompt detail. Future releases cannot remove this variance without replacing the scoring LLM entirely.

Companies adopting the tool will observe identical resumes rejected or advanced based on prompt stochasticity rather than candidate quality.