DAYJOB Healthcare

Each task places an agent in a realistic healthcare workspace with records such as longitudinal patient charts, imaging and lab reports, payer contracts, claims queues, dispensing logs or batch records. The agent must produce work such as a physician handoff, a clinical appeal or a regulatory workpaper. Each task is a short workplace request backed by a multi-file workspace. An agent (OpenHands SDK harness on Kaggle) must produce finished deliverables, which a judge model grades against an expert-written checklist of binary criteria. A trial passes only when every criterion passes, and each model gets five attempts per task; the score is the share of passing trials. The track has 50 tasks across five subfields and 2,785 rubric criteria, with a median of 49 per task.

Scores are low by design: models satisfy most individual criteria but rarely all of them. Results depend on the Kaggle agent harness and the judge model, and Surge AI keeps a separate leaderboard of its own. Kaggle reports close agreement between the two, not identity. The tasks are simulated workspaces, not live patient care, and the score does not establish clinical safety.