In the NEJM Image Challenge study, Claude 3.7 succeeded when physicians failed in 123 of 272 cases (45.2%), while physicians succeeded when Claude 3.7 failed in only 8 cases (2.9%), a model-advantage-to-physician-advantage ratio of 15.4:1.
Sources
- Multimodal large language models challenge NEJM image ... - PMC (pmc.ncbi.nlm.nih.gov)
- https://www.nature.com/articles/s41598-026-39201-3 (nature.com)
- https://www.researchsquare.com/article/rs-7175790/v1 (researchsquare.com)
- https://www.researchgate.net/publication/395133210_Comparison_of_Multimodal_Large_Language_Models_and_Physicians_for_Medical_Diagnosis_Using_NEJM_Image_Challenge_Cases_Cross-sectional_Study (researchgate.net)
More in Technology
- Google's smile-detection model used a racial classifier trained on four race subgroups and a gender classifier with two categories.
- The paper 'Crowdsourced wireless spectrum anomaly detection' was authored by Sreeraj Rajendran, Vincent Lenders, Wannes Meert, and Sofie Pollin.
- Rocketdyne supplied 33 engines for each Saturn V rocket, providing engines for every stage except the lunar module's descent engine.
- A laptop belonging to former IRS Criminal Investigation (IRS-CI) agent Chris Janczewski held the cryptographic keys used to seize $3.6 billion in stolen bitcoin.
- Twenty-five manufacturers produced leading-edge chips 20 years before 2021, but by 2021 only TSMC (Taiwan), Intel (United States), and Samsung (South Korea) had facilities capable of producing the most advanced chips.
- JWST's mid-infrared instrument (MIRI) is less sensitive than the telescope's other instruments.