Consultant dermatologist Magnus Lynch tested GPT-6 Astra and Claude Fable 5.1 vs fine-tuned Inception v3 (2015 CNN) on 200 biopsy-proven lesions from MILK10k (clinical + dermoscopic images, six diagnosis classes).
Results: CNN 125/200 correct first choice (62%) vs 84 and 85 (42%) for the LLMs. Second-choice allowed: 173 vs 134/138. Melanoma first choice: 13/30 vs 8 and 11. LLMs showed bias (GPT-6 over-called basal cell carcinoma; Fable 5.1 over-called actinic keratosis). Example melanoma misclassified as seborrhoeic keratosis by both LLMs; CNN correct on dermoscopy.
Unsafe-to-discharge lesions: LLMs kept 96–97/100; CNN 77 at preset threshold (98–99 when matched). Safe discharge: LLMs discharged 22–32/100 benign; CNN 76/100 at preset—raising false-negative melanoma discharges at default threshold (18/30) vs 2–4 for LLMs.
Patient-style prompt (clinical photo only, “need a doctor?”): LLMs advised doctor visit in 98–99% of 100 additional melanomas.
CNN trained on ISIC (68k images), H100, ~20 minutes; runs in browser via WebGPU demo. Data/code released. Author notes MILK10k public since 2025 (possible LLM training exposure); CNN held out patient/lesion leakage.
Conclusion: small specialized CNN beat frontier general models on this dermatology benchmark; regulated skin-AI products still outperform Inception v3.