Consultant dermatologist Magnus Lynch reports that a fine-tuned Inception v3 correctly identified 125 of 200 skin lesions, compared with 84 for GPT-6 Astra and 85 for Claude Fable 5.1. His self-published comparison puts an older, specialized convolutional neural network ahead of two general-purpose multimodal models on a narrowly defined medical-image task.
Lynch presents the headline result as 62% versus 42% first-choice accuracy. But the most consequential detail sits below that headline: the CNN’s higher diagnostic accuracy did not automatically translate into safer decisions about which lesions could be discharged without treatment, biopsy or follow-up.
This is one clinician’s published evaluation, not a clinical trial or independent validation. Its value lies in the documented setup, the reported release of evaluation materials, and the opportunity to examine where diagnostic accuracy and clinical caution diverge.
The CNN Leads on First-Choice Diagnosis
All three models chose among the same six diagnoses. The two language models could also answer “unsure,” and were asked for confidence, a second choice and a discharge decision.
Lynch’s reported results were:
| Model | Correct first-choice diagnoses | Accuracy calculated from counts |
|---|---|---|
| GPT-6 Astra | 84/200 | 42% |
| Claude Fable 5.1 | 85/200 | 42.5% |
| Fine-tuned Inception v3 | 125/200 | 62.5% |
The counts put the CNN about 20 percentage points ahead of either LLM. They also make clear what the headline’s rounded figures represent.
The advantage varied substantially by diagnosis. Inception v3 correctly identified 37 of 50 benign moles and 26 of 50 seborrhoeic keratoses. GPT-6 Astra identified 16 and 13, respectively; Claude Fable 5.1 identified 33 and six.
The CNN did not win every category. GPT-6 Astra correctly identified all 25 basal cell carcinomas, compared with 24 for the CNN. Both LLMs also scored higher on actinic keratosis.
Melanoma remained difficult for all three: the CNN named it correctly in 13 of 30 cases, against eight for GPT-6 Astra and 11 for Claude Fable 5.1. Even the highest-scoring system therefore missed melanoma as its first-choice diagnosis more often than it identified it.
That category-level result matters more than a broad claim that one type of AI “understands” skin cancer better. The published evidence supports a narrower conclusion: this particular specialized pipeline classified these images more accurately overall.
The 200 Lesions Form a Difficult Test Set
Lynch drew the lesions from MILK10k, a public dataset released through the International Skin Imaging Collaboration in 2025. Each lesion had two views: a clinical close-up and a dermoscopic image taken through the illuminated magnifying instrument used to inspect skin structures.
Every diagnosis was confirmed by histopathology, meaning examination of sampled tissue. The test set contained:
- 30 melanomas
- 25 basal cell carcinomas
- 25 squamous cell carcinomas, including Bowen’s disease
- 20 actinic keratoses
- 50 benign naevi, or moles
- 50 seborrhoeic keratoses, benign warty growths
Lynch grouped the first four categories as 100 lesions that should not be discharged, and the two benign categories as 100 that could be.
The biopsy requirement creates an important selection effect. Even the benign lesions had prompted someone to obtain tissue. These are therefore more diagnostically challenging than a random collection of ordinary harmless moles.
The equal unsafe/benign split also differs from whatever mix a particular clinic or screening service might encounter. It makes the comparison easy to inspect, but its overall accuracy cannot simply be carried over to another patient population.
Specialized Training Shapes the Comparison
Inception v3 is an architecture first published in 2015. Lynch used a version pretrained on everyday photographs, then fine-tuned all its layers on 68,011 clinical and dermoscopic images from the ISIC Archive.
According to his method description, that fine-tuning took about 20 minutes on one H100 GPU. The architecture’s age should therefore be read alongside its recent, task-specific training.
Lynch says he excluded the 200 test lesions and every other photograph belonging to the same lesion or patient from the CNN’s training and validation data. The test cases were also excluded from model selection and the setting of its initial threshold. Patient-level exclusion is important because another photograph of the same lesion could otherwise make a nominally separate test misleadingly easy.
The LLMs received both images, no additional clinical information, and identical prompts at default settings. Neither refused a case.
The CNN also processed both views, but combined their scores with most of the weight assigned to dermoscopy.
My reading is that this compares complete task-specific setups, not architectures in isolation. Specialized training, image weighting and the output procedure all contribute to the CNN’s result. The LLMs were general-purpose systems responding to a clinical prompt.
That distinction strengthens the practical lesson: a general-purpose frontier model should not be assumed to outperform an older specialist simply because it has broader capabilities. It also limits what this experiment can say about other CNNs, other LLM prompts or purpose-built medical systems.
Discharge Decisions Change the Safety Picture
A diagnosis and a discharge decision answer different questions. A model can name the wrong cancer type while still recommending that a lesion receive attention. It can also mistake a melanoma for a benign growth and recommend no further action.
Lynch’s opening example illustrates the second failure. Both LLMs reportedly classified an invasive melanoma on the ear, under 0.8 mm thick, as a seborrhoeic keratosis and said it could be discharged. The CNN identified melanoma.
Across the full test, however, the CNN’s initial discharge behavior was much less reassuring than its accuracy lead suggests.
The following safety figures come from explainx.ai’s secondary account of Lynch’s analysis. That account summarizes the same evaluation; it is not an independent reproduction.
| Discharge measure | GPT-6 Astra | Claude Fable 5.1 | CNN at preset threshold |
|---|---|---|---|
| Unsafe lesions retained, out of 100 | 97 | 96 | 77 |
| Benign lesions discharged, out of 100 | 32 | 22 | 76 |
| Melanomas discharged, out of 30 | 3 | 4 | 18 |
At that preset threshold, the CNN discharged substantially more benign lesions correctly, but it also discharged 18 of 30 melanomas. That would be an unacceptable miss rate for a real discharge service.
The LLMs were more conservative. They retained nearly all unsafe lesions, while discharging relatively few benign ones. In a clinical workflow, retaining more benign lesions could mean more assessment, procedures or follow-up, depending on how the system was used.
A threshold determines how much evidence a scoring system requires before taking an action. Changing it can trade fewer dangerous discharges for more benign lesions being retained.
The secondary account reports that adjusted CNN thresholds retained 98 or 99 of the 100 unsafe lesions, with one or two melanomas discharged. Those figures describe different operating points from the preset result. They do not mean the underlying image classifier suddenly became more accurate at naming diagnoses.
Frequently Asked Questions
3 questions
1Did the CNN Beat Both LLMs?
Yes, on first-choice diagnosis in this test: 125 correct answers versus 84 and 85. That result does not establish superior clinical safety.
