Unable to reproduce CNN scores

[ciaran-coleman]

Hello,

I am attempting to reproduce your results but am seeing differences in the WSI CNN count results. Example WSI NA5001_2AB:

• My results. Cored: 41, Diffuse: 288, CAA: 649
• Your results. Cored: 7, Diffuse: 410, CAA: 595

All packages I have matched, e.g. version 3.4.1 of libopencv/opencv/py-opencv, version 0.3.0 of torch, pyvips 2.1.2 with libvips 8.2.2. I should say that I am on a Windows 10 machine so this could very well be the problem, but just wanted to check first that I haven't missed something else.

When running notebook 2.2) I get the same AUC-PRC scores and so it seems the model is working correctly and giving the correct confidence.

I have also checked the stain normalization is working as intended by comparing 256x256 patches from the training data set provided in the Zenodo link to the equivalent cropped images produced when processing the WSIs with notebooks 1.1)-1.3). So I believe the WSIs are correctly stain normalized.

Would it be at all possible to see your prediction confidence heatmap for the example above so that I can inspect? I suspect this is where the differences lie that lead to the discrepancy.