Mitosis Detection Results: A Comparative Analysis of Published Pretrained Models and Reproduce Training Approaches Using Domain Adversarial RetinaNet

[shbkukuk]

Hello, firstly, I'd like to express our gratitude for the opportunity to contribute to the advancement of mitosis detection and the associated challenges. We are currently engaged in scientific research focused on mitosis detection, and in our pursuit, we've come across the MIDOG21 and MIDOG22 datasets, which are renowned for their extensive and diverse content.
Our attention was drawn to a proposed 'baseline' methodology known as Domain Adversarial RetinaNet, which we believe holds promise for our objectives. We endeavoured to employ this methodology by utilizing both the published pretrained model and conducting our own training and inference processes. During this endeavour, we ensured consistency by employing the same MIDOG slides for training purposes.

However, upon comparing the results obtained from the pretrained model and our own trained model, we encountered discrepancies. On the left, we have the results obtained from the published pretrained model, while on the right, we present the results from our trained model. In your paper [link], a threshold of 0.64 is mentioned, which we adhered to in both cases. Additionally, we meticulously examined the docker code for inference and the implementation for patching to ensure conformity in parameters and methodology.
Despite these efforts, the observed disparities persist, prompting us to seek clarification and possibly identify areas where our approach may require refinement. We are keen to address these challenges and enhance the efficacy of mitosis detection methodology the below you can see our training process metrics and loss. We could not see any not expected situation. This model got fit.

epoch     train_loss  valid_loss  pascal_voc_metric_by_distance_da  BBloss    focal_loss  domain_loss  total     acc       AP-mitotic figure  time    
0         0.785450    0.983057    0.346603                          0.202706  0.779731    0.000620     0.736207  0.977500  0.346603           09:26     
Better model found at epoch 0 with total value: 0.736207127571106.
1         0.542444    0.610519    0.690280                          0.142023  0.468416    0.000080     0.457749  0.992000  0.690280           07:01     
Better model found at epoch 1 with total value: 0.45774924755096436.
2         0.450877    0.469345    0.808177                          0.127178  0.342132    0.000035     0.351947  0.994000  0.808177           06:05     
Better model found at epoch 2 with total value: 0.35194727778434753.
3         0.331436    0.378063    0.868382                          0.099184  0.278862    0.000017     0.283518  0.996000  0.868382           05:42     
Better model found at epoch 3 with total value: 0.2835179567337036.
4         0.327526    0.312038    0.895164                          0.081804  0.230220    0.000014     0.234004  0.994500  0.895164           05:26     
Better model found at epoch 4 with total value: 0.23400427401065826.
5         0.266648    0.284434    0.899722                          0.074652  0.209772    0.000010     0.213308  0.996500  0.899722           05:12     
Better model found at epoch 5 with total value: 0.21330773830413818.
6         0.258341    0.299516    0.921162                          0.068209  0.231297    0.000011     0.224619  0.996500  0.921162           05:06     
7         0.254432    0.251472    0.906898                          0.060052  0.191414    0.000005     0.188594  0.996000  0.906898           05:00     
Better model found at epoch 7 with total value: 0.18859417736530304.
8         0.237483    0.261316    0.906031                          0.066721  0.194590    0.000005     0.195978  0.995500  0.906031           04:58     
9         0.224526    0.231317    0.916301                          0.056809  0.174500    0.000008     0.173473  0.993000  0.916301           04:56     
Better model found at epoch 9 with total value: 0.17347344756126404.
10        0.222153    0.237260    0.916550                          0.061097  0.176161    0.000002     0.177942  0.998000  0.916550           04:56     
11        0.236798    0.223430    0.937565                          0.052226  0.171199    0.000005     0.167563  0.995500  0.937565           04:56     
Better model found at epoch 11 with total value: 0.16756345331668854.
12        0.196236    0.230294    0.925636                          0.060864  0.169403    0.000027     0.172672  0.985000  0.925636           04:54     
13        0.197627    0.226272    0.937331                          0.057529  0.168739    0.000004     0.169697  0.992500  0.937331           04:52     
14        0.214247    0.219538    0.940474                          0.056912  0.162621    0.000005     0.164645  0.994000  0.940474           04:53     
Better model found at epoch 14 with total value: 0.1646449863910675.
15        0.175549    0.224067    0.952182                          0.048183  0.175883    0.000001     0.168048  0.999000  0.952182           04:53     
16        0.189039    0.218559    0.913038                          0.052150  0.166405    0.000004     0.163912  0.994500  0.913038           04:52     
Better model found at epoch 16 with total value: 0.16391249001026154.
17        0.172550    0.204367    0.937197                          0.048504  0.155858    0.000005     0.153267  0.992500  0.937197           04:51     
Better model found at epoch 17 with total value: 0.15326663851737976.
18        0.187031    0.214273    0.905593                          0.047507  0.166760    0.000005     0.160696  0.994500  0.905593           04:52     
19        0.186904    0.221622    0.948327                          0.054655  0.166962    0.000005     0.166208  0.994000  0.948327           04:52     
20        0.202084    0.249369    0.919916                          0.050631  0.198738    0.000001     0.187025  0.998500  0.919916           04:51     
21        0.156743    0.193121    0.942234                          0.044848  0.148140    0.000133     0.144607  0.968500  0.942234           04:52     
Better model found at epoch 21 with total value: 0.14460746943950653.
22        0.182503    0.199949    0.944177                          0.050305  0.149634    0.000009     0.149946  0.992500  0.944177           04:50     
23        0.180590    0.197966    0.948764                          0.049220  0.148709    0.000037     0.148410  0.971500  0.948764           04:51     
24        0.164153    0.169926    0.942608                          0.046240  0.123659    0.000027     0.127398  0.976500  0.942608           04:53     
Better model found at epoch 24 with total value: 0.12739767134189606.
25        0.193322    0.188513    0.941290                          0.045644  0.142856    0.000014     0.141360  0.989500  0.941290           04:51     
26        0.180413    0.197304    0.951649                          0.052441  0.144830    0.000032     0.147921  0.987000  0.951649           04:50     
27        0.159821    0.206688    0.940152                          0.057649  0.148869    0.000170     0.154719  0.936000  0.940152           04:52     
28        0.158653    0.288118    0.949677                          0.054382  0.233663    0.000073     0.215960  0.969000  0.949677           04:53     
29        0.156991    0.266293    0.907901                          0.047837  0.218218    0.000239     0.199302  0.919000  0.907901           04:51     
30        0.162830    0.195992    0.928350                          0.049219  0.136453    0.010320     0.128934  0.594000  0.928350           04:51     
31        0.180620    0.189446    0.945017                          0.042536  0.135710    0.011201     0.122484  0.470500  0.945017           04:51     
Better model found at epoch 31 with total value: 0.12248370051383972.
32        0.189680    0.204291    0.930798                          0.044373  0.148940    0.010978     0.134007  0.521000  0.930798           04:51     
33        0.171617    0.249374    0.915211                          0.051483  0.182120    0.015771     0.159431  0.340000  0.915211           04:52     
34        0.172175    0.272676    0.914635                          0.055209  0.202369    0.015098     0.178085  0.379500  0.914635           04:52     
35        0.197487    0.209198    0.936295                          0.045366  0.148199    0.015633     0.129541  0.334500  0.936295           04:51     
36        0.160086    0.207027    0.943772                          0.050380  0.145067    0.011580     0.135006  0.522500  0.943772           04:52     
37        0.168441    0.213366    0.932930                          0.044194  0.153431    0.015741     0.132478  0.314500  0.932930           04:51     
38        0.180116    0.196610    0.958032                          0.043068  0.137225    0.016317     0.118902  0.298000  0.958032           04:52     
Better model found at epoch 38 with total value: 0.11890240758657455.
39        0.190928    0.203397    0.942083                          0.047760  0.139312    0.016325     0.123979  0.343000  0.942083           04:51     
40        0.163907    0.222326    0.932640                          0.044493  0.162745    0.015088     0.140340  0.311500  0.932640           04:51     
41        0.163990    0.212104    0.934992                          0.048361  0.147599    0.016143     0.130827  0.303500  0.934992           04:51     
42        0.179682    0.220284    0.949341                          0.049924  0.154292    0.016068     0.137094  0.299000  0.949341           04:52     
43        0.161919    0.200178    0.947384                          0.046068  0.141197    0.012914     0.127535  0.423500  0.947384           04:51     
44        0.156455    0.235900    0.927150                          0.047400  0.172466    0.016034     0.148866  0.340500  0.927150           04:50     
45        0.154492    0.225369    0.941197                          0.042586  0.166588    0.016195     0.140686  0.277000  0.941197           04:50     
46        0.167239    0.217614    0.936051                          0.047130  0.154223    0.016261     0.134754  0.321500  0.936051           04:50     
47        0.142298    0.197371    0.943239                          0.042996  0.138553    0.015822     0.120340  0.323000  0.943239           04:50     
48        0.181488    0.319136    0.903015                          0.055773  0.246466    0.016898     0.209781  0.306000  0.903015           04:49     
49        0.183666    0.253149    0.939159                          0.047853  0.191290    0.014006     0.165351  0.427000  0.939159           04:50     
50        0.167431    0.261053    0.910725                          0.047439  0.197972    0.015642     0.168417  0.334500  0.910725           04:46     
51        0.156896    0.220033    0.936470                          0.046075  0.159330    0.014628     0.139425  0.370500  0.936470           04:46     
52        0.171971    0.204539    0.925098                          0.041754  0.148185    0.014599     0.127856  0.432000  0.925098           04:47     
53        0.161871    0.220858    0.927294                          0.047065  0.158594    0.015199     0.139046  0.350500  0.927294           04:46     
54        0.159982    0.208867    0.938260                          0.040258  0.151700    0.016909     0.127060  0.294500  0.938260           04:47     
55        0.153658    0.209263    0.943699                          0.039806  0.155189    0.014268     0.131979  0.387500  0.943699           04:47     
56        0.185696    0.208002    0.950126                          0.041127  0.152948    0.013927     0.131630  0.439000  0.950126           04:48     
57        0.144182    0.179400    0.950784                          0.040102  0.123425    0.015874     0.106771  0.312000  0.950784           04:47     
Better model found at epoch 57 with total value: 0.10677138715982437.
58        0.176117    0.202111    0.947194                          0.047481  0.137643    0.016986     0.121858  0.334500  0.947194           04:48     
59        0.144845    0.177567    0.940757                          0.045533  0.116166    0.015869     0.105405  0.287500  0.940757           04:47     
Better model found at epoch 59 with total value: 0.10540502518415451.
60        0.156287    0.227368    0.944799                          0.038740  0.172753    0.015875     0.142744  0.374000  0.944799           04:48     
61        0.160707    0.184147    0.950982                          0.045141  0.123691    0.015315     0.111309  0.314000  0.950982           04:48     
62        0.150024    0.200020    0.957444                          0.043341  0.142159    0.014520     0.124606  0.396500  0.957444           04:47     
63        0.162003    0.201243    0.946226                          0.038524  0.145131    0.017587     0.120154  0.282000  0.946226           04:47     
64        0.138322    0.182657    0.948850                          0.040711  0.127395    0.014552     0.111528  0.417000  0.948850           04:47     
65        0.144133    0.200995    0.934477                          0.042571  0.143671    0.014753     0.124928  0.352500  0.934477           04:48     
66        0.127142    0.190230    0.943350                          0.041578  0.132417    0.016235     0.114262  0.309000  0.943350           04:47     
67        0.155443    0.240632    0.937342                          0.042395  0.182529    0.015707     0.152986  0.341500  0.937342           04:47     
68        0.144572    0.171364    0.945882                          0.039459  0.116960    0.014945     0.102369  0.352500  0.945882           04:48     
Better model found at epoch 68 with total value: 0.10236901044845581.
69        0.161703    0.195337    0.940093                          0.047317  0.132942    0.015078     0.120115  0.362500  0.940093           04:48     
70        0.144342    0.336308    0.949807                          0.040224  0.281974    0.014110     0.227539  0.370500  0.949807           04:47     
71        0.144123    0.180164    0.965901                          0.043903  0.121420    0.014840     0.109152  0.338500  0.965901           04:47     
72        0.141794    0.208556    0.938842                          0.043725  0.147690    0.017140     0.126421  0.285500  0.938842           04:57     
73        0.149875    0.186927    0.958077                          0.045622  0.124439    0.016865     0.110681  0.288000  0.958077           04:55     
74        0.139127    0.236158    0.940866                          0.045136  0.175814    0.015209     0.150503  0.379000  0.940866           04:46     
75        0.121149    0.283810    0.922071                          0.045367  0.222432    0.016011     0.184839  0.290000  0.922071           04:47     
76        0.151578    0.210009    0.955442                          0.044814  0.148047    0.017148     0.127498  0.247500  0.955442           04:47     
77        0.141308    0.246644    0.941830                          0.041339  0.189283    0.016022     0.156944  0.299000  0.941830           04:47     
78        0.142363    1.521045    0.928240                          0.059404  1.439179    0.022461     1.101476  0.334000  0.928240           04:47     
79        0.133982    0.174231    0.943210                          0.042523  0.117101    0.014607     0.105111  0.421500  0.943210           04:48     
80        0.142308    0.211360    0.933219                          0.039926  0.156366    0.015068     0.132151  0.323500  0.933219           04:47     
81        0.141441    0.190722    0.947818                          0.044007  0.130053    0.016661     0.113884  0.328000  0.947818           04:47     
82        0.140780    0.225875    0.941220                          0.042266  0.166093    0.017516     0.138752  0.295000  0.941220           04:47     
83        0.160125    0.198652    0.932029                          0.041318  0.141601    0.015734     0.121455  0.363500  0.932029           04:47     
84        0.132766    0.185648    0.948422                          0.044723  0.125424    0.015501     0.112109  0.313500  0.948422           04:46     
85        0.147027    0.204072    0.934485                          0.044901  0.144499    0.014672     0.127379  0.357500  0.934485           04:48     
86        0.155001    0.198236    0.934426                          0.042693  0.139123    0.016420     0.119941  0.310000  0.934426           04:47     
87        0.129572    0.187880    0.927613                          0.039618  0.132512    0.015750     0.113348  0.318000  0.927613           04:46     
88        0.119754    0.219112    0.947549                          0.047004  0.155832    0.016276     0.135851  0.315500  0.947549           04:47     
89        0.123231    0.236715    0.938996                          0.041993  0.179774    0.014947     0.151378  0.361000  0.938996           04:47     
90        0.133879    0.238884    0.955843                          0.040154  0.182933    0.015797     0.151518  0.296500  0.955843           04:47     
91        0.150030    0.211925    0.936782                          0.043796  0.152163    0.015966     0.131003  0.309500  0.936782           04:47     
92        0.132356    0.188554    0.950641                          0.042847  0.130123    0.015585     0.114143  0.341000  0.950641           04:48     
93        0.109335    0.237328    0.950354                          0.041139  0.179571    0.016618     0.148914  0.255500  0.950354           04:47     
94        0.114027    0.211238    0.944948                          0.043280  0.151887    0.016071     0.130305  0.293000  0.944948           04:47     
95        0.140208    0.216891    0.938503                          0.045852  0.153699    0.017340     0.132323  0.307500  0.938503           04:41     
96        0.110975    0.281596    0.913604                          0.045401  0.219968    0.016228     0.182799  0.271500  0.913604           04:39     
97        0.137016    0.178768    0.950799                          0.038754  0.124168    0.015847     0.106345  0.304000  0.950799           04:39     
98        0.134312    0.208802    0.943562                          0.044228  0.149622    0.014952     0.130435  0.385000  0.943562           04:36     
99        0.136337    0.178562    0.962772                          0.039064  0.122319    0.017179     0.103858  0.258000  0.962772           04:40     
100       0.122893    0.183780    0.940092                          0.043687  0.122852    0.017242     0.107662  0.221000  0.940092           04:39     
101       0.134159    0.196005    0.956618                          0.038617  0.141561    0.015828     0.119306  0.396500  0.956618           04:39     
102       0.114882    0.225477    0.963191                          0.039846  0.169361    0.016271     0.140634  0.331500  0.963191           04:40     
103       0.125989    0.203896    0.918097                          0.035930  0.153425    0.014542     0.127474  0.389500  0.918097           04:39     
104       0.120106    0.184987    0.965017                          0.040252  0.129131    0.015604     0.111433  0.339500  0.965017           04:40     
105       0.136907    0.204692    0.957192                          0.038823  0.149435    0.016433     0.124760  0.254500  0.957192           04:40     
106       0.147546    0.291626    0.951552                          0.041585  0.234377    0.015664     0.191307  0.326500  0.951552           04:41     
107       0.372598    0.201593    0.947389                          0.046870  0.139780    0.014943     0.125045  0.357500  0.947389           04:41     
108       0.121295    0.189618    0.953973                          0.039269  0.134829    0.015520     0.115053  0.340500  0.953973           04:41     
109       0.109155    0.177106    0.956813                          0.037708  0.122441    0.016958     0.103153  0.252000  0.956813           04:41     
110       0.111040    0.212543    0.955270                          0.042335  0.150066    0.020142     0.124158  0.188500  0.955270           04:40     
111       0.125227    0.201385    0.947159                          0.038268  0.145709    0.017408     0.120575  0.279500  0.947159           04:40     
112       0.114964    0.201367    0.939172                          0.044779  0.140439    0.016149     0.122765  0.284500  0.939172           04:40     
113       0.112271    0.220891    0.930691                          0.048382  0.155217    0.017292     0.135407  0.316500  0.930691           04:39     
114       0.120581    0.190376    0.931197                          0.038950  0.135539    0.015886     0.114981  0.333500  0.931197           04:40     
115       0.116819    0.212193    0.952629                          0.042305  0.154009    0.015879     0.131357  0.320500  0.952629           04:41     
116       0.151188    0.195947    0.952565                          0.039917  0.140683    0.015347     0.120103  0.326000  0.952565           04:40     
117       0.113476    0.230768    0.945555                          0.036548  0.179414    0.014806     0.147165  0.410500  0.945555           04:39     
118       0.098747    0.236197    0.941359                          0.042683  0.177102    0.016412     0.148427  0.249000  0.941359           04:40     
119       0.119915    0.209336    0.958719                          0.042855  0.151412    0.015069     0.130632  0.373500  0.958719           04:40     
120       0.114365    0.181502    0.965282                          0.040438  0.125013    0.016052     0.108036  0.309500  0.965282           04:41     
121       0.117941    0.197289    0.941160                          0.040243  0.140833    0.016213     0.119594  0.290000  0.941160           04:41     
122       0.096139    0.552260    0.943592                          0.035965  0.500501    0.015794     0.386555  0.314000  0.943592           04:40     
123       0.127164    0.175807    0.959735                          0.038272  0.122145    0.015389     0.104924  0.363000  0.959735           04:40     
124       0.096579    0.209624    0.957389                          0.041025  0.153727    0.014872     0.131192  0.399500  0.957389           04:41     
125       0.118317    0.183046    0.951172                          0.039188  0.129143    0.014715     0.111533  0.403000  0.951172           04:41     
126       0.103399    0.205166    0.942465                          0.040970  0.148367    0.015829     0.126174  0.314000  0.942465           04:41     
127       0.101435    0.194475    0.955980                          0.036799  0.141671    0.016005     0.117848  0.281000  0.955980           04:41     
128       0.102088    0.187990    0.945449                          0.035907  0.134858    0.017225     0.110849  0.284000  0.945449           04:42     
129       0.098437    0.856650    0.955877                          0.037770  0.802758    0.016122     0.614274  0.256500  0.955877           04:42     
130       0.096596    0.185124    0.958848                          0.040357  0.131326    0.013441     0.115321  0.421500  0.958848           04:42     
131       0.100013    0.206874    0.938703                          0.036957  0.154297    0.015620     0.127821  0.318000  0.938703           04:41     
132       0.107007    0.201512    0.951709                          0.039248  0.146260    0.016004     0.123128  0.313000  0.951709           04:42     
133       0.113583    0.184317    0.946862                          0.037286  0.130789    0.016242     0.109814  0.284500  0.946862           04:41     
134       0.097388    0.184101    0.958423                          0.038728  0.128322    0.017051     0.108237  0.240500  0.958423           04:42     
135       0.085454    0.197828    0.956510                          0.040150  0.141635    0.016042     0.120297  0.292500  0.956510           04:42     
136       0.105328    0.183154    0.956231                          0.040204  0.127476    0.015474     0.110287  0.346000  0.956231           04:49     
137       0.096226    0.215639    0.940306                          0.038909  0.159970    0.016760     0.132399  0.292500  0.940306           04:54     
138       0.098154    0.183644    0.961170                          0.037469  0.130214    0.015961     0.109802  0.298000  0.961170           04:51     
139       0.093971    0.201110    0.943272                          0.041491  0.143804    0.015814     0.123157  0.328000  0.943272           04:48     
140       0.083270    0.189820    0.958757                          0.037109  0.140375    0.012336     0.120777  0.513000  0.958757           04:51     
141       0.098914    0.182260    0.954193                          0.037346  0.129928    0.014987     0.110468  0.378000  0.954193           04:50     
142       0.077391    0.219728    0.951431                          0.040115  0.163940    0.015673     0.137368  0.337500  0.951431           04:49     
143       0.096102    0.168688    0.943726                          0.035525  0.116736    0.016427     0.097769  0.290000  0.943726           04:49     
Better model found at epoch 143 with total value: 0.09776861220598221.
144       0.094300    0.189119    0.942578                          0.035220  0.136192    0.017707     0.110852  0.277500  0.942578           04:47     
145       0.092052    0.162262    0.945067                          0.039021  0.107420    0.015820     0.094011  0.299000  0.945067           04:42     
Better model found at epoch 145 with total value: 0.09401094913482666.
146       0.088274    0.187951    0.944375                          0.037093  0.133730    0.017129     0.110988  0.215500  0.944375           04:42     
147       0.095171    0.204087    0.956507                          0.035841  0.152058    0.016187     0.124737  0.270500  0.956507           04:42     
148       0.088965    0.206030    0.936347                          0.038430  0.149573    0.018027     0.122975  0.242000  0.936347           04:42     
149       0.097582    0.192821    0.944700                          0.038650  0.137276    0.016895     0.115050  0.252000  0.944700           04:42     
150       0.096938    0.185611    0.956914                          0.044542  0.124933    0.016136     0.110971  0.264000  0.956914           04:41     
151       0.086486    0.245326    0.938044                          0.039749  0.189695    0.015882     0.156201  0.348000  0.938044           04:42     
152       0.077055    0.222055    0.949954                          0.037463  0.169020    0.015572     0.139291  0.308500  0.949954           04:41     
153       0.083291    0.185139    0.953293                          0.039853  0.128375    0.016912     0.109259  0.272500  0.953293           04:42     
154       0.094479    0.179118    0.941715                          0.036134  0.126636    0.016348     0.105729  0.255000  0.941715           04:41     
155       0.083417    0.192812    0.947840                          0.037576  0.138610    0.016627     0.115512  0.234500  0.947840           04:41     
156       0.093022    0.175765    0.955580                          0.034677  0.125116    0.015972     0.103873  0.319000  0.955580           04:41     
157       0.100234    0.204923    0.944169                          0.038705  0.151343    0.014874     0.127662  0.388000  0.944169           04:41     
158       0.072664    0.237324    0.949212                          0.038611  0.182809    0.015903     0.150162  0.282000  0.949212           04:41     
159       0.086279    0.209794    0.947191                          0.041186  0.152529    0.016078     0.129209  0.261000  0.947191           04:41     
160       0.095580    0.175932    0.960151                          0.036544  0.123670    0.015719     0.104441  0.331500  0.960151           04:41     
161       0.078650    0.194813    0.956329                          0.040617  0.137913    0.016283     0.117614  0.275000  0.956329           04:41     
162       0.085086    0.200761    0.948708                          0.041245  0.143302    0.016214     0.122196  0.306500  0.948708           04:41     
163       0.084111    0.212135    0.948223                          0.039692  0.156472    0.015972     0.131151  0.285000  0.948223           04:41     
164       0.093526    0.196454    0.938987                          0.037314  0.143250    0.015890     0.119533  0.291000  0.938987           04:41     
165       0.087800    0.191732    0.937500                          0.038256  0.137535    0.015942     0.115901  0.283500  0.937500           04:41     
166       0.090100    0.214589    0.945305                          0.037079  0.161629    0.015881     0.133150  0.274000  0.945305           04:41     
167       0.088303    0.195450    0.954410                          0.036996  0.142788    0.015667     0.119171  0.348000  0.954410           04:41     
168       0.069431    0.187785    0.949960                          0.039381  0.132866    0.015538     0.113647  0.359500  0.949960           04:40     
169       0.086145    0.180941    0.957234                          0.037594  0.127677    0.015670     0.108283  0.331000  0.957234           04:41     
170       0.073757    0.221506    0.954738                          0.038582  0.166900    0.016025     0.138086  0.297000  0.954738           04:41     
171       0.098995    0.209120    0.952302                          0.037685  0.155223    0.016212     0.128469  0.292000  0.952302           04:41     
172       0.081653    0.202188    0.947124                          0.037304  0.148981    0.015903     0.123811  0.293500  0.947124           04:40     
173       0.077004    0.212156    0.941345                          0.034953  0.161438    0.015765     0.131529  0.323000  0.941345           04:40     
174       0.072520    0.192403    0.953131                          0.037626  0.138811    0.015966     0.116362  0.303000  0.953131           04:40     
175       0.081657    0.185333    0.943647                          0.036804  0.132911    0.015618     0.111669  0.333000  0.943647           04:40     
176       0.073755    0.214307    0.951647                          0.040487  0.158114    0.015706     0.133244  0.308500  0.951647           04:40     
177       0.073110    0.245446    0.949097                          0.035771  0.193520    0.016155     0.155814  0.276500  0.949097           04:50     
178       0.083714    0.226262    0.942498                          0.039632  0.170825    0.015806     0.142037  0.309500  0.942498           04:46     
179       0.072813    0.211184    0.945893                          0.038751  0.156497    0.015936     0.130500  0.326000  0.945893           04:44     
180       0.081544    0.207568    0.955574                          0.036302  0.155196    0.016070     0.127553  0.295500  0.955574           04:44     
181       0.070548    0.210342    0.963228                          0.035853  0.158099    0.016390     0.129075  0.255000  0.963228           04:43     
182       0.078953    0.193935    0.947417                          0.036749  0.140748    0.016438     0.116685  0.253000  0.947417           04:44     
183       0.082916    0.198010    0.956960                          0.036024  0.145684    0.016302     0.119979  0.256000  0.956960           04:43     
184       0.074588    0.249540    0.944067                          0.039287  0.193984    0.016269     0.158684  0.270000  0.944067           04:41     
185       0.082477    0.215001    0.955480                          0.036390  0.162589    0.016022     0.133212  0.325000  0.955480           04:41     
186       0.078518    0.213460    0.947624                          0.036022  0.161354    0.016084     0.131948  0.299000  0.947624           04:50     
187       0.078276    0.212622    0.943323                          0.034626  0.161943    0.016054     0.131373  0.313000  0.943323           04:59     
188       0.102532    0.289659    0.929094                          0.036452  0.237158    0.016049     0.189158  0.302000  0.929094           04:59     
189       0.103186    0.294890    0.960607                          0.042828  0.236151    0.015911     0.193323  0.312000  0.960607           04:54     
190       0.084295    0.227399    0.942876                          0.038936  0.172526    0.015937     0.142660  0.338500  0.942876           04:54     
191       0.080735    0.198275    0.958823                          0.035594  0.146504    0.016176     0.120397  0.290000  0.958823           04:57     
192       0.081973    0.214822    0.949846                          0.035695  0.163057    0.016070     0.132994  0.296500  0.949846           04:57     
193       0.093528    0.236086    0.942166                          0.038842  0.181096    0.016148     0.148805  0.286500  0.942166           04:44     
194       0.076362    0.241348    0.949359                          0.036218  0.188955    0.016175     0.152704  0.283500  0.949359           04:47     
195       0.080417    0.330995    0.958705                          0.036631  0.278227    0.016137     0.220007  0.283000  0.958705           04:43     
196       0.084029    0.226262    0.948254                          0.038368  0.171705    0.016189     0.141365  0.276500  0.948254           04:46     
197       0.081667    0.210475    0.954809                          0.037743  0.156563    0.016168     0.129562  0.291000  0.954809           04:46     
198       0.076088    0.211807    0.928583                          0.039940  0.155625    0.016241     0.130432  0.269500  0.928583           04:46     
199       0.080302    0.223791    0.945904                          0.036197  0.171338    0.016257     0.139394  0.262500  0.945904           04:45     
Saved model as DA_RetinaNet 

Our aim is comparing our proposed methodology with yours.
So Could you help me with issue?

[maubreville]

Dear Süha Berk Kukuk,

so from what we can see the model apparently was fitted properly, but it is hard for us to tell from here because of which reason the model fails in inference.
Some suggestions might be:

• Normalization is not set up properly (i.e., equal to training)
• Data loader uses BGR instead of RGB data format
• Model selection did not work properly

Please be reminded that the threshold was optimized on the validation set and 0.64 is only the suitable value for our model, yours will be very likely different. This threshold also hugely depends on the sampling scheme that you use and on the loss function.

To us, it looks as if the non-maximum suppression is not working as intended. You might also want to evaluate the loss itself on the hold out set just to see if the data loader works in both cases similarly, or run inference on the training set, which should yield overly optimistic but sensible results.

Best regards,

Frauke and Marc