Regarding the Accuracy Issue in Open-Set

[Wenbo-Qin]

Hello, thank you for your repo. I used your repo to run the Open-set mode for MSSA and MFSAN, but I found that their prediction accuracy for the unknown class is 0%. The final results map all unknown classes into existing categories and do not output -1 as expected. I’m not sure how to modify the code to improve the model’s prediction performance for unknown classes. Could you please provide some guidance? Thanks so much.

[Feaxure-fresh]

Actually, the open-set mode is useful only for methods with open-set classification ability, such as IWAN, AFN, and UDA, which typically output -1 for unknown classes. For standard methods like MSSA and MFSAN, the accuracy for unknown classes is 0%, as they lack the ability to classify them. Our code also provides their accuracy and Closed-set accuracy (which excludes categories present in the target domain but not the source domain) for comparison.

[Wenbo-Qin]

Thank you. I found the disadvantage of that so I added an OpenMax module for each model used in closed-set so that it can be used to classify the unknown classes :p