How can I get the evaluation benchmark ？

[xdliu1998]

Hi @guxd

It is mentioned that two developers manually labelled relevant/non-relevant results for the top 50 stackoverflow questions used as benchmark. Can I get the manually labelled 10 results of each query？

Thanks，
lxd

[guxd]

please check #8

[xdliu1998]

please check #8

I have got the labeled results in keras/results and train the model.
What I need is a sorted code segment or code segment id corresponding to 50 queries. Such as
query :convert an inputstream to a string
result: segment1,segment2,...segment10
Can I get this?

[guxd]

Unfortunately, the raw returned results were not stored. The results were labeled from the demo server directly. We are trying to set up the original demo service to reproduce the returned results shown in the paper.