Performance of cell-level clustering

[Nathalia Kim]

Hello,

I've been trying to reproduce the results of cell-level clustering from the paper.

According to Table II, for dataset A, your method achieved purity = 0.855, entropy = 0.750 and f-score = 0.863.

I have run your code with different configurations, but for some reason, I can't achieve the same results as those reported in the paper.

• According to the paper, the model was trained with n_epoch = 10, batchsize = 64, lr = 2e-4

  • With these parameters, I get purity = 0.511, entropy = 1.548 and f_score = 0.589

• Using the defaults of the code: n_epoch = 50, batchsize = 10, lr = 1e-4

  • I get purity = 0.725, entropy = 1.137 and f_score = 0.732

I also ran the code with other configurations to investigate:

• n_epoch = 10, batchsize = 64, lr = 1e-4

  • purity = 0.535, entropy = 1.451, f_score = 0.617

• n_epoch = 50, batchsize = 64, lr = 1e-4

  • purity = 0.668, entropy = 1.087, f_score = 0.689

• n_epoch = 50, batchsize = 10, lr = 2e-4

  • purity = 0.725, entropy = 1.145, f_score = 0.733

It seems that for all the runs, my results are significantly lower than the results you got. Could you please provide me some guidance on what could be causing this discrepancy? Any thoughts would be highly appreciated!

Thank you,
Nathalia

[bohu615]

Hello Nathalia,

I would suggest you produce generated images for each cluster and investigate whether it is a good representation. You can also divide images into clusters and investigate visually.

It is difficult to evaluate performance by arbitrary parameters.

Could you also run longer time? Please also refer to the learning curve.

But the most straightforward way is to see if the produced images are a good representation.

Best,
Bo

[Nathalia Kim]

Hello Bo,

The parameters I mentioned in my last comment were only to serve as examples, but I actually trained the model with all the parameter values from the paper and the default values of the code and wasn't able to achieve the same results as reported in the paper. So my question wasn't meant to be about arbitrary parameters, but why do you think this might be the case even using these two sets of parameters? (paper and defaults)

Thank you,
Nathalia

[bohu615]

I cannot answer questions like:

'Why the results are not good enough' or 'I changed the parameters this way but the results are not good enough.'

I can answer questions such as:

'This is my cell-level representations, what went wrong?' or 'the codes are outdated what should I do to make it work.'

I'm not provided with enough information. I don't know how to answer you specifically.

I'm working on other projects so please raise questions about the mathematics, codes, and experiment details specifically. I cannot answer questions just given the numbers. I can only answer questions if I'm provided enough information.