Since training use sentences similar to gold target as label 1, others as label 0. And it will lead to the imbalance of label 1 and label 0. It may cause a ratio which maybe1 vs 6 or even more. And we only use normal BCE loss. Is it a little strange that training still works?