In　object　detection,　the　imbalance　problem　often　occurs　when　the　number　of　training　samples　of　different　categories　varies　greatly,　or　multiple　loss　functions　need　to　be　minimized　which　is　harmful　to　the　performance　of　the　detector.　In　this　paper,　we　consider　that　the　imbalance　problem　can　be　implied　by　the　imbalance　of　gradient　distribution.　To　address　these　imbalance　issues,　we　analyze　the　gradient　of　cross-entropy　loss　and　propose　balanced　cross-entropy　(BLCE)　loss　and　balanced　binary　cross-entropy　(BBCE)　loss　for　solving　objective　imbalance　and　class　imbalance　issues　respectively.　The　BLCE　loss　significantly　reduces　the　overall　classification　loss　and　keeps　the　classification　loss　and　regression　loss　balanced.　Furthermore,　the　BBCE　loss　automatically　down-weight　the　contribution　of　inliers　during　training　and　rapidly　focus　the　model　on　outliers.　Ablation　studies　on　object　detection　and　image　classification　demonstrate　the　effectiveness　of　our　loss　function.　We　replace　the　corresponding　losses　in　Libra　R-CNN　and　evaluate　our　detector　on　the　COCO　test-dev.　Our　results　show　that　Libra　R-CNN　can　surpass　the　accuracy　of　many　existing　state-of-the-art　detectors　when　training　with　our　balanced　loss.　©　2020,　Springer　Nature　Switzerland　AG.