Existing　saliency　detection　evaluation　metrics　often　produce　inconsistent　evaluation　results.　Because　of　the　widespread　application　of　image　saliency　detection,　we　propose　a　meta-metric　to　evaluate　the　performance　of　these　metrics　based　on　the　preference　of　an　application　that　uses　saliency　maps　as　weighting　maps.　This　study　uses　content-based　image　retrieval　(CBIR)　as　the　representative　application.　First,　we　perform　CBIR　using　image　features　extracted　from　deep　convolutional　layers　of　convolutional　neural　networks　as　well　as　saliency　maps　computed　by　various　saliency　detection　algorithms　as　the　weighting　maps　over　queries.　Second,　we　establish　the　preference　order　of　the　saliency　detection　algorithms　in　the　CBIR　application　by　sorting　the　mean　average　precision.　Third,　we　determine　the　preference　order　of　these　algorithms　using　existing　saliency　detection　evaluation　metrics.　Finally,　our　meta-metric　evaluates　these　metrics　by　correlating　the　preference　order　in　the　CBIR　application　with　that　determined　by　each　evaluation　metric.　Experiments　on　three　publicly　available　datasets　show　that,　of　24　evaluation　metrics,　the　traditional　metric:　area　under　receiver　operating　characteristic　curve　is　the　best　metric　for　a　CBIR　application.　©　2018,　Springer　Science+Business　Media,　LLC,　part　of　Springer　Nature.