Image　Super-Resolution　(SR)　techniques　improve　visual　quality　by　enhancing　the　spatial　resolution　of　images.　Quality　evaluation　metrics　play　a　critical　role　in　comparing　and　optimizing　SR　algorithms,　but　current　metrics　achieve　only　limited　success,　largely　due　to　the　lack　of　large-scale　quality　databases,　which　are　essential　for　learning　accurate　and　robust　SR　quality　metrics.　In　this　work,　we　first　build　a　large-scale　SR　image　database　using　a　novel　semi-automatic　labeling　approach,　which　allows　us　to　label　a　large　number　of　images　with　manageable　human　workload.　The　resulting　SR　Image　quality　database　with　Semi-Automatic　Ratings　(SISAR),　so　far　the　largest　of　SR-IQA　database,　contains　12　600　images　of　100　natural　scenes.　We　train　an　end-to-end　Deep　Image　SR　Quality　(DISQ)　model　by　employing　two-stream　Deep　Neural　Networks　(DNNs)　for　feature　extraction,　followed　by　a　feature　fusion　network　for　quality　prediction.　Experimental　results　demonstrate　that　the　proposed　method　outperforms　state-of-the-art　metrics　and　achieves　promising　generalization　performance　in　cross-database　tests.　The　SISAR　database　and　DISQ　model　will　be　made　publicly　available　to　facilitate　reproducible　research.　©　1999-2012　IEEE.