This　paper　proposes　a　privacy-preserving　retrieval　scheme　for　JPEG　images　based　on　local　variance.　Three　parties　are　involved　in　the　scheme:　the　content　owner,　the　server,　and　the　authorized　user.　The　content　owner　encrypts　JPEG　images　for　privacy　protection　by　jointly　using　permutation　cipher　and　stream　cipher,　and　then,　the　encrypted　versions　are　uploaded　to　the　server.　With　an　encrypted　query　image　provided　by　an　authorized　user,　the　server　may　extract　blockwise　local　variances　in　different　directions　without　knowing　the　plaintext　content.　After　that,　it　can　calculate　the　similarity　between　the　encrypted　query　image　and　each　encrypted　database　image　by　a　local　variance-based　feature　comparison　mechanism.　The　authorized　user　with　the　encryption　key　can　decrypt　the　returned　encrypted　images　with　plaintext　content　similar　to　the　query　image.　The　experimental　results　show　that　the　proposed　scheme　not　only　provides　effective　privacy-preserving　retrieval　service　but　also　ensures　both　format　compliance　and　file　size　preservation　for　encrypted　JPEG　images.　©　2017　SPIE　and　IS&T.