The　increasing　demand　for　on-device　deep　learning　necessitates　the　deployment　of　deep　models　on　mobile　devices.　However,　directly　deploying　deep　models　on　mobile　devices　presents　both　capacity　bottleneck　and　prohibitive　privacy　risk.　To　address　these　problems,　we　develop　a　Differentially　Private　Knowledge　Distillation　(DPKD)　framework　to　enable　on-device　deep　learning　as　well　as　preserve　training　data　privacy.　We　modify　the　conventional　Private　Aggregation　of　Teacher　Ensembles　(PATE)　paradigm　by　compressing　the　knowledge　acquired　by　the　ensemble　of　teachers　into　a　student　model　in　a　differentially　private　manner.　The　student　model　is　then　trained　on　both　the　labeled,　public　data　and　the　distilled　knowledge　by　adopting　a　mixed　training　algorithm.　Extensive　experiments　on　popular　image　datasets,　as　well　as　the　real　implementation　on　a　mobile　device　show　that　DPKD　can　not　only　benefit　from　the　distilled　knowledge　but　also　provide　a　strong　differential　privacy　guarantee　(=2$)　with　only　marginal　decreases　in　accuracy.　©　2020　ACM.