Contact-rich　manipulation　tasks　are　difficult　to　program　to　be　performed　by　robots.　Traditional　compliance　control　methods,　such　as　impedance　control,　rely　excessively　on　environmental　models　and　are　ineffective　in　the　face　of　increasingly　complex　contact　tasks.　Reinforcement　learning　(RL)　has　now　achieved　great　success　in　the　fields　of　games　and　robotics.　Autonomous　learning　of　manipulation　skills　can　empower　robots　with　autonomous　decision-making　capabilities.　To　this　end,　this　work　introduces　a　novel　learning　framework　that　combines　deep　RL　(DRL)　and　variable　impedance　control　(VIC)　to　achieve　robotic　massage　tasks.　A　skill　policy　is　learned　in　joint　space,　which　outputs　the　desired　impedance　gain　and　angle　for　each　joint.　To　address　the　limitations　of　the　sparse　reward　of　DRL,　an　intrinsic　curiosity　module　(ICM)　was　designed,　which　generates　the　intrinsic　reward　to　encourage　robots　to　explore　more　effectively.　Simulation　and　real　experiments　were　performed　to　verify　the　effectiveness　of　the　proposed　method.　Our　experiments　demonstrate　that　contact-rich　massage　skills　can　be　learned　through　the　VIC-DRL　framework　based　on　the　joint　space　in　a　simulation　environment,　and　that　the　ICM　can　improve　learning　efficiency　and　overall　performance　in　the　task.　Moreover,　the　generated　policies　have　been　demonstrated　to　still　perform　effectively　on　a　real-world　robot.　©　2015　IEEE.