When　network　is　undergoing　problems,　such　as　DDoS　attack,　component　failures,　etc.,　the　detection　of　heavy　flows　(e.g.　heavy　hitters　and　heavy　changers)　is　much　more　critical.　However,　it　has　been　increasing　challenging　to　ensure　the　accurate　detection　of　heavy　flows　while　dealing　with　massive　network　traffic　volume,　diversified　traffic　distribution　and　the　stringent　memory　requirement.　Although　recent　research　efforts　like　LD-Sketch　are　scalable　for　diverse　network　traffic,　they　depend　on　excessive　memory　to　maintain　high　accuracy,　such　that　they　fail　to　work　well　when　the　memory　is　limited.　We　propose　RL-Sketch,　a　adaptive　sketch　using　reinforcement　learning　in　detecting　heavy　flows.　It　predicts　potential　heavy　flows　based　on　the　statistics　of　network　traffic,　to　achieve　both　high　accuracy　and　scalability　with　minor　memory.　Trace-driven　evaluation　shows　that　RL-Sketch　achieves　higher　accuracy　than　state-of-the-art　sketch-based　technologies　with　up　to　17.79×　accuracy　gain,　while　maintaining　high　robustness　in　extreme　conditions.　©　2019　IEEE.