As　the　workloads　and　service　requests　in　cloud　computing　environments　change　constantly,　cloud-based　software　services　need　to　adaptively　allocate　resources　for　ensuring　the　Quality-of-Service　(QoS)　while　reducing　resource　costs.　However,　it　is　very　challenging　to　achieve　adaptive　resource　allocation　for　cloud-based　software　services　with　complex　and　variable　system　states.　Most　of　the　existing　methods　only　consider　the　current　condition　of　workloads,　and　thus　cannot　well　adapt　to　real-world　cloud　environments　subject　to　fluctuating　workloads.　To　address　this　challenge,　we　propose　a　novel　Deep　Reinforcement　learning　based　resource　Allocation　method　with　workload-time　Windows　(DRAW)　for　cloud-based　software　services　that　considers　both　the　current　and　future　workloads　in　the　resource　allocation　process.　Specifically,　an　original　Deep　Q-Network　(DQN)　based　prediction　model　of　management　operations　is　trained　based　on　workload-time　windows,　which　can　be　used　to　predict　appropriate　management　operations　under　different　system　states.　Next,　a　new　feedback-control　mechanism　is　designed　to　construct　the　objective　resource　allocation　plan　under　the　current　system　state　through　iterative　execution　of　management　operations.　Extensive　simulation　results　demonstrate　that　the　prediction　accuracy　of　management　operations　generated　by　the　proposed　DRAW　method　can　reach　90.69%.　Moreover,　the　DRAW　can　achieve　the　optimal/near-optimal　performance　and　outperform　other　classic　methods　by　3~13%　under　different　scenarios.　©　2022　IEEE.