With　time-varying　workloads　and　service　requests,　cloud-based　software　services　necessitate　adaptive　resource　allocation　for　guaranteeing　Quality-of-Service　(QoS)　and　reducing　resource　costs.　However,　due　to　the　ever-changing　system　states,　resource　allocation　for　cloud-based　software　services　faces　huge　challenges　in　dynamics　and　complexity.　The　traditional　approaches　mostly　rely　on　expert　knowledge　or　numerous　iterations,　which　might　lead　to　weak　adaptiveness　and　extra　costs.　Moreover,　existing　RL-based　methods　target　the　environment　with　the　fixed　workload,　and　thus　they　are　unable　to　effectively　fit　in　the　real-world　scenarios　with　variable　workloads.　To　address　these　important　challenges,　we　propose　a　Prediction-enabled　feedback　Control　with　Reinforcement　learning　based　resource　Allocation　(PCRA)　method.　First,　a　novel　Q-value　prediction　model　is　designed　to　predict　the　values　of　management　operations　(by　Q-values)　at　different　system　states.　The　model　uses　multiple　prediction　learners　for　making　accurate　Q-value　prediction　by　integrating　the　Q-learning　algorithm.　Next,　the　objective　resource　allocation　plans　can　be　found　by　using　a　new　feedback-control　based　decision-making　algorithm.　Using　the　RUBiS　benchmark,　simulation　results　demonstrate　that　the　PCRA　chooses　the　management　operations　of　resource　allocation　with　93.7　percent　correctness.　Moreover,　the　PCRA　achieves　optimal/near-optimal　performance,　and　it　outperforms　the　classic　ML-based　and　rule-based　methods　by　5　similar　to　7%　and　10　similar　to　13%,　respectively.