The　growing　number　of　complex　and　heterogeneous　Internet　of　Things　(IoT)　applications　has　imposed　a　high　demand　for　scarce　communications　and　computing　resources.　To　meet　this　stringent　requirement,　it　is　desirable　to　develop　large-scale　highly　adaptive　online　resource　allocation　strategies　to　streamline　existing　network　operations.　Deep　reinforcement　learning　(DRL),　which　combines　the　merits　of　reinforcement　learning　and　deep　learning,　is　capable　of　addressing　complex　decision-making　tasks,　thus　enabling　efficient　online　resource　allocation.　In　this　article,　we　present　a　DRL-based　resource　allocation　framework.　We　begin　a　discussion　on　DRL　basics　and　review　its　several　recent　applications.　Then,　we　develop　two　new　DRL　algorithms　that　facilitate　unlocking　the　potential　of　DRL　and　offer　viable　solutions　to　many　more　complex　resource　allocation　problems.　The　first　one　tackles　an　optimization　problem　exposed　to　mixed　(discrete　and　continuous)　action　spaces　and　bound　by　a　number　of　highly　non-linear　quality-of-service　(QoS)　constraints.　The　second　one　extends　the　single-agent　DRL　to　a　more　challenging　multi-agent　DRL　by　introducing　a　novel　semi-distributed　architecture.　Finally,　we　discuss　the　challenges　and　future　visions　of　applying　DRL　to　real-world　IoT　networks.