In　the　field　of　computer　vision,　Video　captioning　is　a　very　important　and　meaningful　task,　which　could　automatically　generate　textual　descriptions　of　contents　from　videos.　It　is　a　challenging　problem　due　to　the　difficulties　of　understanding　the　objects　and　activities　in　a　video.　Benefit　from　the　rapid　development　of　deep　learning　technology,　e.g.　sequence　to　sequence　model,　video　captioning　task　has　achieved　very　accurate　results.　However,　there　are　two　serious　flaws,　the　first　is　that　the　pre-trained　deep　models　are　often　used　as　visual　feature　abstractors　as　the　training　is　highly　time-consuming,　so　the　feature　generalization　performance　generated　by　these　pre-trained　Encoder　is　limited　when　we　directly　employ　those　networks　in　video　captioning　tasks.　The　second　is　that　each　frame　in　the　video　is　processed　separately,　ignoring　the　correlation　of　video　data　in　the　time　dimension.　In　this　work,　we　propose　video　captioning　model　with　attention-memory　module　to　explore　the　role　of　capturing　temporal　correlations　which　with　sequence　to　sequence　model　as　the　background　and　showing　the　importance　of　temporal　structure　to　vision　tasks　by　adding　the　correlation　of　videos　when　extracting　features　and　enhancing　the　time-memory　capability.　Our　experiments　are　based　on　two　most　famous　benchmark　datasets　in　the　field　of　video　captioning:　MSVD　and　MSR-VTT.　Then　employ　BLEU　and　METEOR　to　evaluate　the　accuracy　of　the　description　generated　by　different　methods.　Finally,　the　experimental　results　confirm　that　the　proposed　model　could　make　significant　improvements　in　description　results　compared　with　the　baseline　models.　©　2021　IEEE.