With　each　advancement　in　internet　technology,　new　security　challenges　arise.　The　prevalence　of　malicious　programs　continues　to　increase,　which　makes　it　crucial　to　detect　and　address　them　effectively.　Many　researchers　focus　on　solving　different　datasets　by　using　deep　learning　methods　and　make　significant　progress.　However,　these　strategies　must　be　continuously　improved　to　adapt　to　the　latest　data.　In　this　paper,　an　improved　model　based　on　CNN-LSTM　is　proposed　to　detect　and　classify　malware　programs,　named　malDetect　I.　At　the　same　time,　the　Transformer　Encoder　module　is　also　modified　based　on　model　Bert　to　adapt　to　the　classification　task.　Lastly,　two　models　are　compared　with　prediction　results　on　evaluation　indicators.　The　data　used　in　this　paper　is　the　Windows　API　sequence　extracted　after　dynamic　operation.　The　text　processing　methods　are　also　suitable　for　processing　sequence　data.　The　experiment　uses　Word2Vec　and　two　different　learning　rate　strategies,　and　the　improved　model　accuracy　is　9.83%　higher　than　the　original　CNN-LSTM　model.　The　model　integrated　the　BiLSTM　model　with　the　Self-Attention　mechanism,　named　malDetect　II,　is　11.46%　higher　than　the　basic　model　CNN-LSTM　and　2.82%　higher　than　the　Transformer　Encoder　classification　model.　©　2024　IEEE.