This　paper　develops　a　neural　network　optimization　algorithm:　The　rectified　L2　regularization,　which　can　be　used　to　train　ternary　neural　networks　with　weights　of　all　layers　constrained　to-1,　0　and　+1.　It　will　analyze　how　to　set　the　learning　rate　and　penalty　coefficient　during　the　training　phase.　Compared　with　previous　approaches,　the　rectified　L2　regularization　algorithm　can　be　directly　implemented　on　the　open　source　machine　learning　framework,　such　as　TensorFlow　and　PyTorch.　The　accuracy　of　the　MNIST　and　Fashion-MNIST　test　datasets　is　99.40%　and　92.21%,　respectively,　which　is　close　to　the　state-of-the-art　accuracy　of　full　precision　neural　networks　with　the　model　compression　rate　guaranteed.　©　2019　IEEE.