Automatic　optical　inspection　technology　(AOI)　is　a　visual　inspection　technology　that　has　developed　rapidly　in　recent　years.　The　high　speed　and　accuracy　of　AOI　can　greatly　enhance　the　efficiency　of　modern　industrial　production.　However,　when　this　technology　is　applied　to　optical　components　inspection,　it　encounters　a　challenge　that　the　specular　highlight　induces　over　or　under　exposure　during　the　imaging　process,　and　further　results　in　a　low　imaging　contrast　and　unclear　defect　details　of　the　targets.　To　solve　this　problem,　a　dark-field　polarization　imaging　setup　based　on　a　division-of-focal-plane　polarization　camera　was　adopted　to　achieve　high　contrast　defect　images.　Meanwhile,　algorithms　based　on　the　improved　LeNet-5　convolutional　neural　network　were　developed　to　recognize　the　defects.　An　accuracy　above　99.5%　was　obtained　for　the　distinction　of　defective　and　non-defective　samples,　and　an　accuracy　of　94.4%　was　reached　for　the　various　defect　classification.　Our　work　demonstrated　an　effective　application　of　polarization　imaging　and　machine　learning　in　AOI　of　optical　components　manufacturing.