Defect　detection　plays　a　crucial　role　in　ensuring　the　safety　and　longevity　of　structures,　with　defect　region　classification　particularly　beneficial　for　focusing　efforts　on　potential　defect　areas.　Traditional　deep　convolutional　neural　networks　(DCNNs)　based　defect　classification　networks　still　have　a　high　number　of　parameters　and　computational　demands,　making　them　unsuitable　for　embedded　systems.　This　paper　proposes　the　Adaptive　Prior　Activation-Based　Binary　Information　Enhancement　Network　(AOIE-Net),　which　significantly　reduces　computational　requirements　by　binarizing　weights　and　activations.　Specifically　designed　for　steel　defect　detection,　AOIE-Net　optimizes　the　binary　quantization　process　and　enhances　feature　representation　to　improve　the　performance　of　BNNs　in　steel　defect　detection　tasks.　AOIE-Net　introduces　a　Dual　Batch　Normalization-based　Information　Enhancement　Block　(DBN-IEB)　and　an　Adaptive　Binary　Activation　Independent　Optimization　(ABA-IO)　method　to　reduce　computational　complexity　while　boosting　classification　accuracy.　Experimental　results　demonstrate　that　AOIE-Net　outperforms　state-of-the-art　binary　neural　network　models　on　CIFAR-10,　ImageNet,　and　the　NEU-CLS　steel　defect　dataset,　achieving　classification　accuracy　of　90.6%,　72.1%,　and　99.4%,　respectively.　The　proposed　method　offers　an　efficient,　low-complexity　solution　for　real-time　defect　classification　in　large-scale　structural　inspections　and　holds　significant　potential　for　practical　applications.