In-network　machine　learning　is　a　promising　technology　that　offloads　machine　learning　models　onto　programmable　data　planes　to　enable　intelligent　decision-making　by　programmable　devices.　Such　advancement　empowers　security　applications　(e.g.,　intrusion　detection)　to　adapt　to　dynamic　network　changes　in　real　time　and　make　rational　decisions.　Existing　research　deploys　neural　network　models　in　a　distributed　way　on　programmable　data　planes,　with　the　aim　of　performing　real-time　inference　using　network-wide　compute　resources.　However,　existing　research　primarily　focuses　on　model　implementations,　with　little　attention　paid　to　the　negative　impact　on　the　efficiency　and　robustness　of　in-network　applications　introduced　by　the　inference　process.　We　propose　NDIF,　a　framework　for　performing　in-network　neural　network　inference　in　a　distributed　manner.　NDIF　enables　in-network　inference　on　arbitrary　programmable　devices,　with　each　device　autonomously　managing　its　inference　workload　based　on　available　resources.　Moreover,　new　inference　schemes　can　be　easily　deployed　by　writing　entries　into　programmable　devices　to　adapt　to　network　changes.　These　benefits　improve　the　efficiency　and　stability　of　the　in-network　inference　process,　thereby　enhancing　the　efficiency　and　robustness　of　in-network　applications　built　based　on　neural　network　models.　The　experiments　on　the　use　cases　of　anomaly　detection　and　packet　classification　demonstrate　that　NDIF　outperforms　previous　inference　frameworks　across　various　quality　of　service　(QoS)　metrics　while　maintaining　a　reasonable　cost.　©　2025　Elsevier　Ltd