Establishing　reliable　correspondences　by　a　deep　neural　network　is　an　important　task　in　computer　vision,　and　it　generally　requires　permutation-equivariant　architecture　and　rich　contextual　information.　In　this　paper,　we　design　a　Permutation-Equivariant　Split　Attention　Network　(called　PESA-Net),　to　gather　rich　contextual　information　for　the　feature　matching　task.　Specifically,　we　propose　a　novel　＂Split-Squeeze-Excitation-Union＂(SSEU)　module.　The　SSEU　module　not　only　generates　multiple　paths　to　exploit　the　geometrical　context　of　putative　correspondences　from　different　aspects,　but　also　adaptively　captures　channel-wise　global　information　by　explicitly　modeling　the　interdependencies　between　the　channels　of　features.　In　addition,　we　further　construct　a　block　by　fusing　the　SSEU　module,　Multi-Layer　Perceptron　and　some　normalizations.　The　proposed　PESA-Net　is　able　to　effectively　infer　the　probabilities　of　correspondences　being　inliers　or　outliers　and　simultaneously　recover　the　relative　pose　by　essential　matrix.　Experimental　results　demonstrate　that　the　proposed　PESA-Net　relative　surpasses　state-of-the-art　approaches　for　pose　estimation　and　outlier　rejection　on　both　outdoor　scenes　and　indoor　scenes　(i.e.,　YFCC100M　and　SUN3D).　Source　codes:　https://github.com/x-gb/PESA-Net.