Classical　multi-channel　algorithms　usually　employ　minority　discrete　channels.　It　is　considered　that　the　channels　are　independent　and　their　multi-mountain　like　distribution　is　neglected.　Double-layer　particle　filter　for　target　tracking　with　multi-channel　energy　extreme　points　set　method　is　proposed.　Visual　target　is　firstly　decomposed　to　multiple　frequency　and　angle　channels.　And　then　energy　extreme　points　set　is　calculated　as　target　feature.　Double-layer　particle　filter　algorithm　framework　is　established.　Target　motion　state　is　globally　constrained　by　first-order　particle.　Second-order　particles　represent　the　accessory　multi-channel　features　and　cluster　to　feature　peak　by　local　optimization　method.　Multi-channel　features　are　examined　and　then　second-order　particles　are　weighted.　The　result　is　feedback　to　first-order　particles　as　weighting　parameters.　Thus　global　constraint　and　multi-feature　reliability　distribution　are　balanced.　Experimental　results　show　that　the　algorithm　runs　steadily　with　moderate　computational　complexity.　Compared　to　correlation　methods,　the　average　tracking　error　is　decreased　by　about　0.5　pixels.