Kernel　target　alignment　is　a　very　efficient　evaluation　criterion.　It　has　been　widely　applied　in　kernel　optimization.　However　the　traditional　kernel　methods　that　based　on　the　Kernel　target　alignment　optimize　the　kernel　function　mainly　with　batch　gradient　descent　algorithm.　This　kind　of　methods　has　to　scan　through　the　entire　training　set　at　each　step,　which　is　much　too　costly.　The　On-line　learning　algorithm　exactly　solve　above　problem.　At　each　step,　on-line　learning　algorithm　only　need　one　example　then　discarded　after　learning,　which　make　on-line　learning　algorithm　fast,　simple,　and　often　make　few　statistical　assumptions.　Thus,　in　this　paper,　we　propose　a　novel　method　to　optimize　the　Gaussian　kernel　with　on-line　learning.　We　formulate　the　objective　criterion　for　kernel　optimization　based　on　kernel　target　alignment.　The　objective　criterion　can　be　proved　to　have　a　determined　global　minimum　point.　Then,　we　use　the　on-line　learning　algorithm　to　optimize　the　formulated　kernel　function.　In　addition,　in　order　to　get　an　appropriate　learning　rate　for　the　algorithm　to　accelerate　the　convergence　rate,　we　use　an　adaptive　rate　learning　method　to　optimize　the　kernel　function.　Finally,　we　evaluate　the　empirical　performance　of　the　proposed　kernel　optimization　method　on　ten　diverse　datasets.　The　experimental　results　show　that　the　proposed　method　is　more　effective　than　the　state-of-the-art　kernel　optimization　algorithms.　©　2015　IEEE.