The　road　centerline　extraction　is　the　key　step　of　the　road　network　extraction　and　modeling.　The　hand-craft　feature　engineering　in　the　traditional　road　extraction　methods　is　unstable,　which　makes　the　extracted　road　centerline　deviated　from　the　road　center　in　complex　cases　and　even　results　in　overall　extracting　errors.　Recently,　the　road　centerline　extraction　methods　based　on　semantic　segmentation　employing　deep　neural　network　greatly　outperformed　the　traditional　methods.　Nevertheless,　the　pixel-wise　labels　for　training　deep　learning　models　are　expensive　and　the　postprocess　of　road　segmentation　is　error-prone.　Inspired　by　the　work　of　human　pose　estimation,　we　propose　DeepWindow,　a　novel　method　to　automatically　extract　the　road　network　from　remote　sensing　images.　DeepWindow　uses　a　sliding　window　guided　by　a　CNN-based　decision　function　to　track　the　road　network　directly　from　the　images　without　the　prior　of　road　segmentation.　First　of　all,　we　design　and　train　a　CNN　model　to　estimate　the　road　center　points　inside　a　patch.　Then,　the　road　seeds　are　automatically　searched　patch　by　patch　employing　the　CNN　model.　Finally,　starting　from　seeds,　our　method　first　estimates　the　road　direction　using　a　Fourier　spectrum　analysis　algorithm　and　then　iteratively　tracks　the　road　center-line　along　the　road　direction　guided　by　the　CNN　model.　In　our　method,　the　CNN　model　is　trained　by　point　annotations,　which　greatly　reduces　the　training　costs　comparing　to　those　in　semantic　model　training.　Our　method　achieves　comparable　performance　with　the　state-of-the-art　road　extraction　methods,　and　extensive　experiments　indicate　that　our　method　is　robust　to　the　point　deviation.　©　2008-2012　IEEE.