When　the　existing　automatic　portrait　coloring　algorithms　are　directly　applied　to　scene　sketches,　distortion　phenomena　are　caused,　such　as　wrong　colorization　and　checkerboard　artifacts,due　to　the　diversified　line　semantics　of　scene　sketches.　To　address　this　issue,　an　automatic　colorization　algorithm　with　anime　effect　for　scene　sketches　is　put　forward.　The　structure　of　U-Net　generator　in　the　existing　automatic　portrait　coloring　algorithms　is　improved　and　enhanced　based　on　the　conditional　generative　adversarial　network.　A　double-layer　information　extraction　U-Net(DIEU-Net)　is　designed　for　automatic　anime　effect　colorization　of　scene　sketches.　Firstly,　the　double-convolution　sub-module　prominence-information　extraction　of　a　scene　sketch(IESS)　is　designed.　Then,　a　module　integrating　double-layer　IESS　and　residual　structure　is　inserted　into　different　stages　of　the　proposed　generator.　Thus,　the　global　learning　ability　of　the　generator　on　important　features,　like　colors　and　positions　related　to　the　sketch,　are　enhanced,　and　the　network　degradation　problems　caused　by　vanishing　gradients　as　the　network　deepens,　are　alleviated.　Moreover,　the　deconvolution　in　U-Net　is　replaced　by　the　operations　of　convolution　and　upsample　to　suppress　the　occurrence　of　the　checkerboard　artifacts.　Experimental　results　show　that　the　proposed　algorithm　performs　well　in　avoiding　the　distortion　phenomenon　and　achieves　more　reasonable　and　natural　coloring　effect　than　other　algorithms.　Furthermore,　the　proposed　algorithm　can　be　applied　to　automatic　anime　coloring　of　various　types　of　scene　sketches.　©　2020,　Science　Press.　All　right　reserved.