Crack　detection　is　significant　for　the　inspection　and　diagnosis　of　concrete　structures.　Various　automated　approaches　have　been　developed　to　replace　human-conducted　inspection,　many　of　which　are　not　adaptive　to　various　conditions　and　unable　to　provide　localization　information.　In　this　paper,　an　end-to-end　semantic　segmentation　neural　network　based　on　U-net　is　employed　to　detect　crack.　Due　to　the　limited　number　of　available　annotated　samples,　data　augmentation　is　employed　to　avoid　overfitting.　The　adopted　network　is　trained　by　only　200　images　of　512　×　512　pixels　resolutions　and　achieves　a　satisfactory　accuracy　of　99.56%　after　37　epochs.　The　output　is　an　image　of　the　same　size　as　the　input　image　where　each　pixel　is　assigned　a　class　label,　i.e.　crack　or　not　crack.　It　takes　about　7s　to　process　an　image　of　designed　size　on　CPU.　Combined　with　sliding　window　technique,　our　model　can　cope　with　any　image　of　larger　size.　Comparative　experiment　results　show　that　our　model　outperforms　traditional　Canny　and　Sobel　edge　detection　methods　in　a　variety　of　complex　environment　without　extracting　features　manually.　©　Springer　Nature　Switzerland　AG　2018.