This article investigates the effect of the cast surface topography on the fatigue strength of the cast aluminum alloy AlSi8Cu3, which is equivalent to EN AC-46200 and A380 according to ASTM. Fatigue tests utilizing small-scale specimens under bending at a load stress ratio of R = 0 reveal a significant influence of the cast surface layer, leading to a reduction of the nominal fatigue strength at ten million load cycles by about one half. Fracture surface analysis highlights that crack initiation occurs because of both surface roughness and micropores within the cast surface layer. A numerical simulation procedure to assess the influence of the surface roughness on the fatigue strength is presented. Thereby, a three-dimensional mesh of an optically scanned surface topography is set up and a local fatigue strength reduction factor is numerically computed based on the critical distance approach by Taylor. The results reveal that the presented method slightly overestimates the fatigue strength compared to the experiments; however, the determined values lead to a sound accordance if the scatter of the fatigue test data points is considered.