In this study, we evaluate a coronal mass ejection (CME) arrival prediction tool that utilizes the wide‐angle observations made by STEREO's heliospheric imagers (HI). The unsurpassable advantage of these imagers is the possibility to observe the evolution and propagation of a CME from close to the Sun out to 1 AU and beyond. We believe that by exploiting this capability, instead of relying on coronagraph observations only, it is possible to improve today's CME arrival time predictions. The ELlipse Evolution model based on HI observations (ELEvoHI) assumes that the CME frontal shape within the ecliptic plane is an ellipse and allows the CME to adjust to the ambient solar wind speed; that is, it is drag based. ELEvoHI is used to perform ensemble simulations by varying the CME frontal shape within given boundary conditions that are consistent with the observations made by HI. In this work, we evaluate different setups of the model by performing hindcasts for 15 well‐defined isolated CMEs that occurred when STEREO was near L4/5, between the end of 2008 and the beginning of 2011. In this way, we find a mean absolute error of between 6.2 ± 7.9 and 9.9 ± 13 hr depending on the model setup used. ELEvoHI is specified for using data from future space weather missions carrying HIs located at L5 or L1. It can also be used with near‐real‐time STEREO‐A HI beacon data to provide CME arrival predictions during the next ∼7 years when STEREO‐A is observing the Sun‐Earth space.