Locating the minimum energy structure of molecules, typically referred to as geometry optimization, is one of the first steps of any computational chemistry calculation. Earlier research was mostly dedicated to finding convenient sets of molecule-specific coordinates for a suitable representation of the potential energy surface, where a faster convergence toward the minimum structure can be achieved. More recent approaches, on the other hand, are based on various machine learning techniques and seem to revert to Cartesian coordinates instead for practical reasons. We show that the combination of Gaussian process regression with those coordinate systems employed by state-of-the-art geometry optimizers can significantly improve the performance of this powerful machine learning technique. This is demonstrated on a benchmark set of 30 small covalently bonded molecules.