'Downhill' - whilst it may seem easier - often has the meaning of heading for an inevitable end. For example, life is sometimes seen as a mountain...we climb up the slope towards middle age and, once past that. "It's all downhill"...ie finishing up in the grave! The idea is that there is no going back and the end is inescapable.
This is a trick question... Where I'm from, anyway, "it's all downhill from here" is an indication that you have already past the obstacle, and that it really is easy from here on out.