In fact, the average distance of Earth from the sun is increasing by about 15 cm per year:That’s roughly half a foot. It’s also fairly close to one trillionth the average separation.
If this distance were the only factor that determined the average temperature of the Earth, the Earth would be cooling at a tiny rate. In fact, the Earth has been warming quite a bit faster than this hypothetical cooling could be.
But taking your question literally and using the fact that radiant power drops off as the inverse square of the distance and using the binomial approximation to get the change in temperature, it would be about 6 x 10^-10 degrees C.
That explains why we can’t detect any change from the small change in distance per year.