Sugrufy Your Xbox Controller
I don't know how but the rubber grip on the xbox controller joysticks wears off. Especially, the left joystick.
Fortunately, Sugru can fix that.
Apply Sugru
First make sure the joystick is clean. Remove any residue rubber.
Open a (mini) pack of sugru and roll it into a sphere.
Flatten it and place on joystick.
Smooth-en out the edges and the surface.
Finish
The final result should look the the attached photo.
Make sure the Sugru dries before you use your controller.