I agree, but having gotten so used to AP, I will frequently just crank it in the direction I need to give me the shutter speed I want rather than switching.
Problem for me switching to manual, is I forget to switch back. Live view has saved me from that now that I have gone Mirrorless. I used to chimp shots and couldn't figure out what ‘the camera was doing wrong’ when I forgot and left it in manual.