Interesting. Do you use Powertop or TLP, by any chance? Some power utilities will turn on USB power saving if there is no activity on a USB port for a while, which can cause issues with USB mice. Generally I turn off that specific setting, or I believe there might be some way to whitelist certain USB devices to not have this sleep behavior.
I am not sure, then. If you are on a laptop, you might try one or the other of those utilities regardless simply because they can improve battery life, but that is a separate issue.