Hold on the server. AC4 has some issues with servers, it is harder to install and I had trouble with backup. I have to agree that the issue is probably wireless.
At home, I found wireless N made a huge difference. You might be able to upgrade to N standard for about $100 by shopping around, but first I would try that John above has recommended to see if that is even necessary.
