Most slowness for those with decreased memory comes from the RAM devoted to the cache. So, rebooting the server gives the memory back to the OS. Filling up SQL's cache doesn't slow down your computer except due to the RAM being used. So, if you only have 300MBs allotted to it, and it is filled, rebooting will not give back as much memory.
