SP3 is everything plus a few fairly inconsequential security updates. There are only four new updates. Removing SP3 will only remove the four extra updates. You can remove it from Add/Remove programs.
Personally, I would try it with SP3. I know I read what H. O. wrote, but is that confirmed as a bug -- not being able to install with SP3? Sorry, if that is misleading. Try it with one computer. If you don't have VM, the new Acronis Home -- I think version 11 has this really cool feature on it where you can do anything and test it before clicking a button to make it permanent.
