You can't put home versions of OS on a domain. At least not XP and I don't think Vista's. Basically, that is what makes them a home version. Microsoft leaves out the networking piece. It's probably on the disk just won't let you install it. Personally, if you are starting over, I would just get WIN7 Business and be done with it.
What bit OS are you using. 32 or 64? If 64, then I would increase the RAM. If 32, you can't.
I will help along the way.
