I could be wrong, but II is in a separate folder and not in a database. So I would think you are just pulling it directly from that folder. Since, I don't use II, I don't see why AC would cache those every time you pull a patient. It would make more sense for AC to access the document when you need it.
If you are sitting right at the server then it doesn't matter if it is peer to peer or Client/Server or anything. If it is on an SSD and it is slow then that is concerning. I am running mine in a VM with 2 GBs of RAM, and it pops right up. But, not over the network.
But, if you want to test it, make a dummy patient, make one note with only a CC, no meds, FH, exam, anything. No II. Then pull it. If it is slow, then it is AC.