I would agree with the ID assessment of the culture results. When you have different organisms in different bottles, it sounds like a contaminant. When both organisms are skin colonizers, that seems like a slam-dunk. Gene, I hope it wasn't you who didn't wash well before drawing the cultures.
On the other hand, I wouldn't ignore the apparent partial esponse to the antibiotics.
Keep in mind that CAT scans of the abdomen are not "all-seeing". They can certainly miss diverticulitis as well as a colon cancer with micro-perforation. Was there diverticulosis on the CT?
74 year old man with right lower quadrant pain, acting like he is infected with a partial response to antibiotics? I would do a colonoscopy, looking for one of the above two diagnoses.
I guess to this hammer, they all look like nails.