Never really had a need for color. It's up to you, most medical charts we have are printed from a Laser printer which is black and white anyways. Occasionally we will find blue ink on the paper but we are okay with not preserving the integrity of the pen ink.
Also, a lot of people use ScanSnap which auto-detects color and switches appropriately and increases the resolution if there is an image.