Its possible that your Harvey O. Carroll is really Harvey A. CArroll. I've seen that letter 'A' in census'records and it looked remarkably like an "O". Also, it is very confusing with the "Harvey" and "James'" names. They seem to alternate for several generation. So Harvey's fathers name was James, then James' father's name was Harvey, etc.
Do you have any dates on your Harvey? Please email me directly. Thanks