Here is what I have found so far.
1850 Owensboro, Daviss, Ky p. 347a
John H. bennett, 22, merchant, MD
A.F., 18, Ky
1850 Upper or Northern Dist. 2, Hardin, Ky, p. 312a
Thomas H. Duncan, 42, Merchant, $3000, Ky
Sally A., 12
William M., 2
1860 2nd Dist. elizabethtown, Hardin, p. 20
Thomas H. Duncan, 51, merchant
Joseph D., 17
Thomas H., 15
William M., 12
John L., 9
John Harry/Murry (?), 5
Agnes Bennett, 26, Milliner
Samuel M, 7
Bob D., 4
1870 Elizabethtown, Hardin, p. 341 a
Charles W. Yager, 32, Ret.(retail)merchant
Samuel Bennett, 17
Fox Duncan, 25, ret. merchant
Wallace Duncan, 22, clerk in store
Nancy English, 20, black, domestic servant
1880 Elizabethtown, Hardin, ed 60, p. 241b
Charles W. Yeager, 41, farmer
Agnes F., 45
Robert D., 23 locomotive fireman, Ky/MD/Ky
Thomas B., 12
R.F. Gardner, 26, clerk on R.R., MD
Fanny, 20, Ky
Frank Johnsan, 22, telegraph operator, La
Mary Thomas, 22, domestic servant
J.P. Hobsopn, 28, lawyer, 28
George Allhand, 23, telegraph operator, IN
Joseph Orr, 27, works in machine shop, PA
James M. Rihn, 26, retail grocer, Ky
Harry German, 19, works in machine shop, KY
Wiley Eadis, 26, black, domestic servant
*the most inportant info on the 1880 census is that robert D. bennett is now called Yeager and he lists his father as being born in Maryland.
1910 Elizabethtown, Dist. 6, Hardin, ed 60, p. 21b
Charles W. Yeager, 72, widowed, milliner merchant
christine E., daughter, 16, KY/Ky/NY
(Pearl?) Woodridge, niece, 22, milliner clerk
Julia ?, sister-in-lwa, 33, widow, she has had 2 children, 2 are living, IL/Denmark/Denmark, milliner
George W., McGinnis, 62, blacksmith
Mary, 57, dressmaker
*Looks like Charles married again after Agnes died? This census was very faded and hard to read. I looked at 3 different versions but this was as much as I could get.
Today I went to the National Archives and looked in the 1900 soundex for Charles but I couldn't find him. Not sure why unless he got skipped in the index or maybe the transcriber got his last name wrong. Again, some of the index cards were faded and there were lots to page through so it's possible I missed them. Heritage Quest is about to come out with a 1900 index so if you can wait, I'll take a look using that version.
I also found on FamilySearch website that Charles Yeager married 1/31/1861 in Hardin Co. There was no wife listed.
If I were you I would write to the Hardin co. Courthouse and see what they have by way of marriage records. They may also have a death record for John Bennett. I know it's not as fast as doing it over the internet but sometimes it's the only way you can get what you want. Most courthouses don't take long getting back to you.
If John Bennett served on the Union side in the Civil war and Agnes applied for a pension there copuld be quite a lot of information on John contained in that file. The downside about these pension files is that they are pricey($39)and they can take awhile to get back. I have sent for several and ALL have provided a wealth of info that I would not have been able to get anywhere else. Well worth the cost and wait in my view. The trick is finding out if a pension was applied for in his name.
Where did you get the information that he was in a prison camp in IL? If he was held in IL then he was most likely a Confederate. The only John bennett I could find in a prison camp was a John H. bennett of NC who was held at Elmira Prison, New York. Doesn't sound like yours. Maybe the person who gave you the information about his death in a prison could tell you where they got the information.
That's about it.