Also found them in the 1880 census. They are in District 6, McNairy County, TN.
It has James Kizer age 56 (b. 1824) Mary Ann age 37 William F age 8 Josiah age 5
It lists James' birthplace as Alabama. His parents are listed as b. NC.
Mary's birthplace is listed as Miss.
I also found the marriage listing for James and Mary. Looks like they married 27 March 1872 in McNairy.
There is a James Kizer listed in the 1850 (this is the one that is living next to my John Kizer, and I assume they are related somehow...either brothers, cousins, uncle/nephew). Anyway....not sure if this is your James, but the age is similar and birth location. Here's the info:
James Kizer age 23 Clarinda age 21 James age 11months Margaret age 29 Eliza age 6 Michael age 70
I think it's likely that this is your James. The age matches and both list Alabama as birth location. I think Michael is probably his father (and in this census, it lists NC as his birth location, which matches up what James said in the 1880 census).
James came from Alabama either with his wife Clarinda (or married in McNairy co, TN). Something probably happened to her and the son (most likely they died). I'm guessing that this Margaret is probably a sister-in-law to James and Eliza is her daughter. James re-married to Mary Ann Williams in 1872. We can try looking in the 1860 and 1870 census records. I'll tell you that much of the McNairy county census records in 1870 are poor quality. Very hard to read...so might be a challenge.
A lot of this is just an educated guess on what happened. I think it's unlikely that James married for the first time at age 48. Unfortunately, the farther back you go...the harder it is to find birth/death records.
I'm not sure I'll (or anyone else) will be able to connect your Kizer line with my Kizer line. However, I'm certain there is a connection. It can not be coincidence that James Kizer moves right next to John Kizer. I've been looking for that connection, mainly because I want to figure out if the Michael living with James is also John's father.