I don't think the Isaac in 1870 is the one you are looking for - that one was 28 years old and his wife's name was something like Elgean, age 23, daughter Esther, age 2.
I did find yours in 1860 in Canton, Van Zandt Co., TX: Isaac Christmas, age 58, b. MS Nancy, age 56, b. LA B.S., age 16, b. MS William S., age 14, b. MS
Here they are in 1850 - Harrison Co., MS: Isaac, age 49, b. MS Nancy, age 48, b. LA Francis (male), age 20 John, age 17 Rebeca, age 12 Benjamin, age 7 William, age 4 Henry Brannon, age 28 Martha Brannon (daughter??), age 22 William L. Brannon, age 4 Elizabeth Brannon, age 3 James Brannon, age 3 mos.