I really think it's the same family in 1860, 1870 and 1880.
In 1860, John Gray, Sr. is married to Sarah Phillips who he married in 1852. She must have been his 2nd wife since the son, John, was born before that date. Sarah was Alex's mother. In 1870, John Gray, Sr. is married to Lilley and had 2 younger children, Julian and Ardonia. Lilley was also most likely the mother of Margaret "Mattie," Elizabeth "Bessie" and Adolphus/Adolph.
There are problems with this hypothesis. John Gray's age in 1870 is off by 8-10 years from that given in 1860 and 1880 and there doesn't seem to be a marriage for John Gray to a Lilley in Union Parish. I guess Lilley could be a daughter-in-law who was living with John Gray along with her 2 children.
I think that the deed records should clear up some of this. I have never seen deeds from Louisiana but I'm hoping that the wife had to sign deeds when her husband sold land. That's the way it worked in some other states.
I looked at the card catalog of the Mesa Family History Library. They have a book of Union Parish, LA marriages. You need to go through that book and see if there are any marriages for these Grays. It's probably the same book that I had a lookup from, but I asked for John Gray marriages. Maybe he was listed as J Gray or something similar when he was married.