Thanks for responding. I, too, have always thought Miles Everett "Babe" Gibson was somehow connected to Seaborn Gibson, but I don't know how. I recently learned from an 1863 Confederate Draft Registration in Warrington, Escambia Co., FL, that James Madison Garner's brothers, Joseph E. and Thomas G. Garner, were born in Dade Co., AL. I haven't been able to connect these Garners with any of the Dade Co., AL Garners in the 1840 or 1850 census.
I also found that in 1910, Joseph E. Garner (James' brother) was living in Santa Rosa Co., FL next door to another Gibson - this one was the widow, Mary, age 53, and her children. There are too many Gibson/Garner events to just be coincidence.
I found Seaborn Gibson in 1850 in the household of his father, Edward M. Gibson (b. 1807 GA) and wife, Nancy F. Lowery (b. 1814 GA)in Henry Co., AL. My gut tells me that Susan/Susanna (b. 1809-1815 GA - mother of James Madison Garner) is either a sister of Edward Gibson or a sister of Nancy Lowery. Why else would she allow her young adolescent son to move away and stay with another family if they weren't related? I haven't been able to find anything to support this yet. I also found out that Edward Gibson was the son of Manning Gibson (b. 1765 SC - d. after 1850 Henry Co., AL). FYI, the Georgia Gibson that James Madison Garner was living with in 1870 was the sister of their next-door-neighbor, Seaborn Gibson.
I think it is very interesting that you mentioned someone named Silas in your family. In 1870, Susan Garner was living with the Silas Nipper family next door to Seaborn Gibson. I haven't been able to trace this Nipper family back any farther.
If you come up with any more info or ideas that might help, I would love to hear it.