Randy, I have been working on a James M. Adams, GA and AL also.He is a different one, but I did some looking into yours while I was working on mine.
Your James was a little conflicted. 1850 census shows he was born in SC.1860 census says GA. 1870 says GA and the 1880 says he and his parents were born in VA.At least Sarah was SC all the way.
I am descended from James Adams, Jr. who came from VA to Elbert Co, GA in the 1790's along with his brother Thomas.There were several James who were children and grandchildren of those two, but they are accounted for.
It seems that every Adams family in the southeast had a son named James.Very hard to keep them apart.
There were not too many Roarke's around. I found none in GA in the early 1800's.There were some in York Co., SC.There were Donaldson's there also.Also some James Adams. The Roarke was probably a family name and maybe from the Donaldson side.
There were Absolom Adams in the GA/SC area, but none that I know of in my James Adams line.
There were two different lines of Adams families in Dewey Rose, Elbert Co, GA (population about 100).There are a lot of similar names in the two lines, but two completely different lines of Adams.They were from VA also.I think that there were some Absolom's in that family.
It looks like the Edwin Adams who was in the Donaldson household with James and Sarah in the 1850 census may have been married to the Georgia.
I found a D Adams and Amamda in Macon Co.,AL in 1860 and the same family in Talbot Co, GA in 1870 with the father a Dinkin Adams.Somewhere on Ancestry.com in the 1860 census someone had shown the Amanda as being Georgia Amanda Ann Snipes.I don't know if this is the same family as was with the Donaldson's in 1850, but they appear to be about the right age and.
There may be some leads in there.You may have found all of this already.
Do you know of any male descendant of James M. Adams who is an Adams and might be willing to participate in the DNA testing.That might go a long way in at least getting you back to the correct Adams line.And there are a lot of them.