You list counties of his residence, so I'm not sure whether you have Ezra's parents, or not.
1880 census Polk Co, GA, shows Ezra's brother Joseph:
NameRelation Marital Status Gender Race Age Birthplace Occupation Father's Birthplace Mother's Birthplace Ezra JAMESSelfMMaleW26GAFarmerGAGA Caroline JAMESWifeMFemaleW30ALKeeping HouseGAGA Minnie L. JAMESDauSFemaleW6MGAGAAL Robt HUGHESOtherWMaleW26WALESMinerWALESWALES Joseph JAMESBrotherSMaleW24GAWorks On FarmGAGA
1870 Cobb Co, GA, here's Ezra and brother Joseph with their parents and family:
James, John W., 40, b GA - land 600/personal 150 " Sarah, 37, b GA " Esry, 17, b GA " Joseph, 14, b GA " Annah, 13, b GA " Thomas, 8, b GA " Andrew, 7, b GA " Noah, 4, b GA
1860 census Cobb Co, GA:
James, John, 32, b GA - land 1000/1450 " Sarah, 28, b GA " Ezzra[sic], 7, b GA " W. J., 5, b GA " Louisa, 3, b GA James, Noah L., 13, b GA [Usually when someone with same surname is listed out of age order, they aren't the child of the couple.Note John has a 4 yr old Noah in next census (1870)Probably named for this 13 yr old Noah.Note difference in personal value, John may have owned slaves.]
1880 census Cobb Co, GA, (John still in Cobb Co.)
John JAMESSelfMMaleW52GAFarmerNCGA Sarah JAMESWifeMFemaleW50GAKeeping HouseSCSC Lou JAMESDauSFemaleW22GAAt HomeGAGA Thomas J. JAMESSonSMaleW19GAFarmerGAGA J. Andrew JAMESSonSMaleW16GAFarmerGAGA Noah L. JAMESSonSMaleW12GAFarmerGAGA Samuel E. JAMESSonSMaleW8GAGAGA
Note, John shows his father born NC, on the 1860 census Cobb Co, GA, there is a Cyrus James 60 b NC.You might follow up on him, see if John is with him on the 1850 census.I do not have access to online 1850 census index.