Since you mention the name Merit was hard to read, I assume you were looking at the hand written copy of the 1880 census.You might check the version on the FamilySearch site.They show James M. age 55, making him born about 1825, (not 1807) and that would come closer to matching him on the 1870 census where he is listed as 49, being born about 1821.Human nature was the same then as now, and people didn't always "age" 10 years from census to census.Notice his wife did, though.(^_^)
If wife N.C. on the 1870 was "Nancy", Annie is one of the nicknames for Nancy, so Annie on 1880 might be the same wife.As for birth location difference, I've seen people show different birth locationn for themselves on every census.It mightmake a difference who answered the door the day the census enumerator called.I was probably in my late teens when I learned my father was born in a completely different state then what I thought.I certainly was old enough to have answered the door if an enumerator had called.Grandma thought they should move, but turned out too much rain and Grandpa couldn't get enough work, so they moved back.During that short move out of state, my Dad was born.Everyone before and after my Dad was born in the same state, and they all grew up there, accounting for the stories I heard about them, and why I assumed he also was born there, too.
Now, for some more variations, here is the 1870 census Talladega Co, AL (Clay formed 1866, one of the parent counties was Talladega):
Browning, J.M., 33, b NC
" Anastasia, 29, b GA
" John, 12, b GA
" James, 10, b GA
" Isaac, 7, b GA
" Mary 6, b GA
" Nancy 2, b GA
Remember on the 1870 census when John, living next door to James, had three young Blackstocks living with him?There is a Blackstock family on the same page as J.M. in 1860.Wife born NC, maybe she's a sister to J.M.??Their children born GA, too.
1900 census Clay Co, AL:
Browning, James, hd, b Oc 1824, 75, M 54 yrs, b NC, NC, NC
" Annis S., wf, b Oct 1830, 69, M 54 yrs, ? ch, 10 lvg, b Ga, GA, GA
" Merit L., son, b Apr 1869, 32, wid, b AL, NC, GA
Same page is son Isaac B. with large family.
Happy Hunting (^_^)