My guess would be that Mary Down who married Elisha Evans is the same Mary Dow you found in 1860 with son James age 7.
She is the right age according to the 1870 census and that would make sense of the visit to James and Susan.
I am looking for death records for any/all of them to see if it matches up.
At familysearch.org there is a death of a Llewellyn Dow age 63, died March. 30, 1896 Silver Ridge, ME - father is shown as WIlliam Dow, mother's name is not shown he is shown as married
Llewellyn is in 1880 census for Silver Ridge, Aroostook, ME age 46, wife Jane 41, sons Alonzo C. 15, Willie W., 12, and George E. 10
and 1870 Township 2, Aroostook, ME, age 32, wife Jane 26, Alonzo 7, Willie 3, and john 1.
likemany census entries, there ages are not consistent from one census to next there is a death for a Jane Dow born about 1840, died Silver Ridge in 1882- shows husband as S. Dow, but no image so cannot see what the image looks like
so it seems likely he married around 1860/61 - I didn't find him in 1860
there is a marriage for a Lewellyn Dow and a Cynthia J. Brown Sept. 16, 1861 Penobscot, ME so could possibly be them - if Jane is her middle name ?
also a marriage for Eliza E. Dow and Albion A. Bangs, Nov. 15, 1857 West Gardiner, kennebec, Maine - parents names are not shown
By the way, while Eliza and Albion and children Mary T. and William are with her parents William and Mary Dow in 1860 they are also found for that same year in his parents house so they were listed twice - with his parents theyre in West Gardiner, Kennebec - dated July 27, 1860
with the Dow family the date was July 7, 1860- both these census entries areon ancestry.com
also on ancestry is birth ofMary T. Bangs born Sept. 5, 1858 West Gardiner, ME I did not find birth for son William
Albion is with his parents in 1870 but Eliza is not with him and neither are Mary or William.There are several young women in the household but looking back at 1860 they all seem to be his sisters, except in 1870 there is also a girl named Olive, age 7
there is a birth for an Olive Ella Bangs May 16, 1875? (that is how the entry is written) - born Gardiner, parents Alvin and Hannah so I wonder if it is supposed to be 186o's not 70's
Albion died in 1914 - death shows correct names for parents so it is right fellow, and he is shown as widowed
Back to Mary Evans, she is in 1880 census in Palermo, age 75, widowed
Elisha is listed in the 1880 census mortality schedules. He died in Oct. so that would be of 1879, of consumption in Somerville, Lincoln ME
his death is also at familysearch.org, showing he died Oct. 17, 1879
I'm sorry I cannot find anything to prove for sure that Mary Dow who married Elisha Evans is mother of James.