You probably have most of the following, but maybe a fragment or two may be of use. I also suspected that Jane H. Sharman, in 1851, might be James, but the enumerator seems quite definite that she was a daughter. I suspect that you need to look at the larger family more closely and would recommend the 1850 census of Washington Co., Maine, the 1861 census of Charlotte Co., and, perhaps, probate records. The marriage registers of the period might help as well.
A peculiarity I noticed is that, in the marriage of children and grandchildren of James Sherman is that they were born in McAdam. My first thought was that James necessarily was from York Co., but also noting the connection to the railway (Andrew was a section man) I suspect that McAdam was readily accessible from Charlotte Co. When Andrew's son, Donald Andrew Sherman, married, he gave his father's birth-place as Woodstock, but his own birth-place as McAdam. Andrew's son, Walter Clarence, gives his birth-place as McAdam, but Andrew's as Meredith Settlement.Thus where Andrew was actually born may be a mystery. Mary Ann's son, John Gibson Blair, also was b. McAdam. This probably attests more to the mobility of a railway family than to an indication of a place-of-origin. The point is that, in spite of surface indications, James Sherman could easily be from Charlotte Co., and should be there in 1851.
JAMES HOWARD SHERMAN b. ca. 1844-7, d. 1909, m. SARAH JANE DOUGLAS (b. ca. 1852-4), d/o Howard and Mary Ann Douglas.
The 1851 census of St. David Parish has:
Sharman Valentine M Head 46 Birth Far (Prop.)
Sharman Thankful F Wife 46 Birth
Sharman John M Son 19 Birth
Sharman William M Son 18 Birth
Sharman Abagail F Daughter 15 Birth
Sharman Valentine M Son 14 Birth
Sharman Warren M Son 11 Birth
Sharman Matilda F Daughter 9 Birth
Sharman Jane H. F Daughter 4 Birth
Examining the image of the original, Jane H. is definitely Jane H., female and a daughter; but whether this information is, in fact, correct is another question. Valentine Sherman m. St. David Parish 25JAN1825 Thankful Scott.
The 1871 census of St. David Parish, Charlotte Co., NB, as on GenWeb, seems to have disappeared but an index survives and a part of this is:
Alexander Sherman 15
Almedia Sherman 15
Anna Sherman 15
Edward F. Sherman 15
Frederick Sherman 15
James W. Sherman 15
Jerusha Sherman 15
John Sherman 15
David A. Sherman 14
Margaret Sherman 14
Stephen B. Sherman 14
The names were alphabetical but I have rearranged them slightly to group them by page. It is possible it was one family with it beginning on page 14.
I see little correspondence in the forenames to those in 1851 suggesting a 2nd Sherman family in the parish. Possibly the family of Stephen Sherman in St. David Parish in the tax list of 1844, as was Valentine Sherman. Both were listed in 1843 as well (with Valentine as Sharman which seems to be a legitimate variation in spelling.) An 1867 electoral list has an 'Elishia' Sherman. Elisha appears to have died about that time as, in the 1865 directory, is Mrs. Elisha Sherman, but in 1867 it is just Elisha. Elisha m. 1865 Elizabeth Kelvie/Kelvey and in 1868 Ellen Staples. In 1874 he m. Elizabeth Sinclair. These marriages are likely those of a younger Elisha as in 1823, there were in St. David Parish Stephen Sharman, John Sharman (John Sherman m. Mary 'Scot' 1818) and Elisha Sharman. This is all very disjointed and incomplete. Supposedly these are members of the family discussed at:
and apparently this family hops back and forth across the international border. In 1831 there were 'Elishaw', Stephen and Valentine Sharman. Presumably they were one family but there seems to be little to connect them. It does appear, however, that James H. Sherman has a much larger family in which to seek parents. The James W. Sherman, in 1871, appears intriguing. Probably the James W. Sherman who m. 14SEP1885 Sophronia Linton. He was b. ca. 1863 in 1881 and the s/o John and Jerusha Sherman. This suggests two families in 1871, with John and Jerusha's on page 15 and another beginning page 14.
James (36) and Sarah J. (27) are in St. James' Parish, Charlotte Co., NB in 1881. Next door is the family of Howard 'Douglass' (60) and his wife, Mary A. (48), possibly Sarah J.'s parents. (It is possible that they actually spelt the name, Douglass, which, considering its origin, 'dubh-glass' (black-grey), is not unreasonable.) They were, in St. James' Parish in 1871, however, Howard (45) and Mary Ann (44) Douglas, and a daughter was Sarah Jane Douglas (19.) In 1901, Walter, James and Sarah's son, lives with his siblings next to his grandmother, the widow Mary A. Douglas (b. 1MAR1824.)
They are in St. James' Parish in 1891. James (as is all the family) is given as b. NB.
1. WALTER C. SHERMAN b. Charlotte Co., NB 27MAY1873 (ca. 1874-8), d. York Co., NB 5MAY1923
2. HOWARD VALENTINE (VOLITINE?) SHERMAN b. St. Stephen Parish, Charlotte Co., NB (or McAdam, York Co., NB, or Andersonville, Charlotte Co., NB, or Lawrence Station, Charlotte Co., NB) 15JUL1877 (ca. 1874-8, 15JUL1878 in 1901, 187 on headstone), d. York Co., NB 6JUN1929, bur. Rockland Cem., York Co., NB, m. Charlotte Co., NB 24DEC1912 MARGARET BLANCHE GIBSON (b. Lawrence Station, Charlotte Co., NB 13FEB1885 (ca. 1884, 1885 on headstone), d. 1968, bur. Rockland Cem., York Co., NB), d/o Frederick Collins and Mary Ann (Anderson) Gibson.
Howard boarded in McAdam in 1901, a section-man for the railway.
3. MARY ANN SHERMAN b. St. Croix, Charlotte Co., NB 15DEC1879 (ca. 1879), d. 26JUL1926, m. Charlotte Co., NB 9NOV1898 JOHN FREDERICK BLAIR (b. St. Croix, Charlotte Co., NB ca. 1873), s/o John Russell and Margaret King (Gibson) Blair
4. ANDREW CAMERON SHERMAN b. Meredith Settlement, Charlotte Co, NB (or McAdam, York Co., NB, or Woodstock, Carleton Co., NB, or Andersonville, Charlotte Co., NB) 22SEP1884 (15DEC1880 in 1901, ca. 1883-5, 1882 on headstone), d. 1969, bur. Rockland Cem., York Co., NB, m. Charlotte Co., NB 28JAN1913 AMY HELEN MARSHALL (b. Watt Junction 1895 (1899 on headstone), d. 1973, bur. Rockland Cem., York Co., NB), d/o Michael and Susan (Hamilton) Marshall
4. MABEL SHERMAN b. ca. 1881
5. LAURA BELL SHERMAN b. 22MAR1883 (ca. 1884), m. Maine 19JUL1902 WILLIAM F. PIERCE
When Andrew Cameron Sherman applied for a certificate of birth in 1939 his aunt, Laura B. Densmore, attested to the accuracy of the information. This would be Laura B. Densmore b. 13AUG1860 in 1901, wife of Adam Densmore (16JUN1848.) Birth records make her to be Laura Belle Douglas/Douglass, therefore Sarah's sister.
When married, Laura lived in Calais, Maine, but William was from St. Stephen, NB.
6. MARTHA MATILDA (TILLY) SHERMAN b. Meredith Settlement, Charlotte Co, NB (or Merdiville?, NB, or Marysville, St. Mary's Parish, York Co., NB?) 1AUG1889, (ca. 1885, ca. 1886-8) in St. James' Parish, m. Charlotte Co., NB 22OCT1906 PETER PETERSON (b. Milltown, St. Stephen Parish, NB 20DEC1884 (ca. 1885)), s/o Andrew and Christina (Christianne) (Rosendale) Peterson.
Tillie Sherman was a domestic to Charles R. amd Mary M. Gibson in St. James' Parish in 1901, possibly in-laws. Similar domestics in the parish were Sherman Sherman (22SEP1884) and Volly Sherman (male) in the house next to the one Sherman was in, age 14 years.
7. MABEL SHERMAN b. 18MAY1887 (ca. 1886.)
Also a servant in St. James' Parish in 1901 for Milford P. and Blanche H. Christie. 'Servant' or 'domestic' was often a euphemism for someone taken in when their parents had died.
7. NINA JANE SHERMAN b. ca. 1887, m. 1st Maine WALTER LIBBY; m. 2nd Maine C. D. HAYFORD