In the 1861 census, the family is in Richmond Township, Lennox County, Canada West (later Ontario), but Elijah and the younger children are not listed with the family, but instead on a separate page, living with the family of Lawrence Cramore (Philip, aged 14; Elijah, aged 10, Wesley, aged 6, and Ellen, aged 3). In the 1871 census, Elijah (aged 20) is at home with his parents, along with older siblings, Ann E., aged 30; Philip, aged 22, and younger siblings, Wesley, aged 15; Eleanor, aged 13.
The 1851 census was actually delayed and not taken until early 1852, and Elijah does show up, also in Richmond Township:
Hall, Fillip, farmer, born England, W. Methodist, aged 49, married
Hall, Mary, born England, aged 37, married
Hall, Mary, born Canada West, aged 16
Hall, William, laborer, born Canada West, aged 14
Hall, Ann, born CW, aged 12
Hall, Ruth, born CW, aged 10
Hall, Thomas, born CW, aged 8
Hall, S. Jane, born CW, aged 6
Hall, P. Henry, born CW, aged 5
Hall, Eeliga, born CW, aged 1
The earliest surviving census in England is 1841, so Philip won't show up in any English census.