As Sandra says, you definitely need marriage certificate to confirm his father's name.
The following may be of interest in the meantime: The first sighting of Edwin S in the Pontypridd area seems to be on the 1871 census as below. He seems to be born c.1850 which is consistent with the ages you have on the later censuses. He is a collier.
Birthplace given as Wiltshire and on the original census image it appears to say WESTBURY LAYE which must, I think, be WESTBURY LEIGH which is a specific hamlet, now part of Westbury. (See streetmap.co.uk).This seems a very specific piece of info which could surely only have been given by Edwin himself. So I think you are probably right in having Westbury as the focus in searching for his origins.
1871 census KNIGHT, James HeadM 26 1845 Worcestershire KNIGHT, Catherine Wife F 24 1847 Gloucestershire KNIGHT, John F Son M 2 1869 Glamorgan KNIGHT, Thomas H Son M 0 1871 Glamorgan TANNER, Edwin S Lodger M 21 1850 Wiltshire Piece: 5378 Folio: 36 Page: 6 Registration District: Pontypridd Civil Parish: Llanwonno Municipal Borough: Address: Cardiff Rd, Llanwonno County: Glamorganshire