Thanks to Chris Glass I am now aware of the ELGEY message board. I now think that Albert is decended from John Thomas ELGEY and Betsy ANDREW. John Thomas was born in Burstwick, Holderness and may be the son of a William ELGEY who is listed there as School Master. Also the IGI shows a William ELGEY bapt in Scecking with Burstwick April 4th 1827 son of Anne but no father mentioned. Parish Registers need checking of course.
Betsy ANDREW was the daughter of Thomas ANDREW and Charlotte HARPER of Swanland, my great grand parents. John Thomas and Betsy had their eldest son named George Harper ELGEY and he was born in Swanland.
Both the 1881 and 1901 census have wrongly listed the names as ELGAY and ELSIE