Sorry it has taken me so long to reply, I haven't visited this site for quite some time.
OK as to the topic. My father, Jack Impey lived in Hitchen Road High Town. His Grand father, (x5) Samuel Impey lived in High Town Road. My father tells me there were more Impey's living in Olive Street High Town. As far as he knows they were not (direct) relatives. Although I suspect they must have been at some point.
My father was in the Royal Navy (HMAS Matchless) on minesweepers. He worked at Vauxhall after the war and emigrated to Australia in 1958. He says he new a lot of Impey's in Luton (that he incorrectly) says were not related but he never knew of another Jack Impey. He does recall meeting a Merchant seaman during the war named Impey, but he thinks he came from Eaton Bray.
My guess is the family of Impey's that lived in Olive Street High Town may have some direct relationship to your Jack Impey.
Hope this helps and once again sorry for the delay in responding.