Looks like his family called him Joseph to being with, and maybe he liked Joe better.
Since his death record shows he was born in 1879, and the 1880 census shows he was 1 yr. old, the 1879 date looks more promising.What did his draft registration say?Surely, he would have given that information.
I cannot find him in the 1900 census, have you found him? That's the one where the month and year of birth are given.
I doubt you will find a birth certificate that early.Church records in Orange Co.??