Are you sure on the birth year for Ellen? The 1850 census has an Ellen Allen age 31 with Isaac in Freeport. Which would make her born about 1819. And the 1860 census for Freeport also has Elenor as age 41 so again 1819 would be about her birth date. Maybe Ezekiel is your ancestor as the Harmon genealogy suggests!