I found a copy of the records of the Center Street Church in Machias which show an Esther Albee as a member in August of 1811. That sort of follows the Machias Vital Records which also show the parents of the Albee children as William and Esther.
Do you suppose her name was Esther Allen Dillaway, and she sort of adapted her middle name to a one she liked better?For instance, my middle name is Jane, which I've been using since I was 13 because I do not like my given name.
As a matter of note, Polly (Mary?)Chaloner joined the same Church in 1810, and a Sarah Dillaway was a member in 1782.