Looking at the Agee Register by Louis N. Agee, I see that there was anAgee family in Madison Co., KY. They were descended from Mathieu - Anthony - John - Jacob to Adler Agee who married Rachael Masters 1/31/1821.They had 7 children and several of them married into the Masters family. I do not see a Lovie Agee or Charles Masters, but they must be in the records somewhere for Madison Co., KY. Mathieu Agee was the immigrant to VA with the French Huguenots in 1700.
Jacob Agee appears to have been surveying for the Burchett family and married a daughter, Ailsey, in Henry Co., VA in 1800. Then he went to Wayne Co., KY and Robertson Co., TN.His son, Adler was in Madison Co.
I would check census records. Good luck. Sandra Due