Virginia, Prominent Families, Vol. 1-4 Ambrose Madison, of Gloucester Co., Va., was the son of John Madison, Jr., and Isabel Todd. He married (August 27, 1721) Frances Taylor, daughter of James Taylor, of Orange Co., Va. They had three children: Volume IV Chapter XIII Hite, Madison, Fontaine and Maury Families. Madison Family.