Sorry, but to the best of my knowledge the Lygon Family of England and the Ligon Family of America are not and never were Huguenots. The first Lygon in England came from Normandy to England in 1066 with his kinsman, William the Conqueror. Huguenots left France in the 1700s.
Now, it is a fact that Mary Ligon the daughter of John and Mary (Moseley) Ligon did marry Matthew Agee, the son of Matthew Agee, a Huguenot refugee born near Nantes, France.
Also it was Richard Ligon who as a surveyor for Henrico Co., VA. laid out the Huguenot settlement at what became Manikintowne, extending below what is now Falling Creek in Chesterfield, VA westward up James River about 20 miles west of Richmond.
The Lygon Family in England were closely associated to the Royal Families.