I do not believe it is possible that your Lydia is descended from Daniel Holyfield & Mary Pye's family directly as all of their children were married in Georgia, one of which was Lydia,but she was born Aug. 19, 1800 and an apparent twin to my descendant born the same year;Willis Holifield.
There was a brother (or cousin) of Daniel Holyfield's named Valentine Holyfield who married Sussanah in 1774,Wake County,NC, then they moved to Surry County,NC.Daniel Holyfield apparently moved to Surry County,NC as well after he was married in 1778.A Rutherford,NC historian of Surry County,NC wrote that shortly after Surry County was formed in 1790; a Valentine & Daniel Holyfield arrived there to settle.Both of them served in the Revolutionary War.The name Lafayette is all but certain the name given after the famed French General-Marque de Lafayette whom Benjamin Franklin met along with another French Officer while in France when Franklin negociated to purchase the French Army's obsolette model 1766 Muskets & matching pistols.(I own one of those French model 1766 Pistols which were issued to Officers & NCO's during the Revolutionary War)Benjamin Franklin wrote a letter of recommendation to George Washington for the Marque de Lafayette and his fellow Officer (can't remember his name) to come over and train our Army's Solders, mostly who were as our ancestor's, Farmers.So when the two French Officers arrived at Valley Forge in December 1777; they each handed their letters of recommendation to George Washington which Washington recognized Benjamin Franklin's wax seal, and read the letters which recommended them an Officer status in our Army under Washington in order to train our Solders in combat tactics & discilpine.But Lafayette requested he be commissioned a General (even though he was only 25) and to the surprise of many, Washington did just that.By winters end of 1778, our Army of Famers and the like, emerged trained & desciplined,which became a turning point in the War with the British.And as history goes, in 1782, the French arrived just in time to block a sea escape of the British, thus ending the war with the British surrendering.Lafayette became a hero and often came back to visit the United States as a hero of Liberty.Thus the name Lafayette was remembered well by Daniel Holyfield, a Rifelman in Capt. Charles West's Company, the 3rd (Rifle) Company of the 3rd Virginia Regiment of the Contenintal Line.The 3rd Virginia Regiment were among the 2400 Solders who crossed the Deleware River with Washington on Christmas night 1776 to win the battle of Trenton,NJ against the Hessians.They would also fight in the battle of Priceton,NJ.Daniel Holyfield's enlistment shows he extended his enlistment past his 1 year obligation as requested by Washington of almost all of his remaining Solders on Dec.30, 1776,as Daniel Holyfield has a pay record for mid Feb. 1777 for 3 & 1/3 Dollars for 2 & 1/2 months service.So he must have collected his pay and went home,because starting in Jan. 1777, you had to sign up for 3 years and Daniel married in 1778.
Valentine Holyfield married Susannah in 1774 in Wake County,NC where the older generation of Holyfield's moved in the 1755-1761 time frame.Wake County,NC was formed from Orange & Johnston Counties (if I remember right) in 1770.This is where William,alias Gideon & Ralph moved to at that time,as mentioned previously.After Valentine Holyfield & Sussanah married, they moved to Surry County,NC.So since all of Daniel Holyfield's children married in Georgia, then Lydia must be descended from Valentine & Sussanah's family.Daniel Holyfield & Mary Pye then moved to Burke County,NC which McDowell County,NC was formed from.It seems that nephews via Valentine Holyfield & Sussanah must have moved nearby to Yancey,Buncombe & Watauga sp? (which is where Valle Crucis is located).So it seems that Lydia migrated "North" from 1830 in Yancey & Buncomb Counties to Watauga County,NC by 1850. So you must be correct in that the names Lydia & Wiey were respected Aunts & Uncles.
Spruce Pine has a Holifield Cemetery there:
MITCHELL COUNTY, NC - CEMETERIES - Hollifield Cemetery
This data is taken from the Historical Records Survey of North Carolina.
The survey was done in the 1930s and 1940s, and included only graves dating
from before 1914.
Information has been reformatted from the original by Patrick Barrett (2003),
working from a microfilm of the original typescript (FHL US/CAN Microfilm #882939).
HISTORICAL RECORDS SURVEY OF N. C.
Worker: Warren H. Pritchard
July 28, 1939
SPRUCE PINE, N. C.
LOCATION: Located just behind the home of Mr. Avery Hollifield, on
Highway No. 26, about four and a half miles south of Spruce
Pine, N. C. In fairly good condition.
Hollifield, Daniel M.15 Feb. 186629 Dec. 1905
Hollifield, Joe6 Aug. 189815 Feb. 1904
Hollifield, John26 Apr. 189227 Jun. 1894
Hollifield, Mannie7 Dec. 1886
Hollifield, Margaret M.10 Aug. 183623 Feb. 1907
Hollifield, Sheldon P.4 Nov. 18371 Apr. 1906
Hollifield, William A.18341912
Hollifield, William D.2 Apr. 187331 Mar. 1900
Hollifield, Willie C.1 Mar. 19015 Jan. 1902
Daniel Holyfield & Mary Pye lived in Loudoun County,VA;Surry County,NC;Burke/McDowell County,NC;Oglethorpe & Jasper Counties,Georgia;Auburn,Perry & Sumter Counties,Alabama.
Well, I hope this helps!