I found two James Madison Gilleys in my line, but none married to Virginia.My line begins with John I. Gilley, circa 1770 to 1856, and his wife Mary Barger.One of their offspring, and they had a bunch of them, was George Washington Gilly, born in 1808 and my Great Great Grandfather. One of this sons was George Washington Gilly, Jr. who had Jasper by Eldora Miles. He had other children by five wives.Jasper was my Grandfather and marrwho was my Grandfather, who had George Alexander Gilly, my Dad.
The first James Madison Gilley (1838-1923) I see is the son of Jessee Gilley and the grandson of John I. and Mary. But my list says James was married to Anna Barron and Bettie Wolf.
I couldn't find the other North Carolina Gilleys you spoke of.
I think the Virgina Gilleys started out in North Carolina then made their way to Wise County and Lee County VA.
I don't think this helps much. There is someone in Virginia doing an outline or book about the family. I learned of her from James M. Gilley of Lynchburg, VA (He's 93 and doing great). He is the grandson of the James Madision Gilley I spoke of. He and I have been trading information for a couple of months.Let me know if any of this fits.