In reading Don Fritz's last post something struck me that might support his position. A Taliaferro James was living in Caroline (or Spotsylvania) Co., Virginia during the first decades of the 18th century.
I have never been able to trace this Taliaferro James, but have always assumed that he was somehow related to Sherwood James, who lived in Caroline during this time. It may be that he was the son of Ann Johnson, who Don states married (first) a Hugh James.
This is all off the top of my head so my dates &c. may not be entirely accurate.