What I have gathered in my Towles research is that you are related to Henry and the Virginia Towles. Henry was pretty much the founder of the family.
In my readings of some Towles genealogy I have noted that the Towles family had settled in Edgefield, SC. One of my distant relatives, a Virginia Towles, (I am of the Kentucky Towles) was married in Spartanburg.
I think there is little doubt about your relation.
Beware:There are several Henry, Oliver, and various spellings of Stokeley (Stockely) Towles in the family.