Cindy, as you're probably aware, the early census records didn't list the Wives or Childrens' names til 1850, but evidently, James Agee must have died sometime after 1820. I found on the 1830 Spartanburg Co, SC Census a PATSY AGEE listed as Head of Household & she's living in close proximity to Grace Agee Harrison's other children.(She is the ONLY Agee listed in Spartanburg in 1830, along with 1 male 15-20, 1 female 10-15, one female 20-26, & Patsy, 45+.) I think I'm right on those ages, but may be off some.
So, I would assume James Agee's wife to be Patsy, if that helps.