Hi Shirlene I have no personal knowledge of an Elizabeth Duffy, however a quick scan of the Tallapoosa Alabama Censuses revealed that in 1860 in Beat 14 P O Tehepeka Dwelling 1070, Family 980, a James Boulware age 61 born S C and Frances age 60 born S C resided, with a Mary Patterson age 18 born in Al, and a Francis Boulware b. Al.
Living in the Residence next door, Dwelling 1069, Family 979 was a H F Freeman age 24 born Georgia, married to Jane age 16 born AL. With A 10 months old child, James, and living in the same household was an Elizabeth Duffy age 30 born in N C
I was unable to locate any of the Boulwares or Duffys on the 1880 Tallapoosa Alabama Census. Hope this helps If you think this is a part of your Family Unit email me personally. leslie