Patsy Hartless--N Cooper--Duff-- Amherst Va

I am new to the Hatless and Cooper lines.I am trying to find the parents of these children, who were living with Patsy Cooper in the 1860 Census in Amherst County Va.

Patsy Cooper 72
Jeremiah J 17
James H 15
Patsey 14
Sarah C 7

I believe that Jeremiah J Cooper later changed his name to Jeremiah J Duff who married Eliza A ?

Also, James H Cooper changed his to James H Duff he married Mary A Mason 12-27-1867 in Rockbridge Co.

Was Pasty their grandmother or Ggrandmother?Was thier father a Duff? I have been unable to find James until 1900.
I found Jeremiah in the 1880 census with the Duff name.

We have been trying to locate the parents of James H Duff and have only found him in 1900 & 1910.

I found a marriage for a James H Cooper b 1846 to Mary A Mason b 1846 in 1867.I went back to Amherst and found Jeremiah and James with Patsy Cooper.I am thinking their father must have been a Duff and they later took his name.

Mary Agnes Mason was the daughter of Peter H Mason and Dianna Sorrells.She was buried in Green Hill cemetery in Rockbridge.
Can anyone help with this?????