Rhodene - I would be surprised if Daniel Pitts (b. 1804) had ancestors in Robertson Co. People crossed back and forth from Logan Co.,Ky and RobertsonTn (which is close to the Ky line)- possibly business reasons - certainly I could be wrong -
Like you I found little on these individuals in 1870/1880 - If it was just one or two I moght say the census taker missed them, but it seems to be several of them which leads me to think they moved away.
Did find the family of Willie F. Mason - In 1870 census she appears to be the daughter of J. W.(John) and Jessy Mason. They have a daughter "Willie" aged 8 who I feel sure is the one you were looking for. In the 1880 census. John W. Mason and family are in Robertson Co.other children are in the house, but not Willie probably because she was married to John Pitts by that time.
In 1880 census in Logan Co., KY,like you, I found George Pitts and his family, Reuben Pitts and family, and his brother William R. is living with them. Anna certainly may be living with her new husband(Collins) but I would guess she may be dead since William is living with his brother - an assumption of course.
In the 1910 census for Montgomery Co., TN I found George Pitts age 55, wife Elizabeth 50 and their family. Looks like they moved back to TN from Logan Co.KY. I feel this is the same George as he says he was born Tennessee, that his father was born SC and his mother was born in KY. If he lived as late as 1910, there may be a death certificate for him in Tennessee (if he died there) and it might have his mother's maiden name on it.
Are you trying to trace a particular descendant of Daniel Pitts - or are you looking for his ancestors?