The year of 1880 doesn't tell us much. Do you know when either Joseph or Ida were born? How about when and where were their daughters born? Something that would narrow the playing field for more common surnames.
I ask becasue I found a Joe Turner on the TX Dth Idx, with wife named Ida. BUT, he is listed colored. Could that be your Joseph?