I agree that John D is most likely the son of Samuel M. Along with birth place correlations and migration similarities, there is a round-about census tie
1860 Houston, TX Mathews, S M [L M] 51 SC Luisa[levisa] 27 GA E A. 24 AL female Mary 22 AL William 18 AL Robert 16 AL Lockwood 10 male AL[Francis Lockwood] ******** Coronitia [Cordelia] 8 AL Nancy 6 AL
1880 Houston TX Mathews S M 71 SC Levisa 67 GA Burson, Mary L. TX gr dau
same household-new family Mathews, F L 30 AL SC GA********* Elizabeth J. 24 TX Jessie May 4 TX dau Willie C. 1 TX dau
in 1900 Francis Lockwood Mathews is living with John Duke and shown as his brother.
1900 Angelina, TX Mathews, John 68 Sarah 62 wife Ryan, Willie E. 20 gr dau Mathews, Francis L. 50 Brother married**********
I have found nothing to support Levisa being a Hollingsworth, or any other maiden name.