I noticed that you mentioned a Henry Levy (b 1825) as a brother to your family member.
My ggg grandparents were Henry Levy and Julian "Julia" Jackson Spencer. He was born in 1825 according to burial records in Nashville, TN. In the 1860 Nashville Census he is listed as a brick layer and he is living with his wife and children "Alinora, Rossetta, John and Mary Ann". In the 1880 census their daughter Mary Ann "Annie" Levy Norman who was married to John Bell Norman lives nearby. Annie and John Bell's two sons were named Charles and Allison which made me wonder about a connection with James Allison Levy.
What are your thoughts that Henry Levy would be one of James Allison Levy's children? Could this be the Henry you mentioned as a brother to Parker and James Allison Levy. Have you learned anything else about the family during those early years in Tennessee, or can you help me confirm my Henry Levy is related to your James Allison Levy?