I think that Ellen (Lillard) Payne must have been the child of Walker Lillard's first marriage.She was born in 1865 and it appears that Walker Lillard remarried to a woman named Harriet in about 1870.
Walker Lillard was a member of Co. H of the 68th U.S. Colored Infantry in the Civil War.He was mustered in as a private and mustered out as a corporal. You can look at the National Archives website at http://www.archives.gov/genealogy/military/http://www.archives.gov/genealogy/military/ to see how to get copies of his military records, and those may give some further clues to you.
On the 1860 Slave Schedules, there are two white men in Missouri named Lillard (James in Lewis Co. and John in Jackson Co.) who were both from Kentucky and who own male slaves born in about 1837 (Walker's approximate year of birth).It is possible that one of these men brought Walker from Kentucky to Missouri.