First thing I would do is order a copy of his Original SS application.'He'would filled out the info listed about 1940's.Who knows what parents he listed.Would be very interesting. Since you have Ludie & her husbands SS #, you can also send for their Original SS applications.I already sent you the link to order this.
You may not find 'Adoption papers'.Children were often turned out & other families would take them in. A 'timeline' for him might be of help, placing each little tidbit of info (by date & location) as you find it. Birth, Census, Marriages, birth of his children,family lore, Military. Be sure to note who/where you got the info from. Slowly the pieces will start to build a picture of his life. You can also check in the County you found him in on the 1930 census.They may have guardianship or adoption papers.I'll check to see if I can find an address for you in my reference book.
FYI:I would only include one SS request per envelope.