An 1858 Rock Island County marriage record shows James Manley married to Elizabeth Smith. However, she may have married a Smith after her marriage or possibly before her marriage to James.
The 1946 death record for their daughter Sarah Elizabeth, wife of Lawrence O'Neill, shows her mother's maiden name as Smyth (see Illinois deaths and still births at familysearch.org). It shows her father James Manly was born in County North, Pennsylvania which is probably Northampton County although this is by no means certain. Her mother is listed as born in County Intrim, Ireland which is probably County Antrim.
The 1870 and 1860 censuses (both times in Rock Island County)show James was born closer to 1821. The 1880 census shows David's father also born in NJ and mother in PA. I could not find David in any other census records.
A Civil War record shows James Manley from Rock Island County enlisting in Company I, 81st Illinois Infantry. His service record may or may not have exactly where he was born.
I suggest tracking down as many of the records on this family as possible. One of them may have more detailed info such as obituaries. Check with a a local library or historical society to see if they can help you.