This does sound like the same James Wagg.I show him as married to a Mary Buck abt. 1819 and having two children, Mary Ann, born abt. 1819-1820; and David born abt. 1823.He then married Sarah Crisp and had children James William, 1826; William, 1828; Louisa 1831; Lucy 1833; Richard 1845; Emily 1837; Susannah Sarah 2 Jan 1841 (my gggrandmother), Harriet 1842, Helen 1843, Henry Barrington 1845 and John 1847.
I have your Mary Ann as having married James Jary (b. 1818) and having children Elizabeth, Sarah Ann, John William and Alfred all born in Heacham.I have more information and can post to you if you would like.