I was interested in your messages. I have the 1851 and 1861 Somerset census C.D.'s. There was no sign of Edmund in 1851, but I did see John & Elizabeth Gunning at 4, Church Street, Bath in 1851. It said: 350 GUNNING John 64 Baker Bath, Som. Elizabeth 57 Wilts Harnington
The "Harnington" is probably incorrectly transcribed as I can't see that on a list of Wiltshire villages I use, and the writing on the census is poor. If you E-Mail me at firstname.lastname@example.org then I will send you the 1851 page its on, and you can make your own mind up.
I did see in Pallot's Index in 1806 a marriage in Bath (st James) between John Gunning and Elizabeth Bishop, though the timing does not fit at all with the ages in the censuses. It would make Elizabeth about 12 or 13 at marriage! Pallot also lists Edmund Gunning marrying Sarah Talbot in Bath (St James) in 1809.
Edmund Gunning married Jane Austin in the Pancras district in the first quarter of 1858 (Volume 1b Folio 172 if you want the certificate). The 1861 census shows Edmund Gunning and Jane with what are obviously Jane's cildren from a previous marriage - Joseph J aged 6 and Sarah J G aged 3.
Edmund died in 1871 - 1st quarter (Vol 1b Folio 86) aged 56. In the 1871 census, Jane, a 43 year old widow, had her children Edmund Austin aged 18, and Sarah G Austin aged 13 with her. It is only in 1881 that Sarah became Sarah Gunning living with her mother. Jane died in 1891. There is no sign that Edmund and Jane actually had any children!