As a bit of an afterthought, I had a look at what GENUKI has to offer on the subject of Swindon registration district and discovered that, in 1899, it was renamed from Highworth to Swindon.
This means that if Arthur was born before 1899, you should be looking for a birth in the Highworth district - not Swindon. The GRO (General Records Office) index volume number for this district is 5a. So if you decide to search the microfilmed St. Catherine's indexes, you should concentrate on HOG* entries with index numbers beginning 5a.
Taking the whole thing one step further, I then checked to see if FreeBMD (http://freebmd.rootsweb.com) had any records which closely matched your query. (FreeBMD is a site dedicated to compiling an online archive of the St. Catherine's index and is only partially complete, so far.)
I found the following reference, which looks quite hopeful, although I wouldn't start counting chickens at this stage.
Births - March Qtr. 1856 Surname|First name(s)|District|Vol|Page Hogarth|James Arthur|Heghworth*|5A|8
*The area covered by volume 5a was (1852-1899) as follows: Bishopstone, Blunsdon St. Andrew, Castle Eaton, Chiseldon, Draycot Foliat, Hannington, Haydon Wick, Highworth, Inglesham, Liddington, Little Hinton, Redbourne Cheney, South Marston, Stanton Fitzwarren, Stratton St. Margaret, Swindon, Wanborough, Wroughton.'Heghworth' above, is clearly a typo, therefore.
If you decide to take a chance and buy the birth cert with this index number, let me know and I'll send you a copy of the application form with instructions for applying to the ONS.