The 1854 marriage is the only one coming up in stat records. So his previous marriage must have been before 1st April 1845 when non-RC stat marriage records started, or possibly outside Ireland. If it was in Ireland you would really need to know where it took place and the denomination to trace it through parish records. Only a small fraction of church records from that are on-line.
Tradition was to marry in the bride’s church, and thereafter she’d attend her husband’s. I notice from the 1854 marriage that Alexander’s occupation was a farmer. He is unlikely to have been a farmer in central Dublin, and so I suspect that’s where his wife came from, and he normally lived elsewhere. Presumably somewhere rural.
The childrens births are all before the start of statutory birth registration (1.1.1864) so you won’t find birth certs for them.