The "O" appellation signified "of." It is similar to the Germans use of "von" and Welsh use of an "ap" prefix.
Originally, many, if not most, Gaelic Irish surnames had an "O." However, to Anglicise the name, many dropped the "O" entirely. Many who stilled used it, threw it overboard when they immigrated to the Americas and elsewhere.
Since Ireland achieved independence, there has been a revival of its use.