Hi - First problem, your query is too general and broad to be readily researched. You’ll want to focus on tracing the ancestors of Alfred Olsen himself, but given the scant information you provide, we can’t tell if Alfred Olsen is born in Norway in 1850 or in South Africa in 1950. Both are real possibilities for a grandfather of a living person writing a query.
You will need to develop as much information as you can about “your” Alfred Olsen, then share it with volunteer researchers, and from there trace back one generation at a time.
When you fill in the blanks in your information, remember to include full place names. Is this Knysna, Western Cape Province, South Africa? Did Alfred Olsen live in Knysna? Usually records that are found in family history research are not stored in town archives, one must look to the records of a larger governmental body. That is why a full place name is often useful. Your volunteer researchers in America (on Genforum) and in Norway (in Norway-Heritage Forum, if you post there) will not be automatically familiar with place names in South Africa, as you are.
Research will involve Date of birth (DOB) for everybody, place of birth, and other facts that are noted in public records.
What are Alfred Olsen’s birth date and place of birth and death? Do you have his death certificate, marriage certificate, military enrollment, or any other record that shows his birth date, birth place, parents’ names? Who are his siblings and did any of them emigrate to SA? Who are their spouses? Did Alfred’s parents live in SA, were they born in Norway and they emigrated to SA? That is, does he have any Norwegian family members with him in South Africa?
Genforum is a primarily USA site, and the most-used research tool is Ancestry.com. This is a huge database and includes some international records, but its deepest databases are American. Its databases of records in South Africa are not sufficient to identify your grandfather and answer your question about his roots (at least I found nothing when I searched).
Others in South Africa searching for “Olsen.” The name Olsen just means the son of someone named Ole. Like finding all the kids born to a Charles in South Africa - it wouldn’t tell us much about their relationship. (I am referring to the traditional - not modern - naming practice in Norway.)