There are a variety of companies where one can have yDNA testing done. Family Tree DNA happens to be the largest company and consequently has the largest database to compare results once you have results for your own sample (or that of a male relative). I also happen to know the most about this company because I am a volunteer Admin for several yDNA surname projects.
To get specific information about the costs and also to answer questions that you might have, go to http://www.familytreedna.com/http://www.familytreedna.com/ and poke around at various links. In the upper right hand corner of the Home Page is "Surname Search" - type in Wiley and follow the links.
If your mother was a Wiley then you are right, her brother would be an excellent participant for the line. Any male Wiley cousins could also participate in this surname project.
That are a variety of different tests that can be useful for genetic genealogy but the one that you would want would be one of the "Y-Chromosome DNA Test". You can have a simple basic best done that uses only 12 markers. Slightly more complex would be a 25 marker test; then a 37 marker test; the most complete would be a 67 marker test. Each test is more expensive that the other. If you want to start small, you can do that and then upgrade at a future date.
If your mother's elderly brother is willing and is your preferred relative to participate, you might just want to get started with the 12 marker test.