Re: your question #1. Did Edward IV indeed father an illegitimate daughter with Elizabeth Lucy?
Yes, according to several sources Elizabeth was his illegitimate daughter. CP VIII:274, while mentioning her and her marriage, only commented on her brother (or half-brother) Arthur, when it said his mother could have been Elizabeth Lucy, Jane Shore, or Elizabeth Waite. BP:2174 says she is a natural daughter of Edward IV, by Lady Elizabeth Lucy. Sheppard, "Royal Bye Blows," NEHGR 121:187 stated "Edward IV, son of Richard , Duke of York, is said to have had many bastards, but only two have been identified by the writer:
1. Arthur Plantagenet, Viscount Lisle
2. Elizabeth, married Sir Thomas Lumley knt., …"
That he had many bastards is footnoted; the author citing Francis Sandford (1630-1694), "The Genealogical Dictionary of the Kings of England." The title may be a variation or the same as another of his works, "A genealogical history of the kings of England, and monarchs of Great Britain," (1677).
Your question 2. Did this daughter indeed marry Thomas Lumley?
Yes, according to all three of the above sources, CP, BP, and Royal Bye Blows, same citations. One, Sheppard's article "Royal Bye Blows," mentioned some confusion between Sir Thomas Lumley and his grandfather, Thomas Lumley, Lord Lumley, but stated the former was more likely to have married Elizabeth. Also that he derived much of this information again from Sandford, op. cit., from Clay, "Extinct and Dormant Peerages of the Northern Counties of England, p., 130, " and from the D.N.B (Dictionary of National Biography).
Your question 3. If known, who are the descendants of Elizabeth and Thomas Lumley? In particular, the line that allegedly descends to Reginald Foster, one of the founding fathers of Guilford, CT.
I wish I could be of more help here but none of my sources lists this connection. Elizabeth and Sir Thomas had, along with other unnamed children, a son Richard, the fifth Lord Lumley [BP:2174]. Unfortunately it's much the same as with your question 4 (Is it known how many other illegitimate children were sired by Edward IV and their mothers names?). That there were several is mentioned in both sources, but their names and lines of descent were not provided by either.
Lastly, you are right to suspect the validity of this information. The only way you can verify the lineage you found on a web page is if the author provided valid source citations to quality sources. This is exactly why source citations are provided; so you can verify and evaluate the information yourself. If there are no valid, detailed, source citations, the information itself is virtually useless and shouldn't be trusted or passed along. The only possible use for such information is as clues to follow up on and research yourself--assuming you have the resources and inclination to do so. I still have more research to do in Guilford myself, but this is on hold until I find an opportunity to go there (I'm in the opposite corner of the US) and find what's available from local information resources (of which there are several in this case). Hope this helps.