Radagast the Brown

In a 1954 passage in Unfinished Tales he (Tolkien) says "of all the Istari, one only remained faithful... for Radagast, the fourth, became enamoured of the many beasts and birds that dwelt in Middle-earth, and forsook Elves and Men". (1)

(This is patently unfair to Radagast, no a bad fellow as wizards go, who lent his aid to the watch on Sauron, and played a small but crucial - and completely faithful - part in the Great Years. But it illustrates Tolkien's pessimism.)

In 1954 he was uncertain about the Blue Wizards, but in a letter of 1958 he says of them "I fear that they failed, as Saruman did, thought doubtless in different ways; and I suspect they were founders or beginners of secret cults and 'magic' traditions". (2)

However, in The Peoples of Middle-Earth they get a happier ending; they "must have had great influence on the history of the Second Age and Third Age in weakening and disarraying the forces of [the] East".

(1) Unfinished Tales, p.390

(2) The Letters of J.R.R. Tolkien, p.280

(3) The Peoples of Middle-Earth, p.385

Richard Sturch: 'On Tolkien and Williams'
The Charles Williams Quarterly - 118 (Spring 2006)