Tolkien’s imagination worked in a philological way. Apart from the well-known fact that his created world of Middle earth grew out of invented languages as he asked himself the question ‘What kinds of people would speak like this?’, there is a process or sequence of thinking that must develop in people interested in words and their histories, which starts with an existing word and works backwards, using a set of known rules or laws about sound changes and the way in which words evolve, to establish an earlier, consistent version of the word. In this way, philologists generate a web of words and construct whole languages long since vanished from the present day world. This, in Tolkien, transferred over to the way in which his characters and stories developed imaginatively.
For example, Aragorn - at first, for Tolkien as he wrote The Lord of the Rings, simply a shadowy stranger whom the hobbits meet in ‘The Prancing Pony’, with Tolkien knowing no more about him than they did - is taken by Tolkien’s imagination on a journey back through time, following the already-established rules and broad guidelines of his world, until he is ‘discovered’ to be the one and true heir of Isildur, destined to be king over the Reunited Kingdoms of Arnor and Gondor. Then his cloaked nature and the gradual revelation of his role take on true dramatic power in the story.
A similar process must have occurred in relation to the wizards of Middle earth: Gandalf was at first, it seems, no more than a sketch of a well-known ‘wise old man’ archetype, based initially (Tolkien said) on a postcard he had seen while on holiday in Switzerland, and drawing for depth of character on other wizard types, like Merlin and the Norse concept of Odin, wandering the world disguised as an old man. But in true Tolkien style, it was not to end there: the wizard was drawn back into a deeper, wider history which involved ancient ages, gods and indeed, the One God, Iluvatar.
Though none of this is made clear within The Lord of the Rings, there is a chapter dealing with wizards in the book Unfinished Tales. Tolkien’s thoughts about them are presented within the chapter ‘The Istari’. Tolkien died in 1973, taking his further thoughts with him, but in a note in Unfinished Tales, dated by his son as probably coming from 1972, Tolkien sets out their origins and purpose, referring to them as ‘Maiar', or lesser angels.
We must assume that they [the Istari] were all Maiar, that is persons of the 'angelic' order, though not necessarily of the same rank. The Maiar were 'spirits', but capable of self-incarnation, and could take 'humane' (especially Elvish) forms... Now these Maiar were sent by the Valar at a crucial moment in the history of Middle Earth to enhance the resistance of the Elves of the West, waning in power, and the Men of the West, greatly outnumbered by those of the East and South. It may be seen that they were free each to do what they could in this mission; that they were not commanded or supposed to act together... and that each had different powers and... were chosen by the Valar with this in mind.
Another note in Unfinished Tales about Gandalf specifically, says:
It was believed by many of the 'Faithful' that 'Gandalf' was the last appearance of Manwë himself... But I think it was not so... To the overthrow of Morgoth he sent his herald Eonwë. To the defeat of Sauron would he not then send some lesser (but mighty) spirit of the angelic people, one coeval and equal, doubtless, with Sauron in their beginnings, but not more? Olórin was his name. But of Olórin we shall never know more than he revealed in Gandalf.
Here we can see Tolkien’s imagination toying with the Norse idea that the old man was actually the disguised leader of a pantheon of gods, as per the stories about Odin. But, as he says, ‘I think it was not so’ - in other words, imaginatively Tolkien felt it wouldn’t work to have Manwë, the Elder King, who had been put in charge of the world by Eru the One God, come down to that level of the narrative and get involved with the logistics of the story himself.
There’s also a brief narrative of a council of the Ainur in Unfinished Tales which is incomplete and partly illegible, but which clearly demonstrates the messengers to be of the Maiar:
It was resolved to send out three emissaries to Middle-earth. 'Who would go? For they must be mighty, peers of Sauron, but must forgo might, and clothe themselves in flesh so as to treat on equality and win the trust of Elves and Men. But this would imperil them, dimming their wisdom and knowledge, and confusing them with fears, cares, and weariness coming from the flesh.' But two only came forward: Curumo, who was chosen by Aulë, and Alatar, who was sent by Oromë. Then Manwë asked, where was Olórin? And Olórin... asked what Manwë would have of him. Manwë replied that he wished Olórin to go as the third messenger to Middle-earth... But Olórin replied that he was too weak for such a task, and that he feared Sauron. Then Manwë said that was all the more reason why he should go...
This is Gandalf taken back, along the ‘philological track’, to what might seem imaginatively, philologically, to be a valid origin: ‘Olórin’, Gandalf’s original name, is Quenya, and its meaning is associated with the Quenya word olos or olor, meaning ‘dream’ or ‘vision/of mind’. What is it that the old man archetype needs most of all to fulfil his function in any narrative? It is the ‘vision’ which sees the Bigger Picture. The figure of the wise old man with the stick, common to more fiction than you might believe, is the one who extrapolates to the other characters, and to the reader, what is ‘really going on’; he is the one who can see (as described at length in How Stories Really Work). From this seed of a concept, as far as Tolkien was concerned, grew the wizard in the wide-brimmed hat with the twinkle in his eye, who would eventually manage the affairs of the Third Age to such triumphant effect.
To take all the elements of a story, at first sketched out loosely but then worked ‘backwards’ to find their internally credible roots, and create a web of believable, consistent and interlinked characters and events is the work of a true sub-creator. Tolkien’s background and history as a philologist, then, should not be viewed as separate from his creative side, but rather as an inextricable part of it and largely responsible for its success.