Horror stories belong in the Irony quadrant of my genre circle, as outlined in my book How Stories Really Work.
Irony - and its subset horror - draw their power from two main sources:
1. The expectations arising from the conventional tropes of the Epic genre, in which events are presented as real and external and the subjective state of the characters (and thus of the audience) is less a focus than the outward action and movement of the story. Epics place a great emphasis on linear vacuums - what happens next? - and the resulting momentum; Ironies, on the other hand, question the reality of events, delving deeply at times in to the subconscious minds of the characters (and thus the audience’s minds) and making readers and audiences question what is being presented to them as ‘real’.
2. Death. Death is the ultimate ‘vacuum’, the great unknown, the barrier from beyond which even our greatest loves make no return (unless you’re a Christian, as I am). Ghost stories in particular draw upon this, given that their material is to do with the threshold of death itself.
In Ironies, the emphasis is placed upon mystery and moral vacuums: what is going on under the surface? And is this morally right? Linear vacuums - what happens next? - are interwoven with the mystery, hence the power of the step-by-step approach of horror, or the time countdown.
The haunted house trope is interesting because it is often as though the house itself is alive: such are the magnitude and number of vacuums surrounding it that it almost becomes a ‘character’, indeed, sometimes almost a protagonist.
I prefer ‘eucatastrophes’ myself, as outlined by Tolkien: stories in which there is a sudden ‘turn’ towards the Light. My favourite scary books or films are ones where the horror is finally expunged through some kind of heroic sacrifice, and the story reverts from being an Irony back into an Epic, with good triumphing in the end. But while Death remains mysterious, horror stories will exist in some form and will have their pull.