The satisfying twist, argues Tobin, rests on this.
An entirely unpredictable narrative element frequently qualifies not as a satisfying twist, but as an unsatisfying non sequitur. A narrative that would avoid this pitfall must include some elements early on that are endowed with some significance that will only be visible later. However, this significance must, in retrospect at least, seem to have been available from the start, or, when the reader looks back, she or he will not be satisfied. (Tobin, 2009, p. 157)This is what Tobin identifies as the narrative rug-pull - an authorial trick which undermines expectations but keeps the reader content that this undermining has been done 'in the spirit of fair play' (p. 157). \This is of course particularly important in detective fiction. Such a twist means that the reader has to reinterpret the information which they already have in a way that changes their entire understanding of the situation.
Tobin gives the following quotation from Dorothy L. Sayers to sum up this dilemma for the mystery writer:
The reader must be given every clue - but he must not be told, surely, all the detective's deductions, lest he should see the solution too far ahead. Worse still, supposing, even without the detective's help, he interprets all the clues accurately on his own account, what becomes of the surprise? How can we at the same time show the reader everything and yet legitimately obfuscate him as to its meaning? (Sayers, 1929, p. 97)The embedded perspective
So, Tobin argues, the author can take advantage of the curse of knowledge by aligning a narrative with an embedded viewpoint - a strong narrator, in other words, so that the reader strongly identifies what they know with what the narrator knows - or claims to know. The stronger the belief in the narrator's perspective, the 'more vulnerable he or she will be to a narrative rug-pull. This is why unreliable narrators remain and especially dependable resource for setting up this kind of surprise twist' (Tobin, 2009, p. 166). And Dr Sheppard in The Murder of Roger Ackroyd is a particularly unreliable narrator!
Tobin reports that the twist at the end of The Murder of Roger Ackroyd created 'genuine sensation' when it was published (p. 166) even though it wasn't the first time this kind of structure or revelation had been used in crime fiction. (She suggests Poe's 'The Tell Tale Heart' as a comparator.) She quotes two contemporary reactions:
Willard Huntington Wright (1927): 'the trick played on the reader in The Murder of Roger Ackroyd is hardly a legitimate device of the detective-story writer'
vs
Dorothy L. Sayers (1929): 'this opinion [Huntington Wright's] merely represents a natural resentment at having been ingeniously bamboozled. All the necessary data are given.'
Because Dr Sheppard is the narrator, Tobin argues, the reader's perspective is strongly aligned with his, which enables the surprise at the end. She suggests it is particularly unsettling for the reader to discover that a trusted narrator has been unreliable - in other words, when what we thought we knew, turns out to be a drastic misinterpretation of the facts.
There is a further reason, Tobin suggests, that this twist is so effective in this case:
Another reason that Christie's version of this twist was so effective and surprising, of course, was the way that it played on other conventions of the genre. The narrator of Roger Ackroyd fills a conventional role, that of the faithful sidekick exemplified by Dr John Watson, the friend and confidant of Arthur Conan Doyle's famous detective Sherlock Holmes. As the narrator of the Holmes stories Watson serves as both the detective's assistant and the reader's proxy. He is clever enough to follow Holmes' reasoning when it is explained to him, but never so excessively brilliant that the explanation would be unnecessary. He is capable and faithful; if he were not, the perceptive Holmes would surely not rely on him. (Tobin, 2009, p 167)
As a result, we trust Sheppard because we think we know who he is - not in terms of the narrative, but in terms of the genre in which that narrative falls.
Tobin, V. (2009) 'Cognitive bias and the poetics of surprise', Language and Literature 18 (2), pp. 155-172