And what worried Matthew was that the difference impressed him as being something that should not exist.

Anne had a brighter face, and bigger, starrier eyes, and more delicate features than the other; even shy, unobservant Matthew had learned to take note of these things; but the difference that disturbed him did not consist in any of these respects.

Then in what did it consist?
Matthew was haunted by this question long after the girls had gone, arm in arm, down the long, hard-frozen lane and Anne had betaken herself to her books.

He could not refer it to Marilla, who, he felt, would be quite sure to sniff scornfully and remark that the only difference she saw between Anne and the other girls was that they sometimes kept their tongues quiet while Anne never did.

This, Matthew felt, would be no great help.
He had recourse to his pipe that evening to help him study it out, much to Marilla's disgust.