Yet there was one present who recognized him as soon as he entered, and watched him with strong interest.

The interest was not friendly, however, as might be inferred from the scowl with which he surveyed him.

This will not be a matter of surprise to the reader when I say that the observer was no other than Fairfax, whose attempt to rob Colonel Preston had been defeated by Andy.
He recognized the boy at once, both from his appearance and his voice, and deep feelings of resentment ran in his breast.

To be foiled was disagreeable enough, but to be foiled by a boy was most humiliating, and he had vowed revenge, if ever an opportunity occurred.

For this reason he felt exultant when he saw his enemy walking into the eating house.
"I'll follow him," he said to himself, "and it'll go hard if I don't get even with him for that trick he played on me." But how did it happen that Andy did not recognize Fairfax?
For two reasons: First, because the adventurer was sitting behind him, and our hero faced the front of the room.