and so far as I remember they became imaginative about the Temple, and fancied that the Unknown God of it would help them to regain their private affairs: one of them wanted to get back to his girl, another to his favourite pub, another to his money-making, another to his collection of miniatures.

And they used to sit and look at the Temple day after day and expect something to happen.

When the ship came at last they wouldn't go into it because they couldn't bear to think that something should happen at last and they not be there to see it.

Oh yes, one of them went back, I remember.
But his actual meeting with his girl was so disappointing in comparison with his long expectation of it in front of the Temple that he took the next boat back to the island ...

but he never found it again.