maybe it has to be a memory specific to the person - like if he were to casually mention something he remembered about brian to debbie, he wouldn't forget it, but if he were to tell brian it would pop.

or maybe it's a memory that only he would know, so if he was talking to someone about the king of babylon contest, and he mentioned the blond twink who won, that would be something anyone who was there would remember and he wouldn't lose it.

but clearly the speaking out loud triggers it.... argh!