There is an idea or saying that elephants never forget or that they have very good memories First article in google about this.

So to say you have a memory of an elephant could mean your memory is very good.

The twist is that this can also mean you remember seeing an elephant, which is what the second line implies.

An old saying that Dads are likely to have heard is “an elephant never forgets”. So “I have the memory of an elephant” is saying “I never forget.”

Unless you’re making a Dad joke.

In which case it means “I remember seeing an elephant once”.

Must be one of those giant Costco mayonnaise jars.