I prefer a toothpick. Wood won’t scratch the metal or cause a short, but it’s still stiff enough to scrape any lint that’s stuck. And lint sticks a bit to the toothpick, so that helps get it out.
Neither will the plastic of a floss pick. And the floss pick is narrower so there is much less risk of deforming the interior parts of the plug. Also, less risk of splinters.