Don’t use them unless they’re part of a name, like M&S or M&Ms. They look old-fashioned at best, affected at worst.
The symbol is a ligature of the letters in et, Latin for ‘and’. The word ‘ampersand’ comes from the fusion of ‘and per se and’ which means ‘[the symbol which] by itself [is] and’. Don’t worry, we don’t follow either.