You don't really have to, but I always do because mine tend to crumble after rigorous scrubbing. I believe that you only need to change erasers once they fall apart, so I usually only tear off a piece that I know I'll need for the pony. Still, I scrub kind of rough to make sure that the pony gets clean, so sometimes the little particles can get stuck in their hair.
I'm pretty sure that they don't contain any cleaning agents, just that they are made as a super, super fine sandpaper and it's the abrasion that takes the grime off. Because of that, I don't believe that any harmful chemicals can stay on the pony once they had a bath.