The color is coming out or the yarn is? Are you working with yarn or with unspun cotton? Brushing unspun cotton or wool will make tufts come out in the brush. I'm not sure what will stop that other than using a longer staple (a staple is the length of the individual hairs of wool, how long they let it grow before shearing the sheep). I'm not much of a customizer so I'm not sure what you artsy types do, but working with unspun wool is a hobby of mine. Let me do a little research and I'll see what I can come up with for you.
Is this a color question or a cotton/wool question? If we are talking about color coming out, when working with wool try adding a little bit of vinegar to the dyebath or washing it in diluted vinegar before dropping it in the dye bath. Working with cotton needs something else but it escapes my memory at the moment. I'll look that up real quick too. I have alot of wool/cotton and dye books.