Etsy!
Buy extra white, because you start off with that, then add the color to it on top. (White is always the cheapest! I used up a chunk of colors when I started-- which was a waste)
Don't buy from China. X_x; That stuff felts TERRIBLY.
And some stores still carry needle felting stuff but... now the selection is small or gone. :/ I've gotten it from michael's and hobby lobby in the past... but now I usually buy from people who hand dye on etsy/ebay.