While you can buy winter wear at a variety of sources, including cut-rate shops, it's best to shop at places that are targeted to winter sports as these will offer the best designs and quality.
You can always find cheaper workout clothes at cut-rate shops, but while you don't have to get something high-end, it's worth it to spend a bit more and buy clothes with thoughtful design.
The famous shop chain "Sympa" will have bins of cut-rate bras and panties outside, all in patterns and fabrics that can be far more exquisite than anything found elsewhere.
You can also find good rash guards on sites such as Amazon, REI and Sierra Trading Post, the latter especially is known for cut-rate prices on excellent gear.
Bargain Children's Clothing offers just that - children's clothes at cut-rate prices.