I've looked into this in the past.
From what I know, red clover is only a phytoestrogen, although I'm not saying it might not have some anti-androgen traits (I just don't know about any). So you want to look into a legit anti-androgen (I use Saw Palmetto extract, and a lot of people swear by spearmint tea).
Testosterone is what builds muscle in the body. An anti-androgen will prevent the T from turning into DHT. A lot of people think this is why it becomes harder to grow muscle while taking an anti-androgen. However, DHT has nothing to do with creating muscle, so preventing it from making that transformation isn't affecting your growth at all.
ON THE OTHER HAND, when T can't transform into DHT, the body instead transforms it into estrogen. This is what may make it harder for some women to grow muscle while on anti-androgens, because high E makes it harder for the body to build muscle (hence why men are larger than women). I'm still not 100% sure on how much this conversion switch affects most women, so I'd love to see BrazilianBabe ellaborate more again later.

Personally, I've been taking anti-androgens along with estrogenic and progesterone herbs this entire past year. I have still been able to build muscle (I'm finally getting hamstrings now that I've joined a gym), although sometimes I wonder if I were to stop taking my anti-androgens, or even lowering the dose, if that would make a huge difference. On the plus side, the extra estrogen flowing throughout my body from the T conversion has helped me go from my previous banana shape to my current curvy pear shape. Due to my incredibly high T levels, I think that by leveling out my hormones, I've been able to achieve my natural intended shape without even having to worry about exercise (although I must say, the exercise has definitely added to my shape!)
It might work differently in different people's bodies, so you can always do as Babygurl suggested and go a few months on, few months off. Just keep in mind that some people grow a lot of muscle at first and then plateau, while others don't grow at all and then suddenly hit a growth spurt. So this could affect your results as well.
I actually ran out of my Saw Palmetto recently, and don't have the money yet to buy more, so I may be trying this experiment whether I like it or not. :/ Although, unfortunately, without SP, I won't be able to take my maca, as it gives me terrible acne. But once I do buy it again, I may try taking less this time around. Save money AND see if it helps me build muscle!
Good luck in the experiment!