In mammals, the effects of estrogen are mainly mediated by two different estrogen receptors, ERα and ERβ. These proteins are members of the nuclear receptor family, characterized by distinct structural and functional domains, and participate in the regulation of different biological processes, including cell growth, survival and differentiation. The two estrogen receptor (ER) subtypes are generated from two distinct genes and have partially distinct expression patterns. Their activities are modulated differently by a range of natural and synthetic ligands. Some of these ligands show agonistic or antagonistic effects depending on ER subtype and are described as selective ER modulators (SERMs). Accordingly, a few phytochemicals, called phytoestrogens, which are synthesized from plants and vegetables, show low estrogenic activity or anti-estrogenic activity with potentially anti-proliferative effects that offer nutraceutical or pharmacological advantages. These compounds may be used as hormonal substitutes or as complements in breast cancer treatments. In this review, we discuss and summarize the in vitro and in vivo effects of certain phytoestrogens and their potential roles in the interaction with estrogen receptors.