Breast growth is generally permanent after 3 months of being on E.
Just from some quick reading, I would not think of raloxifene as entirely preventing breast growth, but it looks like it would result in less dense breasts (which is why it is prescribed for post-menopausal women to help prevent breast cancer; note that it does not seem to have this effect in pre-menopausal women precisely because they have so much estrogen available - this indicates raloxifene would be most effective at reducing breast density in women with very low or no estrogen).
There is a difference between preventing the breast buds from forming and growing and the breast tissue itself being as dense, I would not count on raloxifene to prevent breast bud growth or permanent breast growth. Also, raloxifene has other side effects and risks associated with it like increasing risk of blood clots that would make me quite worried to be taking it, especially without guidance from medical doctors.
The fat that was distributed to the breasts under estrogen would gradually be redistributed if you went back on T, but this doesn't mean the breast bud growth would be undone, that is the permanent change.
Honestly I think it is unrealistic to consider being on E for only a couple years to get some fat redistribution and then detransitioning intentionally. The fat won't redistribute immediately and might take longer than a couple years to happen, and detransitioning will just put fat back where you didn't want it.
Switching your main sex hormones is not a trivial thing, and there can be mood swings and instability as you change, similar to the onset of puberty. You might consider trying estrogen for a couple months just to see if you feel better on estrogen, and if you do, you might have to make a choice about whether the benefits of staying on estrogen outweigh the risks for you.