SD XL - Slow performance?

Anyone else struggling to run SD XL? Im on an older PC with a 10 gig 3080, 32GB system RAM. Using automatic1111.

Loading up the model alone is a struggle, but I was able to get stable performance (and no nan errors, or out of memory errors) by using med vram and sdp attention (as opposed to xformers, which was having errors after a generating a few images).

The real weird part it will straight up freeze up trying to load the model, or generate an image - unless I minimize my browser. Then suddenly it starts working at a decent 2.8 it/s (1024x768, DPM++ 2M SDE KARRAS, 25 samples). Needless to say, I havent even attempted to use the refiner model, as switching/loading models takes several minutes.

Just curious what everyone's experience with trying to run SD XL so far has been? The output is awesome and playing with it has been pretty fun, but frustrating.