marauding_gibberish142 @ marauding_gibberish142 @lemmy.dbzer0.com

Posts

15

Comments

344

Joined

3 wk. ago

Used 3090s go for $800. I was planning to wait for the ARC B580s to go down in price to buy a few. The reason for the networked setup is because I didn't find there to be enough PCIe lanes in any of the used computers I was looking at. If there's either an affordable card with good performance and 48GB of VRAM, or there's an affordable motherboard + CPU combo with a lot of PCIe lanes under $200, then I'll gladly drop the idea of the distributed AI. I just need lots of VRAM and this is the only way I could think of.

Thanks

Thank you, and that highlights the problem - I don't see any affordable options (around $200 or so for a motherboard + CPU combo) for a lot of PCIe lanes other than purchasing Frankenstein boards from Aliexpress. Which isn't going to be a thing for much longer with tariffs, so I'm looking elsewhere

Thank you, but which consumer motherboard + CPU combo is giving me 32 lanes of PCIe Gen 4 neatly divided into 2 x16 slots for me to put 2 GPUs in? I only asked this question because I was going to buy used computers and stuff a GPU in each.

Your point about networking is valid, and I'll be hesitant to invest in 25Gbe right now

Thanks but I'm not going to run supercomputers. I just want to run 4 GPUs separately because of inadequate PCIe lanes in a single computer to run 24B-30B models

I'm not going to do anything enterprise. I'm not sure how people seem to think of it this way when I didn't even mention it.

I plan to use 4 GPUs with 16-24GB VRAM each to run smaller 24B models.

I don't understand your point, but I was going to use 4 GPUs (something like used 3090s when they get cheaper or the Arc B580s) to run the smaller models like Mistral small.

Who elects these dumb goblins?

It is because modern consumer GPUs do not have enough VRAM to load the 24B models. I want to run Mistral small locally.

Thank you, I'll take a look

Thank you for the links. I will go through them

Aren't Epyc boards really expensive? I was going to buy 3-4 used computers and stuff a GPU in each.

Are there motherboards on the used market that can run the E5-2600 V4 series CPUs and have multiple PCIe Xi slots? The only ones I found were super expensive/esoteric.

Never heard of it. What is it about?

So, any plans to include LibreWolf and TOR browser

Please at least put Librewolf and Mullvad in there. If you're feeling generous, even TOR browser.

Has Waterfox resolved their security issues?

I'm interested too. Any specific job sites for Switzerland, Iceland, Luxembourg etc? I prefer the smaller countries.

Can someone tell me why we care about stories on a messaging app? Don't we have Instagram and its clones for that?

I really wish they would build YACY integration so self-hosters can participate

I'm loving the new code assistants that are popping up. I do find that they help with decent boilerplate code for me

No, I meant the logic where the browser would prompt the user to review and verify the cert for a particular website without consulting a CA. I run some self-signed certs already but I'd love to implement this in my homelab.

This is actually a great idea. Is there an opensource implementation of it?