Skip Navigation

InitialsDiceBearhttps://github.com/dicebear/dicebearhttps://creativecommons.org/publicdomain/zero/1.0/„Initials” (https://github.com/dicebear/dicebear) by „DiceBear”, licensed under „CC0 1.0” (https://creativecommons.org/publicdomain/zero/1.0/)S

SuspiciousCarrot78

@ SuspiciousCarrot78 @aussie.zone

Posts
4
Comments
46
Joined
2 wk. ago

  • Self-hosted Image/Video Gallery Options?

    Jump

  • Just for sake of completion

    https://piwigo.org/

    Pros

    Mature project (around since the early 2000s)

    Lightweight compared to Immich

    Designed as a photo library first, not an AI platform

    Albums, tags, metadata, permissions

    Huge plugin ecosystem

    Runs happily on modest hardware

    Can manage very large collections

    Doesn't demand phone-app-centric workflows (though of course it has a phone to computer app / sync)

    Cons

    Feels more like a traditional photo archive than Google Photos

    Mobile experience is functional rather than slick

    No fancy AI search or face recognition by default (though can add easy enough)

    UI is a bit "classic web"

  • Another reason to self host your own AI

    Jump

  • Huh - cheaper than the P40s (though less VRAM) but larger bandwidth due to HBM2. Good looking out

  • Another reason to self host your own AI

    Jump

  • Good tips - thanks!

    PS: sad to report the 24GB Tesla p40s are now around $250 USD on eBay, so not quite as cheap as I remembered. P4s are still cheap tho, though frankly if you're going that end of town, a 1080 is about on par, less fussy and probably cheaper - it just won't fit in a uSFF.

  • Another reason to self host your own AI

    Jump

  • You probably could. A Tesla P4 or P40 (old data centre cards) are more than up to the job. My Lenovo tiny hosts a P4 (card cost $100 on eBay; the lenovo itself was $200ish) and runs Qwen3.5-35B-A3B at about 20 tok/s. Smaller models are even faster.

    https://www.youtube.com/watch?v=8F_5pdcD3HY

    If you're not bound by the one liter shoebox design, then the P40 is still a great and inexpensive card.

    I think I mentioned elsewhere but right now I'm trying to figure out if I can use a magic packet from the Raspberry Pi to wake up the Lenovo as needed rather than leaving it on all the time.

  • How many and how much are your subscriptions?

    Jump

  • Agree. I know the Pi's are out of favour these days...but they are a cool little machine. I got mine running DietPi and a bunch o crap (the usuals - JF, arr stack, pi hole, syncthing, yadda yadda) and running headless the footprint (power and memory wise) is tiny.

    I joked about the 4xAA batteries thing but iirc, there is actually a Pi-HAT that creates a micro UPS that'll run the pi for maybe three to five hours just on double A batteries.

    Edit: yep

    https://pimodules.com/product/ups-pico-hv4-0-advanced

    or more sensibly

    https://littlebirdelectronics.com.au/collections/raspberry-pi-power-hats/products/raspberry-pi-ups-hat

  • Another reason to self host your own AI

    Jump

  • Agree. And re small models - very agree. In fact I made a ablated version of Qwen 3.5-2B for use with my pi, before thinking a bit harder and realising I can probably code something bespoke that doesn't need a stochastic parrot as a squwake box at all.

    https://huggingface.co/BobbyLLM/polaris-heretic-Q4_K_M-GGUF

    Still, as a SLM, it's perfectly cromulent and does well with tool calling etc which is what I wanted it for.

  • Another reason to self host your own AI

    Jump

  • There's an argument to be had regarding a MoE versus a small dense model. I guess it depends on what exactly you need doing with it. I would be tempted to run a smaller dense model (like a Qwen 3-14B or a Qwen 3.5 9B) as at a reasonable quant, it might fit mostly or entirely on the GPU, thereby giving you excellent speeds.

    PS: I'm actually in the process of designing an expert system (not a LLM) for pretty much the task you described. The intention is that you would still interact with it like a large language model, but the actual brains underneath it would be something more traditional.

  • How many and how much are your subscriptions?

    Jump

  • Which trackers if you don't mind saying? DM me if easier.

  • Selfhosted @lemmy.world
    SuspiciousCarrot78 @aussie.zone

    Another reason to self host your own AI

  • How many and how much are your subscriptions?

    Jump

  • Yep. But that would be 100% CPU, 100% of the time? Real life, it's probably closer to 2w idle and maybe 5-7W under typical load.

    More interesting...I think that technically means you could make a "UPS" for it using what...4xAA batteries?

    Oh man...that would be cool. Stupid but cool.

  • How many and how much are your subscriptions?

    Jump

  • They were, I think. Or we were just younger.

  • How many and how much are your subscriptions?

    Jump

  • Yeah, same. Though at 3-5W ... it really is just a very rough guess. Lemme ShitGPT it. Oh, I was way off

    A realistic Pi 4B-only estimate is about A$8–A$12 per year in electricity, assuming it is on 24/7 and used for Jellyfin streaming around 10–12 hours per week.

    Pi 4B measurements are typically around 2.7–2.85 W at idle, about 5.1 W under moderate server load, and around 6.4 W under full CPU stress. Using Perth/WA’s Synergy Home Plan A1 energy charge of 32.3719 c/kWh, excluding the daily supply charge, that works out very cheaply because the device uses only about 25–36 kWh/year.

    Scenario Assumed usage Annual energy Approx. annual cost

    Mostly idle 3 W 24/7 26.3 kWh A$8.51/year Idle + 12h/wk Jellyfin 2.7 W idle, 5.1 W streaming 25.1 kWh A$8.14/year Heavier Jellyfin/server use 2.7 W idle, 6.4 W streaming 26.0 kWh A$8.40/year Conservative wall-power estimate 4 W idle, 6.4 W streaming 36.5 kWh A$11.83/year

    The bigger swing factor is storage, not the Pi. A USB SSD adds very little; a USB-powered 2.5" hard drive might add a few dollars per year; a powered 3.5" external drive left spinning 24/7 could push the total more into the A$15–A$30/year range.

    So, for the Raspberry Pi 4B itself as a Jellyfin box: roughly A$10/year is a good mental estimate.

  • How many and how much are your subscriptions?

    Jump

  • I remember it being a touch more ...analog...back in the day. ATDT commands and all.

    But yeah, Win 3.11+ trumpet winsock and Free Agent were the shit. Rec.martial.arts was home back then (along with mIRC).

    Lemmy reminds me a bit of the old Usenet fora.

  • How many and how much are your subscriptions?

    Jump

  • Torrent cache? As in seedbox?

  • How many and how much are your subscriptions?

    Jump

  • Use to last me 2-3 months... but my media library is more or less complete now, with little churn. Also, I don't ever go above 1080p.

    I need to check if Radarr / Sonarr works with straight torrents (it must do; I haven't used them for ages / have been using 1337 manually, but I seem to recall torrents being a source).

  • How many and how much are your subscriptions?

    Jump

  • Debatable :) Torrents rely on seeders. I've downloaded movies and TV shows >5 yrs since initial upload via Usenet. Yes, things expire there too (eventually), but when the getting is good, it's uniformly good / fast.

    OTOH, 1337 has been pretty decent to me of late.

    It's tricky. On one hand, Jellyfin and the arr stack are what got me into self hosting. OTOH...torrents are simpler - I can plug my external SSD directly into my router, which streams to NovaPlayer on any android device - nothing else needed. Want a new show / movie? Grab the torrent, punt it across to ssd via samba share. It auto populates.

    https://github.com/nova-video-player/aos-AVP

    It's...simpler. Arguably more elegant / less moving parts.

    Dunno.

  • How many and how much are your subscriptions?

    Jump

  • Yarrr! But it really mostly is Yarr these days. So don't go firing up Trumpet winsock to check Forte Agent :)

  • How many and how much are your subscriptions?

    Jump

  • I was tempted to say $0, but then I thought harder about the problem.

    Technically I do have ongoing costs

    • PAYG costs for Usenet-news (iirc, $22USD for 500GB block)

    https://usenet-news.net/index1.php?url=home

    • News indexer (I think...$60 every 5 years?)

    https://www.nzbgeek.info/

    Electricity (whatever tiny amount raspberry pi sips). At a guess, maybe $50/yr.

    So, amortised over time - very low but not zero. In theory, if I dropped Usenet, it would even lower. And theoretically, I could run the pi off a single solar panel and a diy solar kit but I'm not busy pretending to be Robinson Crusoe just yet. Though... It might be a cool project.

  • Privacy-preserving alternative to Ring cameras (Raspberry Pi Zero 2W)!

    Jump

  • No good deed goes unpunished. The sense of self entitlement some people display is staggering. FOSS project? Well, you should have done x y or z.

    Also, I gave you $3 via Ko-fi, so you need to provide customer support in perpetuity and come to my house and install it. And heaven forbid you try to recoup costs!

    Projects don't just die out - a lot of them are killed (one way or another). For example, I had a fully specced out FPGA design that would capture the signal from Wii GPU and do internal upscaled resolution (think: like what dolphin emulator does but with actual hardware) not just post process sharpening. Total cost under $100 and some know how.

    The amount of flack I copped for it made me shut down the github and work on it for myself. Once it's perfected, I may post about it again but I sure as shit am not compelled to deal with the fucking peanut gallery anymore.

  • Your best local LLM for low-VRAM (6GB)?

    Jump

  • There are many excellent options - far too many to list. So I will briefly say - there are some really nice 4B models (like Qwen3-4B HIVEMIND, Nanbeige, IBM Granite 3B) which you should be able to run at higher quants (Q6 and up) quite nicely. Of course, there are always newer models (Gemma, Qwen3.6 - soon 3.7) etc.

    Best bet is to poke around hugging face, on TheBloke, Unsloth or DavidAUs archives and see what they have in the 3-7B range that tickles your fancy. Don't immediately jump for the newest releases - the old ones are still good. Qwen3-4B 2507 instruct is still a favourite of mine and more recently Qwen3.5-2B shows promise.

  • LocalLLaMA @sh.itjust.works
    SuspiciousCarrot78 @aussie.zone

    Claude? No. Cucumbers? Yes!

  • LocalLLaMA @sh.itjust.works
    SuspiciousCarrot78 @aussie.zone

    "The cost of running LLMs is just too damn high"

  • LocalLLaMA @sh.itjust.works
    SuspiciousCarrot78 @aussie.zone

    Token Speed visualiser

    mikeveerman.github.io /tokenspeed/