Welcome to Incremental Social! Learn more about this project here!
Check out lemmyverse to find more communities to join from here!

What's your server wattage?

I'm in the process of wiring a home before moving in and getting excited about running 10g from my server to the computer. Then I see 25g gear isn't that much more expensive so I might was well run at least one fiber line. But what kind of three node ceph monster will it take to make use of any of this bandwidth (plus run all my Proxmox VMs and LXCs in HA) and how much heat will I have to deal with. What's your experience with high speed homelab NAS builds and the electric bill shock that comes later? Epyc 7002 series looks perfect but seems to idle high.

scarecrow365 ,

I've got a 3 node Proxmox/ceph cluster with 10G, plus a separate Nas. They are all rack mount with dual PSU. Add in the necessary switching, and my average load is about 800w. Throw my desktop (also on 10G) into the mix and it runs 1.1kw.

That's roughly $50-60 extra in electricity costs for me monthly.

Cobrachicken ,

Would be around 300€ in Germany, on a cheap contract.
Limiting myself to one combined NAS/application server atm, with the others turned on only if I want to try sth out.

kylian0087 ,

I ise about the same. But that is more due to the hardware I got being a bit older. 2 dell R710s 1 R510 and a custom build server. Everything is still 1g. In my case electricity is not a big deal due to solar. We produce much more then we can use our self.

johnnixon OP ,

I'm afraid of dumping 500+ watts into a (air conditioned) closet. How are you able to saturate the 10g? I had some idea that ceph speed is that of the slowest drive, so even SATA SSDs won't fill the bucket. I imagine this is due to file redundancy not parity/striping spreading the data. I'd like to stick to lower power consumer gear but ceph looks CPU, RAM, and bandwidth (storage and network) hungry plus low latency.

I ran proxmox/ceph over 1GB on e-waste mini PCs and it was... unreliable. Now my NAS is my HA storage but I'm not thrilled to beat up QLC NAND for hobby VMs.

scarecrow365 ,

My 10G is far from saturated, but I do try and keep things using RAM where possible. I figure that with 100gb of DDR4 in my main server, that should be able to provide enough speed for a 10G link.

I've got ceph running on Intel Enterprise SSDs, so they are pretty quick.

I also tried running ceph on 1G. I found it unreliable as well.

suzune ,
@suzune@ani.social avatar

I recently removed my 25Gbps PCIe dual port cards from my 2 servers because they were using 20W more. My entire rack including 2 UniFi PoE connections uses 90 W now (so 110 W just for having 25 Gpbs).

There is some heat from such cards, but usually it gets transported outside fine. The ones I bought did not come with a fan. I think you cannot operate them without one. The heat sinks get very hot.

thejevans ,
@thejevans@lemmy.ml avatar

I use a Ryzen 5900x, RTX 3080, 2x 10Gbit sfp+ NIC, 128GB ECC RAM, and only 2x 20TB drives at the moment.

For my gateway, I have an Intel N6005 box, I have a managed 2.5/10Gbit switch, and I have a wifi AP.

I have a ton of Proxmox VMs and containers.

All that hovers between 140W to 180W

Decronym Bot , (edited )

Acronyms, initialisms, abbreviations, contractions, and other phrases which expand to something larger, that I've seen in this thread:

Fewer Letters More Letters
AP WiFi Access Point
DHCP Dynamic Host Configuration Protocol, automates assignment of IPs when connecting to a network
DNS Domain Name Service/System
HA Home Assistant automation software
~ High Availability
LXC Linux Containers
NAS Network-Attached Storage
NUC Next Unit of Computing brand of Intel small computers
NVMe Non-Volatile Memory Express interface for mass storage
PCIe Peripheral Component Interconnect Express
PSU Power Supply Unit
PiHole Network-wide ad-blocker (DNS sinkhole)
PoE Power over Ethernet
RPi Raspberry Pi brand of SBC
SATA Serial AT Attachment interface for mass storage
SBC Single-Board Computer
SSD Solid State Drive mass storage
Unifi Ubiquiti WiFi hardware brand
VPS Virtual Private Server (opposed to shared hosting)

[Thread for this sub, first seen 4th Jun 2024, 04:35]
[FAQ] [Full list] [Contact] [Source code]

mesamunefire ,

My pi costs probably around 20 a year lol.

cmnybo ,

The load on my UPS is around 100-140 watts. That includes my server, firewall, switch, starlink and a unifi access point. I would love to get that power consumption down. I only get 4-5 hours of runtime on battery. Also, the room it's in is small and it gets really hot in the summer time.

Retiring ,
@Retiring@lemmy.ml avatar

82.2W average for which I pay 144.6€/a at the moment. That’s for a Ryzen 7 3700X, some hard drives and SSDs and the fiber connection to my basement. I outsourced 90% of media consumption to a VPS though, that’s another 84€/a.

PieMePlenty , (edited )

I run a NUC11 so about 10W. 15-20€ per annum assuming a single tariff at 0.17€ per kwh.
It can use up to 30W but only during heavy load which may be like 8 hours a week. But electricity is also cheaper during off peak hours so it averages to about that (we have 5 tariffs).

Load is NAS, media server, homeassistant and a usb zigbee router, *arr stack.

Power usage was my main concern and wanted something eco friendly.

kokesh ,
@kokesh@lemmy.world avatar

7W I think

just_another_person ,

If you're just running home automation, you do not need an Epyc 🤣

Get a low power anything to just run what you need.

wreckedcarzz ,
@wreckedcarzz@lemmy.world avatar

I just moved my home assistant docker container to a new-to-me Xeon system. It also runs a couple basically idle tasks/containers, so I threw BOINC at it to put it to good use. All wrapped up with Debian 12 on proxmox...

(I needed USB support for zigbee in ha, and synology yanked driver support from dsm with the latest major version, so 'let's just use the new machine'...)

johnnixon OP ,

I looked at Epyc because I wanted to bandwidth to run u.2 drives at full speed and it wasn't until Epyc or Threadripper that you could get much more than 40 lanes in a single socket. I've got to find another way to saturate 10g and give up on 25g. My home automation is run on a Home Assistant Yellow and works perfectly, for what it does.

just_another_person ,

Some unsolicited advice then: don't go LOOKING for reasons to use the absolute max of what your hardware is capable of just because you can. You just end up spending more money 🤑

For real though, just get an N100 or something that does what you need. You don't need to waste money and power on an Epyc if it just sits idle 99% of the time.

johnnixon OP ,

What I need is a 10g storage for my Adobe suite that I can access from my MacBook. I need redundant, fault tolerant storage for my precious data. I need my self hosted services to be high availability. What's the minimum spec to reach that?
I started on the u.2 path when I saw enterprise u.2 drives at similar cost per GB as SATA SSDs but faster and crazy endurance.
And when my kid wants to run a Minecraft server with mods for him and his friends, I better have some spare CPU cycles and RAM to keep up.

just_another_person ,

Get a Drobo if you're that worried about that kind of access then. Make it simple.

Otherwise anything with two NICs is the same thing.

MangoPenguin ,
@MangoPenguin@lemmy.blahaj.zone avatar

You could technically do that from like 2x ~$150 used business desktop PCs off ebay, 10th gen Intel CPU models or around there with Core i3/i5 CPUs.

Throw some M.2 SSDs in each one in a mirror array for storage, add a bit of additional RAM if needed and a 10G NIC. Would probably use about 30-40W total for both of them.

Minecraft servers are easy to run, they don't need much especially on a fairly modern CPU with high single thread performance, and only use maybe 6GB of RAM for a modded one.

You're not asking for a whole lot out of the hardware, so you could do it cheap if you wanted to.

farcaller ,

I run 3900X with a 40Gbit fiber, packed with HDDs and nvmes. The box fluctuates around 90-110W use.

johnnixon OP ,

Where do you find the bandwidth to do all that? NVME eats it up and the 40g too.

farcaller ,

I did ran out of pcie, yeah :-( the network peaks at about 26gbit/s, which is the most you can squeeze out of pcie 3.0 x4. I could move the nvmes off the pcie 4.0 x16 (I have two m2 slots on the motherboard itself), but I planned to expand the nvme storage to 4x SSDs and I’m out of the pci lanes on the other end of the fiber either way (that box has all x16 going to the gpu)

ExcessShiv ,

I'm running my smart home entirely from a single NUC running proxmox with VMs and LXCs for my services. It's pulling ~7W on average

Strit ,
@Strit@lemmy.linuxuserspace.show avatar

Mine is about 8W on average.

It's an Odroid H3 that runs Nextcloud, Jellyfin, AudiobookShelf, a bunch of websites and Home Assistant.

It has 2x Sata SSD's connected.

This setup is not high speed at all, so it's not what you asked about. I just answered the headline question. ;)

If any air ventilation fan turns on in the house it uses at least 3x that power, so I don't calculate the price on my servers power draw as it almost not noticable.

beeng ,

Thinkcenter tiny, 4 external HDDs, a DAC, a raspi3b+, was like 25W I think.

randombullet ,

From the wall I'm pulling 120w

Ryzen 5700G

128GB ram

2tb + 4tb NVMe drive

2 x 20tb HDDs

Unifi Enterprise 24 PoE

Mikrotik RB5009

2 access points

3 cameras

Fiber runs cooler than copper all of my SFP+ are fiber.

theshatterstone54 ,

I feel almost obliged to ask: what are you running on this monster of a setup?

aStonedSanta ,

You know he’s just running docker.

randombullet ,

Mostly for PiHole.

  • All
  • Subscribed
  • Moderated
  • Favorites
  • •
  • selfhosted@lemmy.world
  • random
  • incremental_games
  • meta
  • All magazines
    •