Normally I would say software, or rather a raid-like filesystem like btrfs or ZFS. But in your specific case of funneleing it all through a single usb-c connection it is probably better to keep using an external box that handles it all internally.
That said, the CPU load of software raid it very small, so that isn't really something to be concerned with, but usb connections are quite unstable and not good for directly connecting drives in a raid.
I mean I've been running the setup this way for >4 years and never had any problem with the USB connection, so I cannot attest to "usb connections are quite unstable"...
I supposed that is because the JBOD box was handling the raid internally so short connection issues are not that problematic and can be recovered from automatically. But that wouldn't be the case if you connected everything together with a usb hub and usb to sata adapters and run a software raid on that.
I've been running a 4-disk RAIDz1 on USB for 4 years now with zero failures on one machine and one failure on another where it turned out the USB controller in one WD Elements was overheating. Adhering a small heatsink on it resolved the problem and it's been stable under load for 2 years now. The USB devices have to be decent. AMD's host controllers are okay. VIA hubs are okay. ASMedia USB-to-SATA are okay. I'm using some enclosures with ASMedia and some off-the-shelf WD Elements that also use ASMedia. It's likely easier to get a reliable system if installing disks internally as the PSU and interconnects are much more regulated and any would work well, whereas with USB you have to be careful in selecting decent components.
I read somewhere, years ago, that RAID6 takes about 2 cores, on a working server.
That may have been a decade ago, and hardware's improved significantly since then.
Bet on 1 core being saturated, min, with heavy use of a RAID6 or Z2 array, I suspect..
I'd go with software raid, not hardware: with hardware RAID, a dead array, due to a dead controller-card, means you need EXACTLY the same card, possibly the same firmware-revision, to be able to recover the RAID.
With mdadm, that simply isn't a problem: mdadm can always understand mdadm RAID's.
The argument for hardware RAID has typically been about performance. But software RAID has been plenty performant for a very long time. Especially for home-use over USB...
Hardware RAID also requires you to use the same RAID controller to use your RAID. So if that card dies you likely need a replacement to use that RAID. A Linux software RAID can be mounted by any Linux system you like, so long as you get drive ordering correct.
There are two "general" categories for software RAID. The more "traditional" mdadm and filesystem raid-like things.
mdadm creates and manages the RAID in a very traditional way and provides a new filesystem agnostic block device. Typically something like /dev/md0. You can then use whatever FS you like (ext4, btrfs, zfs, or even LVM if you like).
Newer filesystems like BTRFS and ZFS implement raid-like functionality with some advantages and disadvantages. You'll want to do a bit of research here depending on the RAID level you wish to implement. BTRFS, for example, doesn't have a mature RAID5 implementation as far as I'm aware (since last I checked - double-check though).
I'd also recommend thinking a bit about how to expand your RAID later. Run out of space? You want to add drives? Replace drives? The different implementations handle this differently. mdadm has rather strict requirements that all partitions be "the same size" (though you can use a disk bigger than the others but only use part of it). I think ZFS allows for different size disks which may make increasing the size of the RAID easier as you can replace one disk at a time with a larger version pretty easily (it's possible with mdadm - but more complex).
You may also wish to add more disks in the future and not all configurations support that.
I run a RAID5 on mdadm with LVM and ext4 with no trouble. But I built my RAID when BTRFS and ZFS were a bit more experimental so I'm less sure about what they do and how stable they are. For what it's worth my server is a Dell T110 from around 12 years ago. It's a 2 core Intel G850 which isn't breaking any speed records these days. I don't notice any significant CPU usage with my setup.
I used to use mdadm, but ZFS mirrors (equivalent to RAID1) are quite nice. ZFS automatically stores checksums. If some data is corrupted on one drive (meaning the checksum doesn't match), it automatically fixes it for you by getting the data off the mirror drive and overwriting the corrupted data. The read will only fail if the data is corrupted on both drives. This helps with bitrot.
ZFS has raidz1 and raidz2 which use one or two disks for parity, which also has the same advantages. I've only got two 20TB drives in my NAS though, so a mirror is fine.
If I were to redo things today I would probably go with ZFS as well. It seems to be pretty robust and stable. In particular the flexibility in drive sizes when doing RAID. I've been bitten with mdadm by two drives of the "same size" that were off by a few blocks...
ZFS kicks arse. It's worth learning enough to get a basic array going, with a couple of datasets and encryption. Once you get acquainted with that, you'll be using it for years to come.
Since hardware RAID is not state of the art anymore I will definetly stick with software RAID. I think I will just build a new server for the money, since an 8-Bay USB enclosure costs around 600€ and for that amount of money I can just build a new server with even better performance
Software, software, software! ZFS, mdraid, etc. USB is fine even with hubs, so long as your hubs and USB controllers (USB-to-SATA) are decent and not overheating.
just my 2 cents, if youre going to do raid, buy a thing that will do it...
a nas or enclosure where the hardware does all the heavy lifting.
do not build raided system from a bunch of disks... i have had, and have had friends have many failures over the years from those home brew raids failing in one way or another and its usually the software that causes the raid to go sideways.. mayvbe shits better today than it was 10-20 years ago.
its just off my list. i bought a bunch of cheap nas devices that handle the raid, and then i mirror those devices for redundancy.
Y'all must be doing something wrong because HW raid has been hot garbage for at least 20years. I've been using software raid (mdadm, ZFS) since before 2000 and have never had a problem that could be attributed to the software raid itself, while I've had all kinds of horrible things go wrong with HW raid. And that holds true not just at home but professionally with enterprise level systems as a SysAdmin.
With the exception of the (now rare) bare metal windows server, or the most basic boot drive mirroring for VMware (with important datastores on NAS/SAN which are using software raid underneath, with at most some limited HW assisted accelerators) , hardly anyone has trusted hardware raid for decades.
Y’all must’ve been doing something wrong with your hardware raid to have so many problems. Anecdotally, as an admin for 20+ years, I’ve never had a significant issue with hardware raid. The exception might be the Sun 3500 arrays. Those were such a problem and we had dozens of them.
So what were you doing wrong to have so much trouble with the Sun 3500's?
They were olde by the time I got to administer them and they were failing every so often. They were a pain to work with as I recall, but again no lost data. They would beep with no issues sometimes. One thing that wasn’t their fault was a previous admin had set them up in threes with all but one disk in a raid5 array. They were wondering why performance was crap and one array would just drop every week like clockwork.
Took a while, but I mirrored (VVM) the data off on spare 3500s set up properly. They ran okay then. It was just ancient storage. Glad to work on netapps now. So much smoother and sophisticated storage compared to the olden days. lol
i think any features that flesh out the 'other half' of the fediverse implementation will be a benefit. if this is a popular feature on mastadon, it sounds like a great way to add value to mbin for that audience.
Ok, cool. I was torn whether a new page would be better or to improve the existing microblog page. In any sort that is not "newest" it would not make sense to include boosts, likes, comments, etc. within that page.
When creating a new page I can also try to save the position one was in so it does not reset on a reload
I think a combined view is what people have been asking for, this sounds like a filter on the combined view of sub to me.
I guess it's a bit different if you're wanting to show comments as well. That sounds difficult to do in UX to make the context of a reply make sense, not sure how Mastodon does it, I guess there aren't super complex tree structures? Because the comment they could be replying to could have nothing to do with the original post, you have to show the entire tree some times
I meant everything together, yes. The big thing I mean is that it is a timeline, so strictly chronological and there are events like "someone you follow liked x" or "someone you follow boosted y". And that comments from people you follow appear there as well, like they do on mastodon
I'm still interested personally in fixing the search. Eg. I try to copy and past your mbin url from this thread, into my instance. It I won't give any result. Unless I remove the last 'Mastodon-like-timeline' section from the URI. And try to search again.
I'm still interested personally in fixing the search. Eg. I try to copy and past your mbin url from this thread, into my instance. It I won't give any result. Unless I remove the last 'Mastodon-like-timeline' section from the URI. And try to search again.
I think you only have to add the optional slug parameter to the ap route 🤔
gehirneimer.de
Top