Send feedback to @storage_jm on keybase or leave a comment here
The goal of farming efficiently is to store the largest amount of data in the smallest amount of space with the lowest power. Generally, this involves tradeoffs of cost, availability, reliability, cooling, and noise, based on the size and location of the farm.
Farming in the Chia network is a very lightweight process from the CPU side, so the goal of a farmer platform really revolves around IO connectivity and the number of drives you can attach. This quickly expands to other goals, which we will explore in a few key concepts
Chia Farming Goals
- Maximize the amount of storage capacity you can fit in a given footprint. Real efficiency is measured in TB per cubic meter. In the data center, this is commonly referred to as TB/rack unit. In a desktop, we are going to simplify things and use a number of drives per farmer in a specific form factor. To state, it simply put as many drives in the smallest amount of space possible. For HDDs to pick, avoid temptation on price and use the highest capacity available.
- Spend as little as possible. $/TB attached or $/drive attached since the system cost is amortized over how many drives the farmer can attach the more drives you have per system, the lower the cost per drive.
- Minimize power consumption. We want Chia to be as energy-efficient as possible, so we want the supporting farming platform to minimize the power overhead of the farmer. The drives consume the majority of the power in the farmer.
- Power efficiency. We want the power we consume to be used by the hard drives and the farmer, not to be wasted in power supply efficiency from the regulation of AC to DC power. In desktops, we can simplify how desktops PSUs are already rated for efficiency, in Bronze, Gold, and Platinum. (sweet spot is Gold, taking point 2 into account)
- Minimize noise (measured in dB). If you are hosting your farmer at home, you likely need it to be reasonably quiet. Ignore this if you are putting in colocation, a datacenter, or a remote location where noise is not an issue. The tradeoff for noise is usually with cooling (fan speed) and power.
- Keep drives cool. HDDs last longer (fail less) when they are kept in the recommended temperature operating range. This varies between consumer and data center grade HDDs. To do this requires adequate airflow and fans in the farming case, and also monitoring with SMART.
- Maintain a reliable farmer. We do not want the farming system to go down. We do not want any single failed component on the farmer to take the whole system down. We do not want farming downtime (this is lost time and XCH farming)
- Host connectivity. In general, farming will be either remotely connected through the network (NAS, network-attached storage) or directly attached to the farmer (DAS, direct attach storage)
- Redundancy & RAID. The majority of storage systems are designed with durable storage in mind, and contain redundancy with RAID, erasure code, mirroring, or backups. This is not required in Chia because plots can be recreated upon a device failure and the opportunity cost of using redundant storage vs having that space farming generally outweighs the energy and time to create the plots. This may not be the case for very small farmers with limited plotting power, but the overwhelming majority of users should not need to use any sort of RAID with Chia.
- Nice to haves (these are standard in JBODs that make life much, much easier)
- Hot-swap, field-replaceable drives: the ability to swap a hard drive without taking the system offline (turning it off), generally accessible through the front or top of the chassis
- staggered spin-up: hard drives consume much more power (25W) for the first 10 seconds or so that they are turned on, from something called in-rush current. Compare this with the 3.5-5W required for idle operation in farming per drive. Staggered spin-up spins up only a certain number of drives at a time to limit max power consumption
- slot identification: hard drives fail. When this happens you want to easily be able to identify a slot that contains the failed drive. This is done through a LED locate.
|Storage capacity per farmer||TB, TiB|
|Storage capacity per space||TB/m3 , TB/RU, drives/system|
|Capital Expenditure (cost) of farmer||$/drive, $/TB, $/system|
|Power||TB/W, TB/drive, W per system|
|Power efficiency||PUE for data center, Power supply efficiency % for desktop|
|Operational expenditures (power)||$/TB/month|
|Drive temperatures||Degrees C|
|Reliability||drive failures (AFR, MTBF), system component failures|
|Availability||% uptime (99.9%, 99.99%)|
|Durability||Percent chance of not losing data|
|Blast Radius||Amount of devices and storage capacity lost upon device failure (e.g. one drive, one system)|
As you can see there is a large list of tradeoffs. The good news is, storage is already being deployed in large-scale data centers and for the most part this is a problem that has already been solved, as you will see when we get towards the bottom into enterprise storage arrays, JBODs that are designed to solve all these challenges. The interesting space is somewhere in between. Lots of small desktop farmers don’t need to invest in a JBOD, but they want to know some options for growing beyond a few USB drives.
I do not include any USB in this guide. You will quickly notice that USB external HDDs, like the Seagate 16TB External are less expensive in $/TB than the exact same model of drive in an internal 3.5in form factor (this model has an Exos inside). This is because the market for external storage high capacity drive is larger in the consumer space (today). If you are just getting started, just buy a few externals and start plotting. There are USB-only farms that work, and USB is very handy for hooking up a few drives to a desktop or laptop. External enclosures come with USB natively and many farmers do not want to void the warranty by “shucking” it out of the enclosure. USB enclosures use very low-cost bridge chips that convert USB to native SATA and SCSI (SAS) commands. This is one more place for things to go wrong, incompatibility issues, and in general, just not required. While generally, these USB adapters that come with the external drives are plenty reliable, I don’t recommend them for larger farms. SATA is already a VERY low-cost attach of storage at <$0.50 per drive for cables. If you are only hooking up a few drives, or want to reuse existing hardware, by all means, don’t be afraid to use USB. These are also set up for more aggressive power-saving and may go to sleep when not being accessed for a long time (which means they aren’t farming) so be mindful when setting up a new USB HDD.
Farming for 4-8 drives
If the total drives in the farm are between 4-8, I would recommend a standard mid-tower case for the overwhelming majority of people. There are many good cases that hold a high amount of HDDs. You can also combine this with a basic plotter for a very simple setup. There are dozens of options for great cases (please let me know your favorites!) but here are some of the ones I use regularly.
Desktop cases I recommend:
8 drives whisper silent. Airflow could be better. Very easy to work with. See on the right a combined plotter and farmer in one that I built for a family member, that we are going to populate with 8 drives.
Small form factor, little more challenging to mount drives
SilverStone Technology Premium Mini-Itx/DTX Small Form Factor NAS Computer Case
Dedicated NAS case – small footprint, hot-swap drives. Just wish they would make the pricing more reasonable…
Up to 12 drives in a standard desktop full tower
With up to 12 drives, the options expand significantly. You likely have multiple dedicated plotters at this point. An easy first step is to build out a full tower desktop. This is an easy jump for most PC enthusiasts and builders since you have a familiar desktop case to work with.
With 9 slots out of the box, and plenty of room to mount 5.25in cage like these
Note on using these aftermarket hot-swap 3.5in HDD cages. SAS uses higher signaling voltage, and even with a SATA HDD, I’ve had better luck using these with SAS HBAs vs directly attaching SATA connectors to the back from the motherboard. For what it’s worth, I have 3 different models and I see similar behavior (like 2 or 3 out of the 4-5 bays work) when using SATA from the motherboard. My hypothesis is the signaling voltage drop on the backplane.
Network Attached Storage
Home NAS are great for storing a large amount of drives in a small space and are very power efficient (dedicated backplane and power regulation, instead of inefficient ATX power supplies). NAS storage is generally designed for data redundancy, because home users want to protect the data that they are storing. RAID is not required for farming in Chia because data can be recreated by plotting in the event of a drive failure, so you don’t’ really want to lose the opportunity to farm more space in tradeoff for redundancy! (unless your plotting speed is very, very low)
If you do want to setup a Synology for Chia use, the optimal way is to make each individual drive its own storage pool and volume and use the plugins for farming and container (they exist but are still very new). If you want to make life easy are are ok buying extra storage, you could create a RAID5 of 7+1, but this will impact performance on writes and require a long time to initialize. The advantage of this is having one large array to store the plots and protects from a single drive failure…but we discussed above that if you have a fast plotter there is no need to spend the extra money on redundancy and RAID.
Cons: NAS tend to be more expensive per disk attach than a DIY solution, making other alternatives for farming attractive.
This is a great basic chassis that you can easily get through Amazon prime. My only real complaint is the Molex connectors on the fans instead of regular 4-pin (aka bloody knuckles when building), but this can be easily remedied with aftermarket fans.
with one of these for 12 drives in the front, you can get to 12 drives for $130, with 2 of these you can get to 16 drives for $150 with room for a full ATX motherboard (really tight) or micro ATX board.
Linus has a good video where he shows how to build a high drive count desktop here, 320 TERABYTES in a normal case!! – The DIY 4k editing NAS.
The legendary Chenbro is a 1U server that holds 12 3.5in HDDs. Can be found on eBay for $100-150, with compute included for an integrated farmer. Perfect length SATA and power cables? Yes please!
I first saw this on Craft Computing YouTube
Check out the Chenbro NR12000 1U 12-Bay storage server here: https://ebay.to/2UMPSxz
Hard Drive Rubber Grommets: https://amzn.to/35RPQKU
Drive Mount Screws (6-32 3/8″ Truss Head): https://amzn.to/35Vn2kE
12 drives (2U)
A 2U chassis with integrated power supplies, fans, and SAS backplane can be found on eBay for as little as $100. Sometimes used items have defects and need to be sent back. If this isn’t your thing…spend more money and buy new! These are great chassis for any small scale farmer, storage server, and plotter for easy access to hot-swap drives.
SAS vs SATA infrastructure
SAS is an enterprise storage protocol and interface designed for data center scalability and reliability, which is much more complicated (many more features, options, and logging capabilities) than SATA (which is used in the desktop space as well as the enterprise and data center). While far too detailed for this post, SAS is designed for scalability with many hosts, many targets (drives), and flexible infrastructure (cascading, expansion, etc.). You can access many drives through a JBOD through a single SAS cable. SAS infrastructure, like HBA (host bus adapters) and JBODs are backwards compatible with SATA drives. This is the reason you can buy a SAS enclosure or JBOD, and natively put SATA HDDs in it! SAS also happens to be a lot faster than SATA, with 12Gbps and 24Gbps protocols (vs SATA at 6Gbps) and can tunnel 4 SAS lanes over a single cable.
45 drives (4U) – The cult of the SM45
Known in the farming channel as the SM45, this is a 45 bay 4U JBOD from Supermicro. This was a very popular JBOD and is easy to find replacement drive caddies, power supplies, fans, and other parts. A JBOD does not contain any compute, and needs to be attached to another host. With the SM45 you will need an HBA like the LSI 9200 or 9207 ($20-30 on eBay) and a set of miniSAS external cables (SFF-8088). At 45 drives, 75W idle power, and a used cost of $200-300 (before shipping), this is one of the most attractive options for mid-level farmers. Screwing in the hard drives to the drive caddies (sometimes called sleds or carriers) is a Chia farmer’s right of passage.
90 drives (4U)
The SM90, or SuperChassis 946ED-R2KJBOD houses 90 drives in a 4U chassis. All the drives can be accessed with a single miniSAS HD external cable (SFF-8644) and an LSI 9300-8e HBA. This has 4 power supplies that are fully redundant, two SAS modules with independent paths to a host, and amazing cooling (warning VERY LOUD). This JBOD is absolutely massive, please check the specifications for size and weight before purchasing.
Higher drive count JBODs, 102 drives in 4U!!
Yes, yes, yes…both WD and Seagate have their own JBODs. These are generally 240V only, optimized for their own drives (firmware, management), and harder to find. I wouldn’t recommend them for your average or even serious farmer, unless you are well versed with enterprise hardware.