AMD 8-Core EPYC Home Server Build

AMD’s new EPYC 3201 is a good processor for a near-silent server …without sacrificing performance.

I saw good price on an open box Supermicro M11SDV-8CT-LN4F. It includes an embedded AMD EPYC 3201 8-core processor. I’ve been wanting to try an AMD build for awhile so I grabbed it. This CPU was favorably benchmarked by ServeTheHome matching performance of several Xeon CPUs with a much higher 60-80W TDP. The EPYC 3201 performs just as well (and better in some cases) …but at only 30W TDP.

I don’t really care about power usage. 30W vs 80W might cost me a dollar or two more per year in electricity. But that lower heat makes a difference in cooling requirements. And to me that means less fan noise. You can build a near-silent server with an EPYC 3201 without having to get into anything fancy like liquid cooling.

The last time I had an AMD computer–was 6 years ago. They’ve really been behind Intel in performance for awhile but in the last few years I’ve seen some much better CPUs out of them and thought I’d give it a try.

This an update to my Datacenter in a Box Post which is based on the X10SDV Xeon-D Platform.

I’m planning to use this server in my homelab to expand my Proxmox Virtual Environment from 1 to 2 servers making them into a cluster.

AMD EPYC Server Build Parts List

AMD EPYC Server Build
  • SuperChassis 721TQ-250B. (Amazon) This is my go-to Mini-ITX chassis–every server I build is based on this case. Small compact, includes 4-hotswap drive bays. For my house I prefer these cases over racks because the fan is a large diameter so it can get a lot of airflow without making much noise. Even iXSystems uses this case for their small FreeNAS units.
  • Supermicro M11SDV-8CT-LN4F (Amazon | NewEgg) Motherboard. This includes an AMD EPYC 3201 8-core processor. 4 SATA ports, and Quad gigabit NICs.
  • Samsung DDR4 Registered ECC Memory 32GB (2 x 16GB) (Amazon)
  • Intel Optane M10 16GB (Amazon) in the NVMe slot. These are cheap (sometimes less than $15 pre-owned) and will be used as a ZIL for ZFS write caching. In an application where every write requires a sync (VMware and NFS) I might spend a little more, but for Proxmox this will be fine.
  • Noctua NF-A6x25 PWM (Amazon) – a little extra air flow over the CPU
  • 4 x HDDs. I used some extra drives I had lying around, but if buying new I would suggest the Western Digital Red Pros. See my Hard Drives for ZFS Post

How to assemble AMD EPYC Server

Time needed: 1 hour and 45 minutes.

Assemble all the parts for a Supermicro AMD EPYC Microserver Build. This is a simple build. In fact Eli did most of it–I just supervised.

  1. Remove cover from case

    4 thumb screws at the back, then push the blue tab to release.

  2. Remove the motherboard tray from the chassis.

    5 Screws in the back secure it.

  3. So, Step 3 is here to tell you we forgot the backplate, so complete Step 8, then Proceed to Step 4.

  4. Install Motherboard onto the tray

    Line up the motherboard with the 4 posts, and screw it in. A little bag of screws titled “MB” should have come with the chassis.

  5. Install Memory

    If using 1 stick of RAM populate the closest blue slot.
    If using 2 sticks of RAM (which is what I will be doing) populate the 2 blue slots.

    M11SDV Motherboard

  6. Install NVMe drive

    (Yellow arrow above). Now the board should look like this:

    M11SDV Motherboard with Memory and NVMe SSD

  7. Slide Tray into case… Oh no! We forgot the backplate!

    Supermicro Motherboard Tray

  8. Install the Backplate…

    Take the motherboard back out of the case, unscrew it from the 4 posts, install the backplate (easiest to snap it into the back of the tray). Install the motherboard again, and slide …actually, leave it out, because step 9 is easier with it out.

    Motherboard with Backplate

  9. Remove the Front Cover

    (So, this step is not needed unless you need more airflow over the PCI area, we ended up doing step 20, 21, & 22 instead).

    You’ll find it secured by 4 black tabs like these…

    Black tabs holding front cover

  10. Install the Noctua Fan

    (Again, you’ll probably want to follow steps 20, 21, and 22 instead)

    The AMD CPU uses a “passive” heatsink, even though it’s low TDP it still needs some air. Take the 4 rubber feet and pull them through the holes from the front of the case.

    Supermicro Noctua Fan

    Then pull them through the holes on the fan. Make sure the arrows on the fan are pointing towards the CPU. Now you can slide the motherboard in and put the front cover back on.

  11. Label the hard drive trays

    Always start at 0. So label them with the 0, 1, 2, & 3 stickers. Burn the sticker for 4. You don’t need it. Don’t label the drives 1, 2, 3, 4. You’ll know why when a drive fails and you need to locate it.

  12. Plug in the case fan (black, yellow, red, blue wire) to the FANA port and SATA cables (red) between the motherboard and backplane.

    I should note that when plugging in the SATA cables use the reference on the backplane and match it with the port numbers on the motherboard. SATA port 0 on the motherboard should plug into port 0 on the backplane and so on. Don’t mix them up or you’ll have a hard time locating a failed drive.

    Motherboard Wires

  13. Plug in everything else.

    Just plug in all the wires. Noctua Fan, DC Power, AC power, USB Header (blue).

    Wire up motherboard

  14. Power Test

    At this point I plug it in to power to make sure things come up.

  15. Install Hard Drives

    Use the HDD screws that came with the case. Install 4 screws per drive. Then slide them into the bay.

    Hotswap Hard Drive Tray with drive installed

  16. Cover any open holes in the backplate and plug Ethernet into IPMI port.

    Backplate holes…. I bought this motherboard as an open box so somebody punched out all the holes… this is bad because it messes up the airflow. Cover those up with electrical tape.
    Now plug in the IPMI port (circled in yellow). This is remote management. You will not need to hook up monitor, keyboard and mouse. It’s all done via Ethernet.

    IPMI Port

  17. Plug in Ethernet, Power, and Determine IPMI IP

    Plug in the IPMI port, then plug in power (You don’t need to power on yet, just need power to the board so the IPMI interface lights up). Check your router’s DHCP for the IP of the latest device and that should be your IPMI device. Connect to that IP. E.g. Default credentials are ADMIN / ADMIN (I suggest changing this).

  18. Supermicro IPMI and KVM over IP

    Here you can control your server remotely from your chair while sitting by the fire as if you were physically present. You can power it on, power it off, reset it, bring up a remote KVM, mount an ISO on your computer and boot off of it, etc.

    Supermicro Remote Management IPMI

    There are 3 options for IPMI:
    a. iKVM/HTML5 – You just use a browser. This is the simplest but lacks the ability to remotely mount ISOs from your computer.
    b. Supermicro’s IPMIView tool (must run it as Administrator). This tool can also scan the network for Supermicro IPMI devices.
    c. Use Console Redirection. This is what I usually do. This requires Java 8 LTS. I would use AdoptOpenJDK 8 or Amazon Curreto 8. You may need to associate the .jnlp file with javaws.exe.

    Also, there’s a Supermicro IPMI Android App you can install to have complete control of your server from your phone. The Android app can also scan the network for Supermicro IPMI devices.

    As a final note, I should mention you typically don’t install security updates to keep the out of band management interface updated. The IPMI port shouldn’t ever be public, and ideally should be on a management VLAN away from the rest of your LAN traffic.

  19. Burn In Load Testing

    The next step is to load up something to burn in the hardware–load test it to make sure nothing fails. I like to mine for Cryptocurrency such as Gridcoin or Monero for 1 week. If there are no failures we can consider the server ready. Make sure none of the drives fail, fan noise is tolerable at load, temperature remains okay, check the sensor logs in the IPMI interface to make sure there are no temperature alerts or ECC errors.

    After 24 hours I checked on it and the fan noise and temperature were higher than I’d like. The CPU was at 84C, which is too hot (in my opinion) and what’s worse is the CPU fan is running at 2400RPM making too much noise trying to keep it cool!

    Supermicro Phone app showing Temp High

    We’ll need to find a better cooling solution.

  20. 3D Print a Fan Mount for the M11SDV

    Download a 3D printable STL file for the M11SDV 60x25mm CPU fan holder from Thingiverse and print it out. I used an Ender 3 Pro.

    M11SDV CPU Fan Holder

  21. Install Noctua Fan in CPU Holder

    Slide out the motherboard (wires should be long enough not to have to disconnect anything) and install the CPU holder.

    I glued the legs to the screws using a hot glue gun.

    Noctua Fan in 3D Printed CPU Holder for M11SDV

  22. Slide the motherboard back in and power on.

    3D Printed CPU Fan Holder with Noctua

    Now the readings show 52C with a fan speed of 1200RPM. Much better!

The M11SDV AMD EPYC 3201 server build runs quiet… I am not using a noise reducer on the Noctua fan and I can’t hear it over the ambient noise in my office –unless I put my ear up to the case. This is the perfect server for homelabs where you’re trying to keep the noise level down but still have plenty of compute power.

To compare CPU temperature and fan speed with the Xeon-D, I have three nearly identical server builds except for the different motherboards. Just taking a quick inventory of the IPMI sensors:

  • X10SDV – Xeon D-1540 8 cores – CPU 20% – 74C – 1400RPM , 3100RPM,
  • X10SDV – Xeon D-1518 4 cores – CPU 50% – 78C – 1200RPM, 4900RPM
  • AMD EPYC 3201 8 cores – CPU 80% – 50C – 700RPM, 1200RPM

Now to be fair, the AMD is a generation newer. However even with the latest gen Xeon processors the AMD EPYC requires much less cooling (meaning less noise).

The AMD EPYC is now my favorite microserver CPU.

That’s the build, now I need to finish the burn in test. Next up is a followup with a post on Proxmox.

2019 in Retrospect

We were locked outside. The sliding glass door had locked itself upon being closed. Perhaps it had been slammed just a little too hard. Minutes earlier the kids had shot off poppers. We were celebrating the new year standing out on the back deck; 20 feet off the ground at my in-laws house. It was time to go in and send the kids to bed. That is when we realized we were stuck outside. The sliding glass door wouldn’t budge. There was nobody inside to unlock it. That is how my 2019 started.

Things got better from there.

The adventures continued. I realize middle of February is a bit late for a 2019 Review–but better later than never.

Road Trip in Idaho

Homelab and Tech Projects

If you’re interested in building a Homelab see my Homelab Ideas post.

  • Setup a Proxmox server (blog post on this coming soon!), I’m impressed–my favorite feature is it allows you to run VMs and Containers side-by-side under one control pane making containers as easy to manage as VMs. Also, it natively supports ZFS as a storage backend. Compared to my FreeNAS on VMware server Proxmox is a much simpler setup if you’re just planning on one server for hyper-converged storage. The containers give it higher density, and it has better VM storage performance since it can go direct to disks instead of relying on network storage (this is not to say that I don’t also run my VMware+FreeNAS box, I still love that setup, but it’s good to explore options).
  • Replaced my pfSense firewall with a UniFi USG. My networking equipment (WiFi, Switches, and Router) is now 100% UniFi and I like to see everything under one UI.
  • Containers. I deployed a lot more Proxmox Linux containers than VMs.
  • I’ve been a bit skeptical of Smart devices. We paid extra to get a dryer without Bluetooth back in 2018. But I bought two Smart Devices in 2019:
    • Upgraded our Thermostat to a Nest. It didn’t really save us that much… it appears to have lowered our gas usage by about 4% but it was nearly free after a power company rebate. I think it does a great job at turning down the heat when we are away (I almost always forgot to turn the old one down).
    • When I replaced my Garage Door opener (more on that below) I bought a MyQ Garage Door opener. This has been beneficial…I used to accidentally leave the garage door open all day (and sometimes all night) and now it will alert me if it’s been open more than 15 minutes. But if I miss the alert I set it to automatically close itself at the end of the day. Well worth it. I realize this increases the chance of getting hacked…but when you forget to close it all the time I think THAT risk is higher.

House Projects

This was the year of DIY house projects. I do not consider myself a handyman, but I managed to get some projects done:

  • Replaced our old ceiling fans. The ugly fan in the living room was down to three blades (I think Kris broke the blade on purpose) so I had to bring it down to two blades so it would balance…. it was time to replace it.
  • Replaced the old track-lighting fixtures with LED fixtures in the living room, halls, and kitchen.
  • Organized and Cleaned the Garage. Installed Craftsman VersaTrack and Pegboards along the walls and made lots of trips to the dump.
  • Organized my tools and computer parts… I used to spend half my time on home or computer projects looking around for tools or parts–now they’re in toolboxes! It was expensive to buy enough, but well worth it. I like to systematize so I ended up standardizing on DeWalt TSTAK and Craftsman VersaStack toolboxes (picture of the DeWalt TSTAK below) which stack and latch to each other… what sold me on these is the toolboxes from both brands (made by the same company obviously since both brands are owned by Stanley Black & Decker) are compatible with each other! I can find one brand or the other at pretty much any hardware store.
  • Bought a Craftsman Power tool set–the batteries on my trusty 12-year old DeWalt drill died. When I went to get new batteries at Home Depot it turns out they don’t sell 14.4V batteries anymore so I had to get a new drill! I went back home to research drills. I realized it was much better value in the long-run to get a combo-set and you get a a whole bunch of power tools along with the drill! So I went all out and got an 8-tool Craftsman set. It turns out I’ve found a reason to use every one of them.
  • Replaced the garage door opener. Our old one was was failing to close far enough periodically… I had “fixed” the gears several times but they’d start skipping against each other after several months. I have a rule that if I have to “fix” something 3 times and it needs to be fixed again I replace it.
  • Replaced the hot water heater. I’m a big fan of paying people to do things I don’t want to do. But I could not find a hot water heater guy who would come to my house with any sense of urgency. I even took a vacation day just to sit at home and wait for a plumber who didn’t show up! After 3 days of no hot water and calling multiple places… Out of desperation I ran down to Home Depot, bought a new Hot Water Heater, watched some how to install a hot water heater videos, and installed it myself. Turned out it wasn’t that hard.
  • Re-routed the Sump Pump discharge–it was going out a hose under my garage door so I had to leave the garage door open an inch during the snow melt and rainy seasons. Plus the water just ran down my driveway and ice up. Now I’ve got PVC going out the side of the house under the sidewalk and out through a hole in the curb to the street.
  • Failed at the Front Door–Not everything was a success. Our front storm door got damaged in a storm and now it catches on the frame when closing and I’ve failed about 100 times trying to fix it. Maybe this year.

Books Read in 2019

My main focus of study for 2019 was Entrepreneurship. I enjoyed the $100 Startup, The 4-Hour Workweek, 80/20, Authority, and the Blogger’s Simple Guide to Taxes.

I pick a subject outside of computers each year and try to read 5 books on it. This is just an attempt to stay well-rounded in other areas and sometimes I pick subjects that are practical to my circumstances. In the past years I’ve chosen topics or people such as: Finance, Investing, Photography, Firearms, Archery, Leadership, Management, Real Estate, Early U.S. History, Charles Babbage, GK Chesterton, etc.

Books Read in 2019 From Goodreads
Books completed in 2019

Wrote a Book

LastPass Guide eBook

I started writing my first Book, the LastPass Guide. Kris and Eli helped me a lot. I had it nearly completed by the end of the year but I took a break from it mid-December to make a trip down to California and would pick it back up in January.

The book turned out great. I realized at the end of the year that I spent too much time writing the book compared to marketing. I made a quick course correction and starting focusing on marketing the month before launch and decided to make my 2020 reading topic Marketing.

Blogging 2019 Review

Blogging is my favorite hobby. In 2019….

  • WordPress says has surpassed 1 million visitors/visits/hits/impressions?! I’m not sure what this number means… who knows. But I’m just happy to have a million of something!
  • I had been blogging for 18 years.
  • After much research I switched my email list over to ConvertKit from WordPress’s built in email list. My main reason for choosing ConvertKit is it is designed specifically for creators, writers, and bloggers. This was one of the best changes I’ve made in 2019… after following some of their suggestions my Newsletter started growing and is still growing rapidly. I’m still a believer that email is one of the best forms of communications second only to face-to-face. Email is the best asynchronous method by far. It’s the one form of communication that’s ubiquitous, not controlled by a single entity and is a federated and open standard.
  • Moved hosting from a DIY DigitalOcean server to fully managed hosting at Cloudways (still on DigitalOcean) so I can focus on blogging and let them worry about security updates and patches and performance tuning. The pricing is high so I may move it back at some point but I’m happy where it is for now.
  • Put a lot of effort into improving page load performance–still a lot of room for improvement but I went from a Pagespeed score of 67 to 99 for desktops. Still need to improve mobile. I experimented with AMP but those experiments were mostly failures.

Other 2019 Highlights

  • Dr. Jason Lisle came up for a conference on Astronomy.
  • Kris planned a trip a to Mount St. Helens and we met up with Paul Taylor for some Excursions. This was one of my favorite parts of 2019.
  • We got some Snow Shoes and enjoyed hiking around in the snow!
  • Eli took 3rd place at the Pinewood Derby Contest at Gospel Kidz (similar to Awana). I used a jigsaw to cut out his design but on the rest of it Eli did most of the work himself and had some creative ideas like drilling holes in the wheels to lighten them.
  • We bought a 3D Printer and had a blast learning how to use it… Eli was able to design his own gears in Tinkercad and print them out.
  • We finished up the year with a trip to California… but this time didn’t lock ourselves outside.

Eli walking in Snow Shoes

Whatever your hand finds to do, do it with your might, for there is no work or thought or knowledge or wisdom in Sheol, to which you are going.

Solomon (Ecclesiastes 9:10 ESV)