Eli’s First Gun | Daisy Buck

I bought Eli his first gun, a Daisy Buck Model 105.

Eli holding Daisy Buck BB Gun

It is very similar in function to lever action rifles (lever action works the same, cross-bolt safety is the same) so it’s a great starter gun to teach proper firearm handling and safety.    It was $18 at Walmart but has quality where it counts.  The stock is real wood with an engraved image of a buck.   The barrel and receiver are metal.  The only plastic part is the lever but it’s rugged and shouldn’t break.  If properly cared for I don’t see why it wouldn’t last several generations.

Daisy BB Rifle
Daisy Model 105 Buck

There’s no recoil and it’s quiet enough to shoot in the backyard.  The lever action does take some effort to work.  Eli struggled with it at first but after a couple tries he doesn’t have any trouble cocking it himself.  The gun shoots 0.177 caliber BBs which run a couple of dollars for 2,400 rounds.

I found out Daisy has been around since 1886 with an interesting history.  They’ve been making air guns for a long time.  They started out as a windmill company that started offering a free air rifle to farmers with the purchase of a windmill.  A few years thereafter they stopped making windmills.

Daisy advertises it for age 10 and up, I think Daisy Red Ryderthe age is listed for legal reasons more than size.  For fit I’d say it’s sized for 6 to 10 year olds at 30 inches long.  For younger kids new to guns the shorter length will be easier to manage.  If I was buying for someone over age 10 I’d probably opt for the infamous Daisy 1938 Red Ryder which has a longer stock and is 35 inches long.

BB’s can ricochet so keeping a safe distance from the target as well as safety glasses are a must.  The instruction manual has directions to build a target stuffed with newspapers with magazines at the back that will absorb BBs.  I placed the target in front of my shed but in case he misses the target I set behind the target a sheet of plywood which I angled down to cause any ricochets to bounce down into the ground.  So far this has worked well.  For targets we use milk jugs or print our own.

I think it’s good to teach children how to use guns when they’re mature enough.  It gives them a chance to develop responsibility and character while having fun and learning a useful and important skill.

The NRA has a great Gun Safety Rules page.  Before Eli could have ammo I taught him 3 safety rules:

  1. Always point the gun in a safe direction especially away from people, even when it’s unloaded or the safety is on.
  2. Always leave the safety on until you’re pointed at the target.   I make him turn the safety on before he puts it down or hands it over to me.
  3. Always keep your finger away from the trigger until you’re ready to shoot.

I’ll teach him more as he gains skill.  We treat the BB gun the same as my firearms.   The gun is unloaded after each use and is stored in a safe location where nobody else can get to it, separate from the ammo.  I think it’s good to teach proper gun safety and responsibility from the start.  The best way to do that is by example.

 

 

Ansible Role Rdiff-Backup Script | World Backup Day

Happy World Backup Day!  Here’s a quick little Ansible Role I wrote to automate backup configuration for hordes of servers using Rdiff-Backup from an Ansible inventory file.  If you have no idea what I just said you may want to skip to “I’m Confused” at the very bottom of this post.

Ansible Rdiff-Backup Configuration Diagram

What does the Rdiff-Backup Ansible Role do?

  • Creates a folder on the backup storage server to store backups.
  • Creates a backup script on the backup server.  This script will use rdiff-backup over ssh to backup every server on the list (below) and prune backups older than 1-year (default).
  • Adds/removes servers in an Ansible inventory file to a backup list which the backup script calls as servers are provisioned/decommissioned (the script will not delete backups on a decommission, only stop taking them).
  • Installs the rdiff-backup program on both the client and backup server.
  • Generates an SSH key-pair on the backup server and adds that public key to the authorized key file on each client to allow the backup server to ssh into the clients.
  • Scan ssh-key from client and add it to known hosts on backup server
  • Create a cron job on the backup server to run the backup script once a day.

The rdiff-backup-script role is available on Ansible Galaxy and I’ve uploaded the source to GitHub under the MIT license.

Requirements

You will need:

  1. Ansible (best to be installed on it’s own Linux server).
  2. A Linux server with lots of disk storage to serve as a backup server.
  3. Lots of Linux servers in your Ansible inventory that need to be backed up.

I have tested this on Ubuntu 16.04, but it should work with any distribution (CentOS, RedHat, Debian, etc.) as long as rdiff-backup is in the repositories available to the package manager.

Install the Rdiff-Backup Script Ansible Role

On your Ansible server install the rdiff-backup-script role with:

Create a playbook file, rdiff-bakcup-script.yml with the contents:

Your Ansible inventory file would look something like this:

Run the playbook with:

Once the playbook has run all servers will be configured for backup which will occur at the next cron job run (defaults to 01:43 am).

The above playbook should be added to your site config so it is run automatically with the rest of your Ansible playbooks.  It would also be wise to have something like Nagios or Logcheck watch the logs and alert on failures or stale log last modified dates.

The backup script does not try to create an LVM snapshot and then backup the snapshot.  That would certainly be cleaner and I may add that ability later.  The default settings exclude quite a few files from the backup so make sure those exclusions are what you want.  One thing I excluded by default is a lot of LXC files.  If you’re using LXC you may want them.  Also always test a restore before relying on it.

Obviously, test it in a test environment and make sure you understand what it does before trying it on anything important.

Check your backup strategy

This is a good day to check your backup strategy.  A few things to consider:

  1. System backups are important, not just the data files.  You never know what you’re missing in your Document only backups and restoring service from system backups is much faster than rebuilding systems.
  2. Frequency.  If you can’t afford to lose an hour of work backup at least every hour.
  3. Geographic redundancy.  Local fires, hurricanes, fires earthquakes can wipe out multiple locations in cities all at once.  Keep at least one backup in a separate part of the globe.
  4. Versioned backups.  On Monday you took a backup.  On Tuesday your file got corrupted.  On Wednesday you overwrote Monday’s backups with Wednesday’s backup.  Enough said.
  5. Test restoring from your backups (it’s good to test at least once a year on World Backup Day) to make sure they work.
  6. Encrypt.  Make sure your backups to cloud services, insecure locations are encrypted (but also make sure you have provisions to decrypt it when needed).
  7. Cold storage.  Keep at least one backup offline.  When a bug in your backup program deletes all your live data and your backups you’ll be glad you did.
  8. Keep at least 3 copies of data you don’t want to lose.  Your live version (obviously), one onsite backup that will allow you to restore quickly, and one offsite backup in a far away state or country.

I’m Confused

You might want to backup your computer.  I’d suggest looking at CrashPlan, SpiderOak, or BackBlaze which are all reputable companies that offer automatic cloud backup services for your computer.  The main thing you want to look at for pricing is how much data you have vs the number of computers you have. CrashPlan and BackBlaze charge by the computer but offer unlimited data so they would be ideal if you have a lot of data but few computers.  SpiderOak lets you have unlimited computers but charges you by how much space you use making it ideal if you have little data and many devices.

Practical Fountain Pens

“These scissors don’t work, can you pass me yours?  Thanks.   Hmm… these don’t work either.  Did they work for you?  They did?!  How did you get them to work!”  It is quite embarrassing to be the only kid in class who doesn’t know how to use scissors.  Although it was odd that these scissors didn’t work because my scissors at home work just fine.   My Sunday School teacher eventually found the problem–I was left-handed!  That was 30 years ago.

My experience with pens was similar.  At the end of a writing session I’d have ink smudges all over my writing and ink all over my palm.  I compensated by writing with a hooked hand.

The Fountain Pen

Last year, I discovered the Fountain Pen and Noodler’s Bernanke Ink (an homage to Ben Bernanke who needed fast drying ink to print all that money).   The ink in most cases dries before my palm gets there.  I left ballpoint pens behind and have not looked back.

Fountain Pens

Fountain pens have a few advantages over ballpoints.  The pens last forever and the ink is refillable without the need for disposable cartridges which is more economical in the long run.  There are a variety of inks to choose from and they all have different properties.  Some are waterproof, archival quality, forgery resistant, dry fast, don’t freeze, etc.  But the advantage I like the most is how little effort it takes to write with a fountain pen!  It has been said that the ballpoint pen killed cursive.  On a ballpoint for ink to flow the pen has to constantly move to roll the ball, and in order to do that you must exert pressure.  The trick is you need enough force to roll the ball, but not so much you punch a hole through the paper.  With a fountain pen the weight of the pen is enough pressure for ink to flow.  It took me awhile to train myself not to apply force.   This changed how I write.

3 Practical Fountain Pens

Some fountain pens are extremely expensive, I don’t really see the value in those… but here are three affordable pens, all under $30 that write great.

Pilot Metropolitan (top), Lammy Safari (middle), TWSBI Eco (bottom)

Pilot Metropolitan

Pilot with cap off

Pilot Fountain Pen

The Pilot Metropolitan is the least expensive and the first pen I bought. It includes one disposable black ink cartridge and a converter (a converter is used to “convert” the pen to be able to use ink straight from an ink bottle) pictured below.  In this case squeeze the tweezer things, dip the end of the ink converter into a jar of ink, release the tweezers and it will draw up ink.  Insert that into the feeder and it’s good to write.  This is the only pen I have that comes with a Japanese nib which tend to run finer than German or Italian nibs.  I got a fine and it is the finest so far, even more so than Lamy Safari’s extra-fine.  The Pilot can also be purchased with an extra-fine which would be finer than anything I’d want.

Pilot Metropolitan Apart
Converter

Lamy Safari

Lammy Safari

Lammy Safari Cap Off

The 2nd Pen I bought was a Lamy Safari.  I like this pen because it has a nice matte plastic texture and it’s one of the least flashy fountain pens one can get (if you get black that is).  It’s plain and simple.  Even the nib is black which is unusual.  One nice feature is a window on the side lets you know the ink level.

The Lamy comes with one disposable ink cartridge, but does not come with a Piston Converter so I had to get that separately which adds to the price slightly.  The converter is simple, insert it into a bottle of ink, twist the knob at the end to drive the piston up along with the ink.

 

TWSBI Eco

TWSBI Eco

TWSBI Eco PistonThe TWSBI Eco is the last pen I purchased.  It is my favorite so far being the only piston filler pen which means it doesn’t need cartridges or converters.   Refilling is as simple as placing the pen in a bottle of ink, and twisting the cap to drive the piston creating a vacuum to pull in the ink.  This pen also has more capacity in the ink chamber than any of the other options resulting in fewer refils.  Obviously since it’s transparent this is also the easiest pen to see how much ink is left.

The pen is very economical, by far the simplest and least expensive piston pen on the market and doesn’t seem to sacrifice any quality.

TWSBI Parts

Noodler’s Ink

Noodler's Ink

The only ink I’ve tried is Noodler’s and I don’t really have a reason to try anything else.  So far it’s worked great in all three pens.  I usually write with Noodler’s Bernanke Blue or Black which both dry fast and are my favorite inks.   But the Bernanke inks do feather on cheaper paper so I keep one pen filled with Noodler’s Heart of Darkness which seems to be a nice compromise between anti-feather and fast drying.  Before buying a whole bottle of ink, GouletPen’s sells ink samples, I started with those to find which inks I like best and how it works on the paper I typically write on.  Jet Pen’s Fast Drying Fountain Pen Inks guide is great if you’re looking for fast drying ink… expand each color to see pictures of how it stands up to smearing after 3, 10, and 20 seconds.

 

Ink Samples
Ink Samples

Thoughts so far

So far I have not lost any pens which is good.  After about a year the maintenance on them has been minimal.  About all I do is refill them.  I once let the Pilot sit for several months so it didn’t start, to get it going again I ran water over the nib and it was good to go.  The few times I pick up a ballpoint I notice… it doesn’t effortlessly glide across paper like a fountain pen.  The ink (even Bernanke) doesn’t always dry faster than a ballpoint (depends on the paper), but the angle I write with is shallower so I don’t smudge as much–with a ballpoint I have to write at an almost vertical angle.  I don’t think I’ll be going back anytime soon.  Despite all that some habits are hard to break: I still write with a hook.

If you’re interested in fountain pens I would suggest looking at Goulet Pen’s Founten Pen Education

This post is licensed under the CC BY 4.0 license.

The Mechanical Keyboard

What Is a Rubber Dome Keyboard?

Rubber DomeTake a look at your keyboard.  Push a few of the keys.  It probably feels mushy, and it’s most likely a rubber-dome keyboard.  These weren’t made for typing, they were made to be cheap.  You have this dome of rubber that the key sits on, as you press the key feels mushy until the rubber dome sort of collapses and closes the circuit registering the keypress.

Mechanical Keyboards

Buckling Spring
Buckling Spring

A mechanical keyboard by contrast has a real switch and most give audible and tactile feedback when the switch engages.  These are much more expensive to produce, but they were very common back in the 80s, and are far superior to most  keyboards made today.  When people spent $5,000 dollars on a computer it made sense to put $500 dollars into the keyboard!  On the left is an image of IBM’s infamous buckling spring key switch, which is by many considered the best keyboard switch ever designed.  It simply includes a spring, that when pressed far enough, buckles, causing the spring to hit the wall making an audible click as well as moving the plate to engage the circuit.

MX Cherry Switches

Cherry MX brown: Image from GeekHack forums.

There are a variety of switch types, the most popular and easiest to find keyboard switch on the market today is the Cherry MX mechanical switch.  Cherry style switches can be identified by the plus on the stem which the keycap fits over.  It comes in a variety of switch types for different styles of typing, designated by colors… here are some of the most popular models and how they behave.

Three Most Popular Cherry MX Switches

Cherry MX Blue – An audible click and tactile feedback on engagement.  This switch is popular for typing activities and generally preferred among authors and programmers.  It does make a loud click noise so I don’t own this style but I have typed on blues and they are a fantastic typing experience.  The one thing I don’t care for in this switch is the reset point is a bit higher than the activation point which requires a little more return travel to re-engage.  It’s not really a big deal, just my preference.  Most typists actually prefer this as it prevents an accidental double-strike and most gamers do not like this.

Cherry MX Red – No audible click, and no-tactile feedback.  This is a linear switch with consistent force all the way through.  Because there’s no bump on engagement to “slow you down” and the reset and activation point are the same this switch is very popular with gamers who need to rapidly press a key as quickly as possible.  There is no audible click, but they are still noisy if you bottom out the keys.

Cherry MX Brown – No audible click, and light tactile feedback.  This is my favorite switch, you get tactile feedback, the activation point is close to the reset point and there’s no audible click (which is a bonus if you don’t want to wake people up, however I should mention despite the keys being “silent” they’re still much louder than a rubberdome).  If you’re going to get one all purpose switch (and most of us probably should–switching between different key types all the time probably isn’t that helpful) I think the MX Browns are great.  My only complaint with browns is I feel the tactility could be a little sharper.

Less Common Cherry Switches

Cherry also makes a few other switch types with a stiffer spring that are less common but can be found as an option on high end keyboards.

Cherry MX Green – Similar to the blue but a stiffer spring, this isn’t the same as a buckling spring but probably about the closest you can get in the Cherry switch.

Cherry MX Black – Just like the red but a stiffer spring

Cherry MX Clear – Similar to the brown but a stiffer spring.  If this was more common I would consider using this over the browns since I prefer a heavier switch.

Other Switches

IBM Model M
My old IBM Model M Keyboard

Cherry MX isn’t the only game in town but they’re manufacturing the most keys today.  There are also Topre, ALPS, Kailh, and of course the traditional Buckling Spring Switch. This switch was found in the IBM Model F and IBM Model M.  It includes a spring that sits inside each key, as the key is depressed passed a certain point the spring buckles, causing it to whack the side of the key causing tactile feedback and an audible click as the flipper hits the plate.  While this is a fantastic keyboard it’s very loud.  I had to stop using it when I got roommates.  Unicomp bought the rights and manufacturing equipment from IBM so it is still possible to buy a brand new Model M Keyboard with a modern layout today.

Inside a buckling spring keyboard
Model F Buckling Spring Switch

Key Samplers

Before purchasing a mechanical keyboard.  It’s best to test out the switches.  4-key samplers usually have a Red, Black, Blue, and Brown switch.

Max keyboard testerThe WASD 6-key tester adds Clear and Green.  Generally speaking if you deviate from the most common Red, Blue, or Brown switches your options are going to be more limited and more expensive.

6-key sampler

Things To Consider In a Keyboard

Ghosting

One of the major issues with modern keyboards is ghosting, especially if you’re left handed and play games.  Keyboard manufacturers build keyboards to handle simultaneous keystrokes in the WASD region, which is great if you’re right-handed, but awful if you’re left-handed and using something like OKL; or PL:’ and you find you can’t press O, K, and space at the same time without random keystrokes being sent to the computer.  Nicer keyboards tend to have 6KRO (6 Key Rollover) which means you can press any 6 keys simultaneously and have them all register correctly.  Some keyboards also support NKRO (N-Key Rollover) which means you can literally press every single key on the keyboard at the same time without losing a keystroke.

PS/2 vs USB

PS/2 is superior.  Most modern keyboards have a USB connector, but that doesn’t mean it’s superior to PS/2.  The “legacy” PS/2 port has several advantages.  First it supports NKRO (some higher end keyboards can do NKRO with USB but it’s not as common), and 2nd on USB your computer polls the keyboard periodically thousands of times a second.  The conversation between the keyboard and computer goes like this:

CPU: Hey, any keys pressed?
USB Keyboard:  Nope.
(wait a few milliseconds)
CPU: Hey, any keys pressed?
USB Keyboard:  Nope.
(wait a few milliseconds)
CPU: Hey, any keys pressed?
USB Keyboard:  Nope.
You: Press J
(wait milliseconds)
CPU: Hey, any keys pressed?
USB Keyboard:  Yes, “J” is pressed.

And this can result in a few milliseconds delay between the time you press a key and your CPU realizes it.  By contrast on a PS/2 Connection there is no polling, instead PS/2 sends an interrupt to the CPU:

CPU: if  a is not null then … !!!interrupted!!!
USB Keyboard: Hey CPU!  “A” is pressed NOW!

For me and most people this won’t matter.  Pro-gamers may prefer PS/2.

Keyboard Size

I prefer full size keyboards.  One thing the bothers me is the wasted space above the numpad.  So I always look for keyboards that utilize the space… it’s a great spot to have volume controls like this Ducky keyboard provides.

Happy Hacking KeyboardYou can get smaller keyboards without the numpad/tenkey, or even smaller without the arrows, some like the Happy Hacking Keyboard don’t even come with function keys.  Instead you have to hit modifier keys to get to the keys you want.  I think this is a trick to sell you less keys.  I use all the keys fairly often so I prefer to have a full keyboard without having to use modifier keys.

Key Caps

KeycapsKeycaps are easily swapped out later so not as important.  But the two most common plastic materials are PBT and ABS.  PBT is higher quality, wears slower and is more expensive.  ABS is cheaper and if there is any texture on the keys it wears faster.  Caps often use subliminal dye or are doubleshot (two colors of plastic molded together) for the lettering which means the key markings will practically never wear off.

Layout

The two most popular layouts are US ANSI and International ISO.  Chances are you’ll pick what you’re used to depending on the country you’re in.  The two have a different layout especially when it comes to the position of special characters.  Since I learned to type in the United States I always get the US ANSI layout (I would probably prefer a larger enter key… but I’m not willing to sacrifice the size of a left-shift).

ANSI
ANSI Layout
ISO Layout
ISO Layout

My Favorite Keyboards (so far)…

Ducky

After testing a number of keyboard brands Cooler Master and Ducky are my favorites among keyboards you can still buy today for a reasonable price.  The keyboards are well built, nice and heavy, and are priced well considering their quality and durability.  They both have models with an extra 4 keys above the numpad so as not to waste that area.

Here are the two that I use every day…

Ducky Premier

My current keyboard is an old Ducky Premier with Cherry MX brown switches.  It uses quality PBT keys, and has a nice blue-grey color scheme.  Nothing fancy about it.  This model has been retired but there are plenty of newer Ducky Keyboards.

 

Cooler Master Masterkeys Pro

At work I use the Cooler Master Masterkeys Pro, this is backlit with white LEDs on each key (although I never turn on the LEDs).  Cooler Master also sells a more expensive Cooler Master Masterkeys Pro RGB model which has individually lit RGB LED backlit keys.  You’ll notice no wasted space above the numpad.  I really like the smooth ABS keycaps that come with it.

Coolermaster Masterkeys

Sidewinder X4 (legacy)

Sidewinder X4Lastly, there is one non-mechanical keyboard that is fantastic.  And that is the Microsoft Sidewinder X4.   I must say this is the best rubberdome keyboard ever made.  I like the feeling of it better than the more expensive Topre keyboards.  Unfortunately I wore mine out and they don’t sell them anymore.

Why Mechanical Keyboards?

I like them for the same reason I like buttons, knobs, and switches over touchscreen interfaces.  There is no substitute for physical feedback.  On a rubber-dome you can’t tell when a key engages so you press it all the way and bottom out every time.  With a mechanical keyboard you quickly learn bottoming out your keys isn’t necessary.  Instead you press the key and at some point you feel and hear it pass the activation point so you stop pressing and release it.

Another advantage is the durability, Cherry MX switches are MTBF rated for 50 million keypresses.  Older keyboards like the Model M and Model F still work today despite being over 30 years old.

What about Laptop Keyboards?

Unfortunately most laptops are built with cheap keyboards and short key-travel.  While there are a few exotic models that come with mechanical keyboards, aside from those the best laptop keyboards in general are going to come with Gaming or Business Class Laptops which I mention on my laptop buying guide.  Of those, the Lenovo ThinkPad brand is well known for having the best laptop keyboards in the market.  They’re still rubberdome, but less bad than most.

 

 

Snowcicles | Hoarfrost or Rime?

Beauty in God’s creation.  Sunday morning I walked outside to see all my trees covered in small inch long gravity defying icicles.   But they’re not icicles.

Snow spikes all over treeClick on the images to get a larger size.

Snow SpikesTouching one they’re more like snow than ice.  I’m not sure if this is hoarfrost or rime.  It melted away as soon as the Sun came out.  I’ve never seen anything like it before.

 

 

Merry Christmas 2017

Merry Christmas!

Christmas Trees

In the beginning was the Word, and the Word was with God, and the Word was God. He was in the beginning with God. All things were made through him, and without him was not any thing made that was made. In him was life, and the life was the light of men. The light shines in the darkness, and the darkness has not overcome it.

There was a man sent from God, whose name was John. He came as a witness, to bear witness about the light, that all might believe through him. He was not the light, but came to bear witness about the light.

The true light, which gives light to everyone, was coming into the world. He was in the world, and the world was made through him, yet the world did not know him. He came to his own, and his own people did not receive him. But to all who did receive him, who believed in his name, he gave the right to become children of God, who were born, not of blood nor of the will of the flesh nor of the will of man, but of God.

And the Word became flesh and dwelt among us, and we have seen his glory, glory as of the only Son from the Father, full of grace and truth. (John bore witness about him, and cried out, “This was he of whom I said, ‘He who comes after me ranks before me, because he was before me.’”) For from his fullness we have all received, grace upon grace. For the law was given through Moses; grace and truth came through Jesus Christ. No one has ever seen God; the only God, who is at the Father’s side, he has made him known.

— John 1:1-18 ESV

500th Reformation Day

 

Happy 500th Reformation Day!  The Reformation did not begin with Luther.  Rather it can be traced back as far as 1200 A.D. with Peter Waldo (who was forced to live in hiding), then John Wycliff (labeled a heretic), and John Hus (who was burned at the stake).  All these men realized the Roman Catholic Church teachings were contrary to Scripture, and worked to put Scripture in the hands of common people… which Rome feared would undermine their influence and authority.

After Darkness.  Light.

Gutenberg Press Replica: CC BY SA: https://en.wikipedia.org/wiki/User:Aodhdubh

500 years ago Martin Luther posted the Disputation on the Power and Efficacy of Indulgences, also known as the 95 Theses to the door of the Wittenberg Castle.  Although Luther intended to debate these issues and wanted to reform the already divided Catholic Church, it resulted in the Protestant split from Rome.  Thanks to new technology, the Guternberg press, Luther’s writings were widely distributed.  Although I wouldn’t agree with much of Luther’s ideas, I would agree with him on the main issue of the Reformation about how men can be justified before God.  The Roman Catholic Church erred by teaching that justification is by faith plus doing good works.  That is, that salvation and standing before God is not entirely dependent on God’s work, but by God’s work plus our works.  As a result of that unbiblical teaching you get many of the other issues surrounding the reformation (such as indulgences), but the central theme is how man can be saved.  Luther taught from his understanding of scripture that salvation is by faith alone.  There is no good work a sinful man can do to be saved, but rather it is God who saves man by His grace.  There is nothing one can do to improve their standing with God.  Even the best of our good works are tainted in sin, worthless to God.

Rather, our salvation relies on Christ who died on the cross to take the penalty for our sins, and by faith, believing in that work that Christ did, sinful people can be justified before a holy and righteous God.   The glory for our salvation does not belong to us even partly; but solely to God.

For by grace you have been saved through faith. And this is not your own doing; it is the gift of God, not a result of works, so that no one may boast.  Ephesians 2:8-9 ESV

Reformation Resources

Sola Fide, by Faith alone.
Sola Scriptura, by Scripture alone.
Solus Christus, through Christ alone.
Sola Gratia, by grace alone.
Soli Deo Gloria, glory to God alone.

Jack-O-Lantern

And of course, Happy Halloween: here’s Eli’s third Jack-O-Lantern:

Pumpkin

Jack O Lantern

Ben’s Laptop Buying Guide

Can you recommend a laptop?  It’s one of the questions I’m asked several times a month…  and I realized I should just write a guide.  So here are some options I think are great and things I think you need to consider before buying a laptop:

Business Vs. Consumer Laptops

Most brands have at least two laptop lines.  Consumer class and business class.  Consumer class laptops are generally junk.  Support is usually bad.  Safety isn’t a priority (some consumer laptops have been known to catch fire), generally manufacturers experiment with new features on their consumer lines, consumer laptops sometimes ship with malware, or lots of junk or trial software.  They’re not as rugged, the left hinge will break after a year or two.  Parts are hard to come by so you can’t fix them.  The Wifi cards aren’t Intel so can’t connect to every Wireless network.  Don’t buy them.  Stick with the business class laptops.  It is usually better to buy an old used or refurbished business class laptop than a newer model consumer class.

Marketing is notorious for making things confusing.  It’s not obvious what’s business class and what’s consumer quality.  Here’s the translation for you (I’ve bolded what I believe are the better quality more rugged products):

  • Dell Business Product Lines
    • Latitude = Business / Enterprise
      • 3xxx = budget business laptops, not that great a quality
      • 5xxx = Workhorse
      • 6xxx = I call this the bulky line, but high quality (discontinued)
      • 7xxx = premium ultrabooks
    • Precision = Business Powerful Workstations, High Performance CPUs and GPUs
    • XPS = Premium Consumer line.  They sort of sit between the consumer and business lines.  Great quality, price, and specs but not as rugged as the Latitudes or Precision
  • Dell Consumer Product Lines
    • Inspiron = Consumer Line
    • Alienware = Consumer Gaming Laptops
    • Chromebook = More like netbooks that run ChromeOS instead of Linux or Windows… unless all you need is a browser stay away from these.
  • Lenovo Business Product Lines
    • Thinkpad = Business / Enterprise
      • X = Thin & Light Ultrabooks
      • T = Flagship, thinner than P but more powerful than X.  Best keyboards are found on the T series.
      • P = Powerful Workstations, High Performance CPUs and GPUs (formerly W).
      • E = Small Business laptops –budget, not very good
      • L = Affordable, not as good as T but a step up from E.
      • Yoga = Tablet / laptop convertibles (not as rugged)
      • 13 = 13 inch chromebook or netbook (not as rugged)
      • 11e = 11 inch educational notebooks (not really business class)
      • A = Same as T series but with an AMD processor
    • Lenovo Consumer Product Lines
      • Yoga (not to be confused with “ThinkPad Yoga)
      • IdeaPad = Consumer stuff
      • Lenovo = Consumer
      • Legion = Consumer Gaming laptops
      • Chromebooks = Chromebook or netbooks
      • Yoga Books = tablet type things

I have included Dell XPS and Apple Macbooks for comparison, they tend to be well built machines but I wouldn’t consider them business class.  They’re more in the “prosumer” class.  I generally don’t recommend them but they may be good options if you you’re not moving them around a lot.  If you want OSX Macbooks are obviously going to be the best bet even though you’re not going to get the ruggedness you’d get with a Latitude or ThinkPad.  If you’re going to run Windows or Linux I’d recommend a Latitude or ThinkPad.

Deciphering Model Numbers:

  • Dell Latitudes:
    • 2nd digit after the first number indicates screen size.  The “4” in “7480” indicates a 14 inch screen.
    • 3rd digit indicates the generation, almost matching up with the year.  The 8 in 7480 = 2018 model year (Latitude is off by a year).
  • Lenovo ThinkPads:
    • T470, the first 4 indicates the screen size, 7 is roughly the model year.  Not sure what the last digit means.  Sometimes a model number has a suffix, e.g. T470s or T470p which may differentiate it further (P = powerful, S = slim)

Recommended New Laptops

12.5 inch

Latitude 7280 – Quality Ultra Portal laptop, thin and light.  Rugged and likely to survive a drop from a few feet.  2.8 LBS.

Latitude 7280

ThinkPad X270 – Ultra Portable Laptop.  Great little notebook, runs a little on the thick side (easier to grip) but the advantage is memory isn’t soldered on and has room for a 2.5 inch drive bay.  Has two batteries (internal and external so you can swap the external without powering down) which can go up to 25 hours.  This is by far the most modular 12.5 inch laptop.

ThinkPad X270

Latitude 5280 – Slightly thicker heavier version of the 7280.

Latitude 5280

Macbook – 12 inch Macbook ultra portable.

Macbook 12 inch

(in the 12.5 inch category these screens are small, so 1366 x 768 is okay if you need little larger text, otherwise upgrade to 1920 x 1080)

Used / refurbished options include anything in the Latitude 72xx, Thinkpad x2xx series.  X220 and earlier have classic keyboards which many consider superior.

13.3 Inch ultrabook

Latitude 7380 – Almost no bezel so it’s the same size as most 12 inch laptops, business version of the very popular Dell XPS, the Latitude version adds a little more durability so I would opt for the Latitude 7380 over the XPS 13.

Dell Latitude 7380

Dell XPS 13

XPX 13 – This Dell is the “prosumer” version of the above, it’s high quality but not as durable.  I don’t think it would hold up to being dropped as well as the 7380, but it’s still a good laptop.  WARNING: Some XPS machines don’t ship with Intel branded wireless cards.  Make sure it’s Intel.

 

Macbook ProMacbook Pro 13 – Great laptop, newer ones have an annoying touchbar instead of function keys so watch out for that (unless you want it for some reason).

Used / refurbished options include older gen Dell XPS or a Latitude 7370.  13.3 is a fairly new category so you probably won’t find too many used laptops with this scrern size.

14 Inch Ultrabook (thin and light)

ThinkPad X1 Carbon (5th Gen).  A 14 inch screen in the size of a 12 or 13 inch laptop.  Very sleek, thin and narrow bezel and quite sturdy.  Lightweight, thin, it’s one of the best ultrabooks on the market.  Memory cannot be upgraded or replaced so order it with what you need.

ThinkPad X1 Carbon

Latitude 7480 – Great high quality business laptop.  Memory is upgradable.  With this latest model E-port snap in docking support has been dropped so if you want docking you’ll need a USB-C dock.

Dell Latitude 7480

ThinkPad T470s.  Thicker than the X1 carbon but thinner than a T470.  Still supports snap-in docking and memory can be upgraded.  The chassis is slightly less rigid than on the T470 or X1.  Also one ram slot is soldered on so won’t be upgradable (2nd ram slot is normal)

ThinkPad T470s

 

Used/refurbished options:  Older generation Thinkpad X1, Latitude 74xx, and ThinkPad T4xxs.

14 Inch Workhorse, All purpose laptop

ThinkPad T470 – This is one of the best all around laptops.  It’s not too thin that it’s hard to grip, but thin enough to not be bulky.  Fantastic keyboard, probably the best on the market.  Two batteries, one internal and one external so the external can be swapped out without losing power.  With a 6-cell battery (which will cause a bulge) it can get 20 hours battery life, or opt for a 3-cell that’s flush with the laptop.  For longevity this laptop is the most modular model in the ThinkPad lineup as far as swapping parts so you should be able to make it last longer if anything breaks.  No GPU options for buyers in the U.S.

.ThinkPad T470

ThinkPad T470p – Quad Core for heavy CPU and an NVIDIA 940MX GPU making it one of the most powerful notebooks in the 14 inch category.  Oddly it does not have a USB-C port.

Latitude 5480 – A little thicker than the 7480, Can be configured with Nvidia 930MX GPU.  Latest generation drops the E-docking port.  I use an older version of this laptop at work, the E5470, and at home I use a E5450 with NVIDIA.  Both have been great computers, and the E-Dock (which is now discontinued) is very robust.  Can be configured with quad core processors.

Dell Latitude 5480

ThinkPad T25 Retro – 25th Anniversary Limited edition.  Essentially a high end T470 with an NVIDIA 940MX GPU… and a classic 7-row keyboard.  This is the best keyboard available on any laptop made today.  I believe this is the only ThinkPad on a T470 chassis to have both a GPU and USB-C port.  Unfortunately it’s on the pricey side

ThinkPad T25 Retro

Used/refurbished options are the ThinkPad T4xx and ThinkPad T4xxp, Dell E64xx, Dell E54xx, Dell 54xx.

15.6 Inch “ultrabook”

ThinkPad P51s – thin “ultrabook” equipped with Quad Core processor (can be equipped with a Xeon) and  NVIDIA Quadro GPU

ThinkPad P51s

Precision 5520 – This is one of the few precisions I would consider more prosumer than business class.  It’s a re-branded Dell XPS 15, the screen has almost no bezel and the laptop is the same size as many 14 inch laptops.  I don’t think it would hold up to make abuse because of how thin it is.  However, for a mostly stationary laptop it’s fantastic.  Can be equipped with Xeon E3.  Note that some of these models don’t ship with Intel Wireless cards which may cause problems.  Make sure it’s Intel.

Precision 5520

Dell XPS 15XPS 15 – Same thing as the above.  Note that some XPS models are not shipping with Intel Wireless cards which may cause connection problems.

 

 

Macbook Pro 15

Macbook Pro 15 – Great laptop, newer ones may have a touchbar which I find annoying but can be configured with a normal function key row.

Used / Refurbished options include the ThinkPad P5x series, older gen Dell XPS 15, Precision 5510.  This is a newer category so there won’t be as many older models here.

15.6 Inch Mobile Workstations

ThinkPad T570 – Great business laptop with a 15 inch screen.ThinkPad T570

ThinkPad P51 – Can be equipped with NVIDIA Quadro GPU, and Xeon E3.  Up to 64GB memory.

ThinkPad P51

Precision 7520

Precision 3520 – Can be equipped with Nvidia Quadro GPU, and Xeon E3.

Precision 3520

One thing to look out for is the keyboard layout, some 15 inch models have the keyboard offset to the left to make room for a numpad.  Some people would rather have the numpad and some would rather have a centered keyboard.

P51s Keyboard

Used / Reburbised Models are Precision 75xx, Precision 35xx, and Dell E65xx, ThinkPad P5x, ThinkPad T5xx, ThinkPad W5xx.

17.3 Inch Workstation

ThinkPad P71 – huge.

ThinkPad P71

Precision 7720 – huge.

Precision 7720

Used / Refurbished – Precision 77xx, P7x,  ThinkPad W7xx, Macbook Pro 17 inch.

Buying Used / Refurbished

There are some risks buying used.  USB firmware hacks, malware, etc.  However, it’s a great way to save money and some sellers provide a 1-year warranty.  Most businesses keep ThinkPads and Latitudes for 3-5 years then sell them so you can save a significant amount of money just staying 3 to 5 years behind.  Generally you want to buy the laptop from the guy that always kept it docked so it’s still in great condition.  Keep in mind that the reason businesses cycle through laptops is the productivity lost due to running slower and fixing failing components is greater than the cost of just buying a new laptop proactively.  Just something to keep in mind if you value your time.

It’s probably better to get a used / refurbished ThinkPad or Latitude than it is to buy a new consumer laptop. For newer refurbished items the Dell Outlet, Lenovo Outlet, and the Refurbished Mac store are good places to look.

One of the best places to pick up old refurbished ThinkPads may be WalMart’s website.   Also there are plenty of refurbished and used laptops on eBay and sometimes they can be found on Amazon as well.  If you are not comfortable installing an OS make sure it comes with a fresh install of Windows and the seller is highly rated and offer returns.  Many sellers also offer a warranty.

For used laptops the ThinkPad T, X, P, and W series will be a higher quality than the L and E.  Latitude 5000, 6000, and 7000 will be higher quality than the 3000 series.

To roughly find the age of a computer consider the current models for ThinkPad are T470, the middle 7 roughly means it’s a 2017 notebook.  Same for the Latitude 5480, the 8 means it’s roughly a 2017 year notebook (guess Latitude is +1 on the year).  So if you’re looking around on eBay know that a T440 or Latitude E5440 is roughly a 2013-2014 notebook.  The years don’t quite line up perfectly but gives you a general idea.  Another indicator to look at for age is the generation of Intel processor used (see CPU section below).

ThinkPad Computrace warning for used ThinkPads. Some ThinkPads have a Computrace feature which allows the owner to track down or remotely disable a laptop if lost or stolen.  If enabled only the owner (or one of the previous owners who enabled it) can turn it off.  You’ll want to make sure that is turned off before buying a used laptop or if you get one with it enabled ask the owner to turn it off and if they’re not able return it for a refund.  If you can’t track the previous owner you can call Computrace and they can attempt to contact the owner for you.

Things You Should Consider

Brand.  Dell vs Lenovo.  Dell Latitude has better support, service, and screens.  Lenovo laptops have better keyboards, build quality, and durability.  Both are pretty similar and both brands offer a comparable product in almost every size/model.

Docking Support.  Many laptops have the ability to dock into a “docking station”.  Dell and Lenovo have proprietary docking connectors and docks.  These are great solutions if you’re often working in an office or home.  At my house and office I have a docking station hooked up to dual monitors, ethernet, keyboard and mouse.  It’s convenient to dock in and have a full desktop experience (having multiple screens increases productivity) then undock when I’m on the go.  Not all laptops support docking, but if it’s something you’re interested in be sure to check for that capability.

Customer Support.  When issues occur I’ve found Dell to have the best support, usually after a 30 minute phone call they’ll have a technician scheduled to come out the next day.  Lenovo is 2nd, you’ll get the same result but usually a longer phone call.  In my experience when a Macbook breaks you’re going to be out of commission for a week or two while you send it off for repair.

Warranty.  Basic vs NBD (Next Business Day) Onsite.  Basic warranty usually means a part will get mailed to you, or you’ll ship your laptop and wait a few weeks for it to come back.  When buying new, you have the option to get a more advanced warranty.  If you are in situations where a broken computer can be costly then pay extra to get NBD onsite support.  A technician will meet you wherever you are, at your house, conference, etc. the next business day with a spare part if something needs to be replaced.  For road warriors who can’t have downtime this is a must.  On the other hand, if you aren’t traveling consider the cost of NBD vs just having an extra laptop on hand (perhaps your old laptop) you could use while your main one is under repair.

I generally purchase the cheapest warranty (1 year basic) because I have a spare and if my computer breaks early I’ll just buy a new one.  Over the long run I think this is cheaper.. but if I was a frequent traveler I’d probably opt for a 3 or 4 year NBD warranty.

Ultrabook vs Mobile Workstation.   Ultrabooks are designed to be as thin and light as possible, often because of the smaller size heat can’t be dissipated as quickly so the CPU can’t run at a sustained load for long periods of time without throttling, or a weaker CPU is used.  Most people won’t notice throttling and this is becoming less of an issue as CPUs become more efficient.  The other sacrifice ultrabooks make is shorter key travel so they don’t have a great typing experience, and fewer ports, slots, and extras like GPUs.  Sometimes components like RAM are soldered on and batteries may not be replaceable.

Mobile Workstations can usually be outfitted with more battery, more processing power, more key travel giving them a fantastic typing experience, and are generally easy to service  They tend to be heavier, but generally more durable and more likely to be found with more ports, not throttle under heavy load, can get them with a GPU, and often have trappable batteries.

Ways to save money.  So, in most cases there are several base configurations which can be customized.  I have found in general that Memory and Hard drives are more expensive upgraded through Dell or ThinkPad’s store.  Often it’s cheaper to buy a base configuration unit with the CPU you want and then buy your own memory and hard drive.  For most people swapping out the hard drive will be difficult because the OS will have to be reloaded so may not be worth it.  Sometimes memory is not replaceable so check the laptop your buying to see if it is.  Generally this is possible on the workstations and a hit and miss on the ultrabooks.   If buying a ThinkPad read the ThinkPad Introduction page which has links to Perks discounts.

Wireless Card.  Always get Intel.  If it’s not Intel branded, don’t buy the laptop.

Touchscreen.  I don’t like touchscren but some people do.  Usually both options are available.

Glossy or Matte Screen.  I much prefer Matte, I don’t want to see my own reflection in the screen.  Usually both options are available.

Screen Size and Quality.

Ono of the most popular screen sizes (and my favorite) is 14.4″, it allows for a full-size keyboard (without tenkey) and seems to me to be the right balance between portability and using it like a workstation (faster CPUs, optional GPUs, more key travel on the keys.  The ThinkPad T470 and Latitude 5480 are great workstations in this class, and the Latitude 7480 and ThinkPad X1 Carbon (which is lighter than a lot of 13.3 and 12.5 inch laptops) are great ultraboooks.

For frequent travelers going to 13.3″ or 12.5″ may be better.  If you need a bigger screen or a ten-key then a 15.6″ or 17″ is the way to go.

Dell is going to have better quality screens for brightness and color than Thinkpads in general.  I think 1920×1080 (FHD) screen resolution is pretty decent.  You may want to avoid higher resolutions than that like that because many applications can’t scale properly and become difficult to read.

Apple Laptops have a 16:10 aspect ratio instead of a 16:9.  16:9 is the aspect ratio that movies are in, but in most cases the 16:10 (extra vertical space) would be preferable.

Some newer laptops are coming out with aspect ratios with more vertical space such as 3:2 which is a good compromise between 4:3 and 16:9 but they haven’t made it to mainstream yet.

Keyboard.  The best keyboards will be on the ThinkPads, and the best of those will be on the Thinkpad T series, and the best one on the market today is the ThinkPad 25 but at a high cost premium.  If you use a computer to consume media this won’t matter.  If you’re going to be docked in most of the time it’s not a big deal since you’ll use an external keyboard.  If you type a lot on your laptop the ThinkPads will be better than Dells or Apples.

Keyboard Lighting: Most laptops have a backlit option, if you want it make sure it’s there.  Some older ThinkPads have a “ThinkLight” which is a light on the top of the screen that shines down on the keyboard.

CPU: Stick to the Intel Core i5 or i7 CPUs, whichever is cheapest.  For the most part there is very little difference between an i5 and i7, in smaller computers the i5 will perform as well or better than an i7 because it puts out less heat so doesn’t have to throttle as much.  AMD processors have been behind Intel in Laptops, would consider them 1 or 2 generations behind Intel although they have started to close the gap with the Rzyen processors they’re still a year behind Intel.  I would consider a newer AMD CPU if the price was right but for anything older than 7th gen AMD stick to Intel.

In general, since the i series most CPU generational changes are not that substantial, maybe adding 10-20% boost in performance between each generation so the need to buy a new computer often to get a faster CPU is not particularly great these days.  Most of the gains are around power consumption and battery life.  However, the 8th gen CPUs which should be widely available next year (2018) offer about a 30-40% improvement over 7th gen because of an increase in core count.  You can tell which generation you’re buying by looking at the first number after the “i5” or “i7”  E.g. a Core i5-7600 is a 7th Gen.  The Core i5-8600 is 8th Gen.

Memory: 8GB should be your absolute minimum.  I always get 16GB memory, but I try to buy a laptop with the smallest amount of memory possible and buy extra memory from Amazon.

Hard drive:  The single best thing you can do for computer performance is to get an SSD.  You do need to watch out for size.  NVMe SSDs tend to be faster.  Both will well outperform a normal hard drive.  SSDs are smaller so make sure you get an adequate size.  Minimum of 256GB for most people.  If you are my mother in law maybe a terabyte minimum.

Graphics Card / GPU: Most laptops are not great for gaming.  If you are buying a dedicated gaming laptop most of my recommendations are not ideal and you many want to look at other options.  But if you do play video games you should consider getting a laptop with an AMD or NVIDIA card in it, you’ll be better off than without it.  You’re not going to get the performance from a laptop that you would out of a desktop gaming computer, but you can get pretty far.   Having a GPU usually cuts into battery life but it’s not as bad as it used to be… most laptops can shutdown the discrete video card when not in use and use the built in Intel HD graphics on the CPU which is more battery friendly.  Another option is to get a laptop without a GPU, but use an eGPU enclosure and buy a desktop GPU to put in it… it will connect to your computer via Thunderbolt port.

Batteries.  There are usually a few options for batteries.  Many laptops don’t have removable batteries.  For laptops with removable batteries smaller ones tend to sit flush with the laptop.  Some laptops also offer larger battery packs (and even slices) that make the laptop bulkier but can provide more than 20 hours of battery life.

Some laptops can be adjusted to make the battery last longer by reducing the charge cycles.  E.g.  set your laptop to not start charging the battery until it drops below 80% instead of 95%, and having it charge to only 90% capacity may improve the longevity of the battery quite a bit at the cost of perhaps an hour of battery life of run-time.

DVD Drives.  It’s hard to find newer laptops with DVD drives, but some are available, especially if buying older used models.  Generally you can buy a blu-ray laptop drive and swap it out if you want to watch blu-way video.

Ports.  Consider what ports you will want on your laptop.  Is Ethernet important?  How many USB ports do you need?  What about USB-C?  What about a docking port?  If you present frequently maybe you want a laptop with a VGA port and an HDMI port?  What about SD Card readers?  Headphone jack?  Do you have to use a Smart Card to access certain systems?  In most cases I’ve found I use ports less frequently than I think I would–for me an SD Card reader, Ethernet and a couple USB ports is all I need.

Webcam, Microphone, and Speakers.  If you care about these things google the laptop model you’re looking at plus the word “review” and read a few reviews to see if you can get a sense of the microphone, webcam, and speaker quality.  Some laptops have the webcams placed at the bottom of the screen instead of the top which results in a weird angle when on using video calls.  Also, some laptops don’t have very good speakers so check reviews to see if they’re good, my Dell Latitude E5450’s speaker is so weak I can’t really hear the audio in movies 3 feet away unless there’s absolutely no other noise.

When do new laptop models get released?  It depends, I usually see new Latitudes and ThinkPads announced and released between January and April.  Often new models are announced at electronic shows.   But it depends on whether Intel and all the other suppliers are on schedule so things often get shifted around quite a bit.

Are there other good laptops than the ones you mentioned?  Yes there are.  There are other decent brands, some of the consumer laptops are fantastic.  I don’t know every possible laptop out there at every given moment.  This guide is meant to be more of a generic guide looking at good laptop lines over time, with the availability of NBD support if needed, and docking solutions across a wide range of options from workstations with GPUs to ultrabooks.  For the most part those come from ThinkPad and Dell, but that doesn’t mean a gem isn’t produced under other brands from time to time.

Hope that helps.

 

 

Ben’s Law | The Cost of Being Interrupted

Ben’s Law: within a 4 hour block of time, for each unit of uninterrupted time in hours (t), the value of productivity and creativity is roughly t^2.

An interruption resets t to zero.

Ben's Law: Cost of Interrupting Ben

p = t^2
c = t^2

if t = 1 (1 hour of uninterrupted time) then
p (productivity) = 1 and
c (creativity) = 1

if t = 2 then
p = 4 (4 times more productive then at t = 1) and
c = 4

and so on…

Now, I say roughly, because around the 4th hour–as it gets closer to lunch productivity starts to go down, the curve probably looks more like the below but p or c=t^2 is close enough.

Uninterrupted Development – Ideal 4 hour block of time

The below is very difficult to achieve.  This only happens to me once or twice a month, but when I get a 4 hour block of uninterrupted time I get more done during the last two hours of that block than I do in an entire week!

Writing programs is not at all like rote work, or any job where you’re following a procedure and can just pick up where you left off.  Development is more of a creative task, it requires time to ramp up, load what you’re trying to accomplish in your head.  You can’t always switch into creativity mode on demand and just start coding, you just find yourself one second staring at the code, and the next moment you’re unaware of your surroundings, you’re in the zone and the longer you can stay there the more you can accomplish.  I would say programming is more creative than most people think.  It’s more like painting, or writing a book, or composing music than it is engineering.  Interrupting a programmer is like interrupting a musician in the middle of a song.

Interrupted Development – Real World 4 hour block of time sliced to bits

This is more like the real world, and probably is a better indicator of most programmer’s 4 hour blocks of times.  You can get some work done this way, but it takes about a week to do what could be a day’s worth of work.  A quick interruption sometimes won’t cause enough damage to reset back to zero, but anything over a few minutes will do so.

 

Interrupted Time

See also:

http://www.paulgraham.com/makersschedule.html

This post is licensed under the CC BY 4.0 license.

Benchmarking Guest on FreeNAS ZFS, bhyve and ESXi

FreeNAS 11 introduces a GUI for FreeBSD’s bhyve hypervisor.  This is a potential replacement for the ESXi + FreeNAS All-in-One “hyper-converged storage” design.

Hardware

Hardware is based on my Supermicro Microserver Build

  • Xeon D-1518 (4 physical cores, 8 threads) @ 2.2GHz
  • 16GB DDR4 ECC memory
  • 4 x 2TB HGST RAID-Z, 100GB Intel DC S3700s for ZIL (over-provisioned at 8GB) on an M1015.  In Environments 1 and 2 this was passed to FreeNAS via VT-d.
  • 2 x Samsung FIT USBs for booting OS (either ESXi or FreeNAS)
  • 1 x extra DC S3700 used as ESXi storage for the FreeNAS VM to be installed on in environments 1 and 2 (not used in environment 3).

Environments

E1. ESXi + FreeNAS 11 All-in-one.

Setup per my FreeNAS on VMware Guide.  Ubuntu VM with Paravirtual is installed as an ESXi guest, on NFS storage backed by ZFS on FreeNAS which has raw access to disks running under the same ESXi hypervisor using virtual networking.  FreeNAS given 2 cores and 10GB memory.  Guest gets 1GB memory.  Guest tested with 1C and 2C.

E2. Nested bhyve + ESXi + FreeNAS 11 All-in-one.

Nested virtualization test.  Ubuntu VM with VirtIO is installed as a bhyve guest on FreeNAS which has raw access to disks running under the ESXi Hypervisor.  FreeNAS given 4 cores and 12GB memory.  Guest gets 1GB memory.  Guest tested with 1C and 2C.  What is neat about this environment is it could be used as a stepping stone if migrating from environment 1 to environment 3 or vice-versa (I actually tested migrating with success).

E3. bhyve + FreeNAS 11

Ubuntu VM with VirtIO is installed as a bhyve guest on FreeNAS on bare metal.  Guest gets 1GB memory.  Guest was backed with a ZVOL since that was the only option.  Tested wih 1C and 2C.

All environments used FreeNAS 11, E1 and E2 used VMware ESXi 6.5

Testing Notes

A reboot of the guest and FreeNAS was performed between each test so as to clear ZFS’s ARC (in memory read cache).  The sysbench test files were recreated at the start of each test.  The script I used for testing is https://github.com/ahnooie/meta-vps-bench with networking tests removed.

No attempts on tuning were made in any environment.  Just used the sensible defaults.

Disclaimer on comparing Apples to Oranges

This is not a business or enterprise level comparison.  This test is meant to show how an Ubuntu guest performs in various configurations on the same hardware with constraints of a typical budget home server running a free “hyperconverged” solution–a hypervisor and FreeNAS storage on the same physical box.  Not all environments are meant to perform identically…my goal is just to see if the environments perform “good enough” for home use.  An obvious example of this is environments using NFS backed storage are going to perform slower than environments with local storage… but it should still at the very least max out a 1Gbps ethernet.  This set of tests is designed to benchmark how I would setup each environment given the constraint of one physical box running both the hypervisor and FreeNAS + ZFS as the storage backend.  The test is limited to a single guest VM.  In the real world dozens, if not hundreds or even thousands of VMs are running simultaneously so advanced hypervisor features like memory deduplication are going to make a big difference.  This test made no attempt to benchmark such.  This is not an apples to apples test, so be careful what conclusions you derive from it.

CPU 1 and 2 threaded test

I’d say these are equivalent, which probably shows how little overhead there is from the hypervisor these days, though nested virtualization is a bit slower.

CPU 1 and 2 threaded

CPU 4 threaded test

Good to see that 2 cores actually performs faster than 1 core on a 4 threaded test.  Nothing to see here…

CPU 4 threads

Memory Operations Per Second

Horrible performance with nested, but with the hypervisor on bare metal ESXi and bhyve performed identically.

Memory OPS

Memory MB/s

Once again nested virtualization was slow.. other than that neck and neck performance.

Memory Test

OLTP Transactions Per Second

The ESXi environment clearly takes the lead over bhyve, especially as the number of  cores / threads started increasing.  This is interesting because ESXi outperforms despite an I/O penalty from using NFS so ESXi is more than making up for that somewhere else.

OLTP Test

Disk I/O Requests per Second

Clearly there’s an advantage to using local ZFS storage vs NFS.  I’m a bit disappointing in the nested virtualization performance since from a storage standpoint it should be equivalent to bare metal FreeNAS, but may be due to the slow memory performance in that environment.

Disk Ransom I/O

Disk Sequential Read/Write MBps

No surprises, ZFS local storage is going to outperform NFS

Disk Sequential I/O Well there you have it.  I think it’s safe to say that bhyve is a viable solution for home (although I would like to see more people using it in the wild before considering it robust–I imagine we’ll see more of that now that FreeNAS has a UI for it).  For low resource VMs E2 (nested virtualization)  is a way to migrate between E1 and E3–but it’s not going to work for high performance VMs because of the memory performance hit.