Mount St. Helens

Mount St. Helens
Mount St. Helens

Next May will be the 40th Anniversary of the Mount St. Helens Eruption which occurred on May 18th, 1980. At the time geologists knew very little about volcanoes or the possibility of a lateral blast… it killed 57 people, most in areas outside the restricted zone. It is the most devastating eruption to occur in U.S. History. There have been many volcanic eruptions, but this one happened in an area near modern western civilization so it was well studied and documented.

We got to meet Paul Taylor over at the Mount St. Helens Creation Center and he took us on several excursions.

If I wrote about everything we did there this post would be too long, so here are the highlights.

East Side Excursion

Tree killed by heat from the volcano blast
Tree killed by heat from the volcano blast

The green trees have all grown since the volcano. Following the landslide the pyroclastic flow blasted out at speeds up to 670 mph, knocking down 230 square miles of trees. But the tree above was far enough away it wasn’t knocked down. It was just killed from the heat.

Tree uprooted with main root ball intact
Tree uprooted with main root ball intact

All over the place… and I mean everywhere we can see trees with rootballs uprooted and torn from their roots having been knocked over by the blast. We saw miles and miles of devastation like this. One photo doesn’t do it justice.

Uprooted trees from Mt. St. Helens
More uprooted trees, notice the fallen trees in the background

In the picture below as I was looking at this from a distance I thought there was ice on the right-side of the lake. But looking through my binoculars it’s not ice! Those are logs! We took a hike to take a closer look…

Spirit Lake
Spirit Lake
Hike to Spirit Lake
Kris and Eli going down to Spirit Lake

Hiking down to Spirit Lake

Boy under a fallen tree
Eli under a fallen tree over the path

Family in front of floating logs on Spirit Lake
Ben (me), Kris, and Eli in front of the floating logs on Spirit Lake

The landslide off Mount St. Helens rushed into Spirit Lake at 110+ mph sloshing the water out of Spirit Lake onto a hill with thousands of trees. …which had just been sheared or uprooted many with root ball intact by the initial blast of superheated volcano ash and gas seconds earlier. The landslide rose the lake level by 200ft, then the water returned to the lake taking an avalanche of logs with it. These logs I took a photo of have been floating on the lake for nearly 40 years.

Floating Logs on Spirit Lake
Floating Logs on Spirit Lake

Sonar and divers confirm that many of the logs have sunk and are in various positions at the bottom of Spirit Lake. Since logs sunk at different times they are buried in various layers of sediment.. and they’re spread out all over the lake as if they were a forest. All of this from one event nearly 40 years ago. You’ll notice that this looks very similar to the Yellowstone Petrified Forest where trees are found in different layers of sediment, often with the root ball attached but no roots.

Trees shown in different layers at Mt. St. Helens Spirit Lake
Trees sunk to the bottom and sit vertically. As more sediment accumulates the trees could be mistaken for growing in place at different time periods for different layers, but we know none of the trees grew here. Graphic Credit: Theresa Valentine / US Forest Service

What’s happening in Spirit Lake doesn’t fit the evolutionary narrative of millions of years. This Yellowstone Park article https://www.yellowstonepark.com/things-to-do/yellowstones-petrified-forest still claims that trees found in multiple layers indicate forests over different time periods over tens of thousands of years …and of course it all happened 50 million years ago. The evidence doesn’t bear this out. We know the petrified forest doesn’t need 50 million years to form. We know it can happen very quickly …because we are seeing how it happened today!

Mount St. Helens Creation Center

If you’re in the Mount St. Helens area, you should stop by the Mount St. Helens Creation Center in Castle Rock, WA. There you may find biblical creationist Paul Taylor who is more than happy to answer questions about a variety of topics. There’s a video presentation area where talks are given, comfy chairs and a couch to sit in, complimentary coffee, a few book displays featuring his books as well as other creationists, some free brochures, and a number of exhibits and displays. We were only there an hour in total but could have easily spent half the day there. The center was quite busy before Paul had to shut it down for the excursions.

Benjamin Bryan and Paul Taylor at the Mount St. Helens Creation Center
Paul Taylor and Ben
Eli and Solar System
Eli Found a Solar System display…
Visitors at the Mount St. Helens Creation Center
Visitors at the Mt. St. Helens Creation Center
Seating Area at the Mount St. Helens Creation Center
Sitting area
Noak's Ark Model at the Mount St. Helens Creation Center
Model Ark

Lava Tubes

Between Excursion days we explored the Ape Cave and Lava Tubes

Lava Tubes
Eli Descending a Lava Tube
Lava Tube
Why is it taking you so long, dad?
Lava Tube
Oh good, he made it.
Lava Tube Map
Ape Cave
Ape Cave

West Side Excursion

This is the most popular excursion.

Near Mount St. Helens

This area is a national monument so there is no reforesting or replanting by humans, everything you see is a completely natural which shows how quickly plants recover. A lot of mud flowed through here.

Castle Lake
Castle Lake

Castle Lake didn’t exists before 1980… it was created by the eruption.

Mud from Mt. St. Helens cresting the Johnston Ridge

The landslide that was traveling so fast and far it crested this ridge (the smaller hills are from the landslide). We are walking on the Johnston ridge in this photo near the observatory about 5 miles from the volcano.

Lupine Flowers

Lupine is important to help the area recover, this flower helps other plants grow in volcanic areas by taking nitrogen out of the air and then sharing it with other plants through roots.

Canyon's with Rock Layers
Canyons with Layers much like the Grand Canyon

For as long as I can remember secular scientists have claimed layers of rock such as you see in the Grand Canyon must have formed over millions of years. This is not observational science, but an assumption made to fit the evolutionary narrative. From the Johnston Ridge with a pair of binoculars I can see the layers. If nobody where there to observe the Mount St. Helens timeline secular scientists today would say the rock layers here took millions of years to form just like they say the Grand Canyon did. But for Mount St. Helens we where there to observe it so we know how old these layers are. 25 feet of organized matter was laid down by the volcano in a very short time. How long did it take to form these 200 (conservatively) layers? 3 hours.

Even today, secular scientists will still defend a millions of years timeline for the grand canyon. E.g. https://geology.com/articles/age-of-the-grand-canyon.shtml When I was at the Johnston Observatory I didn’t see any displays or comments discussing the rapid formation of these layers. It’s one of the most fascinating features here but the only display you’ll find on it is at the Creation Center.

Graphic showing Mount St. Helens rock layers compared to Grand Canyon

Paul Taylor Conference at Kootenai Church May 2020

Paul Taylor is coming to speak at Kootenai Community Church in May of 2020 (which coincides with the 40th Year Anniversary of the eruption). If you’re up in North Idaho it will be well worth your time to attend. I expect registration for the conference will open up sometime in 2020 so watch the main website for registration if you’re interested (you can also leave a comment saying you’re interested and I’ll email you when registration opens).

If you’re interested in the Excursions contact the Mount St. Helens Creation Center to book one: https://mshcreationcenter.org/7ws2/visit/excursions/

Final Thoughts

One person that was on the excursion with us was there right before it happened and actually had taken pictures months before the eruption. It was also neat to talk to some of the locals who witnessed the event and how it personally impacted them. I even talked to some people in North Idaho who remember a strange cloud interrupting a sunny day and covering the area with ash. It’s fascinating to listen to all their stories.

This was a great trip, fun for our family and a good learning experience. It was also enjoyable because I didn’t check work email the entire time (I did take my laptop just in case and my coworkers knew they could text or call for anything critical). They didn’t have to contact me once (thanks for everyone who worked hard to make that happen).

During the excursion Paul reminded us that 2 Peter 3 tells us in the last days that scoffers will deny two events: Creation and the Flood.

For they deliberately overlook this fact,
that the heavens existed long ago,
and the earth was formed out of water
and through water by the word of God,
and that by means of these the world that then existed
was deluged with water and perished.

2 Peter 3:5-6

You’ll notice from 2 Peter 3 that overlooking Creation and the Flood isn’t a result of ignorance. It is deliberate. It is not that there isn’t compelling evidence. Evidence is staring the secular scientist in the face. Evidence won’t change them because they already know the truth and suppress it. Rather, it is an issue of lack of belief in Jesus Christ. So while the evidence found at Mount St. Helens has value and confirms the position of biblical creationists, evidence is not what we base our beliefs on. Rather, we base our beliefs on God’s Word. Evidence is not the means that will transform an unbeliever into a believer. Rather God has chosen to use the power of the gospel for that task.

Jupiter – First Attempt at Stargazing with a Telescope

On clear nights I would often take Eli outside to look at the stars when he was a toddler. I told him the names of a few stars, he asked me to tell him the names of all the stars. I found out quickly young eyes are better for looking at stars, he could see a lot more of them than me. This evening we got a chance to look at Jupiter and two of it’s moons using a telescope!

Right now it’s close enough you can see Jupiter’s moons with a good pair of binoculars.

Jupiter and two moons
Jupiter and 2 moons

Finding Jupiter was the easiest part. Early evening it was right where it should be.

Jupiter

The most difficult part was pointing the telescope at that star. We borrowed a telescope (thanks Sean!) and after lots of randomly fiddling with the various undocumented knobs I finally figured out which ones did X another X, Y, Z, Y again, another Y, a yawing Y, and some sort of arc, and got it pointed towards Jupiter!

The earth of course is rotating so I had to re-align it every minute or so. I think it would be a great idea for someone to make a telescope with built in gyroscopes so they will stay pointed in a particular direction.

Setting up a Telescope
There’s a lot of knobs on this telescope. Just tell me when you see something in the scope!
Looking through a telescope
Kris looking through the scope while Eli looks through the spotter scope.
Jupiter and a moon using Pixel Night Mode
Pixel Night Mode

Eli looking through telescope
Trying to move away from the light pollution.
Jupiter and two moons
Another shot of Jupiter and two moons.
Jupiter and moon positions

I’m guessing the moons we saw were Europa and Callisto since Io and Ganymede would have been transiting Jupiter at the time we were looking.

Jupiter and 2 moons

The heavens declare the glory of God,
and the expanse proclaims his handiwork.


Psalm 19:1-6 ESV

I switched to Duplicati for Windows Backups and Restic for Linux Servers

So long, CrashPlan! After using it for 5 years, CrashPlan with less than a day notice decided to delete many of my files I had backed up. Once again, the deal got altered. Deleting files with no advanced notice is something I might expect from a totalitarian leader, but it isn’t acceptable for a backup service.

Darth Vader altering the deal
I am altering the deal. Pray I don’t alter it any further.

CrashPlan used to be the best offering for backups by far, but those days are gone. I needed to find something else. To start with I noted my requirements for a backup solution:

  1. Fully Automated. I am not going to remember to do something like take a backup on a regular basis. Between the demands from all aspects of life I already have trouble doing the thousands of things I should already be doing and I don’t need another thing to remember.
  2. Should alert me on failure. If my backups start failing. I want to know. I don’t want to check on the status periodically.
  3. Efficient with bandwidth, time, and price.
  4. Protect against my backup threat model (below).
  5. Not Unlimited. I’m tired of “unlimited” backup providers like CrashPlan not being able to handle unlimited and going out of business or altering the deal. I either want to provide my own hardware or pay by the GB.

Backup Strategy

Relayed Backups

This also gave me a good opportunity to review my backup strategy. I had been using a strategy where all local and cloud devices backed up to a NAS on my network, and then those backups were relayed to a remote (formerly CrashPlan) backup service. The other model is a direct backup. I like this a little better because living in North Idaho I don’t have a good upload speed so in several cases I’ve been in situations where my remote backups from the NAS would never complete because I don’t have enough bandwidth to keep up.

Now if Ting could get permission to run fiber under the railroad tracks and to my house I’d have gigabit upload speed, but until then the less I have to upload from home the better.

Direct Backups

Backup Threat Model

It’s best practice to think through all the threats you are protecting against. If you don’t do this exercise you may not think about something important… like keeping your only backup in the same location as your computer. My backup threat model (these are the threats which my backups should protect against):

  1. Disasters. If a fire sweeps through North Idaho burning every building but I somehow survive I want my data. So must have offsite backups in a different geo-location. We can assume that all keys and hardware tokens will be lost in a disaster so those must not be required to restore. At least one backup should be in a geographically separate area from me.
  2. Malware or ransomware. Must have an unavailable or offline backup.
  3. Physical theft or data leaks. Backups must be encrypted.
  4. Silent Data Corruption. Data integrity must be verified regularly and protected against bitrot.
  5. Time. I do not ever want to lose more than a days worth of work so backups must run on a daily basis and must not consume too much of my time maintaining them.
  6. Fast and easy targeted restores. I may need to recover an individual file I have accidentally deleted.
  7. Accidental Corruption. I may have a file corrupted or accidentally overwrite it and may not realize it until a week later or even a year alter. Therefore I need versioned backups to be able to restore a file from points in time up to several years.
  8. Complexity. If something were to happen to me, the workstation backups must be simple enough that Kris would be able to get to them. It’s okay if she has to call one of my tech friends for help, but it should be simple enough that they could figure it out.
  9. Non-payment of backup services. Backups must persist on their own in the event that I am unaware of failed payments or unable to pay for backups. If I’m traveling and my CC gets compromised I don’t want to not have backups.
  10. Bad backup software. The last thing you need is your backup software corrupting all your data because of some bug (I have seen this happen with rsync) so it should be stable. Looking at the git history I should be seeing minor fixes and infrequent releases instead of major rewrites and data corruption bug fixes.
Raspberry Pi and 4TB drive on wooden shelf
Raspberry Pi 4TB WD Backup

My friend Meredith had contacted me about swapping backup storage. We’re geographically separated so that works to cover local disasters. So that’s what we did, each of us setup an SSH/SFTP server for the other to backup to. I had plenty of space on my Proxmox environment so I created a VM for him and put it in an isolated DMZ. He had a Raspberry Pi and bought a new 4TB western digital external USB drive that he setup at his house for me.

Duplicati Backup Solution for Workstations

For Windows desktops I chose Duplicati 2. It also works with Mac, and Linux but for my purposes I just evaluated Windows.

Duplicati screenshot of main page

Duplicati has a nice local web interface. It’s simple and easy to use. Adding a new backup job is simple and gives plenty of options for my backup sets and destinations (this allows me to backup not only to a remote SFTP server, but also to any cloud service such as Backblaze B2 or Amazon S3).

Animation of setting up a duplicati backup job

Duplicati 2 has status icons in the system tray that quickly indicate any issues. The first few runs I was seeing a red icon indicating the backup had an error. Looking at the log it was because I had left programs open locking files it was trying to back up. I like that it warns about this instead of silently not backing up files.

Green play icon
Grey paused icon
Black idle icon
Red error icon

Green=In Progress, Grey=Paused, Black=Idle, Red=Error on the last backup.

Duplicati 2 seems to work well. I have tested restores and they come back pretty quickly. I can backup to my NAS as well as a remote server and a cloud server.

Two things I don’t care for Duplicati 2.

  1. It is still labeled Beta. That said it is a lot more stable than some GA software I’ve used.
  2. There are too many projects with similar names. Duplicati, Duplicity, Duplicacy. It’s hard to keep them straight.

Other considerations for workstation backups:

  • rsync – no gui
  • restic- no gui
  • Borg backup – Windows not officially supported
  • Duplicacy- License only allows personal

Restic Backup for Linux Servers

I settled on Restic for Linux servers. I have used Restic on several small projects over the years and it is a solid backup program. Once the environment variables are set it’s one command to backup or restore which can be run from cron.

Screenshot of restic animation

It’s also easy to mount any point in time snapshot as a read-only filesystem.

Borg backup came in pretty close to Restic, the main reason I chose Restic is the support for backends other than sftp. The cheapest storage these days is object storage such as Backblaze B2 and Wasabi. If Meredith’s server goes down, with Borg backup I’d have to redo my backup strategy entirely. With restic I have the option to quickly add a new cloud backup target.

Looking at my threat model there are two potential issues with Restic:

  1. A compromised server would have access to delete it’s own backups. This can be mitigated by storing the backup on a VM that is backed by storage configured with periodic immutable ZFS snapshots.
  2. Because restic uses a push instead of a pull model, a compromised server would also have access to other server’s backups increasing the risk of data exfiltration. At the cost of some deduplication benefits this can be mitigated by setting up one backup repository per host, or at the very least by creating separate repos for groups of hosts. (e.g. a restic repo set for minecraft servers and separate restic repo for web servers).

Automating Restic Deployment

Obviously it would be ridiculous to configure 50 servers by hand. To automate I used two Ansible Galaxy roles. I created https://galaxy.ansible.com/ahnooie/generate_ssh_keys which automatically generates ssh keys and copies the key ids to the restic backup target. The second role https://galaxy.ansible.com/paulfantom/restic automatically installs and configures a restic job on each server to run from cron.

Utilizing the above roles here is the Ansible Playbook I used to configure restic backups across all my servers. This sets it up so that each server is backed up once a day at a random time:

Manual Steps

I’ve minimized manual steps but some still must be performed:

  1. Backup to cold storage. This is archiving everything to an external hard drive and then leaving it offline. I do this manually once a year on world backup day and also after major events (e.g. doing taxes, taking awesome photos, etc.). This is my safety in case online backups get destroyed.
  2. Test restores. I do this once a year on world backup day.
  3. Verify backups are running. I have a reminder set to do this once a quarter. With Duplicati I can check in the web UI, and with a single Restic command it can get a list of hosts with the most recent backup date for each.

Cast your bread upon the waters,
for you will find it after many days.


Give a portion to seven,

or even to eight,
for you know not what disaster may happen on earth.

Solomon
Ecclesiastes 11:1-2 ESV

Dr. Jason Lisle Conference

When Eli was 4, Kris and I needed to keep him occupied while we packed our house up ahead of a move… so I found a video by Answers in Genesis and put it on. Eli discovered an interest in the solar system. He liked it.

A lot.

For the next several years he studied the planets.

Made drawings of planets

Created the asteroid belt…

Made the solar system out of balls…

Made the solar system out of dinosaurs…

Represented it with Legos…

Another Lego Solar System…

Drew Orbits.

His interest in astronomy has not waned so we were excited when we found out Dr. Jason Lisle was coming to our church for a conference. It was held last Friday/Saturday and one of the best conferences I’ve attended.

Dr. Lisle is a Christian astrophysicist. Has held positions at Answers in Genesis, Institute for Creation Research (ICR), and now the Biblical Science Institute. One of my favorite papers by him is about the Anisotropic Synchrony Convention and I’ve enjoyed his debate with Hugh Ross and read a number of Jason Lisle’s Books over the years.

His sessions covered a foundation on Genesis, Astronomy, Science, Fractals, and a few Q&A sessions (he answered Kris’s questions about the multiverse and Kuiper Belt). The conference was recorded so if you’re interested you can watch it below:

Eli’s second love is math, so his favorite part was the session on how God thinks about numbers, which spent a good deal on fractals on Day 2.

Earlier this year while attending a home school curriculum conference I was reminded that there is no subject that can be taught from a neutral perspective. You are either for God or against God. Even math. You can either teach it from a secular perspective which is to ignore God and avoid the question of where math comes from or perhaps try to come up with some explanation about why math exists and even works; or you can teach it from the worldview that that all things including abstract things like math are created by God and therefore have beauty and reflect His nature. By studying math we are discovering what God thinks about numbers and would therefore expect to find beauty in numbers. There is no neutral position. Dr. Lisle’s presentation made this more apparent.


“I was merely thinking God’s thoughts after him.” – Johannes Kepler

How to Get Longer Life Out of Your Dell Laptop Battery

In 2015 I bought myself and Kris Dell Latitude E5450 laptops. 1 year later her battery was fine, however mine lasted 60 seconds on a full charge. I attribute this to Kris often using her computer on battery and not having it plugged in all the time, and me always having my computer in the docking station so it’s constantly charging.

60 seconds of run-time

I lived with a bad battery for 3 years… 60 seconds is enough to run from one outlet to the next without having to power down… which is really all I need. Although I’ll admit 120 seconds would be nice!

Battery Swelling Issue

A couple weeks ago I noticed a crack near my touchpad… and a bulge. My laptop was growing! Or rather, the battery was expanding! The battery pack is about 175% the height of what it should be!

That Dell battery pack on the left is a little swollen….

I quickly waited a few months, and decided that despite the battery still giving me my 60 seconds, this could be a safety or fire risk or my laptop might break if it swells much more, so out of prudence decided to buy a new Dell G5M10 battery. After installing it I went into the BIOS and noticed settings to change how Dell manages the battery! You can opt for faster charging, more run-time, or more longevity.

Here are the batttery life settings.

Charge Time, Run Time, or Lifespan. Pick any 1, sometimes 2.

  • ExpressCharge – Faster charging. This was the default! The problem is the faster you charge a battery the more you cause it to wear out sooner. This makes sense for people on the road who don’t have a lot of time to recharge. But it doesn’t make sense if you’re almost always on AC power like me. This setting probably has a high charge stop up to maximum capacity (100%?) and high custom charge start (95%) so that it’s always ready. I’m not an expert in batteries, but I believe batteries naturally lose power over time so each time it drops 5% of it’s power it charges back up to 100%… those constant charge cycles cause a lot of wear not to mention the battery is being held at full charge which causes it to degrade faster. Running in this setting is giving you the best performance but you’re pushing the limits.
  • Standard – This is the same as ExpressCharge as far as I can tell but a little slower charge. Other than that it’s still going to wear the battery out fast.
  • Primary AC User – Designed to extend the battery lifespan for laptops that are usually plugged in. I assume this does two things: It probably slows down the charge rate, sets the Charge stop to a lower value like 70%, and sets the charge start to around 50% (I’m completely guessing at these numbers). This reduces the number of charge cycles needed to maintain the battery and is generally charging the battery up to levels suitable for long-term storage instead of maximum performance giving you the best lifespan at the cost of run-time. If you want longevity at the cost of run-time this is the setting you want.
  • Adaptive – This is what the default should be! It’s a trade-off between the two. It optimizes battery settings based on how you typically use the computer. Meaning if you’re running on AC power all the time it will act more like the Primary AC power setting, but if it sees you are using the battery a lot it will start behaving like the ExpressCharge.
  • Custom – Could also set custom values

Dell BIOS Settings for Battery Maintenance

Optimizing Battery for both performance and longevity depending on the time of day

This will only drive your battery hard when you might need the run-time, but go easy the rest of the time. If you have a fixed schedule you can tell your Dell laptop what times of day you need more run-time. But then outside of those hours it will maximize longevity.

Dell BIOS Settings for Battery

Well, I’ll be changing my BIOS setting to Primary AC User.

And with my brand new battery I’m liking the new 4-hour run-time again. Now days I walk from outlet to outlet instead of running.

How to Get Longer Life Out of your ThinkPad Battery

If you use a ThinkPad read this KB on How to Increase Your Battery Life by changing the Battery Maintenance settings.

Running Chrony NTP under a VMware Guest

Here’s a quick guide to run an NTP (Network Time Protocol) server using Chrony with a GPS (optional) receiver on a VMware ESXi Guest running Ubuntu 18.04.  I should note this is experimental and something I setup in my homelab temporarily.  For production environments I would run NTP on physical hardware and not VMware.

Create and Configure VM

Be sure to disable Guest Tools Time synchronization by editing the VM settings and uncheck Synchronize guest time with host.

Disable VM Tools Time Synchronization

Set the CPU shares to High… we want the NTP server to have priority if there is processor contention.

High CPU Shares

Install Chrony

I diversified between Ubuntu’s, NTP.org’s and NIST’s time server pools.

That’s it, after restarting the chrony service (service restart chrony) you should be able to get time reports by running:

Why You Shouldn’t Run an NTP Server in a VM Guest

VM’s can’t keep accurate time

I’ve generally found that VMs keep great time inside of VMware.  One thing that can help with this is setting the CPU shares to high so your time server always has a priority.  I ran Chrony in a VM for several weeks, compared it with Chrony on a Raspberry Pi.  Both were acceptable, and both had a smaller standard deviation than public NTP servers over the internet, but the VM had a much smaller standard deviation than the Pi.  That tells me VMs running on better hardware may be better than lesser bare physical hardware at time tracking under certain conditions, and a local NTP server in a VM can be more precise than grabbing time off the internet.

VMs can become out of sync during snapshots, suspend, failover, etc.

I ran a suspend test and this is true.  I paused a VM, waited 10 seconds, then resumed it.  It reported the wrong time to NTP clients for several minutes before it corrected itself from external NTP servers.  Here’s a screenshot of my NTP server being 11 seconds off after a pause!

Chrony after VMware Suspend

This is a valid reason to run an NTP server on physical hardware.  However, I think it is possible to run an NTP server under VMware with the following precautions:

  1. Your NTP servers under VMware should never be paused.  That means they should be excluded from failover (instead of failover it’s better to configure multiple NTP servers for your clients to connect to since it’s better for an NTP server to be down than report a wrong time).
  2. Have multiple NTP servers.  At least three. You’ll notice in the screenshot above Chrony (running on a separate physical machine) flagged the server as not being accurate.  This way if one of your VMs gets paused chrony will switch to another time-source automatically.
  3. Set makestep 1 -1 in the chrony.conf file (this tell chrony that any difference greater than one second will get stepped which allows for faster correction after a resume).

GPS Receiver

This is not really related to VMware.  But I had a GPS receiver so thought I’d see how it works with Chrony….

GlobalSat GPS Receiver

I have a GlobalSat BU-353S4 USB GPS Receiver.  This isn’t the best GPS receiver for accuracy.  For me it’s accurate to within a few hundred milliseconds which is good enough for my experimental purposes but worse than just grabbing time off the internet.  For serious time-keepers you’ll be wanting to use something faster than USB and more accurate than what a cheap GPS receiver can provide.

Configure gpsd

Install Chrony

So, how did I get the values on the refclock line…

The way I came up with my offset of 0.250 is by initially setting the offset to 0.0, restarting chrony, and running chronyc sources -v several times taking note of the offset.  I’d get numbers like +249ms, +253ms, +250ms, etc.

Since my GPS is off by about 250ms I set the offset to 0.250.  Now it’s usually not off by more than 100ms.

Chrony Sources

The 100ms+- variance is not a problem when being combined with other sources, but if it was the only time source I’d be better off tolerating drift than the high variance of GPS for a short period without access to the NTP pools, if I had no internet for several months or an air-gapped network then time via GPS would probably be better than nothing–but a better GPS receiver should be used in those scenarios.

For most networks running chrony in a VM and using a GPS is unnecessary.  It’s better to keep it simple.  I just use the NTP service on my pfSense router and set all the clients to that.

Don’t forget to watch your clocks adjust themselves next Sunday!

Programming Management & Leadership Books

There are plenty of books on managing people; but there are few books targeting management of software development, and even fewer aimed at people who got promoted into leadership positions with no management skills.  I’ve read countless books looking for resources in that area…  I can find plenty of books about how to manipulate people or promote yourself (and I’ve had plenty of training to that affect) but those are not the books I’m looking for.

I want real authentic leadership and practical management.  Below you will find the best of what I’ve found over the last four years. And unlike some “Best Books for Programming Managers” and “Top 10 books on Leadership” lists you’ll find online… I actually read every book listed below. 

I should also note that even if you aren’t in a position of management these books should be beneficial.  Whether you have the position or not, everyone has the opportunity to lead.

Managing the Unmanageable

Managing The Unmanageable Book

“Most successful programming managers are former programmers: They can quickly grasp whether a developer is on track through the most informal of conversations, without having to ferret out the assessment through long strings of questions that can feel pestering.”



Managing the Unmanageable By Mickey W. Mantle and Ron Lichty (2012)

Managing the Unmanageable is the comprehensive handbook to gain a variety of insights and a tool set to manage software development teams.  I didn’t find it lacking coverage on any topic.

It rightly points out how managing programmers is like managing artists–programming is a creative job so you can’t manage that the same way you would manage most other jobs.

It goes over how to build relationships with and manage HR, your boss, other departments, etc.  How to define developer levels, how not to do incentives (which can often be more demotivating than motivating), job descriptions, how to conduct interviews, build culture, motivate developers, etc.  This is a wide book in what it covers.  The vastness of topics is unmatched by any other management book I’ve read.  It may only devote a few pages to some subjects but I haven’t found an area that it doesn’t cover at all. Even for areas it doesn’t go into great depth it references sources for further study.

I think this is the best resource for a new manager to get a comprehensive overview of every topic related to managing programmers.  What I really like about the book is from the experience of the authors it anticipates and provides guidance on a lot of challenges I had to deal with–reading this book helped me proactively plan how to deal with those situations.

For me, reading Managing the Unmanageable is like sitting down at a coffee shop with some seasoned managers and listening to their experience and wisdom.  Today I still use it as reference book.

Peopleware

Peopleware Book on Productive Projects and Teams

“The major problems of our work are not so much technological as sociological in nature.” 

“Most managers are willing to concede the idea that they’ve got more people worries than technical worries.  But they seldom manage that way.  They manage as though technology were their principal concern.  They spend their time puzzling over the most convoluted and most interesting puzzles that their people will have to solve, almost as though they themselves were going to do the work rather than manage it.”



Peopleware: Productive Projects and Teams (3rd Edition) by Tim DeMarco & Timothy Lister (originally published in 1987, I read the 3rd edition published in 2013)

Peopleware, as it’s title suggests is all about the people aspect of managing software developers.  It’s not a generic management book.  Most of it only applies to managing creative and intellectual workers.  It covers why programmers are distinct from and must be managed differently than other types of jobs, such as accountants or manufacturing workers.  The book covers topics like the importance of allowing time to think on the job, giving teams a sense of elitism to increase productivity, creating environments where teams can naturally form and jell, the importance of an interruption free office environment, why the surest way to improve productivity is by focusing on quality.

I learned environmental factors for a programmer cause a 10 to 1 performance difference.  A large section deals with the work environment.  Office design, layouts, how bad cubicles are, the importance of natural light, office size, privacy, etc.  This is a timeless classic.  It would benefit any manager, executive, head of HR, architect, or programmer (even if you aren’t in a management position, this book will help you manage yourself).

The Mythical Man-Month

The Mythical Man-Month

“Why is programming fun? What delights may its practitioner expect as his reward? First is the sheer joy of making things. As the child delights in his mud pie, so the adult enjoys building things, especially things by his own design. I think this delight must be an image of God’s delight in making things, a delight shown in the distinctness and newness of each leaf and each snowflake.”

The Mythical Man-Month: Essays on Software Engineering, Anniversary Edition (2nd Edition) by Fred Brooks (originally published in 1975, I read the 20th Anniversary edition published in 1995)

This is a collection of essays about managing and organizing large software projects. Most important is Brooks’ observation that adding more man-power to a late software project will make it even later. My favorite observation of his was how the most productive teams are smaller because of the communication overhead, you only get fractional gains by increasing the size of large teams. Although pre-Agile, many of his ideas influenced Agile project management. He was well ahead of his time. This is a classic. 

“Adding manpower to a late software project makes it later.”

The Conviction to Lead

The Conviction to Lead

“Whenever Christian leaders serve, in the church or in a secular world, their leadership should be driven by distinctively Christian conviction.”

“Leadership is all about putting the right beliefs into action, and knowing, on the basis of convictions, what those right beliefs and actions are.  This book is written with the concern that far too much of what passes for leadership today is mere management.  Without convictions you might be able to manage, but you cannot really lead.”

The Conviction to Lead: 25 Principles for Leadership That Matters By Albert Mohler, 2014

This was not an easy find. I read fluffy leadership book after fluffy leadership book… and finally read Mohler’s book at my dad’s recommendation.  It has far more substance on leadership than anything else I’ve read.  Where others give you mechanics, tools and methods, Mohler gives you conviction and motivation based on well grounded beliefs.  It is not written just to pastors, nor just to leaders of Christian institutions (although this appears to be the main focus), but also to Christians who happen to be leaders in secular organizations–and that’s quite rare for a book on leadership written by a devout Christian.

Mohler’s book is practical because it provides the foundation for why and how Christians should be leading and the basis for leading in a secular world.  I would say the book is primarily written to C-level, but almost all of it I was able to apply to a smaller realm for lower levels of management if I limited the scope to my area of influence.  This is a good book for any Christian in a position of leadership.

Kindle vs Paper Books

I’ve been using a Kindle for about 6 years.  And have been reading paper books for longer than that!  I have two Kindles, one is the discontinued Kindle Touch, and the other is the newer Kindle Paperwhite.  Here are my thoughts on the Kindle and how eBooks compare to Print Books.

The Kindle Reading Experience

For much of the reading experience I prefer the Kindle.  It’s compact, lightweight, and easy to carry around.  With a kindle I don’t have to awkwardly hold a book open while my other hand is trying to not spill my cup of coffee.  Also when it starts to dim outside and I don’t quite have enough light I can turn on the backlight instead of the house lights.

Kindle Paperwhite vs Book

Backlight

So, e-ink displays don’t have as good of a contrast as real paper.  The reason Amazon calls their latest Kindle the “Paperwhite” is it has a backlight that can sort of match the brightness of paper by supplementing the light from your environment–the idea is you turn the backlight on just enough so that it still looks like it’s reflecting light like a book, but there’s just enough extra light to make it as readable as paper.   This does work, however I think the LED color Amazon chose is a failure.  The pure white LED backlight is too much in the blue spectrum and that’s very obvious when I’m reading under incandescent lights.  It’s okay in natural light but under incandescent lighting it should be warmer to match the surrounding atmosphere  This could affect health if reading right before going to bed.  I hope Amazon fixes this in the next version…maybe it should have RGB bulbs and a sensor to match the ambient light.

In very bright light paper wins out, but if the ambient light is dim as it often is in the Fall in Idaho the Kindle let’s me read a little longer before turning on the house lights.  This probably saves me 1 or 2  cents a year.

Physical Library Size

Kindle Library Size

The Kindle does have the advantage of being able to store my entire Kindle library wherever I am… not only is it smaller than 99% of my books, it can store all of my books in that space.

Fonts

90% of paper book publishers choose great fonts–but some don’t.  For some reason some publishers think their book needs a sans-serif font, or they pick a huge font, or too small a font, or the kerning is not normal.  It bugs me!  If you get the Kindle version you can override the publishers horrible font decision.  As an added bonus the font-size is adjustable so I can read anything without glasses.

Quality

I always prefer a good hardbound paper book to an eBook, however I’ve noticed lately a lot of authors are using cheap (self-publishing?) services–it seems to me the books are printed on demand and the quality is sometimes bad–I’ve had books that–the best way I can describe it is the book feels like I’m holding some ad-hoc document put together at a business conference rather than a book.  I’ll often opt for an eBook if I see the author is using a self-publishing service (not all self-publishing books come this way–I think it’s just a quality control issue so it’s a hit and miss).

Enjoying Books with Others

Eli and Jon reading maps

The social aspect of eBooks is poor.  Often when I’m on an airplane or a friend is at my house they’ll show interest in a book I’m reading or I have on the shelf and it makes a great conversation starter.  You just don’t get that with Kindle books because nobody can see what you’re reading.  Kids love physical books and will spend hours poring over maps, illustrations, and pictures which would be boring on a tablet.  I can easily give a paper book to a friend.  While Amazon has some provision for lending it’s very limited and it’s not as simple as handing your friend a book.

Highlighting and Taking Notes

For highlighting it’s a wash–the Kindle is sometimes a bit finicky when I try to highlight a passage and sometimes gets the wrong portion highlighted but for the most part I can get it.  I always read a book with a pen or pencil but I find underlining a passage without the line going through the words to take a little more effort.  For taking notes in the margin nothing can beat pencil or pen on paper.

Diagrams and Illustrations

Diagrams are pictures are generally bad on eBooks.  For simple graphics it does fine.  But if the book has illustrations they don’t look as great because the screen is smaller and you lose color.

Kindle Lack of Color

Also, the Kindle completely fails at tables… this table below has data that is illegible on the Kindle… it’s too small to read and there’s no way to rotate it into landscape mode.

Kindle Table Fail

 

Flipping Through Pages

The Kindle is useless here.   Even in the flip through the pages mode the e-ink display takes too long to refresh.  A real book is much easier–plus I remember the layout of a page and generally know what I was looking for was in the 1st quarter of the book so can find it in seconds.

Searching

Here the Kindle shines.  If you are looking for a keyword or phrase you can find it very quickly.

Visual Indicators of Progress

Kindle Progress indicatorThis is a big deal.  I am very spacial and use the physical feel of how many pages I have read and how far to go as part of my memory.  This is all lost on eBooks.  With paper books it’s easy to see your overall progress at a glance, and if you want to thumb a few pages ahead to see when the chapter ends it takes half a second.  With an eBook I get something like location 675 or 24%.  That’s meaningless to me.   A progress bar might be nice!  Something visual and not just numbers.  Even web-browsers have scrollbars!

Reading Books as a Group

When reading books for study with others eBooks fail–I tried this once but everyone else was referring to page numbers and I couldn’t get page numbers out of my kindle.

Free eBooks

Amazon has a lot of free Kindle books for Prime members.  I’ve found the free books aren’t really that good so not much of a gain.

Free Classic Books

There are a number of great classic books you can download from the Guttenberg project, this may save you from purchasing a few paper books.

Updates to Books

Some of my more technical books have received free Kindle updates when the author chooses to update the text.  This is a benefit in my mind.  I think it would be better if the Kindle would highlight the differences.

X-Ray

Kindle X-Ray People

One nice feature on the Kindle Paperwhite is the X-Ray.  You can enable it for the page you’re on and it will tell you about the characters and give you some context (if you’ve forgotten the previous chapters or missed it).

Kindle X-Ray Terms

Newspapers

You can read newspapers on the Kindle.  But it’s worthless.  The Wall Street Journal digital subscription is completely separate from the Wall Street Journal Kindle Digital Subscription.  I’m not going to buy a Digital subscription for both my computer and my Kindle.

Synchronization

One great thing about eBooks is I can read them on my Kindle, then bring up the book on my computer to review my highlights while typing up notes–but it’s a hit and miss.  This works for Amazon books I bought from the Amazon store.  But if you buy Kindle formatted books from not Amazon there’s no way to get them to open up in the Kindle for PC program (even though they are available in Kindle for Android).  Very annoying.

So, What’s Better?  Kindle eBooks or Old Fashioned Physical Books?

It really depends.  I like both for different reasons.  I do have a preference for Print Books and mostly because I can visually track progress and visually see the layout of pages and flip through them.  Generally if it’s a book I’ll probably read once I’ll just get what is cheaper… but obviously some I’m going to insist on getting the physical version.  One feature that Amazon does for /some/ books is if you buy a physical, you can get the Kindle version for free, or heavily discounted.  I do hope that this becomes standard practice going forward–that’s the best of both worlds.

Of making many books there is no end, and much study is a weariness of the flesh.   The end of the matter; all has been heard.  Fear God and keep his commandments, for this is the whole duty of man.  For God will bring every deed into judgment, with every secret thing, whether good or evil.

– Solomon, Ecclesiastes 12:12b-14

 

7 Homelab Ideas | Why You Should Have A Homelab

Why You Should Have a Homelab

In 1998 my friend gave me a RedHat Linux CD.  I spent hours each day experimenting with Linux–I loved it.  2 years later I’m in a room with 30 other students at a University applying for the same computer lab assistant job–I’m thinking my chances are grim.  Part way through the mass interview a man walks to the front of the room and asks if anyone has ever used Linux.  I raise my hand–I’m the only one.  He takes me out of the interview for the lab assistant job, introduces me to the department director.  They took me out to lunch.  By the end of the day I had my first job as a Systems Administrator.

Learn things on your own and it will broaden your opportunities.

One of the best ways to learn about systems, applications, and technology is starting a homelab.  A Homelab can give you an enjoyable, low stress, practical way to learn technology.  A homelab will also help you find out the technical areas in which you are interested.  It’s also practical in that you can use it to service your own home.

Here’s 7 Ideas for Your Homelab

1. Router /  Firewall

Ubiquiti EdgeRouter X

The most essential piece of equipment will be your router.  I started out with consumer routers that I’d flash to DD-WRT / Tomato but now I use a virtual pfSense router.  Routers are great to learn about DHCP, DNS, VPN, Firewalls, etc.  I discourage using the router provided by your ISP, they’re usually not very capable and often not secure.  In most cases you can buy a DSL or Cable modem instead of the ISP provided modem combined with the router.  One inexpensive physical router I’d recommend is the Ubiquiti EdgeRouter X.   Ubiquiti provides free software updates (their model is you buy the hardware and the software is free), and you’ll get a handful of advanced features–it’s a very capable router and much better than a typical consumer router–to step up from Ubiquiti you’d be going to pfSense, Juniper or Cisco.

2. Storage

Supermicro StorageThe main reason I started my homelab was storage.  I was taking a lot of family pictures and videos and wanted to save them.  I know there are cloud services, but at the time they were expensive, and then you’re sort of trusting that provider to not delete all your photos or get bought out by a larger company and shutdown.

Then I started using VMware.  I needed faster storage with more IOPS.  One of the best Homelab storage solutions is ZFS.  ZFS takes the best of filesystems, and the best of RAID, and combines them into a software defined storage solution that I’ve not seen any hardware technology able to match.  Two popular free ZFS appliances I like are Napp-It (based on OmniOS) and FreeNAS.  OmniOS is a fork of OpenSolaris and is very robust and has tight integration with ZFS.

FreeNAS LogoI’m currently using FreeNAS which is the free open source version of iX System’s TrueNAS which is used by organizations of all sizes–from small businesses with a few TB of storage to large government agencies with PBs of storage.  FreeNAS has done a great job at technology convergence.  It is both a NAS and a SAN allowing you to try both approaches to storage (I prefer NAS because it takes better advantage of ZFS, but many prefer using SAN and there are benefits and drawbacks to both), it also has many built-in storage protocols:  FTP,  iSCSI, NFS, Rsync server, S3 emulator, SMB (Windows file server), TFTP, WebDav, it can join AD, it can even be an AD DC  (if you like living on the edge) it has a built-in hypervisor (bhyve) to run VMs for whatever you want.  This is now marketed as hyper-converged storage.  All of it is completely free.  You can build your own FreeNAS server like I did, or get started with a FreeNAS Mini from iX Systems.

A few years after I learned ZFS for home, my employer was looking for a new storage solution so having this knowledge and experience was helpful.  I was able to determine one vendor with a traditional RAID solution didn’t handle the RAID-5 write-hole problem properly.

3. Virtualization

VMwareVirtualization allows you to run multiple virtual servers on the same piece of hardware.  VMware is king in the small to mid-size business hypervisor market, and VMware offers their hypervisor for free.  The free version is just like the paid versions except you won’t be able to use some features (most involving high availability and fail-over with multiple servers).  But you can learn most of the concepts and features of VMware.  I’ve tried to use a number of hypervisors but I always come back to VMware.  I consider VMware my basic infrastructure.  From there you can learn about other things like networking, storage, and play with any OS or Linux distribution you want to.

Knowing VMware was hugely beneficial, I’ve implemented it for several businesses, and one of my previous employers.  And knowing how it works means I can discuss the VMware stack intelligently with the ops team.

See my FreeNAS on VMware Guide if you’re interested in running a virtual FreeNAS server inside VMware.

4. Networking

A Homelab without decent networking won’t get you far.  Fortunately if you use VMware you can leverage it to use virtual network switches.  For physical switches I really like the Unifi products.  They are simple enough for non-network engineers like me.  Everything can be configured using the GUI.  Unifi exposes you to managed switches, central management (with the Unifi controller), VLANs, and PoE (Power over Ethernet), port trunking, port mirroring, redundant paths with spanning tree, etc.

Unifi 8 Port SwitchI started with this little 8-port switch (4 are PoE ports).  I also added a 24-port switch so I could learn how to do setup a LAG and configuring VLANs across multiple switches (which was really simple using the Unifi interface).  I also like Unifi’s philosophy–they sell you the hardware but the software is free–which means you don’t pay for maintenance or support but continue to get free updates.  In a homelab you may not need to go crazy on VLANs, but separating your main network from your IoT devices may be prudent.

Learning how to setup VLAN tagging, and link aggregation and understanding how networking works helps me communicate better with the network engineers when discussing design and deployment options–they may be working on Juniper or Cisco equipment but I know the concepts of what they’re doing.

5. Wireless APs

 

Having a robust wireless setup is also a necessity for a homelab.  If you have a large house you get to setup multiple APs and make sure they can handoff connections.  I use a Unifi AP Pro (I just use one because that’s all I need to cover my house, but if you can find an excuse to have 2 or more I’d recommend it since you can practice rolling updates without downtime, wireless handoff, etc.).  These are managed by the same Unifi controller as the switches.  I first gave them a try because I read Linus Trovalds uses Unifi APs, and they seem to be highly rated by tech professionals–and now I don’t think I’d go back to anything else.

I have written more about Unifi Equipment here.

6. Network Monitoring

Icinga

It is hard to maintain a reliable network and application stack without monitoring for failures.  There are hundreds of network monitoring solutions and it really depends on your needs.  The most widely deployed solution is Nagios.  I have had that on my Homelab, but lately I’ve been using Icinga because it’s simple and it integrates into Ansible.

7. Infrastructure Automation

Automating your infrastructure may not make as much sense in a small Homelab, but it does make sense to automate any task you do repetitively or a manual task that could be automated.  For me, this was  installing updates, deploying servers and renewing SSL certificates with Let’s Encrypt.  To manage this I use Ansible which is one of the most well thought out infrastructure automation tools I’ve seen.  Ansible can manage Linux and Windows servers.  Learning infrastructure automation, especially if you do it using version control and CI/CD tools like Azure DevOps (you can get a free account for up to 5 users with unlimited private repositories) is a great thing to learn for your career if you’re interested in the DevOps world.   The book, Ansible for DevOps by Jeff Geerling helped me get started.  I suggest getting the eBook since he has been known to provide updates to the book (not sure if he will continue to provide updates, but just in case).

At work we completely automated the deployment of Linux servers using Ansible–infrastructure as code.  It took a month of investment but it paid off big time with developers now being able to deploy VMware VMs at will with Ansible by making a Git Pull Request, our entire fleet of servers is updated automatically, and our server and configurations are all consistent.  This replaced an old process of waiting several weeks for a VM to be provisioned and configured by hand.

Bonus homelab application server ideas…

  1. Minecraft Server — popular Java game–it’s like playing with Legos and a great way to get your friends together for some casual games.
  2. Mumble Server – one of the best voice protocols for in-game communication.
  3. Emby Media Server — Anyone that has kids realizes those flimsy blu-ray drives aren’t going to last long.  It’s great to store and host movies, home videos, pictures, and audio.
  4. Asterix PBX Server – VoIP Phone server (use Twilio or Flowroute for SIP trunking).  Polycom makes great VoIP phones.  With Twilio SIP Trunking you can have a real landline phone number with E911 capability for a few dollars a month–and if you get multiple phones you can use it as an intercom system.
  5. Web Server (maybe start a blog) — I hosted this blog from a server in my house for years–until my ISP couldn’t handle the bandwidth.  Now days you can also use a service like CloudFlare to act as a CDN which really reduces your bandwidth usage.  Hosting your own blog is a great learning experience and gives you a place to log your homelab experiments, and share solutions to problems.
  6. Automatic Ripping Machine — Get all your Blu-Rays, DVDs, and CDs loaded onto your Emby server
  7. Backup server — I use a CrashPlan Business subscription to backup my FreeNAS server to the cloud (one of the main reasons I use a NAS as this would be less efficient with a SAN).  BackBlaze B2 is another great option to backup FreeNAS.

There are many more areas than I listed, but I think the above is a good baseline to get started.  Pick one area at a time–my homelab was built over many years–often the case is I will improve an area after a piece of equipment fails or I need to replace it for some other reason–that’s a great time to do research.  If you aren’t sure where to start, pick the area that you enjoy the most.  For areas you have no interest the best thing to do is something else–you’re probably not going to be great at something you don’t enjoy.   Certainly a homelab isn’t going to be a substitute for real-work experience.  But it does provide an environment to learn, experiment and enhance your abilities–and the great thing is since it’s your own lab you can learn things that interest you.

I think that’s the largest benefit of a homelab.  To me it’s a playground.  It’s a place put the love of learning into practice.  It’s a place of freedom.  Nobody else is dictating what you do here.  It’s a place to have fun while enhancing your skill.

Do you see a man skillful in his work?
He will stand before kings;
he will not stand before obscure men.     – Proverbs 22:29 ESV

 

 

The Diet of Worms And Five Solas

Happy 501st Reformation Day! (and Happy Halloween)

In 1521 Luther was summoned to the city of Worms to be tried for heresy.  Expecting an opportunity to debate Luther willingly went.  But instead found no discussion would take place.

He was asked to answer two questions:

First question: With his writings on a table in front of him he was asked if the books were his.  He answered in a barely audible voice, “The books are all mine, and I have written more.”

Second question: “Do you defend them all, or do you care to reject a part?”

Luther asked for time to think about it.

He was given one night.

Diet of Worms

The next day, Luther returned and was asked again, “Do you recant?”  Luther tried to explain his views but was quickly shutdown.  The official asked him to answer the question simply.  Luther answered:

Since then your serene majesty and your lordships seek a simple answer, I will give it in this manner, not embellished: Unless I am convinced by the testimony of the Scriptures or by clear reason, for I do not trust either in the Pope or in councils alone, since it is well known that they have often erred and contradict themselves, I am bound to the Scriptures I have quoted and my conscience is captive to the Word of God.  I cannot and I will not retract anything, since it is neither safe nor right to go against conscience.  I cannot do otherwise, here I stand.  May God help me, Amen.

Luther threw up his arms making the gesture of a victorious knight, slipped out and went into hiding.   Obviously, Luther was pronounced a heretic.  The Reformation had started.

The central issue of the Reformation is this question.  How does a person get right with God?

The answer to this question can be summarized into the 5 principles below:

Sola Scriptura – Scripture Alone

Scripture claims to be inspired by God.  It is without error, authoritative, and sufficient.

All Scripture is breathed out by God and profitable for teaching, for reproof, for correction, and for training in righteousness, that the man of God may be complete, equipped for every good work. – 2 Timothy 3:16-17 ESV

There is the ever changing doctrines, beliefs, and ideas of man.  We have various political views, ideologies, philosophies, worldviews, and thoughts that change drastically every few hundred years.   Then there is Scripture.  It stands unchanging.  It has been studied, scrutinized, and examined more carefully than any other publication in history–for 1600 years.  It is God’s Word.  Many claim the Bible is full or errors.  Many claim it has been translated multiple times and the contents changed like the telephone game.  However, the method of Scripture transmission through the ages has a consistency check to detect and correct changes.

Imagine, you create a file.  Then you make 10 copies of that file onto floppy disks, CDs, or USB drives, and give it to your friends, they each make 10 copies and give it to their friends, and it gets distributed throughout the world.  1000 years into the future you will find hundreds, maybe thousands of copies of the data.  They might be imperfect–you may have partial discs, bitrot, on some of the copies someone may have lost power in the middle of a copy and it corrupted or skipped data, some people made malicious changes, not all the copies agree with each other.  But even without the original you can know the contents of the original file.  As long as you have enough copies and a history of those copies you can reconstruct the original.  Even if mistakes are made in all of the first 10 copies of the original (as long as the same mistake is not made in all 10) you can reproduce the original–like a RAIDZ array can reconstruct damaged or missing data by using the parity data from the other drives.

P52Scripture was transmitted through the ages in such a manner that it could not be changed from the original undetected.  Even though some copies of scripture were damaged, missing parts, had a copy mistake, etc.  Most discrepancies are minor and don’t change the meaning of the text or change any doctrine.  With vast geographic distances and no modern means of communication there would be no way to collude a malicious change.  When we do see a difference between various families of manuscripts, we know what changed.  That means we know what the words of God actually are.

I’ve read on random internet forums that the Bible is a translation of a translation of a translation and so much has been lost in all those translations.  That’s not true at all.  Most English Bibles are a direct translation into English made by a team of scholars who have access to the available copies of the manuscripts in the original languages from various regions.  The Bible is translated directly from the original languages written in Hebrew, Greek, and Aramaic into English so you can read it.  The English Bible you have today, if made by a trustworthy translator, is reliable.  God promises His word will stand forever:

“The grass withers, the flower fades,
but the word of our God will stand forever.”  — Isaiah 40:8 ESV

Sola Gratia – Grace Alone

Man cannot save himself.  God saves sinners.  We are absolutely incapable of pleasing God.  We have all sinned against a Holy God.  When God created man, He created us in His image–we are held up to God’s standard of righteousness and because of sin we all fall short.  Ask yourself these questions: have you ever said God’s name in vain, have you ever hated someone, have you ever stolen, have you lied, have you ever looked at a woman (or man) with lust?  Answer those questions honestly.  Have you done any of those?  “Yeah, but everyone else does it…” doesn’t matter about everyone else.  What about you?  If you stood before God today, and He was judging you by the Ten Commandments, would He say you are Guilty or Not Guilty?   I don’t get a pass on this either, I am as guilty of breaking God’s law as anyone.

There is nothing you can do to get a not guilty verdict.  You may decide that from this day forward you’re going to do good, you’ll stop sinning, love others, give away your money, feed the poor, go to church, etc.  It doesn’t matter.  God considers the good things you do like a polluted garment (Isaiah 64:6).  Your “goodness” is worthless to God.  You will still go to Hell.

As it is written:
None is righteous, no, not one;
no one understands;
no one seeks for God.  — Romans 3:10 ESV

One of the most important distinctions between false religions and Christianity is this:  false religions teach that you can earn merit.  Essentially they teach you must do good things to get points from God to earn your salvation–as if life is some sort of video game.  Doing good to get to heaven is probably one of the largest misconceptions about Christianity.  The Bible teaches the exact opposite:  You cannot do anything to save yourself from Hell.   But, there is God’s Grace.  God’s unmerited favor towards mankind:

For by grace you have been saved through faith. And this is not your own doing; it is the gift of God, not a result of works, so that no one may boast. —  Ephesians 2:8-9 ESV

Sola Fide – Faith Alone

By faith, and faith alone are we justified (made holy) before God.  When Christ died on the cross, He paid the penalty for our sins.

Justification is about our legal position towards God.  The means by which we are justified is through Jesus’s death–when God looks at us, He sees the perfect life of Jesus.  He sees that Christ has paid the price for our sins–Jesus’s righteousness is imputed to our account–which means through no power of our own we are declared righteous before God–despite being imperfect.  This is what our faith is in–and by faith we do not mean a blind faith, but confident belief in what God has promised in Scripture, and God’s ability and integrity to keep those promises.  That is what we must have faith in.  And faith alone in Jesus Christ is the means by which we are justified so that we can have eternal life.

Therefore, since we have been justified by faith, we have peace with God through our Lord Jesus Christ. — Romans 5:1 ESV

Solus Christus – Christ Alone

Jesus is the God-Man.  There is none like Him.  He is eternal, existing outside time and creation with God the Father, He is the Creator of the Universe, He is God Himself, the second person of the Trinity.  ~2000 years ago He stepped into His own Creation and became a man and now possesses two natures: God and Man.  Jesus lived the perfect life–the righteous life we could not live as a man.  And took the punishment we deserve for our sins: death.  Christ is the only means of salvation.   Christ’s humility cannot be overstated–we are talking about God the Son, who has always pre-existed with God the Father in the presence of God’s glory coming to earth to be humiliated and die–if Jesus had come to earth as King and ruled the world, lived in the best place and had the finest the world had to offer with servants at His side to take care of any need or task, it would be an insult compared to the glory He deserves–yet he came to die.

A common misconception is Jesus’s death was some work of Satan and something outside of God’s control.  No.  God sent Jesus to earth specifically to die for our sins to offer us salvation from our sins.

Jesus died on the cross, was buried, and then rose again.  There is no other religion, no other method to obtain salvation, other than Jesus to become right with God:

And there is salvation in no one else, for there is no other name under heaven given among men by which we must be saved. — Acts 4:12 ESV

Soli Deo Gloria – To The Glory of God Alone

The salvation of those who love God is an amazing, beautiful work of God.  It is not something we can claim even partial credit for.

I have heard the analogy of salvation being like God throwing a life saver out into the ocean to one screaming for help and all one has to do is reach out and grab it to be saved.  That is exactly what the work of salvation is not like.

We are not drowning begging for help.  Rather, we are all enemies of God, storming His castle, with no inclination of good in us, and God chooses to save us through the Gospel–which is the good news that Jesus Christ died on the cross to take the penalty for your sins.  By believing, by having faith in Him, you will have eternal life.  There is nothing for us to do in regards to our salvation, because the glory is God’s, and God’s alone.

I am the Lord; that is my name;
    my glory I give to no other. —  Isaiah 42:8a ESV

For from him and through him and to him are all things.
To him be glory forever. Amen.  — Romans 11:36 ESV

Jack-O-Lantern