Articles in the musings category

With the announced demise of Pocket by Mozilla I needed to migrate all of my saved articles to 'something else' by the end of the month. I've actually tried to migrate from Pocket a few times over the years. I landed on Instapaper for a while, but it never really clicked for me. I tried a service called Devmarks that Adam G Hill runs, and I really liked it, but for whatever reason I stopped using it. I had also previously tried Raindrop.io ... and I'm not really sure what drove me away from it, but it didn't stick for me at the time.

Since I didn't have a choice about Pocket I did a bit of purusing my options, and finally landed on Raindrop.io again. The process of migration is pretty painless. I just export out the links from Pocket and then import them into Raindrop. No fuss ... no muss. Raindrop even checks for duplicates and allows you to not import them!

So, I imported everything (all 11,500+ articles!) and started to incorporate Raindrop into my workflow. This basically just means saving things to Raindrop instead of pocket, and then checking Raindrop instead of Pocket every week to make sure I'm all caught up on my articles to read.

Over the last weekend I was looking at how all of the imported items in Raindrop were put into the 'archive' collection and decided that I could probably do something about putting them into proper collections.

With the help of Claude Code, I was able to put them into better collections. There were some stragglers and I decided that I could categorize them on my own (there were less than 100).

I started going through these last ones I kept coming across articles for iOS7, or an app that I think I liked in 2015 but isn't on the App store anymore. I came across this article (which I also tooted about on Mastadon) from September 4, 2014 with the title What the Internet of Things Will Look Like in 2025 (Infographic). It's wildly naive, but a fun read nonetheless.

Needless to say it was the only gem in the 100 articles that I went through. I had so many saved articles that aren't 'Evergreen'. I then started looking at some of the articles that had been categorized and came across stuff for Django 1.11, Python 3.8, and other older stuff.

These were great articles when I read them, but I don't know that I need them now. In fact, when I looked at my general workflow for using any read-it-later service, I essentially save it to read later. If it's sitting in my read-it-later service for more than 4 weeks I'll either delete or just archive it.

So really, unless I'm planning on doing something with these articles, I'm not sure that I need to keep them. And that's when it hit me ... I can just delete them. All of them. I don't need to keep them. If they are truly impactful, I can write up something about them in Obsidian. If I really think someone else will get something out of my reaction I can write it up and post it. But, if I'm being honest with myself, this is just digital clutter that isn't "sparking" any joy for me.

So, just like that, I went from having 11,000+ links to having 0. And I have no ragrets.

I'm sure there's some deeper story here about physical things and just letting them go as well, and maybe I'll be able to apply that to my non-digital life, but for now, I'm just going to revel in the fact that I was able to offload this thing and just not ... care? Be sad? I'm not sure what the correct term would be here.

Regardless, it was a good exercise to have gone through, and I'm glad I did.

Posted by ryan on Thu 19 June 2025

I was added to a work email that was requesting a not-so-small new project that was going to need to be completed. The problem that needed to be solved was a bit squishy, but it had been well thought out, and it had an importance to it that was easy to see.

There was still some workflows and data that needed to be reviewed, but overall it was on a good path to having a real project feel to it.

One question still outstanding is, what platform will this project be implemented on? In our EHR, or on a separate web app?

During my weekly project review meeting with the Web Development team I let them know about the potential for this new project and that it would likely need to take priority over one of our current projects. The start is still a couple of weeks away so we have time to plan for it (as much as we can anyway). We looked at the project board and determined a ranking of the current projects. We decided on the project that would likely get bumped if this new one ends up with the web developers. And just like that we had a contingency plan for how to plan for this project given our current constraints.

Now, this project may never make its way to the web development team, but having that conversation with the manager, and then during our standup today, to let the team know that this might be something that will need to be worked on by them felt right. No surprises in a few weeks. No randomness about what projects we'll be working on ... just a bit of planning to prepare for something that might never come.

Eisenhower said, "Plans are nothing, planning is everything."

The team appreciated being in the loop about a potential project and being able to align expectations moving forward. I felt grateful that this was brought to my attention well before it was submitted as a request. The requester now has a bit more information on who to speak with internally, and it really felt like we were working together to solve a problem in a very professional way.

I wish all projects started like this. It would make life way easier and not so much like this

Posted by ryan on Sun 15 June 2025

The 2024-25 season for the Coachella Valley Firebirds ended on May 9th with a 2-0 loss to the Abbotsford Canucks. Overall, that series saw the Firebirds score

• one goal in Game 1
• one goal in Game 2
• five goals in Game 3
• no goals in Game 4

This isn't surprising given exactly how young the Firebirds were this season, but it was disappointing.

Coach Laxdal talked a lot about how young the team was and how on any given night we would have anywhere from seven to nine rookies that were in the starting lineup. And in a team of 24, that's a pretty big portion of guys out there who are very young.

That being said the disappointment is palpable the this is the earliest that the Firebirds have ever exited the postseason. Granted this is only their third year but we are typically used to seeing hockey for another seven weeks. When put into that perspective, it is really disappointing.

Still, I think there were some really bright spots from this year, including Leyton Roed, Jani Nyman, Nikke Kokko, Ryan Winterton, and Ty Nelson.

At the start of the season, I did indicate to a friend of mine (who also has season tickets) that I had pretty low expectations for the Firebirds and may have even indicated I wasn't sure that they would make the playoffs. The Pacific Division has 10 teams and 7 of them make the playoffs. I may have been a bit too pesimisitic in that analysis.

During the first round the Firebirds swept the Wrangerls 2-0. This is great, but they did manage to blow a 3-0 lead in game 1. The were able to win that game, but it took two plus Overtime periods (it ended a few minutes into the third OT).

Game two of that series did see the Firebirds win 2-0 with Nikke Kokko getting his first professional AHL shutout, which was great . But it's also a bummer that it took until the 74th game of the season for him to get his first shutout of the season. ¹

In six games, the Firebirds were 3-3. They scored four goals, two goals, one goal, five goals, one goal, and no goals. They were 0-17 on the power play, and they gave up two, count them two, 3 goal leads.

Needless to say, this was just a hard set of games to watch. The season was hard to watch as a fan. The Firebirds would find ways to lose games. In previous seasons these were the games that they would find some way to win!

There was an article in the Desert Sun that spoke about how proud Coach Laxdal was of the players and how much effort that they gave. And I agree, they did give a lot of effort and he spoke about how young they are.

And again, they are young, and missed their captain Max McCormick for basically two thirds of the season. But they did have some veteran players out there Mitchell Stephens, Brandon Biro, Cale Fleury and Gustav Olofsson. Unfortunately it was just too much to try and overcome.

One of the things that Coach Laxdal also commented on was exactly how much younger next year's team might be. And so while I am again very excited about watching hockey in six months, which is just so long away. I am lowering my expectations for the 25-26 season even lower than they were this year. I'm really hoping we make the playoffs, but won't be surprised if we don't.

And that's going to be okay ... because even bad hockey is still hockey. And I love hockey, and even when they lose, I love watching the Firebirds.

1. During the regular season, there was exactly one shutout by Victor Ostman ↩︎

Posted by ryan on Wed 21 May 2025

The Origin of Systems

When thinking about systems it's easy to think that they have always been there, or been that way. This isn't true of course. The systems that are in place were put there, by people. People that made decisions. Decisions are what I want to focus on here.

In general when making a decision about the implementation of a system you would want to engage with the stakeholders of that system. This of course implies that you can identify at least some of those stakeholders.

But sometimes there aren't any key stakeholders other than regulations, or best practice, or some other nebulous thing that needs to be met. These are the decisions I really want to focus on.

The Illusion of Success

Take a security system for instance. The basic tenets of the security system are that it keeps 'something' safe. If the thing to be kept safe is still safe after the implementation of the security system then the people that implemented the system can claim success. They can look at the evidence that since the security system was put into place the thing has been kept safe.

Of course, it's entirely possible that the thing was never in danger, and that the previous system was doing just fine. In fact, it could be that the security system is actually making it harder to keep the thing safe. It's just harder to see because all you're looking at are potentially meaningless metrics like. Questions like is the thing safe after implementation of the security system don't take into account if the thing was 'unsafe' before? This can lead you to think that the new security system must be responsible for the safety of the thing.

Something else that can be happening is that the security system has caused the people responsible for keeping the thing safe to work more hours, hire more people,who are oftentimes keeping the security system running.

Questioning Purpose

The more we look into a system like this, the more we might ask, "Why is it there?"

There can be a couple of reasons, but I'll focus on one in particular. The person ultimately responsible for keeping the thing safe can show with some kind of metrics that the thing is safe with the new security system, whereas they couldn't under the previous system. There weren't any reports or metrics that showed what was going on, which is why the system was implemented in the first place.

OK ... so that's how some systems can be put in place.

User-Hostile Systems

What about systems that are hard to use, or maybe actively hostile to their users? How do those get put into place? I would argue that the reason we see many user hostile systems in place is because they are decided upon not by their users, but by their ability to meet regulations, AND their ability to maintain by a support system. The consideration of the user is secondary, or maybe not even thought about.

Think about any Enterprise software you've ever encountered. Would you say that it was a joy to use? Would you say that onboarding was simple, and that new employees loved to use it? My guess would be no.

Why Bad Systems Persist

So if the users don't like it, why is it in place? Two reasons:

1. It meets some kind of regulation (this could be a government regulation, but it could also be a regulation of a guild, or union, or something else)
2. It's easy to maintain by the support system

For any software that meets these criteria you are likely to have users that don't like the software, because they are always an afterthought. The primary responsibility of the software developers of these types of systems is always the regulators, and the support infrastructure.

The first because they have to keep producing software that is compliant in order to be sold with a specific rating or seal of approval.

The second because if the support team can't easily support it, they're going to find an alternative solution that they can support.

Conclusion

It's a simple decision of maximizing for the people that enforce the rules (regulators) and that make the decisions (support). The users of the software don't matter. At all.

This is why you will see software for widget processing that could benefit from bulk operations, keyboard shortcuts, or being browser agnostic and they just aren't. The only considerations are: Does it meet the regulations? Is it easy to support? If the answers are yes then the users tend to lose out. They don't matter. If the answer is no, then find a competitor that does and move over to them, even if the current system is loved by your users.

Posted by ryan on Mon 31 March 2025

"If you think technology will solve your problems, you don't understand technology and you don't understand your problems"

~ attrib. Laurie Anderson

From a Toot by Jake Rayson

In a previous post, I wrote about how to ask why without sounding like a jerk. This is a slightly related concept (at least in my head).

Sometimes, as technical people, we are asked to solve problems. The more we dig into them, the more we discover that the problem that needs to be solved isn't a technical one but a people one. In many cases, it's just getting two groups to actually talk to one another.

This can be hard and awkward, so people may want to avoid it. Creating a report telling someone they're doing something wrong is way easier. No hurt feelings! However, I've found that the approach tends to create more problems than it solves.

The situation

The situation is a real one, and I'm obfuscating details to help 'protect the innocent'.

At the start of each year, large amounts of new data are needed to be added to a system. The additions are, by their nature, very manual¹ and so the team responsible for them spends much of their time trying to get the data added.

Another team is highly dependent on this new data being added in order to process their widgets². The widgets get loaded into the system and checked to see if the data from team A is complete. If it isn't, then the widget gets flagged. This flag directs the members of Team B to reach out to Team A to get clarification on the state of the data needed to process the widget.

Only, that's not how Team A understands it. While they are furiously trying to update data, there is some basic data that covers roughly 80% of the widget processing data needs that are already available. So, the vast majority of the time, there is no need for Team B to reach out to Team A because the information they need is available in the system to process the widget.

This understanding was either lost or never communicated effectively so Team B would just email Team A with questions about the widget data and then get their answer and move on. This is despite the fact that the information is available in the system for the members of Team B to review!

The leader of Team A asked me if I could 'update a report' to 'remove some of these widgets so Team A could better focus on the work of adding the data'.

I thought that seemed reasonable, so I asked Team B a few questions and then made a bit more discoveries and found out the actual problem, which was that the information needed by Team B was in the system. Team A just needed Team B to do a better job of looking for it and asking questions about the things that were needed instead of everything.

The Solution

I proposed that the leaders from Team A and Team B get together to talk about the issue.

At the meeting, the leader of Team B was horrified to hear what was happening. They had no idea that many emails were going to Team A about questions that the members of Team B should be able to answer on their own.

This is all well and good, but why did it take a tech person to spot this and get the team leadership together to figure it out?

I wish I knew the answer. I think part of the insight I had was the current pipeline of work, how long it was going to create a report, and the need to have the problem solved sooner rather than later didn't line up. At all.

I needed to look for a potential non-technical solution. The other thing that I think happened here is that I wasn't weighed down by any history of interactions between the Teams. I was just trying to gather information. In gathering information I was able to see what the real problem was and that the only solution that made sense was for the two teams to just talk to each other.

The Outcome

During the meeting, Team B committed to retraining staff and helping to make sure that they only reached out when there was an actual question about the data for the widget production. Team A was thrilled with this solution, and now they can focus on getting the data into the system more efficiently and with fewer interruptions. A win-win, all because a tech guy got some non-tech people to talk to one another.

1. yes I would like to automate this, but one step at a time! ↩︎
2. Not actually widgets ↩︎

Posted by ryan on Thu 06 February 2025

This is the fourth part of my How to Watch a Hockey Game Series. You can catch up on previous articles here

Game Outcomes

In many North American sports when reading the standings there are typically just Wins (W), and Losses (L).¹

Hockey is a bit different. When you look at the standings for Hockey you'll see 4 headers:

• W: Wins
• L: Losses
• OTL: Overtime Losses
• SOL: Shootout Losses

As discussed earlier in this series, if a game is tied at the end of regulation, a five-minute overtime period is played. If either team scores during this Overtime period then the winning team gets a Win, while the losing team gets an Overtime Loss (OTL).

If they're still tied then a Shootout is played. Once a winner is declared in the Shootout they get the Win, while the losing team gets a Shootout Loss.

Because of this, values are assigned to each type of outcome:

This might best be shown with a concrete example.

A Concrete Example

Let's say that the Coachella Valley Firebirds have played 39 games so far. They have won 21 games and lost 13 games. They've also played in 5 games that went into overtime and lost. Their overtime losses are one (1) in the Overtime period and 4 in Shootouts. Their record would look like this:

Coachella Valley Firebirds: 21-13-1-4

Points Calculation:

• Wins: 21 × 2 = 42 points
• OTL: 1 × 1 = 1 point
• SOL: 4 × 1 = 4 points

Total: 42 + 1 + 4 = 47 points

The Firebirds play in the Pacific Division of the Western Conference, and the standings might look like this:

Legend: - GP: Games Played - W: Wins - L: Losses - OTL: Overtime Losses - SOL: Shootout Losses - PTS: Points - PCT: Points Percentage

Winning Percent

There are 2 things to look at in the standings: (1) Total Points, and (2) Winning Percent.

The Total Points we've already spoken about so let's review winning percent.

The winning percent is calculated as the Total Points the team has divided by the total possible points that they could have gotten. The total possible points are calculated as the Games Played x 2 (that is, what are the total number of points that they would have if they won every game they played).

That is

    Winning Percent = Total Points ÷ (Games Played × 2)

For example in the table above, we see that the PCT column for the Firebirds is 0.603. This is calculated by the Points (47) divided by GP x 2 (39 x 2 = 78), that is 47 / 78 = 0.603.

The winning percent allows ranking intra-season when teams haven't played the same number of games. After all games have been played, the rankings are determined by the total number of points a team has.²

Conclusion

You should now be able to parse the standings in a Hockey League and be able to tell how well (or poorly) your team is doing.

This is the end of my series (for now). If there are any other burning questions you have about hockey, reach out to me on Mastodon.

1. Football also has Ties (T) but they are exceedingly rare and are only ever displayed when the first Tie of the season occurs ↩︎
2. Depending on the league there are tiebreakers, but that's outside the scope of this article ↩︎

Posted by ryan on Mon 03 February 2025

In a previous post of this series I laid out some basic rules of hockey. In this post I'll hopefully provide some tips on what to watch during your first few hockey games.

What should I 'watch' though?

This is a tough question and depends on if you're watching on TV or in person.

On TV

If you're watching on TV you're limited by whatever the camera and director are showing you. Hopefully they're pretty good at what they do and they'll help to show you what is interesting. You'll also have the benefit of replays. ¹

Watching the action on TV will be your best bet. The commentators will do a reasonable job of explaining the play. For some of the best NHL broadcasts you'll want to watch a Canadian feed. This might not be an option depending on where you live, but in general, watching a Canadian feed of a Canadian team will be really helpful.

If, for whatever reason, you're watching an AHL game² the best broadcasts to watch, in my opinion, are the Lehigh Valley Phantoms called by Bob Rotruck and Cleveland Monsters called by Tony Brown. Each of these is a single broadcaster doing both the color commentary and the play-by-play ... and they honestly get so excited it's hard to NOT get excited with them.

In Person

For your first in person game, just try and follow the puck as best you can. If for whatever reason you can't do that, pick a spot on the ice to concentrate on, preferably near one of the goalies. Which one? The goalie of the team you're not rooting for is a good choice! Then you can just kind of watch the action there.

Keeping in mind the rules start by focusing on just one rule - either icing or offside - for an entire period. Once you feel comfortable recognizing that rule during gameplay, switch your attention to watching for the other rule in the next period. For example, if you spent the first period watching for icing, spend the next period looking for offside plays.

Hopefully after a full game you're able to see them when icing or offside happen. If not, it just means you'll need to come back and try again 😁.

What not to worry about

Hockey is a fast paced game. No, like really fast. Don't worry too much about anything other than watching for the puck, if you can, and trying to pick up icing and offside. You'll see other stoppages in play when a penalty is called. The refs will make hand gestures to indicate the call on the ice and someone will be sent to the box.

Don't worry about whether or not a fight will break out. They don't always, and if they do, each player will be assessed a major penalty and will spend 5+ minutes in the penalty box.

Don't worry too much about learning the positions. The goalie is an obvious one (that's the person with all of the pads, the bigger stick, and the giant, well painted mask in front of the net), but trying to distinguish between a defender and a center ... like just don't worry about it!

Conclusion

Hockey is an amazing sport to watch, whether in person or on TV. It can take a little bit of time to get used to the fast pace, but hopefully this series has given you some tips to enjoy it and understand what's going on.

1. and refreshments that are much less expensive! ↩︎
2. home of my beloved Coachella Valley Firebirds ↩︎

Posted by ryan on Fri 31 January 2025

Game Structure

Hockey has some stuff in common with live theater. No ... really! 😁

They both have dressing rooms and they both have intermission ... but that is probably where the similarities end.

Each hockey game is split into three 20 minute periods. There is an intermission between each period that lasts 18 minutes. During the intermission the players go back to the dressing room to regroup and chat about the previous period a strategize for the upcoming period.

Out in the arena there are chances for you to get overpriced refreshments, stand in long lines to use the facilities, or just stay in your seat and watch the silly intermission games.

Some examples I've seen of silly intermission games are Fuego Pong (like quarters, but with soccer balls and large 5 gallon buckets), ice bowling where a player is put into a giant slingshot on the ice and hudled towards inflatable bowling pins, and the dress up game.

It's also during this time that the ice is resurfaced by a Zamboni to make it nice and clean for the next period.

If at the end of the third period the game is tied then you're in luck because you get free hockey, also known as Overtime. One thing to keep in mind is that the overtime rules during a regular season game are different than a postseason game.

Regular Season Overtime Rules

At the end of the third period there is a 1 minute 'intermission' and then a 5 minute overtime period starts. The overtime period will feature 3 skaters from each team as well as their goalie.

If a penalty occurs in Overtime (or is carried over from the third period) the period starts with four players on the power play team and 3 on the short handed team.¹

Each team tries to score a goal first. If they do, then they win in overtime. If, at the end of 5 minutes of play, the score is still tied then a shootout happens.

In the shootout each team has 3 chances to score a penalty shot. Essentially a skater from each team has the opportunity to try and score a goal with only the goalie trying to prevent it. If at the end of the three rounds we're still tied, we keep sending out skaters to try and get that penalty shot until one team is victorious. The record for most rounds of a shoot out is 20 rounds in the NHL, and 16 rounds in the AHL.

Postseason Overtime Rules

Postseason overtime rules are a bit different. Basically you just keep adding 20 minute periods until someone scores. Once a team scores they have won that game. The longest overtime in NHL Postseason history went into the 6th overtime and was played in 1936 between the Detroit Red Wings and the Montreal Maroons. The longest AHL overtime was between the Charlotte Checkers and the Lehigh Valley Phantoms which went into a 5th overtime period. This game started at 7:03 pm local and didn't finish until almost 3:00 am local the next day!

In general most hockey games don't get past the first OT period. From The 2006 playoffs through to the 2024 playoffs there have only been 52 games that have gone into a second overtime period (out of 1312).

OK, you've got a few basics 'under your belt'. In the next part I'll try and answer the question, 'What should I watch?'.

1. essentially it would be a short Overtime period and probably pretty boring ↩︎

Posted by ryan on Wed 29 January 2025

I've written a few times before about hockey. I love watching my local sports puck team¹ and really wish more people watched it. So, I'm going to write a beginners guide to watching hockey so that you too, dear reader, can become an avid fan.

Hockey is a pretty fast paced game at the professional level. In the 90s Fox Sports had broadcast rights to hockey in the US and to help its viewers they had a glowing halo on the puck called FoxTrax which allowed fans to more easily find it. This practice was discontinued at some point, and I honestly think it was one of the better innovations that Fox Sports did and really wish that it would make a come back.

The Rules

As a beginner hockey observer there's only three rules that you really need to know to be able to follow the game.

1. Offside²
2. Icing
3. Power Play / Penalty Kill

The Set up

The ice rink can be broken into 3 sections from the perspective of 1 team. Let's assume we have two teams, A and B. Let's root for team A.

1. The Defending zone - This is where team A's Goal is located. It starts right behind team A's goal and goes to the right toward the blue line
2. Neutral Zone - This is the center of the ice between the two blue lines; it also contains a red line that is called 'Center Ice'
3. The Attacking Zone - This is where team A are trying to score. It starts at the OTHER blue line and goes back behind Team B's goal

Offside

Offside is defined as ... actually that's not important. What is important to understand is that a player on the offense cannot enter their Attacking zone before the puck does. If they do, then that player is called Offside. When an Offside happens a face off takes place outside of the Attacking zone (i.e. in the Neutral Zone) where each team will try and gain control of the puck.

Icing

Icing, or icing the puck, is when a player in their half of the ice and shots the puck down the ice towards their Attacking zone and it is NOT touched by anyone before it passes the face off circles in the Attacking zone. When an icing occurs the puck is returned to the defending zone for a face off³. When an icing occurs the team that the icing is called on have to keep all of their players on the ice, that is, they can not send in any substitutions.

Power Play / Penalty Kill

The two rules above, when broken, result in a stoppage of play and a new face off for each team to try to gain control of the puck. Other rules, when broken, will result in a penalty⁴ which sees one, or more, players sent to the Penalty Box⁵. Penalties can either be minor, which result in a two minute penalty, or major, which typically result in a 5 minute penalty⁶.

When a team is on the Power Play they will have 1 or more extra skaters than the other team. The other team's 'missing' players will be in the Penalty Box. The Power Play team, with the advantage, will remain on that advantage until either they score OR the penalty expires. If a team scores while on the Power Play, they are said to have scored a Power Play Goal.

The team that has penalized players is said to be on the Penalty Kill. They are trying to 'kill' the advantage that the Power Play brings to the other team. If the team on the Penalty Kill scores a goal, it is called a Short-handed goal ... because they were short a person, i.e. short handed, when the goal was scored. In the National Hockey League (NHL), American Hockey League (AHL), and most other leagues when a short handed goal is scored the Penalty keeps going until time is over OR a goal is scored by the team on the Power Play. The Professional Women's Hockey Leagure (PWHL) has a rule (which I think is genius) which states that IF a team scores a short handed goal, the Power Play is over.⁷

In the next post I'll talk a bit more about game play.

1. The Coachella Valley Firebirds ↩︎
2. in hockey it is not pluralized like in American Football ... even though in American Football it's not pluralized either! ↩︎
3. This does NOT apply when your team is on a Penalty Kill ↩︎
4. I'll talk more about various penalties in future a post ↩︎
5. it's a small room where players are sent to think about what they did ↩︎
6. There are a few caveats here about game misconduct, but they're not important for an introductory primer ↩︎
7. Now, there are lots of Nuances to the PP/PK write up above, but you don't need to understand them initially to enjoy hockey. ↩︎

Posted by ryan on Mon 27 January 2025

The Roman Colosseum

After the fall of the Western Roman Empire in 497 CE the Colosseum fell into disrepair. Rightfully so! Who can worry about keeping up a giant megalith made by people centuries ago while you're just trying to figure out where your next meal may come from, or the ranging hordes of barbarians showing up and taking the food you did find!

However, during the medieval period, while Rome's population declined dramatically and many ancient structures fell into disrepair or were repurposed, the Colosseum remained a prominent landmark. There are stories that as the centuries progressed, the inhabitants of Rome forgot who built it. While some fantastical legends did develop around it, the basic historical facts of its construction by the Flavian emperors and its original purpose remained part of common knowledge among educated Romans. For the non-educated Roman's there were lots of misconceptions about the colosseum.

The non-education Romans would have created stories¹ about the large building. It was haunted. It was used for pagan rituals and no good Christian would go in. Folklore would rise up around it. As many of us have seen or experienced, in the absence of information, people will make it up.²

The Story of the Legacy System

OK, but why is this important from a technology perspective?

Imagine if you will a large system, built 10 years ago, by a group of developers, that have all left the organization.

No one left knows how it works, or how to make changes to it. Most people don't even really know WHY it's there in the first place.

There isn't any documentation that can be referred to. Either because it wasn't ever created OR it was destroyed by Barbarians, I mean well meaning IT processes that 'clean up' unused files.

So what happens? The people remaining create stories about the system. Stories like the long timer 'Bob' that once caused the entire system to Crash and then an old copy backup copy had to be restored, and months worth of work was lost.

No one ever saw Bob after that. Now we're all afraid to touch any part of it. We mostly leave it alone, and it leaves us alone.

There are stories about another gray beard that actually built the system, but everyone assumes these are just fairy tales.

The stories tell of this Gray Beard busting out the entire system in a weekend, using nothing but a pin to move the electrons into the proper places to get all of the logic to work as expected.

Of course, no one really believes that story, but it encourages people to never want to have to make and changes to it.

The problem here is that it's running on a server with an OS that hasn't been supported for 7 years and there is security mandate to upgrade 'everything' to be on current software

No one wants to be in charge of this project, but someone is going to have to be in charge of it.

What do you do?

The story above isn't real, at least not for me. But it could be.

How many times have you gotten to a system that is old, no one around has any idea how it was built and people mostly just avoid it? Probably more than once.

But how can we avoid this fate? Do we just keep the old timers on until they (or the system) die?

There are options, and they are some of the easiest things to do, but many people don't like to do them.

What is the answer?

Documentation

Documentation. No really, Documentation. Just write it down. For a new project especially. For an old project? Most definitely.

For new projects it's best to just get into the habit of writing good docs³ as you go. If that's doc strings in a method, or a full fledged Knowledge Management System using a documentation framework like diataxis, then so be it.

But write it down. Write down the why's whenever you can. Use something like an Architectural Decision Dsocument to understand WHY you made a technical decision you made. Maybe it's not the best decision, but it's the best decision given a set of constraints.

For existing projects, it can be more challenging. It's possible that NO ONE that created the system is at the organization. It could be that NO ONE that asked for the system to be created is at the organization.

This leads to a bunch of problems to try to solve, but the journey of 1000 miles starts with a single step.

How do you solve it?

Use the helpful Awareness-Understanding Matrix⁴

That is,

• Known Knowns: Things we are aware of and understand
• Known Unknowns: Things we are aware of but don't understand
• Unknown Knowns: Things we are not aware of but do understand or know implicitly
• Unknown Unknowns: Things we are neither aware of nor understand

The Known Knowns may be very small, but it won't be empty.

The Unknown Unknowns might (will probably) be the largest.

The lack of knowledge here represents Risk⁵. Risk to your team, or to your organization. This Risk needs to be handled as much as possible.

Looking at a system with the Awareness-Understanding Matrix can help to risk it properly. Once you've properly risked the system, then you can start writing documentation.

The documentation can take the form of Architectural Review of System X (DRAFT)

The system does these things

1. Thing 1
2. Thing 2
3. Many other things that are still unknown

Sometimes just the act of writing these things out will help you in finding out what you know and what you don't know.

If you're using a documentation framework like diataxis for this, you will want to keep your documentation parts separated (How To, Tutorials, Reference, Explanation). You may start righting a Reference article on the system and realize that you also need to have some, yet to be discovered, Explanation. The issue is that the Explanation still needs to be researched and written.

That's OK! One strategy I've encouraged, and use, is if I'm writing a Reference Article and need to link to a yet to be written Explanation article, is that I'll simply create the yet to be written Explanation article and tag it with Explanation and Stub. This frees me to come back to it later and fill in the details.

The other thing that will need to be done is to figure out who uses the system. Sometimes that's super easy, and sometimes, it's not.

Once you're able to determine who uses the system, you can talk with them about the system and then work to fill in the gaps from above.

Occasionally, you find out who everyone thinks is using the system, and discover that actually, it hasn't been used for 5 years because reasons, and they didn't know who to tell.

Now you can just retire the system using a decommissioning process. You have a technology decommissioning process, right? If you don't, it may be time to look into one!

Back to the colosseum

The inhabitants of Rome never got to a spot where none of them knew why it was built, or who built it. Or even why. But what did happen is that the people with the knowledge may have been parts of groups that were marginalized and therefore their knowledge was discounted or ignored. Because the knowledge was a verbal knowledge and not written down. It was, to use a loaded term, tribal knowledge. EVERYONE just knows the obvious thing.

But the thing is ... obvious things are only obvious in the context they were created. It's obvious what Python is. I mean, why would someone use a snake to write code to get a computer to do a thing. EVERYONE knows I'm talking about the programming language Python ... until they don't.

Just write this shit down. Make sure everyone gets into the habit of documenting. Make the documentation public. And if it's not possible to make all of the documentation public, make as much public as possible.

For the parts that aren't public, make sure they are accessible by the people that will need access to it.

Really, documentation is a means to an end. Sometimes you won't need the documentation. You'll know how the thing works, and it has an obvious API or UI and people just "get it". This can lead to people not writing the documentation because we don't need it.

This is kind of like saying, I've used a seatbelt every day for 30 years and I've never needed it. I don't see why I need to wear it any more.

This might be fine until you're in an accident.

Not writing documentation is fine, until it's needed. And that's the worst time to discover that you need it.

Better to have it and not need it, than to need it and not have it.

1. such as it being a temple to the sun ↩︎
2. Brené Brown ↩︎
3. any docs in this case are good docs! ↩︎
4. This is in no way an endoresement of Donald Rumsfeld. He was a horrible person ↩︎
5. Jacob Kaplan-Moss has a great series on Risk ↩︎

Posted by ryan on Tue 21 January 2025