Types and Programming Languages Part 11 – Principles of good debugging

This is the eleventh part of the Types and Programming Languages series. For your convenience you can find other parts in the table of contents in Part 1 — Do not return in finally

Actually, this could be called “principles of good thinking” as they work in a general case. They refer to how arguments should be proposed, considered, and rejected, which is a bread and butter of any debugging scenario.

Objective, not subjective

You need to think objectively about arguments, not subjectively. Objectively means thinking in general, not personally, not rejecting the argument because of personal beliefs, knowledge, or experience. We can think of four levels of dissecting a statement. Let’s take “some people like potato soup” as an example.

A. Taking it personally

If you immediately think to yourself “I don’t like potato soup” then you make it wrong. It doesn’t matter what your preference is as it doesn’t affect the validity or usability of the argument. The only situation when you can provide your opinion is when you actually prove the argument with it (like by saying “I do like the potato soup”).

B. Comparing with personal experience

Similar to the previous one but targeted at group of people. When you think to yourself “I don’t know anyone who likes potato soup” then you make this mistake. It doesn’t matter if you know someone as your sample is very small and is not representative. Again, you could use this by saying “I know someone who likes potato soup” and this would back the argument, but that’s the only case when your personal experience matters.

C. Taking it in general

This is what we should be doing. The argument is intuitively valid and it doesn’t matter if we (or people we know) back it or not. If we want to disregard the argument then we need to measure it in some way.

D. Judging by knowledge, experience, science, logic etc

This is another level of reasoning about the argument which is much harder and often brittle. If the argument is “some people like stone soup” then we can’t reject it based on A (our preference doesn’t matter), B (preference our of people we know doesn’t matter), C (there can be someone out there in the world who likes the stone soup) but we can confront it with facts, logic, science etc. Why would people like stone soup? How do they eat stones? Do they grind them? How would they prepare such a soup 100 years ago? 1000 years ago? 10 000 years ago? Would it be nutritious? Would it be tasty? Would it be edible at all or healthy?

Based on this reasoning, we can try to argue that it’s super improbable that someone actually likes stone soup. And even if there is such a person, then it’s so rare that the argument has no practical meaning and can be ignored as it most likely doesn’t affect the outcome of the reasoning. Obviously, that’s not a proof and can’t be used as such, but it actually is a level we want to achieve to speed up our reasoning and just focus on the important things.

Adding assumptions

We need to be able to add assumptions to our reasoning and don’t judge them. They are assumptions, when you add them then you take them as dogmas and don’t question.

Imagine that someone tells you “let’s assume that you lose your job tomorrow – what do you do?”. If you think to yourself “that’s a stupid assumption, my job is solid, nothing wrong happens, I feel safe, nobody was fired recently – why would I lose my job?” then you make this mistake. It’s an assumption and so shouldn’t be judged. You just take it for granted and move on.

And if you think to yourself “why would I lose my job? Are you suggesting that you know something I don’t” then you actually make the mistake A from the previous section as you take the argument personally, so that’s even worse.

There are two important things we should be aware of when adding assumptions, though.

First, our brains feel in the presence and cannot understand that something happened in the past or will happen in the future. So if you make an assumption “let’s say that someone offended you and you feel sad” then your brain actuallys start feeling sad immediately. It may not be a conscious feeling, it may have a little to no effect on you, but it may affect you in a longer run. I refer you to psychology, but just want to say that you should be careful which assumptions you want to make for the sake of reasoning as they do affect us. We’re all humans after all.

Second, just like in D in the previous section. When reasoning, we should be able to understand that some assumptions are just “less helpful” when trying to achieve a specific goal (like debugging a very specific problem). It’s important that we start with the most probable assumptions and only then move on to the less probable ones (and ultimately wave a dead chicken).

Removing assumptions

Sounds like the same thing as in the previous section, but it’s not. Removing assumptions may be very hard because we’re so used to them that we don’t understand their consequences and so can’t remove them effectively. Let’s consider “what if we don’t exist” – that’s removing the assumption about our existence. Can you do that? If your first thought is “we obviously exist, because we, erm, exist!” then you just made this mistake. It may be hard to remove assumptions with all their consequences as we may not be aware of all of them.

Understanding that things don’t “work just like that” and there is some principle behind it

If you assume something “just works” magically with no principle then you’re most likely wrong, especially when we’re talking computers. They are all bits and bytes, there is no magic, no “coincidence”, no emerging behavior. All things work because of very simple computation rules. Obviously, there are things which we just can’t measure or we can safely ignore for most of the time (like cosmic ray), but they do happen and do explain the behavior. There is always a risk of a bit flip which ECC doesn’t fix, or an invalid ethernet packet being delivered despite a couple of checksums in the place. However, things do not work “just like that”.

Rules should work both in general and in specific scenarios

If rules or principles you come up with work only for a very specific scenario then they’re most likely invalid or incomplete. This is not how most of the world is constructed and nearly all scientists believe that (and look for a theory of everything). Obviously, the “general vs specific” difference is important and should be included in the theorems, but it’s not about adding a very specific “if” somewhere in the reasoning, but about finding all the factors which change either the micro or the macro scale.

One way of applying this principle is by going to the extremes. If your reasoning is valid then it should hold even in an ideal world, ideal scenario, when all other factors are controlled and set perfectly. If your reasoning fails to work in such a scenario, then it is most likely flawed in some way.

That’s actually a pretty powerful technique of reasoning. First, consider some extreme scenario (=remove all “real” assumptions and add useful assumptions about all things being perfect) which is obviously nonrealistic and impossible in this world. Find the rules and principles. Next, remove assumptions about perfection one by one and add assumptions about real life context. If your principles still work then they’re more probable than principles which work only in “realistic” scenarios, but fail in idealized ones.

Case of accepting and rejecting arguments

There is one more thing to understand, even though it’s much more of a philosophy than of a computer science. Each reasoning is based on arguments (proofs, assumptions, lemmas, definitions…) which all can be rejected. For some of them you just can’t tell if “they’re correct”. You can criticize or provide counterarguments but at some points it’s a matter of preferences. For instance, there are tons of proofs that God exists or if P equals to NP but they are not widely accepted. That’s not a typical thing when debugging, but it’s super important when reasoning about fundamental things. Ultimately, you can argue that nothing exists but your mind or that the world sprang into existence 5 minutes ago, but you most likely won’t be able to prove it.