Agile

Table of Contents

1. Agile

«Users do not know what they want, what they know is what they do not want and will scream at you for shipping it»

Users never understand what technology can do for them, so if you rely on what users say they want, they’re only going to say what they already know and they’re going to miss opportunities
– Dave Snowden - Rewilding agile

1.1. Lean

1.3. Agile aims to create fast feedback loops

  • Between changes in code and tests
  • Between new features and user response

1.4. (Software) Engineers are not fungible

Because attention is not fungible

  1. All engineers do not like doing all things
  2. All engineers are not good at doing all things

This creates some type of division of work, like frontend vs backend, which really has to do more with human psychology than with any technical need

  • https://danluu.com/people-matter/

    The roadmap creation and review process maintains the polite fiction that people are interchangeable, but everyone knows this isn’t true and teams that are effective and want to ship on time can’t play along when the rubber hits the road even if they play along with the managers, directors, and VPs, who create roadmaps as if people can be generically abstracted over.

1.5. Exploratory work and glue work

  • https://blog.bonnieeisenman.com/blog/senior-work/ → Exploratory
    Cynefin Framework Straightforward ↔ Simple/Clear, Difficult ↔ Complicated, Exploratory ↔ Complex
    • Straightforward work is work where !it’s obvious to the team what needs to be solved and how it should be done!.
    • Difficult work requires a deeper understanding of the systems involved. It may involve undocumented, unusual, or subtly complicated systems. !The team is probably clear on what needs to be solved, but it won’t be obvious how to solve it!.
    • Exploratory work is a little different. It’s about framing problems and figuring out what should be done. Initially, both the how and the what are unknown
      Exploratory work can be extremely impactful - a good recommendation here might save your team weeks, months, or years of wasted effort. It’s also easy to bullshit, because it’s difficult to measure or assess. In order to do this effectively, you generally need deep context on the systems involved as well as your team’s priorities. !Figuring out how to solve the problem might be out of scope entirely - the goal here is to define what the problem should be!.
      Exploratory work can overlap strongly with glue work, as described by Tanya Reilly.
  • https://noidea.dog/glue →

1.6. One key trick to becoming a wizard developer – Jessitron

1.6.0.1. A Taxonomy of Yak Shaving – Jessitron
  • Black Yak, aka Attack Yak: you must defeat them to proceed.
    When you feel that your task is 80% complete, these yaks form the next 80%.
    Yaks can be summoned voluntarily, and there are also surprises
    • Be conscious in your yak shaving. Don’t shave its eyebrows.
    • Coordinate your attack
      generativity (as opposed to productivity): the difference in what your team does with you, vs what it would get done without you. It’s about the team, not about me. And it’s about the team in the future, not only right now. Not about going faster, but about going smoother: removing obstacles for everyone, present and future us.
  • Imperial Yaks, aka Yak Stack: one obstacle leads to another and another
    These are the yaks that stack on top of each other. They all deserve to be shaved.
    • Pairing helps. I can be down in the weeds of the syntax and the particular bug while my pair still has part of her brain operating at the level of “why are we doing this again?”
    • If your original task is urgent: look for workarounds sooner. Never abandon the yak silently; create an issue if you don’t have time to fix it. Leave enough information to reproduce the bug. (This is work.), or leaving the code a little clearer than I found it
    • If the yaks you uncovered are dangerous: they might be more important that your original task. Realize you’re doing something important. Update your team and re-orient.
  • Trim Yaks, aka HackHacking Yaks: these can increase your speed at many tasks
    The Trim Yak is not so task-specific; these are the ones that let us work faster generally.
    includes all the ways we change our environment in order to influence our own behavior
    Trim yaks are about increasing our speed over time. It’s hard to know how useful a given Trim Yak will be; we might never use it again or we might use it a million times. Benefit is uncertain, so limit the cost.
    • Timebox! The first time something bugs you, do it by hand. The second time, think about changing it: can you give it 15 min? that might be enough. If it isn’t, then the third or fourth time, give it an hour. Spend time only on what has proven useful.
    • Stay curious! Learn a little bit more than you need to
    • Pairing: it helps to have someone else who is like “yeah we really have to fix this.” But it’s hard to shave Trim Yaks while pairing. Except! Pairing is a great time to share trimming tips and also builds relationships.
  • Royal Yaks, aka Yakkity Yaks: relationships with people help us get work done
    Talking to people is yak shaving; it is an intermediate task that helps you get your official tasks done.
    Conway’s Law says that our system architecture resembles our communication flow
    • The more we know about what people know, the more we know whom to ask for help.
    • We need to know what our teammates don’t know.
      This is how we get distributed decisionmaking with trust: because we can predict each others’ actions.
    • Let our teammates know that we want to know what they know, we want to hear their input. Psychological safety. We each need to feel safe suggesting ideas and asking questions. Google found that this is the #1 determiner of team effectiveness.
    • Your own yak stack
      One way I do this is with my own slack channel, #jessitron-stream. I use it as a notebook. I throw error messages in them when I hit them, and record what I did (especially when changing infrastructure).
  • Golden Yaks, the rarest of all: shave them well enough and change the way we work.
    Golden yaks change behavior. They start by making something so much easier that we do it way more, then they give us an option we didn’t have before or eliminate a tradeoff.
  • Conclusion
    Yaks are a part of life and of progress.
    • Pair: We aren’t in this alone, and we can do it better together. Pairing lets us keep perspective on what we’re doing and why. We can use that space to think about doing it better.
    • Proven usefulness: look for it. Don’t spend a ton of time until you’ve hit a problem a ton of times. Track your yaks and timebox your attacks. Remember it’s about generativity, not all about you. Fill potholes, and shave a little more so your team can share the benefit.
    • Play: Stay curious. Especially learn about the system you work in, both socially and technically. Learn a little extra each time, about your tools, your system, and your companions on this journey. Interact and automate for the wisdom; for changing us, and thereby the world.

1.7. “I’m just an engineer”

1.7.1. Software Dark Ages

Domain Driven Design to the rescue! But:

  1. Old techniques are often forgotten or considered obsolete
  2. Focusing on the wrong things
    In the historical Dark Ages, religion put down science. In Software Dark Ages, infrastructure is putting down important software design techniques.

  3. Software Dark Ages is a very strong self-perpetuating system

    system-thinking.jpg

  4. some people may miss the initial point of DDD
  5. Engin
1.7.1.1. But I’m just an engineer!

Some engineers tell me they are “just engineers”. They don’t care too much about who uses their software or why. They are just implementing, say, JIRA tasks – building services with some bytes on the input and some bytes on the output. Such a mindset leads to a big disconnection between engineers and their clients whose problems we, as engineers, are trying to solve. Without proper communication, it’s much harder to create solutions that help clients in the right way. This is the goal in the end – not simply to process bytes.

1.10. Focus on Soft Skills and core principles rather than in the specifics of programming

1.10.2. Build terrible things. An edict for mid-level engineers   website

  • Her boss tells her she is “focusing too much on the technical side of things. The job also includes team alignment, product support, communicating with other parts of the company, managing up…”.
    She thinks she is so behind technically that technical depth was her limiting factor rather than people management (impostor syndrome?)
  • Learn the fundamentals (what a kernel is, comparison of datbases, stack vs heap, multiprocessing, multithreading, DNS) even if you don’t use them directly in your day to day job
  • If you talk to any senior engineer, they’ll cheerfully enumerate for you every stupid decision they ever made, because making these mistakes is how you become senior.
  • But once you’re on the other side, you want to protect others from making the same mistakes. Your engineers learn to win your approval instead of learning to build things that work and keep working.
    They look to you for opinions because they don’t have many of their own. You prevent them from making the mistakes that you made and learned from, so they do not learn. When truly important projects come along, they are not ready to lead.

So you’re a mid-level engineer now. What to do? I have some suggestions.

  • Start optimizing for decision-making ability. Take on projects where you get to choose what your stack will be, what tools you’ll use, how you’re going to solve problems that come up, how much tech debt you’re going to accrue, what you’re going to build and when.
    You’ll do it all wrong at first, but your instincts will get better at a rate that will surprise you. You’ll fix mistakes where you can, and live with them where you can’t.
  • Be the primary engineer on projects that feel too big. If you can’t do this because somebody won’t let you, go somewhere else. There are a million startups with too few engineers and too much to build.
  • Look for places where things are a little chaotic. Before you have a track record of building and maintaining critical systems, it’ll be hard to talk people into granting you the kind of responsibility you want at a company with lots of structure.
    Even if you manage it, it’ll feel too precious, and you’ll be too careful with it. So go somewhere where responsibility is not scarce.
  • Your goal is no longer to find mentors to teach you everything. That strategy might’ve gotten you from being a junior engineer to a mid-level one, but it won’t make you senior. Start building things on your own, even if it’s scary.
  • Build things quickly. Build things for people. Build things that get used.

:ID: bc5fe470-e235-4f97-8b26-24a57483ecee

Agile/Lean Startup only makes sense when resources are limited
In resourceful companies, it makes sense to first do things and then worry about optimizations
This is a problem because it can lead to corruption
https://www.infosys.com/iki/insights/hybrid-scaling-agile.html

1.12. Conway’s law

Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization’s communication structure.
Likewise, boundaries between organizations have to be designed like APIs

This is only true when humans are acting more or less like computers, processing information, which is kind of true in a company under normal conditions, until you go to the ends of the pareto distributions which include emotions and feelings and all of these things and anthro-complexity kicks in

1.13. The Only Unbreakable Law - YouTube

  • How do Committees invent?
    Some time ago, things could be designed and done by a few people, but now they are so complicated they require a lot of people, which in turn require doing things before you start:
    1. What are you actually doing? Boundaries on the design activity and the system to be designed, placed by the sponsor (limited resources) and by reality (constraints)
    2. What is the structure of the organization that will pursue that design? (companies, committees, consortiums) > 100 people to assign tasks
  • Coordination problems
    A coordination problem arises between groups/teams, separate from the original problem (social, not technological or physical)
    Every time a delegation is made (which it’s possible because of the subgroup structure in the first place) you are excluding certain solutions from the search space
    The system will not have a high-bandwidth communication between those areas (for example between engine and tyres when designing a car)
  • Design alternatives
    Given any design team, you can say which design alternatives cannot be efficiently pursued («conterfactuals?») because of bandwidth constraints (communication at that frequencies would be infeasible)
  • Homomorphism between system graph and org graph
    You may need to collapse or expand some nodes when going from one graph to another, but they are pretty similar
  • A completely rigid organization will only produce copies of itself
  • The org chart is the asymptote
    It’s the “best” that you can do minimizing the number of interfaces
    The “worst” you can do is create internal APIs because of lack of internal communication
    «Is it really the “best” and the “worst” in general?»
  • Connection between three laws
    • Amdahl (execution)
    • Brook (completion)
      No Silver Bullet: “accidental complexity” vs “essential complexity” are in Amdahl’s law “parallelizable” vs “strictly serial”
      Transaction costs / coordination problems become the asymptote, minimum level of communication
    • Conway (structure)
      Unparalellizable means the longest dependency chain
  • Cadence and types of communication
    • Static Interface
    • Versioned Interfaces
    • Documentation
    • Shared code repository can be a form of communication
    • monthly
    • weekly
    • daily
  • Mechanism
    inside any box, communication and thus design iteration is faster than accross the org tree (coherence). People far removed from one another are less likely to explore shared design (coevolution, defined as create solutions by working, solving problems, together)
  • Lower costs will eventually be optimized given enough resorces, but high costs won’t
    Good people, time
  • Conway’s Nigthmare
    Legacy code means system structure becomes a merge of all the previous and the current org chart, unless you do a complete refactor of that subtree
    Delegation now includes time-travel
  • Programmers now think delegation is separatedly good
    Conway’s Law happens within the software with class interfaces/inheritance especially
    Inheritance is bad for the code, it is a way of breaking down problems, a necessary evil like an org chart. We do it because we have to, not because it’s good
    «Really this critique is about multiple inheritance, not OOP as such. Dataclasses in functional programming maybe are better?»
  • A designer faced with a complex problem will be tempted to articifially break it down without knowing the best partition
    Sandboxing cycle
    Libraries, Frameworks/Engines, Package Managers, Virtual Machines, Containers, Microservices are all ways of introducing more (virtual) org charts, boundaries that cannot be optimized across. It’s not good, it’s a necessary evil
  • Conservatism in technology
    When faced with a new technology vs an old proven one (or organization, or method of organizing) you will always choose the old one (I know it works because everyone else is using them, so I will only be as bad as everyone else)
  • Be as lean and flexible as possible
    Both in org chart and code
    Avoid things that avoid code being optimized together (encapsulation, library)
    The Only Unbreakable Law [video] | Hacker News
    «Abstraction have a cost, rather than simply saying they are “bad”»
    «Abstractions cut down the solution space»

1.13.2. How do Committees Invent? (1968) [pdf] | Hacker News

The only way to reduce complexity in a long running project is:

  • diligence to propoerly research the domain and provide defined solution(s)
  • leadership with courage to make the right choice that may have risk
  • comittment from stakeholders to fund the scope of the initiative
  • the team’s grit to see projects through to the end (and not leave things half-refactored)

If any of the above breaks down, you get the problem of half-baked solutions that only end up making things worse by adding additional complexity.

1.15. Team Topologies

https://teamtopologies.com/

1.15.1. Team topologies — optimizing software team size   process

When do teams become big enough to be splitted up? And all the decisions it carries

1.18. 10 Mistakes To Avoid as a Software Development Manager

https://www.reddit.com/r/programming/comments/oytddh/10_mistakes_to_avoid_as_a_software_development/

  1. Get the basics right. Have a development methodology. Structure the teams well, with a good mix of senior and junior. Use issue tracking and version control. Ensure there are code reviews, automated tests and automated deployments.

6.1. Give your developers a good working environment. Open offices are the opposite of that - they need quiet and privacy. So is mandating and locking down their software, giving them standard issue corporate PCs rather than decent developer PCs, overwhelming them with meetings, mandating dress codes and mandating office hours.

  1. Avoid bad hires. Slow members of the team slow the whole team down. Dysfunctional team members destroy team cohesion. If too many bad hires enter your team, you’ll drive the good talent away.

11.1. You can’t fix stupid with process. No amount of Agile, Scrum or any other buzzword methodology will compensate for your bad hires.
11.2. Some turnover is fine. Good developers will not stay forever. Let them go on good terms - they will be advocates for your business and might one day return. Make sure that you can replace them with equally good developers.
11.3. Get rid of bad developers.

  1. Don’t force things down your developer’s throats. Don’t mandate a language, tool, methodology, process (etc) without involving them in the decision process.
  2. Share the fun jobs. Developers, in particular junior developers, can become demotivated if they spend their entire time writing tests or fixing small bugs. Share the interesting work around the team and give junior developers an opportunity to spread their wings.

1.19. Time estimation

1.19.1. Time estimation with maths

https://link.medium.com/S3o9Orlyvcb

  1. It’s easier to tell if task X would take more/less/same time compared to task Y than it is to tell exactly how long they would take. This is because comparing the medians works just as well as comparing the means if the skewness of the curves is roughly the same (which is true for similar tasks).
  2. I usually estimate the inevitable error (mean-median) as a percentage of the task time that goes up/down depending on how comfortable I’m with the dev environment (do I like this language/framework? (40%) Do I have good debugging tools? (30%) Good IDE support? (25%) … etc).
  3. splitting sprints into equally sized tasks, just to create some uniformity in the time estimation process. This allows me to benefit from point 1, it should be easy to tell if two tasks are roughly equal in time. This also makes tasks even more similar so that the hypothesis applies even more perfectly and things become more predictable.

1.19.2. Why Time Tracking is the Secret Sauce for Agile Teams | Pivotal Tracker Blog

https://www.pivotaltracker.com/blog/time-tracking-secret-sauce

  1. More accurate billing
  2. More realistic estimates
  3. Improved processes

1.19.3. The Purpose of Estimation | ThoughtWorks

https://www.thoughtworks.com/insights/blog/purpose-estimation
A common scenario runs like this:

  • Developers are asked for (or given) estimates for upcoming work. People are optimists, so these estimates tend to be too low, even without pressure to make them low (and there’s usually at least some implicit pressure)
  • These tasks and estimates are turned into release plans tracked with burn-down charts.
  • Time and effort goes into monitoring progress against these plans. Everyone is upset when actuals end up being more than estimates. In effort to increase pace with the estimates, developers are told to sacrifice quality, which only makes things worse.

In this narrative, effort put into estimates is, at best, waste - since “an estimate is a guess in a clean shirt”. Usually estimates end up being actively harmful as they encourage FeatureDevotion, a nasty condition where people start valuing ticking off features more than tracking the real outcome of the project.

For me, estimation is valuable when it helps you make a significant decision.
My first example of an estimation-informed decision is allocation of resources. Organizations have a mostly fixed amount of money and people.
Another example is to help with coordination.

So whenever you’re thinking of asking for an estimate, you should always clarify what decision that estimate is informing.
When you do find a decision then knowing it focuses the estimate because the decision provides context. It should also clarify the desired precision and accuracy.

Understanding the decision may also lead you to alternative actions that may not involve an estimate.

But remember with tracking against plans that estimates have a limited shelf life. I once remember a gnarly project manager say that plans and estimates were like a lettuce, good for a couple of days, rather wilty after a week, and unrecognizable after a couple of months.

1.19.4. 11 Laws of Software Estimation for Complex Work

https://betterprogramming.pub/11-laws-of-software-estimation-for-complex-work-c23b6e5e9ec4
11 Laws of Software Estimation for Complex Work:

  1. The work still takes the same amount of time regardless of the accuracy of your estimate.
  2. No matter what you do, estimates can never be fully trusted.
  3. Imposing estimates on others is a recipe for disaster.
  4. Estimates become more reliable closer to the completion of the project. This is also when they are the least useful.
  5. The more you worry about your estimates, the more certain you can be you have bigger things you should be worrying about instead.
  6. When you suck at building software, your estimates will suck. When you’re great at building software, your estimates will be mediocre.
  7. The biggest value in estimating isn’t the estimate but to check if there is common understanding.
  8. Keeping things simple is the best way to increase the accuracy of estimates.
  9. Building something increases the accuracy of estimates more than talking about building it.
  10. Breaking all the work down to the smallest details to arrive at a better estimate means you will deliver the project later than if you hadn’t done that.
  11. Punishing wrong estimates is often like punishing a kid for something they don’t and can’t know yet.

1.20. Features over everything else (featuritis)

1.20.2. Outcomes over Features – the fifth agile value?

https://dannorth.net/2006/10/28/outcomes-over-features-the-fifth-agile-value/

1.22. How to defend good code

1.22.2. https://www.reddit.com/r/coding/comments/qkmuso/how_to_defend_good_code/

This article misses the argument, And it shows by its own definition of “good code”. It says good code is easy and fast to work with.
Well, if that was the case, there would be no pushbacks… smh.
Creating good quality code that is maintainable, readable stable and modular for future growth IS slow.
Its a question of “do we want to be fast now or later?”, and the article completely missed that. Invest in good code now (meaning slower dev in the short term) that will result in faster dev later down the line, OR slopy fast work now, that will incur technical debt will have to pay later.
Most of the time its purely a business decision, not a technical one, And as a proffesional your responsibility is to properly communicate the consequences to C-level.
Its not to say you can’t express your opinion on such decisions, just be aware, that as a programmer, its not under your jurisdiction…
Competitive Startups, many times prefer fast sloppy dev, because they rely on their own expected growth to counter-balance the tech debt. They’ll throw money on the problem later (lots of junior code monkeys, that will sadly grow under this environment of constant sloppiness, but I digress)

  1. Make recommendations and have evidence to support where you are coming from.
    1. Don’t call anyone an idiot in the process
  2. Don’t do anything unethical
    1. Writing shitty code is not unethical unless lives depend on your software
  3. If your supervisor still wants you do continue in a way that you disagree with:
    1. Do what they tell you
    2. (or) Find a new job. There are LOTs of jobs out there in development
    3. (and) Get better at forming a convincing argument for the next time
    4. (and) Genuinely seek to understand the reasoning behind the decision

1.22.3. How to defend good code

https://www.codewithjason.com/defend-good-code/

1.22.3.1. Why good code needs defending

Good code quite frequently comes under fire. Managers explicitly or implicitly pressure developers to cut corners in order to “move fast”.
And sadly, even programmers sometimes argue against writing good code. They say things like “it doesn’t always need to be perfect, because after all we need do to ship”.
The biggest lie in many arguments against good code is that programmers spend too much time gold-polishing their code → our problem is overwhelmingly the opposite one.
Similarly, the problem in the software industry is not that we spend too much time writing good code, but that we spend too much time wrestling with bad code.

1.22.3.2. What “good code” means

Good code is code that’s fast to work with. Good code is easy to understand and change.
If the code can be changed quickly and easily, then it’s good, by definition. If the code is slow and difficult to change then it’s bad. Any specific coding practices like short methods, clear names or anything else are just incidental. The only thing that matters is whether the code can be changed quickly and easily.

  • ↑ This definition ought to be appealing to everyone. Developers and Non-technical stakeholders
  • ↑ Does not need technical expertise: People might not understand exactly what it means for code to be “high quality” or “good”, but they can certainly understand what it means to be able to work quickly.
  • Anytime you have to make a defense for writing good code, it seems smart to remind your “opponent” (who will hopefully become your ally) that your goal is to move fast.
1.22.3.3. Weak arguments for writing good code
  • We should have a clear understanding of why we’re doing it.
  • We should also be able to articulate to others exactly why we’re doing it.
  • “I write good code because it’s more professional to write good code / I believe in craftsmanship”
    • It doesn’t explain why it’s more professional to write good code, its benefits
  • Such appeals are also selfish.
    • They speak to what makes the programmer feel good, not to what benefits the business ⇒ not peruasive
1.22.3.4. Weak arguments for writing bad code

Finally, here are some bad arguments for doing sloppy work and why I think each one is flawed.

  1. “Perfect is the enemy of the good”
    • is a good saying to keep in mind. A bent toward perfectionism can eat up all your time and keep you from shipping something that’s good.

    The typical spectrum of possibilities for a coding change usually doesn’t range from “perfect” to merely “good”. Usually it ranges from “acceptable” to “nightmarish”. a honest version of this saying would be “acceptable is the enemy of the nightmarish”.

    • Refutation: If someone tries to pull “perfect is the enemy of the good” on you, you can say, “Oh, don’t worry, I’m not trying to make it perfect, I’m just trying to make the code understandable enough so I can work with it.” This statement is hard to refute because it appears as though you’re agreeing with the other person. Plus no reasonable person would argue against making the code understandable enough to work with. What you’re saying is also true: you’re not trying to make the code perfect. You’re just trying to make it not nightmarish.
  2. “Users don’t care about code”

    Lack of Second order thinking Yes, obviously users don’t directly care about code. But bad code has negative consequences that eventually become obvious to users.
    Bad code (again, by definition) is slower to work with than good code. When bad code is piled on top of other bad code, the slowdowns become exponential. Changes that should take a day take a week. Changes that should take a week take a month. Users definitely notice and care about this.
    Bad code is also harder to keep bugs out of than good code. Code that’s hard to understand gives bugs safe places to hide. Users are obviously going to notice and care about bugs.

    • Refutation: If someone uses “users don’t care about code” on you, you may be working with someone whose critical thinking skills are so weak that the person is hopeless to try to persuade.
      users don’t care directly about bad code, but users do care about the effects of bad code, like slow delivery and buggy software.
  3. “Your company might go out of business”

    Multiple times I’ve heard something along the lines of this: “If your company goes out of business, it doesn’t matter if the code was perfect.”
    All that’s needed to destroy this argument is a reminder that good code is called good because it’s faster to work with. That’s why we call it “good”.

    • Refutation: Good code is called good because it’s faster to work with. Cutting corners only saves time in the very very short term.
  4. “There’s no time” or “my manager made me do it” or “they did the best they could with the time they had”

    These aren’t arguments for writing bad code but rather excuses for writing bad code.
    No one is holding a gun to your head and making you write shitty code. You’re the steward of your codebase.
    !It’s your responsibility, and no one else’s, to protect the quality of the codebase so that the codebase can continue to be fast to work with.!
    If you consciously choose to take on technical debt, you’ll almost certainly never be granted time to pay back that technical debt. Instead you’ll have to pay interest on that technical debt for the rest of your time with that codebase.
    It’s easy for your boss to tell you to cut corners. Your boss doesn’t have to (directly) live with the consequences of poor coding choices. But eventually when the poor coding choices accumulate and bring development to a crawl, your boss will blame you, not himself.
    Obviously it’s not always easy to fight back against pressure to cut corners. But I think developers could stand to fight back a little more than they do (even if it means being quietly insubordinate and writing good code anyway), and I think developers would benefit greatly from doing so. And so would their bosses and the organizations they work for.

1.22.3.5. My argument for writing good code

My argument for writing good code is very simple: code that’s easy to understand and change is faster to work with. Obviously that’s better.
I’ll also point out something that might not be obvious. Coding choices are multiplicative. The coding choices you make today have an influence over how easy the code will be to work with tomorrow, the next day, and every day after that. Same with the coding choices you make tomorrow. Each day’s choices multiply against every previous day’s choices.
The result is exponential. Poor coding choices every day lead to an exponential slowdown in productivity. Good coding choices unfortunately don’t lead to an exponential speedup, but they do at least avoid the exponential slowdown.
You can think of each day’s code additions as having a score. If you add code that has an “easy-to-change score” of 90%, and you do that three days in a row, then your cumulative score is 0.9^3 = 72.9%. If you add code that has an “easy-to-change score” of 40% three days in a row, then your cumulative score is 0.4^3 = 6.4% (!). This is why programmer productivity doesn’t vary by a factor of just 10X but more like infinityX. Bad code can eventually drive productivity down to something close to 0%.

1.22.3.6. Takeaways
  • Our industry has a much bigger sloppy-code problem than gold-plating problem.
  • Good code is code that’s fast to work with.
  • The popular arguments for writing poor-quality code, although they sound mature and reasonable on the surface, are a result of sloppy and confused thinking.
  • Whether you choose to write good or bad code is your responsibility, and you’re the one who will have to live with the consequences of your decisions.

1.22.4. How to quantify technical debt inflation

https://www.reddit.com/r/programming/comments/txke2m/how_to_quantify_technical_debt_inflation/
Technical debt is at least quadratic in time
From comments (broken window theory):

The worse the code gets the more demoralized the people also get and start to work even slower, get demotivated and in the end quit. So talent retention is something that should be included in this.

1.23. Tips for dealing for Users   process

From Structured Analysis and System Specification by Tom DeMarco

1.24. Critique of Agile

1.24.1. Agile Makes no sense

https://medium.com/hackernoon/agile-makes-no-sense-c8ebbf971012

  • We assume this stuff is obvious and intuitive. It isn’t, unless you’re talking vague high-level/nod-worthy ideas
    • one person’s lightweight process and “open culture”, is another person’s bureaucracy and culture of fear/intimidation
  • Agile is focused on sustainability and long-term viability «which may not be applicable» ⇒ benefits are not immediate, and in the short term you’ll experience some discomfort and dissonance.
    • A focus on quality, for example, truly pays off when the product becomes insanely complex and you can keep innovating.
    • Shipping tiny increments pays off when the customer is extremely satisfied with the end-result (due to learning and pivoting), not when the customer tries the paired down minimalist version after sprint one and gives feedback.
    • If short-termism is your thing (and to most humans it is), a deep focus on Agile isn’t your best hammer (but some blind sprinting may be).
  • No shared understanding. Unclear what “Agile” is.
    • For many, Agile is doing sprints and using Jira. So, they’re already “doing” it. This other stuff is some sort of fringe cult that only works in blog posts.
  • Recommended book: Don Reinertsen’s The Principles of Product Development Flow
  • the challenge of whole-org agility
    agility on the team level is but one part of the puzzle. Even when Agile on the team-level makes sense, you need that other part to make sense. The rest of the org was probably the blocker in the first place.

1.24.3. Agile fails https://link.medium.com/cyHmD2BN4hb

1.24.3.1. Meetings, meetings, meetings

I’m not saying that you shouldn’t communicate with your team or that stand-up is necessarily a bad thing to do. However, if there’s no standup update, why have a meeting? Why disrupt your team’s workflow if nothing important is going to be said or done.

1.24.3.2. Agile Leadership

The main issue with that is that these people have no idea how software development works in the real world.

involved in technical discussions. Often times I’ve spent at least 10 to 15 minutes of meetings trying to explain technical details to someone non-technical. I also end up having to repeat myself constantly because they have no technical foundation they require me to reiterate my points.

Agile leadership also tries to insert far too many people into a meeting.

Keep technical meetings strictly technical

1.24.3.3. Fixing the Issues
  1. Solving the Meetings

    One solution that I’ve found particularly useful is doing asynchronous stand-up meetings.

    One of the primary benefits of stand-up is the last thing that is talked about, the blockers. If there are blockers then it is certainly beneficial to figure those out before your day moves on.

    We send a poll out to the team before stand-up to ask if we should have a stand-up meeting, otherwise write it on the chat

  2. Solving the Leadership

    Ultimately, technical people need to be in charge of technical things. There needs to be more control put in the hands of technical leadership. The way I see it if there is a non-technical Scrum Master on the team they should be a tool in the belt of the technical leadership, not the technical leadership itself.

    Scrum Masters should not be dictating the processes around the code is written or deployed, nor do they need to understand that.

    a non-technical Scrum Master I think they should have limited responsibility

1.24.4. Agile and the Long Crisis of Software

https://logicmag.io/clouds/agile-and-the-long-crisis-of-software/
wrestling match between what managers want software development to be and what it really is, as practiced by the workers who write the code (its habit of sprawling beyond timelines and measurable goals)

1.24.4.1. Turning Weirdos into Engineers

Coding is not a matter of pure logic telling machines what to do, even if the system analysts have specified a high-level architecture
By the late 1960s, software development was facing three crises:

  • not enough programmers
  • make development more predictable
  • managerial necessity to get developers to stop acting so weird. 

industry leaders encouraged programmers to embrace the mantle of “software engineer,”

1.24.4.2. Chasing Waterfalls

Waterfall appealed to managers because “The essence of the software-engineering movement was control: control over complexity, control over budgets and scheduling, and, perhaps most significantly, control over a recalcitrant workforce.”
Universities in 1980 couldn’t fill the faculty positions necessary to train the huge number of students willing to become software engineers.
Despite managers’ efforts to make software development reliable and predictable, it seemed, to have only grown more unwieldy
Rather than clearing the way for software developers to build, waterfall gummed up the works with binders of paperwork and endless meetings.

1.24.4.3. Khakis and Dad Jeans

seventeen middle-aged white guys, dressed in khakis and dad jeans, all obsessed with management: The now-legendary authors of what came to be called the Agile Manifesto
Given the shortage of qualified developers, technology professionals might have been expected to demand concessions of more immediate material benefit—say, a union, or ownership of their intellectual property.
Instead, they demanded a workplace configuration that would allow them to do better, more efficient work.
this revolt against management is distinct from some preceding expressions of workplace discontent: rather than demand material improvements, tech workers created “a new ‘spirit,’ based on cultures, principles, assumptions, hierarchies, and ethics that absorbed the complaints of the artistic critique.”
Developers weren’t demanding better pay; they were demanding to be treated as different people.

1.24.4.4. Organizational Anarchists

Agile meant management ceding control, at least to some extent, to developers themselves. And most large companies weren’t willing to do that, at least not until the 2010s
Some of this popularity had to do with the growth of high-speed internet connections ⇒ allows faster releases
“Agile was openly, militantly, anti-management in the beginning,” but not anti-corporate
The coder of Agile’s imagination is committed, above all, to the project. He hates administrative intrusion because it gets in the way of his greatest aspiration, which is to do his job at the highest level of professional performance.
he wants most of all to be in “the zone”: headphones on, distractions eliminated, in a state of pure communion with his labor.

1.24.4.5. Management’s Revenge   process

Despite velocity chart and disciplined feature-tracking, real progress wasn’t being made because no one really knew what the project was supposed to be like, or what it was for
laboring under Agile may not be the liberatory experience it’s billed as.
software development is in crisis again—but, this time, it’s an Agile crisis.

  • “no true Scotsman” problem.
    Any Agile practice someone doesn’t like is not Agile at all, it inevitably turns out: the manifesto is too vague, doesn’t prescribe any specific activity → insists on its status as a “mindset,” not a methodology

The notion that a competent professional would need to justify his work every day, in tiny units, was absurd to her (a lawyer)

  • Agile encourages developers to see themselves as cogs in a machine.
    They may not be buried under layers of managers, as they were in the waterfall model, but they nevertheless have internalized the business’s priorities as their own.
    “As developers, IT professionals, we like to think of ourselves as knowledge workers, whose work can’t be rationalized or commodified. But I think Agile tries to accomplish the exact opposite approach,”
    Agile conferences notoriously tend to be dominated by managers, not developers.
    The ubiquity of Agile means that it is just as likely to be imposed from above as demanded from below.
    Agile project managers, who are generally embedded in the team as the “product owner,” find themselves pulled in two directions: what’s best for the developers on the one hand, and what they’ve promised to deliver to management on the other.
    Even as the team is pulled in multiple directions, it’s asked to move projects forward at an ever-accelerating pace.
    daily standups become surveillance, pressure to enumerate every task done to justify your work and salary
    story points end up being used as a way to assess performance and used to calculate deadlines
    • The Next Crisis
1.24.4.6. The Next Crisis   process

Agile’s values celebrate developers’ ingenuity and idiosyncratic ways of working.
Problems being broken down into such small pieces turn into limits on autonomy to define what should the work be instead of how to do predetermined work
tech workers are starting to organize to change the direction of their companies’ business strategies
Agile might have played a role in creating a work culture that is increasingly revealed to be toxic for women, people of color, and members of gender minority groups.
pair programming assumes that the two coders are comfortable with each other
retrospectives can descend into a structureless series of accusations, and code reviews in an instrument of harassment
eliminating bureaucracy, hierarchy, and documentation feels great, until you’re the person who needs rules for protection.
ethical arguments are incompatible with Agile’s avowed dedication to keeping developers working feverishly on the project, whatever it might be.
software developers should stop calling themselves “engineers” since they have no code of ethics nor commitments to civil society, only loyalty to the project
Agile is good at compartmentalizing features, neatly packaging them into sprints and deliverables (“information hiding,” a critical way for humans to manage systems that are too complex)
But by turning features into “user stories” on a whiteboard, Agile has the potential to create a “chain of deniability”: an assembly line in which no one, at any point, takes full responsibility for what the team has created.
The Agile Manifesto paints an alluring picture of workplace democracy.

  • misalignment of priorities
    the manifesto makes a strong case that businesses’ products can be strengthened by worker autonomy.
    But they’re just as likely to conflict, as when a project manager is caught between a promise to a client and the developers’ well-being and own priorities.

1.24.5. XP Days 2013: Dan North - Why Agile doesn’t scale and what you can do about it - YouTube

  • Local optimisations don’t roll up
  • Agile doesn’t have an opinion
  • Book Suggestion: Agile adoption patterns - Richard durnall
  • Book Suggestion: The leprechauns of software engineering - ebook
  • Book Suggestion: Beyond budgeting
  • Contextual consistency
  • Scaling is more than just small things bigger
  • Guiding principles, strong leadership
  • Crossing the chasm of credibility is hard

1.25. Gresham’s Law

1.25.1. Gresham’s Law — Why Bad Developers Push out Good Developer and Developers Create Low Quality Code

https://link.medium.com/SgCsaFCUclb
Poor practice will replace good practice if it’s worth the same value e.g. if people don’t get extra reward for good practice, why would the bother doing more work. There Is No Benefit or Incentive for Developers to Create Quality Code on Software Projects.
Bad Development Is Intensely Habit-Forming When Rewarded

1.25.2. Developers Are the Main Danger to Software Development (Law of triviality)

https://medium.com/codex/developers-are-the-main-danger-to-software-development-f82de44aa0cc

The Law Triviality Causes Software Projects to Fail. While focusing on the trivial tasks, the important tasks don’t get done on time and the project falls behind the plan.
Focus on the trivial is at the expense of important

law of triviality
https://www.darkcoding.net/software/facebooks-code-quality-problem/ (more change to the code ⇒ more instability)

1.26. Post-Agile thoughts | Heart of Agile

https://heartofagile.com/post-agile-thoughts/

1.27. Bad Management

1.27.2. Bad managers are a huge problem in tech and developers can only compensate so much

https://www.reddit.com/r/programming/comments/nwfx4l/bad_managers_are_a_huge_problem_in_tech_and/

1.27.3. Business people have turned agile into games rather than dealing with the complexity it attempts to resolve

https://www.reddit.com/r/programming/comments/oco4wk/business_people_have_turned_agile_into_games/

1.27.4. Ignorant managers cause bad code and developers can only compensate so much

https://www.reddit.com/r/programming/comments/oz06jw/ignorant_managers_cause_bad_code_and_developers/

1.27.5. Neil on software - The softer side of software development

Horóscopo para programadores
https://neilonsoftware.com/

1.28. Learning Organization

Learning: an anthro-complexity perspective - The Cynefin Co (Against Systems Thinking)
https://en.wikipedia.org/wiki/Learning_organization
https://ohsystems.io/systems/thinking/learning/organization/knowledge-management.html

1.28.1. https://gitlab.com/kevin.ohsystems/zettconnect

1.28.1.1. Problem Statement

A: In developing ideas or strategies in organizations, thoughts are written and links are posted in team chats, email threads etc. The context and thinking behind the initial idea are easily forgotten and the message thus lost.
An element in research and development that is often neglected is to describe the connections: Why does A link to B, hence the context is lost.
This connection is often part of the message (in a chat for example where a link to an article is posted with a remark such as “this article explains why we might run into problems with our strategy because of unconscious bias”). Such comments mostly get lost in the communication streams, yet they are valuable work and material in that they show and store the thinking someone did.
B: Thought processes (ideas) are much easier to understand and to collaborate on if they are visualized. At same time we want to stick to the principle of storing everything in plain text, so what we write can go into version control (git) and can be fed to machines if we want to do things like natural language processing, search and other computer based processing of our writing (our thoughts).

1.28.1.2. Possible Solution

A combination of systems and languages like Zettelkasten, Business Process Model and Notation, a markup language like markdown, Git version control, a unified process and workplace integration. Integrate with the digital workplace through a central chat application like Slack, using chatbots or slash commands.

1.28.2. https://www.jmaguire.tech/posts/continuous-learning-part-ii/

  • Builds Slack into Dev Process

This final point aims to solve one of the major points of pushback from Upper Management: won’t this slow down our development process. The short answer is no, it may seem like taking time out of our schedule during the week would slow down the development process, but our goal is to actually make it more efficient. The straightforward reasoning for this is that by growing the skill of our engineers we are empowering them to make better decisions, and solve problems more efficiently.

Like most things, there’s more to it than the straightforward answer. When we factor in time for these clubs into our development schedule, we alleviate the time pressure on engineers. Experiments such as The Candle Problem have shown that when people operate under constraints they tend to exhibit less creative solutions and more of a fixed mindset with relation to the objects and problems present. They also help to keep people in the context of work while removing a bit of the time pressure so that their mind can slow down a bit. Generally downtime at home helps slow the mind down, but that means leaving the office and losing context of the problem that they’re currently working on. In addition, by solving other problems in the Code Dojo and The Refactory (and being exposed to them in Book Club/Video Club) we’re able to identify problems with similar unknowns to the problem we’re currently working on and can use that as a potential solution (as discussed by George Pólya in How to Solve It).

1.31. Why None of the Methodologies Works Well in Software Engineering?

1.31.1. Unlike manufacturing, the software engineering world is far from perfect

“Where is the perfect methodology which will let us achieve this manufacturing level precision?” (conveyor belt model)
The obvious answer — there is no such methodology, and there never will be. Period.
The answer is super simple. What is more complex is “Why?”. I think there are two or three fundamental problems that make it’s impossible to come up with anything so repeatable.

1.31.2. The Uniqueness Of Each Product

  • It all feels familiar, but it’s always different.
    • The choice of technologies is different (languages, frameworks, libraries).
    • The way how they are put together is different.
    • The edge cases are different.
    • The features are different.
  • Parts are not highly controlled and design does change on the fly

1.31.3. Engineers Are Not Interchangeable

So, the first thing which a manufacturing process needs is repeatable car parts. Another thing which it needs is repeatable people. You can have a factory if you can easily replace a worker with another worker
But this doesn’t happen with programmers
One person is a guru of backend and super experienced in .NET, and another spent their whole life tinkering with embedded software and dreams in assembly, and the third one can barely write a Hello World. As a result, it’s impossible to randomly switch what they are doing and expect predictable results.
BTW. What I am saying is applicable for QA too. One person is amazing around thinking through all edge cases. Another great with automation, and the third one can barely execute tests written down by somebody else.

1.31.4. Responsibility For One Operation

Manufacturing required significantly deeper specialization and division of labor. And again, in software, we have almost opposite.

  1. First of all, there is no clear separation of all of the tasks that need to be done. Some of the tasks aren’t easily divisible into simpler ones.
  2. And people own a wide variety of tasks to implement some big chunk of the “manufacturing” process.

1.31.5. The World Is Imperfect

What about software engineering?. I am saying that engineers (the most critical tool of the software manufacturing process) work in an unbelievably (and inherently) messy environment.
I am more talking about the messy environment of people relations, changing directions, politics, hype, etc.

1.31.6. Summary

I think a huge chunk of my blog touches precisely on this subject: How to operate in a good way in an imperfect world?
I am not writing about something glamorous, provocative, or viral. It’s more just bread and butter type of things.
Many articles could be summarized as “Here is how people thought it would work in a perfect world. Here is how it actually works in an imperfect world. And here is how to make it a bit saner”.
And here are several articles exactly about such things. How does an imperfect world differ from the dreams of people from ivory towers?

1.32. TDD

1.33. Agile applied to another industries

1.33.1. Ver “The Genius of 3D Printed Rockets” en YouTube

https://youtu.be/kz165f1g8-E
Software manufacturing
Using short feedback loops to prototype using 3d printers

1.33.2. Discográficas (industria grande que se democratiza)

Producción pasa de ser grande a pequeña

  • https://youtu.be/YmaLdu9LKg0?t=19m42s
    En las discográficas hacía falta un montón de gente para producir, distribuir, y promocionar el disco pero ahora lo puedes hacer desde tu casa todo

1.33.3. Ensamblado Virtual

Ver si la pieza encaja con el resto antes de hacerla
Anteriormente con el diseño de planos pasaban estas cosas, ahora con CAD ya no

1.33.4. Wehrmacht, Elon Musk and Unix Software Development (Worse is better)

https://erik-engheim.medium.com/german-soldiers-and-unix-worse-is-better-81752d8fcc78

  • Worse is better - Wikipedia vs The “Right Thing”
  • Simplicity — The design must be simple, both in implementation and interface. It is more important for the implementation to be simple than the interface.
  • Correctness — The design should be correct in all observable aspects. It is slightly better to be simple than correct.
  • Consistency — The design must not be overly inconsistent. Consistency can be sacrificed for simplicity in some cases, but it is better to drop those parts of the design that deal with less common circumstances than to introduce either complexity or inconsistency in the implementation.
  • Completeness — The design must cover as many important situations as is practical. All reasonably expected cases should be covered. Completeness can be sacrificed in favor of any other quality.

—
Examples

  • The Success of The German Wehrmacht
  • Evolution vs Intelligent Design in Biology
  • The Cathedral and the Bazaar Open Source Development
  • Market Economy vs Planned Economy
  • Toyota Production System vs Ford
  • Scrum vs Waterfall in Software Development
  • User Interface Design
  • SpaceX Approach To Rocket Design
  • Developers Favoring Dynamic Typing Over Static Typing

1.35. The Wetware Crisis: the Dead Sea effect : Bruce F. Webster

the more talented and effective IT engineers are the ones most likely to leave — to evaporate, if you will. They are the ones least likely to put up with the frequent stupidities and workplace problems that plague large organizations; they are also the ones most likely to have other opportunities that they can readily move to.
What tends to remain behind is the ‘residue’ — the least talented and effective IT engineers. They tend to be grateful they have a job and make fewer demands on management; even if they find the workplace unpleasant, they are the least likely to be able to find a job elsewhere. They tend to entrench themselves, becoming maintenance experts on critical systems, assuming responsibilities that no one else wants so that the organization can’t afford to let them go.

1.35.4. https://brucefwebster.com/2013/09/30/the-5-books-every-it-manager-should-read-right-now/   book

  • The Mythical Man-Month
  • Death March
  • Waltzing with Bears: Managing Risk on Software Projects
  • Facts and Fallacies of Software Engineering
  • Perfect Software and Other Illusions about Testing

1.36. Slowed canonical progress in large fields of science | PNAS

https://www.pnas.org/content/118/41/e2021636118
The size of scientific fields may impede the rise of new ideas. Examining 1.8 billion citations among 90 million papers across 241 subjects, we find a deluge of papers does not lead to turnover of central ideas in a field, but rather to ossification of canon. Scholars in fields where many papers are published annually face difficulty getting published, read, and cited unless their work references already widely cited articles. New papers containing potentially important contributions cannot garner field-wide attention through gradual processes of diffusion. These findings suggest fundamental progress may be stymied if quantitative growth of scientific endeavors—in number of scientists, institutes, and papers—is not balanced by structures fostering disruptive scholarship and focusing attention on novel ideas.
https://www.pnas.org/content/118/41/e2021636118

Author: Julian Lopez Carballal

Created: 2024-09-16 Mon 05:28