> Note that the Pull Requests were introduced by the Open Source Software community. It allows core teams to maintain a system and accept contributions from the outside world. The outside world is generally perceived as low-trust. Therefore, we want to verify those contributions. Pull Requests are pretty helpful in that regard. However, in a closed-source corporate setting, the Pull Request model essentially indicates that the team owns the codebase, but it is not allowed to contribute.
I think this is a very lopsided way of looking at pull requests. They are about a lot more than just trust. Reviewing and being reviewed is a great way of learning from colleagues, making common practices gel in a team, and keeping up to date with changes to the codebase. It’s not just a barrier.
> money stuck into the system. It is stuck because the organisation invested considerably in creating all this code on parallel branches. However, as long as this code is not merged into mainline, deployed in production and released to users, it does not generate any revenue. Therefore, it is money stuck in the system. But, because we have less WIP, we create less inventory. As such, we have less invested money stuck in the system.
I’ve been telling people that work in flight should be minimised for a long time, but this is the first time I’ve heard the inventory metaphor and I like it. It’s a similar metaphor to tech debt.
> I think this is a very lopsided way of looking at pull requests. They are about a lot more than just trust. Reviewing and being reviewed is a great way of learning from colleagues, making common practices gel in a team, and keeping up to date with changes to the codebase. It’s not just a barrier.
I am old enough that my first 2 jobs were subversion-based (repo side, client side git-svn) and both teams didn't care about branching. It might have to do with how awkward subversion branches felt.
Anyway, we would commit (in git lingo pull-rebase & commit) directly and we basically maintained the id of the last reviewed commit and jointly did PR reviews commit-by-commit with the code on the projector in the office.
We had a joint look at code, everyone voiced their feedback ("I don't understand a variable name like `xzc`"), "where is the unit test", "I read in a blog post recently you are supposed to not use classes...". etc. pp. Sometimes fixes would be pushed right in the PR review session so you had the variable renamed 4 commits further.
Anyway, in retrospect it worked surprisingly well at helping the team to develop a joint understanding of values & virtues that the team would like to maintain in their code base. This might of course be nostalgia of a dev looking back into their junior years.
When we finally got pull-requests, we really felt thrown into the future. It was just great. But after a while I started to miss the direct conversations about code with fellow humans.
And honestly I couldn't tell whether PR really improved the quality of the code base in the long run. They lowered the probability of bad code being committed to the code base, but also lowered the probability for a dev to just fix awkward things while they stumbled over them.
I miss that too. There’s so many little micro-improvements that are possible when you’re reading code— clarifying a variable name, expanding a comment, adding a test case, and it’s unfortunate when those things are lost under the inertia of it being a pain in the butt to create PRs and get them reviewed.
At the same time, I expect your PM is delighted that you’re not wasting time getting distracted with all that yak shaving nonsense and are instead working on the next burndown ticket that has been assigned a t-shirt size and the appropriate number of story points.
Same here. At my previous employer, we’d pair regularly, even on PRs. Larger blocks of work would warrant a mini-standup so the team could all review together in real-time (after an async review with comments). It’s an efficient use of time while acknowledging the need for real-time, humane communication.
> When we finally got pull-requests, we really felt thrown into the future. It was just great. But after a while I started to miss the direct conversations about code with fellow humans.
Why is this disjoint though?
Our newer colleagues are currently bringing in their first bigger contributions and changes to the config management. We're using short-ish lived feature branches of a week or two with pull requests.
This is good, because I can spend some time understanding what they are doing and I can prepare some ideas and examples for improvements, before we have a call to talk about these changes.
I'm also entirely willing to move my bigger changes into branches and a PR and spend some afternoon with the team to talk about good practices, like structuring commits, naming, ansible code structure, ... and see if other people enjoy that as well. Management wants more stability and broader understanding of our code bases, so moving slower and reading and discussing more code seems up that alley.
I don't like the inventory analogy. Code changes are the aggregation of keyboard events. At what point do these events become "inventory"? Every key press, every point at which the code compiles, every day, when the feature is done? It is still very clearly work in progress if it exists on a branch or behind a feature flag.
Since people usually start out writing code on their own, the tendency is to wait until the feature is done before committing. Therefore, some pressure should exist to try to get things merged because that is actually optimal on a team. However, it doesn't need to be dogmatic or drag in analogies from the manufacturing industry.
Technical debt is also a bad analogy. There is just the current state, the desired state, the set of activities required to go from one to the other and their associated cost. Sure, every now and then something hacky is done in order to (for example), land a customer which you plan to get back to (i.e. take on debt and pay it back later). It isn't that common however so the analogy almost never applies.
How about we just stop using analogies in software altogether?
My rule is if I’ve had a WIP branch for more than a day without making progress on it, I ask one of my coworkers if they’d like to tag in on it, or my boss to see if he feels like looking at it with me. It’s actually a really hard thing to do, there’s this inertia to reaching out that I hate feeling mired in - but every single time I’ve done it, it’s been okay. It’s really the times when I haven’t reached, in retrospect, when things have not gone okay.
How is Gerrit doing these days; is it any good for breaking away from GitHub? I'm looking for a git/mailing-list solution a small business could use on-prem but not any of that Gitlab nonsense (to throw people off-balance, if you will)
GitLab is pretty decent; I like the reviewing a bit better than GitHub, for example how non-code comments still form threaded discussions, or how you can set it to block merge until all discussions are marked “resolved”.
Yes GitLab has a lot to it, but much of it is optional; you won’t be hurting if you just stick to the code repo part.
Used trunk-based dev a lot. Not a fan. It sounds good on paper, but in practice, it gets messy fast and code quality is hard to maintain, especially when teams are remote.
What actually works is trunk-based deployments — keep main always deployable, and ship from there. Simple.
PRs, are underrated. They’re great for sharing context. You get inline comments, CI runs, you can test stuff in isolation by spinning up infra, and teammates actually see what’s changing.
Stacked diffs make juggling multiple PRs manageable. And yeah, PR reviews can slow you down, but honestly, I think that's a plus. Slowing down just enough to have another human look at the code pays off more often than not.
I don’t understand why PRs are being discussed here at all – doesn’t anyone serious use trunk based development with code reviews? How is that different from a PR? Seems like we need to disentangle the discussion of whether code review is good from whether feature branches vs trunk based dev are good. Those are completely unrelated questions to me.
I think there are a lot of interesting questions about using feature flags (a baby branch) vs actual branches. Personally I’m pro flags and anti branches, after a lot of experience in developer tools and CI.
Definitely feature flags are the way to go. Having those code paths integrated early doesn't cost much and makes it basically free when you want to "merge" the feature into default.
This does require your software to have a decent architecture such that feature flags aren't littering every part of your codebase, though. Ideally you want something like a whole module/plugin being enabled/disabled right at the entrypoint of your program/module. But this also pays dividends in the long run.
There are different flavours of TBD. Some use short-lived feature branches, some commit directly to master. The article advocates for the latter.
PRs basically have a hard-requirement on branches (or equivalent, like fork), because the code that is being requested to pull needs to be available somewhere. The article also advocates for not using pull requests.
However people who take this position also often advocate for post-merge code review. And more advocate for pair programming or mob programming, which they consider to represent code review as well. So branchless TBD isn’t incompatible with code review, just code review as it is commonly practiced.
I'm pretty sure trunk-based development includes using feature branches and MRs/reviews etc. I didn't get the impression we were always supposed to commit directly to master (although that is possible for trivial things which reduces overhead).
If your team is very small then strictly creating feature branches and pressing the merge button immediately because there's nobody left to review it is silly. I like that it gracefully degenerates into just committing directly to the trunk when it makes sense.
This works with git (or DVCS in general) because your local "master" or whatever is a short lived branch. People seem to forget but with git you are branching every time you clone. Obviously committing broken stuff and WIPs directly to the trunk isn't going to work.
I wonder how all these absolutist methodologies really work when you take subjectivity and our general inability to measure anything reliably away? I say this because we've tried all of them after someone read an article on a blog saying it'd solve all our problems. And we always have the same problems afterwards despite people promoting that it solved them. Sometimes even worse ones.
The root cause in my company is we hire the lowest cost outsourcing outfit we can find, which hires inexperienced juniors and pretends they are senior and has a huge staff turnover because who the hell wants to work like that for more than 12 months. But of course changing the repo structure and process will compensate for that.
My observation is that a good engineering team will self-organise a methodology that works. The methodology is unimportant compared to getting a good engineering team. I am fortunate that I work in a small niche pocket in the organisation that has good engineers and has resisted normalising with the process churn of the rest of the organisation.
We run a "dark ages" team really. We were on SVN until 2020-ish, because, well it worked fine for us. Now we're on github on commandment but it is used like SVN. We have light feature branches. Master is always stable. Releases are tagged off master. At no point do we use any other github features not even PRs. CI is makefiles. Push to ECR/S3 by human. Deployment is makefiles. Comms is email because Slack is too distracting and demands immediacy. Issue tracking is a 17 year old JIRA project with a default workflow. We have the lowest defect rate, the lowest time to market and highest ROI per head in the entire org and we don't have or need a platform/sre/devops team supporting us.
We are a disliked because we didn't do anything for over 10 years other than deliver software.
In my (only 2 decades of) experience, almost all "processes" and "methodologies" exist to compensate for lack of skill, care and/or understanding. As you say, what/how you do may sound outdated to others but it works because you care about what you do.
I think this applies to many "best practices" and "standards" as well. I could mention many examples of old systems/apps written by single/few deeply committed and skilled people were replaced by "better" and more "modern" alternatives that worked less well and that took a team of people to maintain. The latter used all the right tools, processes and practices but they were just not artists, for lack or a better word.
We don't do standups. Drop 15 mins in whoever's calendar and use the phone works. Yes we're that backwards! Every alternate Friday lunch time we meet up in person though.
After a couple of decades in the Danish software industry I've come to accept that there is only trunk-based development (COME AT ME!). Feature branches, set releases and so on are fake safety gaps that people add when they don't trust themselves and/or their developers. These days I view it similar to someone who won't deploy changes on fridays because they fear for their weekend. If you're too afraid to deploy on a friday, then you should not be allowed to deploy on a monday. Similarily, if you're not brave enough to push changes to production, then your changes aren't ready.
Of course I don't actually think there is a religion on how to do these things, but I've never seen any particular quality added by people doing quarterly releases from a feature branch. For the most part I think this part at the very beginning of the article explains a lot of the reason as to why that is:
> When we do trunk-based development, the WIP we commit gets used, before any actual user sees it, by our whole team.
In my eyes all of it comes down to management much more than deployment strategies. The disadvantage of feature branching is that it's very easy for management to cut costs on testing. The disadvantage of trunk-based development is that it requires functional teams of people who can actually work together and accept the "norms".
Our team has thorough, high quality, automated tests. In the past 5 years we have never released such a badly broken system that emergency remedial work was required. We still don't release on Fridays. It's just common sense, risk management and (most importantly) respect for the developers home and family time.
I'd argue that it depends on your users. From a business perspective, it's a lot safer for us to release on fridays because failures would not affect a lot of people over the weekend. Of course I do live in a country where you'd have to agree to fix it over the weekend, and if you do, you'd be compensated with new days off + money.
Having literally no branches might work for tiny projects, but none of this is the case for any decently-sized software.
If you want to release a piece of software to a customer who will be using it as-is for some time (which is one possible business model), then you need a version of your software where all features that the customer sees are working well. To do that, you need to fix even minor bugs in those features without adding any (visible) new features to the product.
However, this doesn't scale. You can't pause all development of new features while someone is re-aligning a few UI elements and cleaning up error messages that were too confusing. So, you have to create a split in the software: one part is getting ready to ship, another is work in progress for future releases.
Now, you can create this split using git branches, or using feature flags. So yes, trunk based development can work, you can absolutely build up the new experimental features into the software you're about to ship, but you'll have to hide them behind a feature flag that is disabled - a runtime branch. Or, you can have a release branch where only bug fixes are allowed, and submit major new work on the main branch. There are pros and cons to either solution, but whatever you do, you need stable vs unstable branches somewhere.
And, of course, sometimes you'll have to release hotfixes, and there you won't get away without a true SCM branch, since you can't just give your customers your latest and greatest software just to patch a security issue in libcurl for a 2 year old release.
> The disadvantage of feature branching is that it's very easy for management to cut costs on testing.
Have to agree with this. In a large (lower than 2/3rd in the Fortune 500 list) company with cost cutting as a constant mantra, our ever shrinking team hasn’t had a testing/QA team for several years. We never got to automating a lot of tests either. The developers create something and test it manually as per their limited knowledge. Then it’s the end users who do the bulk of the testing.
> When we do trunk-based development, the WIP we commit gets used, before any actual user sees it, by our whole team.
And of course internal teams love also doing QA on the side of their existing jobs ;) I think that’s fair enough to ask from developers and project/product managers but a lot of others on the team basically have to suffer lower productivity to test your code.
It probably depeds on the kind of software you create, but in our case:
When i add non-trivial code I want others of my team to review it first. When it is on trunk, it gets less easier to refactor as others already might use it. In this sense using a MR is really good way to get some quality checks before it goes to main and needs to be supported (in most cases) indefinitely. Next, we have system tests that i really want to avoid breaking on main and getting EVERYONE stuck.
The problem arises if I keep that branch for more than a week. Merge conflicts get harder, the financial investment is not leveraged, etc etc.
The problem is with long lived branches. This is something you should avoid, you can take it to the extreme, no branches. But extreme is almost never good.
PS CC is really good at rebasing and resolving merge conflicts.
I think the article misunderstood the debate... the debate is (or at least was) about gitflow, and specifically about having git branches per environment. So there are git branches for dev, staging, etc, all the way to prod. And git changes become deployed on the respective branch.
Trunk based development is about having a single master (sorry, "main"), and generating deployable artifacts from there, and then the remaining environments have only deployments of versioned artifacts.
Unless I'm missing some new debate about the value of PullRequests, but that sounds extreme.
> With trunk-based development teams, managing technical debt becomes a non-issue. It just happens.
Unless it doesn't happen. Issues that are dealt with automatically are those that would break the build, or make tests or linting checks fail; everything else can simply accumulate without anyone "wasting time" on it.
Technical debt is often recognized retroactively (e.g. there is a new foreign customer and not enough time to write thousands of neglected translated labels and messages).
I'm a big fan of trunk based development in the Churchillian sense. It's the worst form of development, except for all the others.
I don't think it's for every project, or even every feature in a given project. But there's a niche it works best for, and that niche happens to represent a massive percentage of all software engineering.
The niche is, you're building a web app. It's some shitty Web 8.0 version of a Craigslist or something. No one wants to pay for formalized testing. No one really wants to spend a lot of time on quality. Arguably it's not that important to users. The year is Enshittified 2030 and they're lucky if they can even log into their online banking, let alone slide into someone's DMs on your platform.
What's the solution? TBH it's not tripling down on good engineering process. (Don't get me wrong, good engineering process absolutely has a role, I'm merely suggesting that as you move from flight control software to online classifieds to your neighbor's recipe blog, that role diminishes in favor of less formal feedback.) The solution is radically forcing the code into the light of day in front of people who can say or do something about the biggest problems. The minor bugs will probably slip through this net, but the really nasty ones WILL get worked out if people on your team have to deal with them first, followed by an 'insider' circle of users, finally followed by the general public.
That translates to certain buzzwords actually representing great ideas in this type of development, such as: feature flags, CI, pilot rollouts, and... trunk-based development.
Just get the code/feature out in front of a couple people who matter, as early as the day it was written, and keep expanding until you're public.
The idea that you can apply lower-quality engineering practices in some systems is totally wrong, I think. Good engineering practices will help you retain good developers, will allow you to maintain a sustainable development pace and will prevent you annoying your users so much that they stop using your system. I can't imagine any commercial project where this doesn't matter.
Isn't this just another representation of the fallacy that it's possible to deliver faster by cutting quality? That's the kind of thing I expect to hear from ignorant stakeholders who know nothing about developing software, not on HN.
> The idea that you can apply lower-quality engineering practices in some systems is totally wrong, I think.
> Isn't this just another representation of the fallacy that it's possible to deliver faster by cutting quality?
If we use a real world analogy. You don't use the same engineering quality practices to build a sandwich as you do to build a space probe.
It would be a mistake to x-ray verify connections between parts in a sandwich, but it would most likely be a mistake not to for a space probe.
The engineering practice that needs to be done correctly is judging the error risk, predicting the consequences of errors, and mitigating errors where the cost of mitigation does not outweigh an estimate of the expected cost of damage from the error, including work to make things right.
If you ship a sandwich where the spread was unevenly applied, inspection likely could have detemined this before shipping, but the cost to have another person carefully review each sandwich as it is being made outweighs the cost of the ocassional dry spot on the bread... Which may not be noticed or may not be bothersome... Worst case, a sandwich can be remade.
Otoh, if an assembly comes apart in space on the way to another planet, it can't be fixed and the mission might be lost.
If it takes weeks, months, or years for changes to be deployed, it makes a lot of sense to invest in procedures that improve quality before shipping, as responding to issues is expensive.
If it takes minutes or seconds to deploy changes, measuring quality in production becomes more reasonable. Some changes are such that errors are likely to result in expensive cleanup, and those need extenstive testing before release... But things where the effects of errors will be mild can be pushed and if errors appear, a rapid response is often fine. It takes experience/wisdom to know which changes need more qualification and which are safe to try... and experience hopefully reduces errors too.
> The idea that you can apply lower-quality engineering practices in some systems is totally wrong, I think.
But we obviously do this - practices are significantly different between beta release webapps and software on rivers we send to Mars. And that’s a good thing, the tradeoffs are wildly different.
> Isn't this just another representation of the fallacy that it's possible to deliver faster by cutting quality?
Do you think there’s anything that would increase quality on a project that you are on but also slow it down?
It’s not always true - more haste less speed - but it’s not always false either.
After having worked four years with trunk based development and enjoying it very much, I worked on a project were it was required to have one branch per feature for review purposes. It caused lots of delays, temporary work arounds, merge conflicts, moving changes between branches, bugs being introduced, due to reviews being delayed. I got the impression it made development twice as slow and did not improve code quality.
It's funny as i get older to see how all these patterns just repeat themselves. When i started out i worked in a team that used subversion and it was by default trunk-based development flow. no code reviews except the single big changeset from all devs. it was how things were done back then by many teams. Then there was a shift towards feature-request-based development with the rise of git and github. trunk-based development was now seen as "unprofessional" in corporate. totally fine and i was there to teach this mind shift.
while the feature-branch-based flow solved some problems it introduced many more like stall branches and out-of-sync code. this became the default as most devs simply weren't aware or had the discipline to merge early and cut their "tickets" to an appropriate size.
mostly for compliance reasons in corporate and big teams i would still advise feature-request-based flow. but as another colleague noted here - it slows you down and if it's a one or two people shop / product you will reap more benefits by doing basic trunk-based flow.
What they are describing as "Trunk-Based Development" seems a little extreme, and frankly, I'm not sure how achievable it is. Generally, when I've seen/implemented TBD it has involved Pull Requests and code reviews (they seem to imply it shouldn't). With their definition, I think a few issues arise:
* Even if you think you can have high quality without PRs, in many industries, it's part of security/compliance.
* They say that PRs undermine shared ownership, when actually, PRs done right can enable shared ownership (yes, I can't just merge whatever I want, but also, I get to help ensure no one merges whatever they want).
They also say that "no branches are created". In reality, I don't see how you do any modern development without branches. If anything, contributors need a local branch that is then merged to the trunk (unless, somehow, everyone is live-editing the main branch?). The idea of TBD is just to have smaller, short-lived branches that are frequently merged into main (very doable with pull requests).
Perhaps I can shed some light into your questions as I work on TBD since 3 years now.
In our case there are no Pr's, we push to master, the test suite is triggered, the dev environment receives the new code. It moves to other environments if the tests pass. We release to master using a canary instance that gets only a small share of the traffic, if our metrics for success fail then we rollback and start a hotfix.
Yes you can have as many branches you want in your local but in the remote there is only one.
You make sure it works first. Are you opening PRs with bugs “all the time”? Why? Nothing is ever bulletproof, but bugs shouldn’t be the norm in new code.
It is indeed extreme and very limited to writing web applications in a specific context. That is not how I have been working for the past 20 years, and his solution would be destructive in my daily job.
One of his previous article (https://thinkinglabs.io/articles/2022/05/30/on-the-evilness-...) takes it one step further. It seems that everyone works on everything at once and people step on each other toes all the time. They would definitely improve by using the feature branches that they shun.
Last but not least, he's talking about quality, high-trust, or on-demand build which are topics unrelated to a branching strategy. Quality is done with local tools like linters and CI to prevent bad merges, and on-demand build can be done with tags in any branching strategy. The whole article is confusing actually.
> everyone is working on a single mainline
It's the same for every strategy. And TBD mentions release branches, so... he's wrong in the very first sentence.
I do like TBD though because it's a clean way to handle release and feature branches, but that's it.
> It is indeed extreme and very limited to writing web applications in a specific context
It’s not. I’ve used TBD for mobile apps and it’s been great. I do prefer short-lived branches with PRs (which is another form of TBD), but it’s wrong to say that their approach is limited to web apps in a specific context.
It definitely is. He's saying that "we uncover more problems sooner" which can only be done on systems such as web sites where the tests and their results are done very fast. Where I work we can have weeks of testing to make sure that every feature has not introduced a bug.
I'm not criticizing TBD which I like, I criticize his thoughts that it's related to TBD where in reality the whole article talks about his specific job and the organization of his team, which again is unrelated to any branching strategy.
So are you saying I just imagined using this for non-web projects?
> He's saying that "we uncover more problems sooner" which can only be done on systems such as web sites where the tests and their results are done very fast.
I don’t understand why you think this. Uncovering problems quickly is one of the major points of continuous integration and it’s in no way limited to the web.
> Where I work we can have weeks of testing to make sure that every feature has not introduced a bug.
This does not mean that branchless TBD is limited to the web, it means that it’s not suitable for your project.
You misunderstood everything to the point I think you're doing it on purpose.
I like TBD, but the article is NOT about TBD.
> are you saying I just imagined
I use TBD for everything and I like it. I was talking about the writer of the article, not you. The article is, again, NOT about TBD and the writer, most likely an overpriced consultant, is very confused about branching strategies, and most likely believe that a daily standup or some pair programming is part of that strategy. His other articles confirm this.
You said it was limited to web projects, I said I had used it in non-web projects and it wasn’t limited to web projects, and you said “it definitely is”. How else am I meant to interpret that? You are directly saying that what I have personally experienced isn’t possible.
> The article is, again, NOT about TBD and the writer, most likely an overpriced consultant, is very confused about branching strategies, and most likely believe that a daily standup or some pair programming is part of that strategy.
This does not resemble the article we are commenting on.
> If anything, contributors need a local branch that is then merged to the trunk (unless, somehow, everyone is live-editing the main branch?).
The term originates in the svn or earlier era (hence "trunk"), when "commit" and "push" were the same operation. So there wasn't really such a thing as a local branch, just uncommitted changes. And svn was able to commit changes to any files that no one else had committed to yet, it wasn't whole-repo versioning like git that required you to be fully up-to-date before pushing.
I cannot imagine any advantage of this extreme description of trunk based development over trunk based development with commit stack workflows. You dont have long lived wip but you have draft commits that can mature and be worked on in parallel (also critical for coding agents) constantly auto rebasing on each other. i cannot count the number of bugs that i found before something landed on trunk just because i let “finished” work mature in draft for a bit while i work on the next task. this is especially important if the features have overlap or include ui changes that could have glitches undetectable by tests. similarly you don’t have prs but you have review stacks that are tested and evaluated in any amount of detail required by the context BEFORE they become the new trunk tip. (only fast forward, no merge commits)
A decade in software development taught me: it always depends. It’s never a good idea to speak in absolutes.
Your development workflow depends on your product, on your team, their skills, on your organization and so on.
As with everything you need to carefully evaluate, if Trunk-based development is a good fit for your product, your team and your organizational structure. Sometimes it fits well, sometimes it’s not a good fit.
I think a lot of people dismiss gitflow and its feature-branched model as too complicated. Yes, it feels so slow, right?
But it gives inexperienced teams a very clear and documented workflow, especially so, if you need to support multiple versions of your product in the wild. It clearly dictates how to do bugfixes and hotfixes, that need to be done in multiple releases of your product.
Could you achieve the same with Trunk-based development? Maybe? Could an inexperienced team come up with it on their own? I don’t think so.
I don't get this trust discussion. I don't trust myself not to do a mistake, why should I trust somebody else.
Also - while pair programming is great - it does not replace a code review. When you do pair programming you start working on the problem at the same time, you get tiered and annoyed of the problem at the same time. You want to go to lunch break at the same time. All reasons to slip in a sloppy solution and convince each other it is actually ok.
While doing code review you get a fresh perspective on the matter.
> Note that the Pull Requests were introduced by the Open Source Software community.
"Pull requests" by that name maybe. But code reviews existed way before that. As soon as code was written, programmers started to ask for peer reviews of their code.
OK it is faster as article claims but I want to see the tickets, team composition, how long they work together, team dynamics, assholes blocking PRs vs assholes pushing stuff to trunk „because they know better”.
Amount of work that was done and amount of bug reports and anyone can write that their customers are happy.
If they worked previously on PRs some numbers how that was going compared to what they do now.
For me it is just fluff and I wasted time reading it.
> When we do trunk-based development, the WIP we commit gets used, before any actual user sees it, by our whole team.
I think this is where I'm stuck. I rarely work in projects where my team can 'use' the code in any meaningful way. When I wrote code for accountant my team wasn't going to be able to tell if the wash sale calculations went wonky.
Had to look up what trunk-based development is. It's where you have only one eternal integration branch, possibly longer lived release branches and short-lived feature branches.
Yes, this is the best way. Having multiple eternal branches like git flow is complete nonsense. Just don't do it. You do not need anything so ridiculous.
I joined one team that used eternal branches to deploy to various environments (staging, prod etc). What a nightmare! You could just leave a commit out of prod and be none the wiser.
With a single integration branch you either do continuous deployment to prod or you use tags to mark releases. No need for another branch. Release branches are there if you need to patch things but don't want to release everything on master. You just tag a release on the release branch.
I disagree with the linked article that says DVCS has made more branching common. Branching was never difficult, merging was. Git in particular made short lived branches a thing because now they are cheap enough to merge every day.
Is it really true that Google has no feature branches? Sounds unlikely to me. How do you do code reviews for example if everything goes straight to main?
Yes, Google uses trunk based development for almost everything. Even Android is now on trunk based development (previously a branchy mess) and heavily uses sets of feature flags for promotion / demotion through quality lifecycles.
I’m not sure why TBD should have anything to do with code reviews. You aren’t able to submit to main without first uploading a change or stack of changes to code reviews, then you submit the changes after they are approved. That has absolutely nothing to do with your branching model. Just think of it as a rebase request instead of a pull request. PR was always the wrong terminology anyway.
> Just think of it as a rebase request instead of a pull request. PR was always the wrong terminology anyway.
I kinda feel like your terminology is mixed up... You start off describing branches, but then say "pull" request and that it was the wrong terminology?
A pull request is requesting a pull from a separate forked repo.
A merge request is requesting a merge from a branch within the same repo.
And:
> You aren’t able to submit to main without first uploading a change or stack of changes to code reviews, then you submit the changes after they are approved.
How is this functionally different from a merge request? It seems like manually doing the exact same thing.
I once worked at a company (4 engineers including the CTO at the time). The CTO urged everyone to push to master, and called me out sarcastically asking, "Why don't you push to master?"
I responded, "Because that's stupid."
I'm not very eloquent and I don't work there anymore. They are currently valued over a billion dollars (for now).
TBD as described here is what everyone basically used to do in the olds with SVN (because branches were so hard). PRs are objectively better, just have to complete them on a regular basis. The article conflates PRs with long running branches.
They were also slow because, if memory serves, creating a branch involved creating a logical copy everything (files and history) instead of just tracking the delta using pointers/copy-on-write/whatever-clever-thing locally. It was on the order of tens of minutes in one project I worked on and, needless to say, we stopped doing it. Merging the branch was also (as noted) a headache when this all ran in reverse. Merge conflicts were also much more common, though I can't remember why just now -- presumably because synchronizing changes was equally slow and painful, rebasing was not an option and people couldn't be bothered.
It's hard to emphasize just what a leap forward Git and Mercurial were. In addition to quick and cheap branches, you could also commit changes while working offline! The distributed nature of these tools was also revelatory and is underutilized in the modern age.
I don't remember it being particularly slow on a large code base. It was tricky however, I can't remember the details but it was only ever done if absolutely needed. So, according to this article the bad old days of svn are actually what they want.
> With trunk-based development, we naturally commit (and push) more frequently to mainline. If we commit repeatedly to a remote mainline, we produce more frequent builds. If we produce more frequent builds, we uncover more problems sooner, allowing us to fix them when they are still small and easy to fix. This enables us to Build Quality Into the product instead of testing quality later, hence leading to better quality.
Strawman, cicd can be set up such that branches build and test too.
I’m having a hard time figuring out what they mean by trunk based development here. They’re too deep in, you gotta zoom out a bit and tell folks what the problem is.
My experience is you have to juggle a lot of flags and need to sometimes pay more to keep your changes isolated. Theres also the issue of how you test these in your dev env, since you can usually only have a few permutations active
The article title isn't misleading at all: they list the (subjective) benefits without listing any of its disadvantages. Sure, that list is useful for some trivial projects.
But good luck selling that to some regulated companies where basic risk assessment is a must. Tell a financial institution you'll _never_ fuck their money transfer pipeline, because you don't do code reviews, you "deliver value" even on Fridays and your methodology avoids all sort of errors or malicious modifications.
IMHO, trunk based development removes some safety. Software development is a complex task and we all risk to break things even on a good day. You're not smarter by removing safety from the process.
If the whole team is available in one physical office, a centralized model like perforce can run circles around the various git flows. You typically see this in places like game development.
Being able to lock files centrally is a massive advantage as long as you have to be at the office to do it. This forces conflict resolution in a much more direct and immediate way than the passive aggressive PR submissions that are eventually reviewed for conflicts after all parties have already burned untold amounts of energy on potentially unmergable code.
> Note that the Pull Requests were introduced by the Open Source Software community. It allows core teams to maintain a system and accept contributions from the outside world. The outside world is generally perceived as low-trust. Therefore, we want to verify those contributions. Pull Requests are pretty helpful in that regard. However, in a closed-source corporate setting, the Pull Request model essentially indicates that the team owns the codebase, but it is not allowed to contribute.
I think this is a very lopsided way of looking at pull requests. They are about a lot more than just trust. Reviewing and being reviewed is a great way of learning from colleagues, making common practices gel in a team, and keeping up to date with changes to the codebase. It’s not just a barrier.
> money stuck into the system. It is stuck because the organisation invested considerably in creating all this code on parallel branches. However, as long as this code is not merged into mainline, deployed in production and released to users, it does not generate any revenue. Therefore, it is money stuck in the system. But, because we have less WIP, we create less inventory. As such, we have less invested money stuck in the system.
I’ve been telling people that work in flight should be minimised for a long time, but this is the first time I’ve heard the inventory metaphor and I like it. It’s a similar metaphor to tech debt.
> I think this is a very lopsided way of looking at pull requests. They are about a lot more than just trust. Reviewing and being reviewed is a great way of learning from colleagues, making common practices gel in a team, and keeping up to date with changes to the codebase. It’s not just a barrier.
I am old enough that my first 2 jobs were subversion-based (repo side, client side git-svn) and both teams didn't care about branching. It might have to do with how awkward subversion branches felt.
Anyway, we would commit (in git lingo pull-rebase & commit) directly and we basically maintained the id of the last reviewed commit and jointly did PR reviews commit-by-commit with the code on the projector in the office.
We had a joint look at code, everyone voiced their feedback ("I don't understand a variable name like `xzc`"), "where is the unit test", "I read in a blog post recently you are supposed to not use classes...". etc. pp. Sometimes fixes would be pushed right in the PR review session so you had the variable renamed 4 commits further.
Anyway, in retrospect it worked surprisingly well at helping the team to develop a joint understanding of values & virtues that the team would like to maintain in their code base. This might of course be nostalgia of a dev looking back into their junior years.
When we finally got pull-requests, we really felt thrown into the future. It was just great. But after a while I started to miss the direct conversations about code with fellow humans.
And honestly I couldn't tell whether PR really improved the quality of the code base in the long run. They lowered the probability of bad code being committed to the code base, but also lowered the probability for a dev to just fix awkward things while they stumbled over them.
I miss that too. There’s so many little micro-improvements that are possible when you’re reading code— clarifying a variable name, expanding a comment, adding a test case, and it’s unfortunate when those things are lost under the inertia of it being a pain in the butt to create PRs and get them reviewed.
At the same time, I expect your PM is delighted that you’re not wasting time getting distracted with all that yak shaving nonsense and are instead working on the next burndown ticket that has been assigned a t-shirt size and the appropriate number of story points.
Same here. At my previous employer, we’d pair regularly, even on PRs. Larger blocks of work would warrant a mini-standup so the team could all review together in real-time (after an async review with comments). It’s an efficient use of time while acknowledging the need for real-time, humane communication.
> When we finally got pull-requests, we really felt thrown into the future. It was just great. But after a while I started to miss the direct conversations about code with fellow humans.
Why is this disjoint though?
Our newer colleagues are currently bringing in their first bigger contributions and changes to the config management. We're using short-ish lived feature branches of a week or two with pull requests.
This is good, because I can spend some time understanding what they are doing and I can prepare some ideas and examples for improvements, before we have a call to talk about these changes.
I'm also entirely willing to move my bigger changes into branches and a PR and spend some afternoon with the team to talk about good practices, like structuring commits, naming, ansible code structure, ... and see if other people enjoy that as well. Management wants more stability and broader understanding of our code bases, so moving slower and reading and discussing more code seems up that alley.
I don't like the inventory analogy. Code changes are the aggregation of keyboard events. At what point do these events become "inventory"? Every key press, every point at which the code compiles, every day, when the feature is done? It is still very clearly work in progress if it exists on a branch or behind a feature flag.
Since people usually start out writing code on their own, the tendency is to wait until the feature is done before committing. Therefore, some pressure should exist to try to get things merged because that is actually optimal on a team. However, it doesn't need to be dogmatic or drag in analogies from the manufacturing industry.
Technical debt is also a bad analogy. There is just the current state, the desired state, the set of activities required to go from one to the other and their associated cost. Sure, every now and then something hacky is done in order to (for example), land a customer which you plan to get back to (i.e. take on debt and pay it back later). It isn't that common however so the analogy almost never applies.
How about we just stop using analogies in software altogether?
My rule is if I’ve had a WIP branch for more than a day without making progress on it, I ask one of my coworkers if they’d like to tag in on it, or my boss to see if he feels like looking at it with me. It’s actually a really hard thing to do, there’s this inertia to reaching out that I hate feeling mired in - but every single time I’ve done it, it’s been okay. It’s really the times when I haven’t reached, in retrospect, when things have not gone okay.
How is Gerrit doing these days; is it any good for breaking away from GitHub? I'm looking for a git/mailing-list solution a small business could use on-prem but not any of that Gitlab nonsense (to throw people off-balance, if you will)
GitLab is pretty decent; I like the reviewing a bit better than GitHub, for example how non-code comments still form threaded discussions, or how you can set it to block merge until all discussions are marked “resolved”.
Yes GitLab has a lot to it, but much of it is optional; you won’t be hurting if you just stick to the code repo part.
> this is the first time I’ve heard the inventory metaphor
Interesting, I was under the impression that this analogy was a core component of "lean software".
So it is! It’s been a while since I’ve read anything on Lean. Thanks to your nudge, I found this Spolsky article that expands on the metaphor:
https://www.joelonsoftware.com/2012/07/09/software-inventory...
Used trunk-based dev a lot. Not a fan. It sounds good on paper, but in practice, it gets messy fast and code quality is hard to maintain, especially when teams are remote.
What actually works is trunk-based deployments — keep main always deployable, and ship from there. Simple.
PRs, are underrated. They’re great for sharing context. You get inline comments, CI runs, you can test stuff in isolation by spinning up infra, and teammates actually see what’s changing.
Stacked diffs make juggling multiple PRs manageable. And yeah, PR reviews can slow you down, but honestly, I think that's a plus. Slowing down just enough to have another human look at the code pays off more often than not.
I don’t understand why PRs are being discussed here at all – doesn’t anyone serious use trunk based development with code reviews? How is that different from a PR? Seems like we need to disentangle the discussion of whether code review is good from whether feature branches vs trunk based dev are good. Those are completely unrelated questions to me.
I think there are a lot of interesting questions about using feature flags (a baby branch) vs actual branches. Personally I’m pro flags and anti branches, after a lot of experience in developer tools and CI.
Definitely feature flags are the way to go. Having those code paths integrated early doesn't cost much and makes it basically free when you want to "merge" the feature into default.
This does require your software to have a decent architecture such that feature flags aren't littering every part of your codebase, though. Ideally you want something like a whole module/plugin being enabled/disabled right at the entrypoint of your program/module. But this also pays dividends in the long run.
There are different flavours of TBD. Some use short-lived feature branches, some commit directly to master. The article advocates for the latter.
PRs basically have a hard-requirement on branches (or equivalent, like fork), because the code that is being requested to pull needs to be available somewhere. The article also advocates for not using pull requests.
However people who take this position also often advocate for post-merge code review. And more advocate for pair programming or mob programming, which they consider to represent code review as well. So branchless TBD isn’t incompatible with code review, just code review as it is commonly practiced.
I'm pretty sure trunk-based development includes using feature branches and MRs/reviews etc. I didn't get the impression we were always supposed to commit directly to master (although that is possible for trivial things which reduces overhead).
Is everyone talking about the same thing here?
There are different flavours of TBD. Some use short-lived feature branches, some commit directly to master. See:
https://trunkbaseddevelopment.com/
If your team is very small then strictly creating feature branches and pressing the merge button immediately because there's nobody left to review it is silly. I like that it gracefully degenerates into just committing directly to the trunk when it makes sense.
This works with git (or DVCS in general) because your local "master" or whatever is a short lived branch. People seem to forget but with git you are branching every time you clone. Obviously committing broken stuff and WIPs directly to the trunk isn't going to work.
I wonder how all these absolutist methodologies really work when you take subjectivity and our general inability to measure anything reliably away? I say this because we've tried all of them after someone read an article on a blog saying it'd solve all our problems. And we always have the same problems afterwards despite people promoting that it solved them. Sometimes even worse ones.
The root cause in my company is we hire the lowest cost outsourcing outfit we can find, which hires inexperienced juniors and pretends they are senior and has a huge staff turnover because who the hell wants to work like that for more than 12 months. But of course changing the repo structure and process will compensate for that.
My observation is that a good engineering team will self-organise a methodology that works. The methodology is unimportant compared to getting a good engineering team. I am fortunate that I work in a small niche pocket in the organisation that has good engineers and has resisted normalising with the process churn of the rest of the organisation.
We run a "dark ages" team really. We were on SVN until 2020-ish, because, well it worked fine for us. Now we're on github on commandment but it is used like SVN. We have light feature branches. Master is always stable. Releases are tagged off master. At no point do we use any other github features not even PRs. CI is makefiles. Push to ECR/S3 by human. Deployment is makefiles. Comms is email because Slack is too distracting and demands immediacy. Issue tracking is a 17 year old JIRA project with a default workflow. We have the lowest defect rate, the lowest time to market and highest ROI per head in the entire org and we don't have or need a platform/sre/devops team supporting us.
We are a disliked because we didn't do anything for over 10 years other than deliver software.
In my (only 2 decades of) experience, almost all "processes" and "methodologies" exist to compensate for lack of skill, care and/or understanding. As you say, what/how you do may sound outdated to others but it works because you care about what you do.
I think this applies to many "best practices" and "standards" as well. I could mention many examples of old systems/apps written by single/few deeply committed and skilled people were replaced by "better" and more "modern" alternatives that worked less well and that took a team of people to maintain. The latter used all the right tools, processes and practices but they were just not artists, for lack or a better word.
Not well
- Posted from my 45 minute long standup
My commiserations.
We don't do standups. Drop 15 mins in whoever's calendar and use the phone works. Yes we're that backwards! Every alternate Friday lunch time we meet up in person though.
It's not that bad I typically get a lot done. Nobody can sign me up for meetings I need to participate in while I'm in standup.
After a couple of decades in the Danish software industry I've come to accept that there is only trunk-based development (COME AT ME!). Feature branches, set releases and so on are fake safety gaps that people add when they don't trust themselves and/or their developers. These days I view it similar to someone who won't deploy changes on fridays because they fear for their weekend. If you're too afraid to deploy on a friday, then you should not be allowed to deploy on a monday. Similarily, if you're not brave enough to push changes to production, then your changes aren't ready.
Of course I don't actually think there is a religion on how to do these things, but I've never seen any particular quality added by people doing quarterly releases from a feature branch. For the most part I think this part at the very beginning of the article explains a lot of the reason as to why that is:
> When we do trunk-based development, the WIP we commit gets used, before any actual user sees it, by our whole team.
In my eyes all of it comes down to management much more than deployment strategies. The disadvantage of feature branching is that it's very easy for management to cut costs on testing. The disadvantage of trunk-based development is that it requires functional teams of people who can actually work together and accept the "norms".
Our team has thorough, high quality, automated tests. In the past 5 years we have never released such a badly broken system that emergency remedial work was required. We still don't release on Fridays. It's just common sense, risk management and (most importantly) respect for the developers home and family time.
I'd argue that it depends on your users. From a business perspective, it's a lot safer for us to release on fridays because failures would not affect a lot of people over the weekend. Of course I do live in a country where you'd have to agree to fix it over the weekend, and if you do, you'd be compensated with new days off + money.
Having literally no branches might work for tiny projects, but none of this is the case for any decently-sized software.
If you want to release a piece of software to a customer who will be using it as-is for some time (which is one possible business model), then you need a version of your software where all features that the customer sees are working well. To do that, you need to fix even minor bugs in those features without adding any (visible) new features to the product.
However, this doesn't scale. You can't pause all development of new features while someone is re-aligning a few UI elements and cleaning up error messages that were too confusing. So, you have to create a split in the software: one part is getting ready to ship, another is work in progress for future releases.
Now, you can create this split using git branches, or using feature flags. So yes, trunk based development can work, you can absolutely build up the new experimental features into the software you're about to ship, but you'll have to hide them behind a feature flag that is disabled - a runtime branch. Or, you can have a release branch where only bug fixes are allowed, and submit major new work on the main branch. There are pros and cons to either solution, but whatever you do, you need stable vs unstable branches somewhere.
And, of course, sometimes you'll have to release hotfixes, and there you won't get away without a true SCM branch, since you can't just give your customers your latest and greatest software just to patch a security issue in libcurl for a 2 year old release.
> The disadvantage of feature branching is that it's very easy for management to cut costs on testing.
Have to agree with this. In a large (lower than 2/3rd in the Fortune 500 list) company with cost cutting as a constant mantra, our ever shrinking team hasn’t had a testing/QA team for several years. We never got to automating a lot of tests either. The developers create something and test it manually as per their limited knowledge. Then it’s the end users who do the bulk of the testing.
> When we do trunk-based development, the WIP we commit gets used, before any actual user sees it, by our whole team.
And of course internal teams love also doing QA on the side of their existing jobs ;) I think that’s fair enough to ask from developers and project/product managers but a lot of others on the team basically have to suffer lower productivity to test your code.
It probably depeds on the kind of software you create, but in our case:
When i add non-trivial code I want others of my team to review it first. When it is on trunk, it gets less easier to refactor as others already might use it. In this sense using a MR is really good way to get some quality checks before it goes to main and needs to be supported (in most cases) indefinitely. Next, we have system tests that i really want to avoid breaking on main and getting EVERYONE stuck.
The problem arises if I keep that branch for more than a week. Merge conflicts get harder, the financial investment is not leveraged, etc etc.
The problem is with long lived branches. This is something you should avoid, you can take it to the extreme, no branches. But extreme is almost never good.
PS CC is really good at rebasing and resolving merge conflicts.
I think the article misunderstood the debate... the debate is (or at least was) about gitflow, and specifically about having git branches per environment. So there are git branches for dev, staging, etc, all the way to prod. And git changes become deployed on the respective branch.
Trunk based development is about having a single master (sorry, "main"), and generating deployable artifacts from there, and then the remaining environments have only deployments of versioned artifacts.
Unless I'm missing some new debate about the value of PullRequests, but that sounds extreme.
> With trunk-based development teams, managing technical debt becomes a non-issue. It just happens.
Unless it doesn't happen. Issues that are dealt with automatically are those that would break the build, or make tests or linting checks fail; everything else can simply accumulate without anyone "wasting time" on it.
Technical debt is often recognized retroactively (e.g. there is a new foreign customer and not enough time to write thousands of neglected translated labels and messages).
I'm a big fan of trunk based development in the Churchillian sense. It's the worst form of development, except for all the others.
I don't think it's for every project, or even every feature in a given project. But there's a niche it works best for, and that niche happens to represent a massive percentage of all software engineering.
The niche is, you're building a web app. It's some shitty Web 8.0 version of a Craigslist or something. No one wants to pay for formalized testing. No one really wants to spend a lot of time on quality. Arguably it's not that important to users. The year is Enshittified 2030 and they're lucky if they can even log into their online banking, let alone slide into someone's DMs on your platform.
What's the solution? TBH it's not tripling down on good engineering process. (Don't get me wrong, good engineering process absolutely has a role, I'm merely suggesting that as you move from flight control software to online classifieds to your neighbor's recipe blog, that role diminishes in favor of less formal feedback.) The solution is radically forcing the code into the light of day in front of people who can say or do something about the biggest problems. The minor bugs will probably slip through this net, but the really nasty ones WILL get worked out if people on your team have to deal with them first, followed by an 'insider' circle of users, finally followed by the general public.
That translates to certain buzzwords actually representing great ideas in this type of development, such as: feature flags, CI, pilot rollouts, and... trunk-based development.
Just get the code/feature out in front of a couple people who matter, as early as the day it was written, and keep expanding until you're public.
The idea that you can apply lower-quality engineering practices in some systems is totally wrong, I think. Good engineering practices will help you retain good developers, will allow you to maintain a sustainable development pace and will prevent you annoying your users so much that they stop using your system. I can't imagine any commercial project where this doesn't matter.
Isn't this just another representation of the fallacy that it's possible to deliver faster by cutting quality? That's the kind of thing I expect to hear from ignorant stakeholders who know nothing about developing software, not on HN.
> The idea that you can apply lower-quality engineering practices in some systems is totally wrong, I think.
> Isn't this just another representation of the fallacy that it's possible to deliver faster by cutting quality?
If we use a real world analogy. You don't use the same engineering quality practices to build a sandwich as you do to build a space probe.
It would be a mistake to x-ray verify connections between parts in a sandwich, but it would most likely be a mistake not to for a space probe.
The engineering practice that needs to be done correctly is judging the error risk, predicting the consequences of errors, and mitigating errors where the cost of mitigation does not outweigh an estimate of the expected cost of damage from the error, including work to make things right.
If you ship a sandwich where the spread was unevenly applied, inspection likely could have detemined this before shipping, but the cost to have another person carefully review each sandwich as it is being made outweighs the cost of the ocassional dry spot on the bread... Which may not be noticed or may not be bothersome... Worst case, a sandwich can be remade.
Otoh, if an assembly comes apart in space on the way to another planet, it can't be fixed and the mission might be lost.
If it takes weeks, months, or years for changes to be deployed, it makes a lot of sense to invest in procedures that improve quality before shipping, as responding to issues is expensive.
If it takes minutes or seconds to deploy changes, measuring quality in production becomes more reasonable. Some changes are such that errors are likely to result in expensive cleanup, and those need extenstive testing before release... But things where the effects of errors will be mild can be pushed and if errors appear, a rapid response is often fine. It takes experience/wisdom to know which changes need more qualification and which are safe to try... and experience hopefully reduces errors too.
> The idea that you can apply lower-quality engineering practices in some systems is totally wrong, I think.
But we obviously do this - practices are significantly different between beta release webapps and software on rivers we send to Mars. And that’s a good thing, the tradeoffs are wildly different.
> Isn't this just another representation of the fallacy that it's possible to deliver faster by cutting quality?
Do you think there’s anything that would increase quality on a project that you are on but also slow it down?
It’s not always true - more haste less speed - but it’s not always false either.
After having worked four years with trunk based development and enjoying it very much, I worked on a project were it was required to have one branch per feature for review purposes. It caused lots of delays, temporary work arounds, merge conflicts, moving changes between branches, bugs being introduced, due to reviews being delayed. I got the impression it made development twice as slow and did not improve code quality.
It's funny as i get older to see how all these patterns just repeat themselves. When i started out i worked in a team that used subversion and it was by default trunk-based development flow. no code reviews except the single big changeset from all devs. it was how things were done back then by many teams. Then there was a shift towards feature-request-based development with the rise of git and github. trunk-based development was now seen as "unprofessional" in corporate. totally fine and i was there to teach this mind shift.
while the feature-branch-based flow solved some problems it introduced many more like stall branches and out-of-sync code. this became the default as most devs simply weren't aware or had the discipline to merge early and cut their "tickets" to an appropriate size.
mostly for compliance reasons in corporate and big teams i would still advise feature-request-based flow. but as another colleague noted here - it slows you down and if it's a one or two people shop / product you will reap more benefits by doing basic trunk-based flow.
What they are describing as "Trunk-Based Development" seems a little extreme, and frankly, I'm not sure how achievable it is. Generally, when I've seen/implemented TBD it has involved Pull Requests and code reviews (they seem to imply it shouldn't). With their definition, I think a few issues arise: * Even if you think you can have high quality without PRs, in many industries, it's part of security/compliance. * They say that PRs undermine shared ownership, when actually, PRs done right can enable shared ownership (yes, I can't just merge whatever I want, but also, I get to help ensure no one merges whatever they want).
They also say that "no branches are created". In reality, I don't see how you do any modern development without branches. If anything, contributors need a local branch that is then merged to the trunk (unless, somehow, everyone is live-editing the main branch?). The idea of TBD is just to have smaller, short-lived branches that are frequently merged into main (very doable with pull requests).
Perhaps I can shed some light into your questions as I work on TBD since 3 years now.
In our case there are no Pr's, we push to master, the test suite is triggered, the dev environment receives the new code. It moves to other environments if the tests pass. We release to master using a canary instance that gets only a small share of the traffic, if our metrics for success fail then we rollback and start a hotfix. Yes you can have as many branches you want in your local but in the remote there is only one.
And how is that different than just doing a pr, and merging to trunk?
It feels like you are just canceling a lot of pipeline tests all the time, assuming there's a cadence of 10+ pushes per day.
> In our case there are no Pr's, we push to master
Just to clarify, are you saying there is no code review where another engineer looks at your change before you merge to main?
I’ve never seen a TBD project work that way. I can’t imagine how it would work for anything but the smallest teams.
In our case we are 3 devs.
I have been in projects where I am the sole dev, and I still create PRs and self review them before merging.
I do this because I am human and make mistakes.
> we push to master, the test suite is triggered
If you have a bug due to the last commit, don't you have to write fixes all the time? How do you review the code?
You make sure it works first. Are you opening PRs with bugs “all the time”? Why? Nothing is ever bulletproof, but bugs shouldn’t be the norm in new code.
It is indeed extreme and very limited to writing web applications in a specific context. That is not how I have been working for the past 20 years, and his solution would be destructive in my daily job.
One of his previous article (https://thinkinglabs.io/articles/2022/05/30/on-the-evilness-...) takes it one step further. It seems that everyone works on everything at once and people step on each other toes all the time. They would definitely improve by using the feature branches that they shun.
Last but not least, he's talking about quality, high-trust, or on-demand build which are topics unrelated to a branching strategy. Quality is done with local tools like linters and CI to prevent bad merges, and on-demand build can be done with tags in any branching strategy. The whole article is confusing actually.
> everyone is working on a single mainline
It's the same for every strategy. And TBD mentions release branches, so... he's wrong in the very first sentence.
I do like TBD though because it's a clean way to handle release and feature branches, but that's it.
> It is indeed extreme and very limited to writing web applications in a specific context
It’s not. I’ve used TBD for mobile apps and it’s been great. I do prefer short-lived branches with PRs (which is another form of TBD), but it’s wrong to say that their approach is limited to web apps in a specific context.
It definitely is. He's saying that "we uncover more problems sooner" which can only be done on systems such as web sites where the tests and their results are done very fast. Where I work we can have weeks of testing to make sure that every feature has not introduced a bug.
I'm not criticizing TBD which I like, I criticize his thoughts that it's related to TBD where in reality the whole article talks about his specific job and the organization of his team, which again is unrelated to any branching strategy.
> It definitely is.
So are you saying I just imagined using this for non-web projects?
> He's saying that "we uncover more problems sooner" which can only be done on systems such as web sites where the tests and their results are done very fast.
I don’t understand why you think this. Uncovering problems quickly is one of the major points of continuous integration and it’s in no way limited to the web.
> Where I work we can have weeks of testing to make sure that every feature has not introduced a bug.
This does not mean that branchless TBD is limited to the web, it means that it’s not suitable for your project.
You misunderstood everything to the point I think you're doing it on purpose.
I like TBD, but the article is NOT about TBD.
> are you saying I just imagined
I use TBD for everything and I like it. I was talking about the writer of the article, not you. The article is, again, NOT about TBD and the writer, most likely an overpriced consultant, is very confused about branching strategies, and most likely believe that a daily standup or some pair programming is part of that strategy. His other articles confirm this.
You said it was limited to web projects, I said I had used it in non-web projects and it wasn’t limited to web projects, and you said “it definitely is”. How else am I meant to interpret that? You are directly saying that what I have personally experienced isn’t possible.
> The article is, again, NOT about TBD and the writer, most likely an overpriced consultant, is very confused about branching strategies, and most likely believe that a daily standup or some pair programming is part of that strategy.
This does not resemble the article we are commenting on.
> If anything, contributors need a local branch that is then merged to the trunk (unless, somehow, everyone is live-editing the main branch?).
The term originates in the svn or earlier era (hence "trunk"), when "commit" and "push" were the same operation. So there wasn't really such a thing as a local branch, just uncommitted changes. And svn was able to commit changes to any files that no one else had committed to yet, it wasn't whole-repo versioning like git that required you to be fully up-to-date before pushing.
I cannot imagine any advantage of this extreme description of trunk based development over trunk based development with commit stack workflows. You dont have long lived wip but you have draft commits that can mature and be worked on in parallel (also critical for coding agents) constantly auto rebasing on each other. i cannot count the number of bugs that i found before something landed on trunk just because i let “finished” work mature in draft for a bit while i work on the next task. this is especially important if the features have overlap or include ui changes that could have glitches undetectable by tests. similarly you don’t have prs but you have review stacks that are tested and evaluated in any amount of detail required by the context BEFORE they become the new trunk tip. (only fast forward, no merge commits)
A decade in software development taught me: it always depends. It’s never a good idea to speak in absolutes.
Your development workflow depends on your product, on your team, their skills, on your organization and so on.
As with everything you need to carefully evaluate, if Trunk-based development is a good fit for your product, your team and your organizational structure. Sometimes it fits well, sometimes it’s not a good fit.
I think a lot of people dismiss gitflow and its feature-branched model as too complicated. Yes, it feels so slow, right?
But it gives inexperienced teams a very clear and documented workflow, especially so, if you need to support multiple versions of your product in the wild. It clearly dictates how to do bugfixes and hotfixes, that need to be done in multiple releases of your product.
Could you achieve the same with Trunk-based development? Maybe? Could an inexperienced team come up with it on their own? I don’t think so.
See, it always depends.
I don't get this trust discussion. I don't trust myself not to do a mistake, why should I trust somebody else.
Also - while pair programming is great - it does not replace a code review. When you do pair programming you start working on the problem at the same time, you get tiered and annoyed of the problem at the same time. You want to go to lunch break at the same time. All reasons to slip in a sloppy solution and convince each other it is actually ok.
While doing code review you get a fresh perspective on the matter.
> Note that the Pull Requests were introduced by the Open Source Software community.
"Pull requests" by that name maybe. But code reviews existed way before that. As soon as code was written, programmers started to ask for peer reviews of their code.
- Design and code inspections to reduce errors in program development. 31 December 1976. https://ieeexplore.ieee.org/document/5388086
But I guess that today's programmers think that software engineering was created in 2010.
OK it is faster as article claims but I want to see the tickets, team composition, how long they work together, team dynamics, assholes blocking PRs vs assholes pushing stuff to trunk „because they know better”.
Amount of work that was done and amount of bug reports and anyone can write that their customers are happy.
If they worked previously on PRs some numbers how that was going compared to what they do now.
For me it is just fluff and I wasted time reading it.
> When we do trunk-based development, the WIP we commit gets used, before any actual user sees it, by our whole team.
I think this is where I'm stuck. I rarely work in projects where my team can 'use' the code in any meaningful way. When I wrote code for accountant my team wasn't going to be able to tell if the wash sale calculations went wonky.
Had to look up what trunk-based development is. It's where you have only one eternal integration branch, possibly longer lived release branches and short-lived feature branches.
Yes, this is the best way. Having multiple eternal branches like git flow is complete nonsense. Just don't do it. You do not need anything so ridiculous.
I joined one team that used eternal branches to deploy to various environments (staging, prod etc). What a nightmare! You could just leave a commit out of prod and be none the wiser.
With a single integration branch you either do continuous deployment to prod or you use tags to mark releases. No need for another branch. Release branches are there if you need to patch things but don't want to release everything on master. You just tag a release on the release branch.
I disagree with the linked article that says DVCS has made more branching common. Branching was never difficult, merging was. Git in particular made short lived branches a thing because now they are cheap enough to merge every day.
Is it really true that Google has no feature branches? Sounds unlikely to me. How do you do code reviews for example if everything goes straight to main?
Yes, Google uses trunk based development for almost everything. Even Android is now on trunk based development (previously a branchy mess) and heavily uses sets of feature flags for promotion / demotion through quality lifecycles.
I’m not sure why TBD should have anything to do with code reviews. You aren’t able to submit to main without first uploading a change or stack of changes to code reviews, then you submit the changes after they are approved. That has absolutely nothing to do with your branching model. Just think of it as a rebase request instead of a pull request. PR was always the wrong terminology anyway.
> Just think of it as a rebase request instead of a pull request. PR was always the wrong terminology anyway.
I kinda feel like your terminology is mixed up... You start off describing branches, but then say "pull" request and that it was the wrong terminology?
A pull request is requesting a pull from a separate forked repo.
A merge request is requesting a merge from a branch within the same repo.
And:
> You aren’t able to submit to main without first uploading a change or stack of changes to code reviews, then you submit the changes after they are approved.
How is this functionally different from a merge request? It seems like manually doing the exact same thing.
I once worked at a company (4 engineers including the CTO at the time). The CTO urged everyone to push to master, and called me out sarcastically asking, "Why don't you push to master?"
I responded, "Because that's stupid."
I'm not very eloquent and I don't work there anymore. They are currently valued over a billion dollars (for now).
TBD as described here is what everyone basically used to do in the olds with SVN (because branches were so hard). PRs are objectively better, just have to complete them on a regular basis. The article conflates PRs with long running branches.
> because branches were so hard
They were also slow because, if memory serves, creating a branch involved creating a logical copy everything (files and history) instead of just tracking the delta using pointers/copy-on-write/whatever-clever-thing locally. It was on the order of tens of minutes in one project I worked on and, needless to say, we stopped doing it. Merging the branch was also (as noted) a headache when this all ran in reverse. Merge conflicts were also much more common, though I can't remember why just now -- presumably because synchronizing changes was equally slow and painful, rebasing was not an option and people couldn't be bothered.
It's hard to emphasize just what a leap forward Git and Mercurial were. In addition to quick and cheap branches, you could also commit changes while working offline! The distributed nature of these tools was also revelatory and is underutilized in the modern age.
I don't remember it being particularly slow on a large code base. It was tricky however, I can't remember the details but it was only ever done if absolutely needed. So, according to this article the bad old days of svn are actually what they want.
> With trunk-based development, we naturally commit (and push) more frequently to mainline. If we commit repeatedly to a remote mainline, we produce more frequent builds. If we produce more frequent builds, we uncover more problems sooner, allowing us to fix them when they are still small and easy to fix. This enables us to Build Quality Into the product instead of testing quality later, hence leading to better quality.
Strawman, cicd can be set up such that branches build and test too.
I’m having a hard time figuring out what they mean by trunk based development here. They’re too deep in, you gotta zoom out a bit and tell folks what the problem is.
My experience is you have to juggle a lot of flags and need to sometimes pay more to keep your changes isolated. Theres also the issue of how you test these in your dev env, since you can usually only have a few permutations active
The article title isn't misleading at all: they list the (subjective) benefits without listing any of its disadvantages. Sure, that list is useful for some trivial projects.
But good luck selling that to some regulated companies where basic risk assessment is a must. Tell a financial institution you'll _never_ fuck their money transfer pipeline, because you don't do code reviews, you "deliver value" even on Fridays and your methodology avoids all sort of errors or malicious modifications.
IMHO, trunk based development removes some safety. Software development is a complex task and we all risk to break things even on a good day. You're not smarter by removing safety from the process.
If the whole team is available in one physical office, a centralized model like perforce can run circles around the various git flows. You typically see this in places like game development.
Being able to lock files centrally is a massive advantage as long as you have to be at the office to do it. This forces conflict resolution in a much more direct and immediate way than the passive aggressive PR submissions that are eventually reviewed for conflicts after all parties have already burned untold amounts of energy on potentially unmergable code.
[dead]