Thanks for the insightful post. I've been thinking about doing a similar thing for my CI testing. It seems like all of this depends on categories that are truly isolated, such that you are confident that changing one category will not affect another. Have you been able to maintain these clean categories like billing/checkout/search/firmware? Or have you had to deal with the occasional cross-cutting concern? I suspect the categories don't have to be perfectly isolated as long as you can catch a cross-cutting issue quickly via bisection and it doesn't happen frequently.
We use a hermetic build system (see the part about Bazel) so this is not a concern. We sometimes have the opposite issue: wide-ranging integration tests that prevent independent changes from commuting. That is solved by tactical decisions like dropping scopes in the merge queue (allowing logical conflicts) for certain tests, in exchange for adding post-merge coverage for them.
Thanks for the insightful post. I've been thinking about doing a similar thing for my CI testing. It seems like all of this depends on categories that are truly isolated, such that you are confident that changing one category will not affect another. Have you been able to maintain these clean categories like billing/checkout/search/firmware? Or have you had to deal with the occasional cross-cutting concern? I suspect the categories don't have to be perfectly isolated as long as you can catch a cross-cutting issue quickly via bisection and it doesn't happen frequently.
We use a hermetic build system (see the part about Bazel) so this is not a concern. We sometimes have the opposite issue: wide-ranging integration tests that prevent independent changes from commuting. That is solved by tactical decisions like dropping scopes in the merge queue (allowing logical conflicts) for certain tests, in exchange for adding post-merge coverage for them.