Disable the check for branches to be up to date with master

[dandel10n]

I think we should consider disabling the check for branches to be up to date with master as it is more painful than helpful. Consider this scenario:

• We have 3 PRs which are up to date
• One gets merged
• Now we have 2 PRs which are out of date despite having no conflicts
• Both get updated in some way (rebase or by clicking the button - doesn’t matter)
• Both authors wait for the ci build
• One eventually gets merged
• Now we have 1 PR which is outdated

This feels like an endless cycle of updates and waiting for CI to report green. For example at the moment we have 18 PRs and the queue will become bigger the more components we add.

I suggest instead сonsider adding new (probably optional?) or modifying existing CI jobs to run tests against code merged into master (if we don't do this already). This way on the old PRs we can just run optional jobs/re-run tests before merging.

[tommcclean]

Yeah I find it frustrating too; I think it has something to do with the visual tests needing to run fresh every time. But it would be interesting to actually see if that is necessary. If it isn't possible, perhaps we can automate the merging of master and have the builds run as fast as possible so we don't notice it so much.

[siggerzz]

Originally it was put in place to ensure that when visual regression was ran, it was using the latest version of master to prevent screenshots getting out of sync. If we can achieve this without having the GitHub check, that'd work fine.

I'm just not sure how reliable the visual regression tests would be without this check.

[Preet-Sandhu]

worth doing a spike on this to see if we have alternatives, we can also have a chat with Percy team to find out how/if this can be achieved!

[dandel10n]

@siggerzz Can we run visual regression tests again after the merge? This way there will be still a risk that merged PR changed something undesired but it will be easy enough to trace and fix but we won't have to wait for days to be able to merge PRs.

[siggerzz]

We do run them after merge against master. This is the build that's 'auto approved' and used to set the baseline images.

[dandel10n]

Oh, true...

[ashleynolan]

I completely agree that it's a pain – I don't think that we should be putting the main branch in a posiiton where it could ever be failing visual regression, as doing that causes people to lose faith in the tests altogether.

I think part of the solution to this will be speeding up our build when we can do proper concurrency (whenever we get the Circle licence). Hopefully then, rebuilds will be a lot less expensive from a time perspective.

I think the other issue we have atm is that there is no true order to how we merge our PRs – it's just a case of whoever hits merge fastest once they have a passing build. For me, we should be a bit stricter in this regard – if someone has been waiting 6 hours to merge and someone else has been waiting for an hour, then the one waiting longer should be merged first. I'm not sure how we manage that though – I'm not sure if we can automate this potentially.

Thinking out loud – I wonder if there's a way of writing a github action that could take care of some of this for us (so if a PR is in a certain state, with 2 reviews and no other PRs in the same state are waiting to be merged, then auto-merge the PR and auto-updateany PRs with the new master). I think that would at least take some of the manual pain away, but I've now idea if that's possible in practice. We'd need a spike to see what options there are.