Feature request: concurrency to queue all jobs waiting on a group

[Shereef]

Current behavior

Job A arrives, finds nothing in its concurrency group and runs
Job B arrives, finds job A running in its concurrency group, looks for anyone else waiting (with the intent to kill them), finds nothing, and waits.
Job C arrives, finds job A running in its concurrency group looks for anyone else waiting (with the intent to kill them), finds job B, kills it, and waits.
Job A finishes, job C is started.

Desired behavior

• Job A arrives, finds nothing in its concurrency group and runs
• Job B arrives, finds job A running in its concurrency group, and waits.
• Job C arrives, finds job A running in its concurrency group, and waits.
• ....etc all jobs just wait in line

Sorry if this was discussed before I tried to search but didn't find anything.

Our current implementation

To give it more context:

• We have 5 team environments
• PRs push to the author's team env
• After merge main has a github action to push to all team environments
• the groups are unique to the team env it pushes to
  • We can't deploy to the same env from 2 github actions that's why concurrency is based on envs

Imagine this scenario:
We have 5 PRs for Team A
We merge to main and update the 5 PRs
All 5 will want to deploy to the env + the main branch deploy that is ongoing
the last PR to merge is going to be queued
all other PRs will be cancelled and have to be retried manually

Suggested solutions

one of the below would work

• cancel-in-queue: false a new command to disable cancelling queuing jobs
• cancel-in-progress: false would not cancel anything and leave everything in queue

[ethomson]

Hi @Shereef - thanks much for the feedback. Our initial design was focused on deployment scenarios, but we want to address the use case that you've outlined. I appreciate you outlining this so nicely and providing these suggestions. I hope that this is an area that we can address in the future.

Thanks again.

[miguelb-gk]

Any updates on this?

[finolacahill]

@chrispat @tuves
Is anyone following this highly desirable feature since ethomson left Github?

[finolacahill]

@chrispat @tuves Once again asking if anyone is following up on this feature since ethomson left Github?

[finolacahill]

@ankneis could you see if anyone has anyone taken this on since ethomson left Github please?

[lkraav]

2026: ➕ 1, or 💯 maybe

I'm also at a loss for how to at least work around this shortcoming: multiple workflows, incl. unit tests etc, depend on having an up-to-date node-modules/ cache asset. So when 4 different test runners execute on a large PR, whichever gets there first could do the check-and-maybe-update dance, other should wait until it's their turn to check - definitely not cancel the whole workflow.

All these turnstile etc workaround ideas proved to simply not work out.

Now a bunch of main and production branch commits display a confusing ❌ because workflow engine is a weakling here.

Help?

[JimiC]

The desired behavior here, I believe, would be for jobs with the same run_id to be canceled (either in_progres or in_queue), whenever a new incremental run_id is present on the same ref.

[justin-pierce]

The OP is asking for jobs/workflows to wait and execute sequentially instead of canceling. Canceling existing workflows when a new workflow is triggered is already supported:

https://docs.github.com/en/enterprise-server@3.3/actions/using-jobs/using-concurrency

When a concurrent job or workflow is queued, if another job or workflow using the same concurrency group in the repository is in progress, the queued job or workflow will be pending. Any previously pending job or workflow in the concurrency group will be canceled. To also cancel any currently running job or workflow in the same concurrency group, specify cancel-in-progress: true.

If your issue is specific to manually rerunning the same workflow, seems like a separate issue.

[JimiC]

The said behavior is problematic as it's canceling it's own jobs in the same run. I have 6 jobs to run in each commit and they cancel each other in the same run.

[justin-pierce]

Can your issue be solved by using concurrency on the workflow level instead of the job level? Afaik concurrency on the job level is meant more for branching paths (like their fallback value example). Sorry if I'm misunderstanding your usecase though

[JimiC]

Any previously pending **job** or workflow in the concurrency group will be canceled this is what's problematic.
And because jobs of the same commit share the same group they get canceled.

[nelson6e65]

I think I just need the same, but add to a queue instead of cancel the workflow. I meant, don't run un parallel but in serial instead

This will help, for example, use the same build result or cache in multiple runs for the same commit. The first one will create the artifacts/cache and the next one will be faster

[JimiC]

That will complicate things even further.

[Anonymous-Coward]

Cache management is orthogonal to concurrency. Cache management across multiple jobs is something the workflow writer has to think about. Controlled concurrency among jobs in the same concurrency group, which is what this feature request is about, is not possible with what GHA provides out of the box.

[justin-pierce]

Was surprised to see that canceling previous runs was the only option officially supported -- waiting and executing sequentially is what I need and seems to me like a clear usecase (canceling a deployment could result in a broken state)

So far I've found this Action: https://github.com/ahmadnassri/action-workflow-queue

But ideally, workflows sitting waiting for previous workflows to finish wouldn't consume action minutes in an official feature.

[JimiC]

What I needed was exactly what https://github.com/styfle/cancel-workflow-action provided

[mro-pbraswell]

We implemented something similar with a run step, but that means an entire self-hosted runner is occupied while it's waiting for earlier workflow runs to finish.

It's disappointing that the choice is between scaling up our self-hosted runners or surpassing our build minutes quota for github-hosted runners or building a separate queue outside of github actions to handle this kind of functionality.

[duro]

We have a long running workflow that goes through several jobs to deploy AWS infrastructure. If one of them was cancelled mid-way, or deploys happened on to of one another, this would cause some pretty serious issues.

We definitely need a way to queue jobs in sequence without consuming action minutes

[koreyGambill]

Brainstorming - Eventually I will probably have to implement a specific self-hosted group. Maybe like 25 runners on a t3.small. Their job would be to handle queues. So I'd have a workflow or job dedicated to queing, and my larger runners (c7a.xlarge) wouldn't have to sit and wait for 15 mins at a time - instead I can have a single t3.small for all repos.

[liry]

@justin-pierce the action-workflow-queue has an issue when there are multiple workflows started in the same second. See #45.

@koreyGambill exactly, but do you know about some simple yet reliable queuing solution for that?

[ghost]

Would be great to be able to queue workflow runs (ideally without consuming action minutes). Was surprised that new runs cancel existing ones by default.

[ashwinv11]

We hit this limitation while testing out the Merge Queue feature that is currently in limited beta. Our use case is ensuring that only one workflow is running for the PR at the top of the merge queue. When concurrency is set at the workflow level (for the merge_group event type) and multiple PRs are queued within the merge queue, the system cancels queued check suites once a new PR is added to the queue. It would be fantastic if we had the ability to disable canceling queued jobs!

[akagr]

If you're using self-hosted runners, a workaround/hack we used was to label one of these runners specifically, and then make all the jobs that need to be in a concurrency group just run on this labelled runner instead. No concurrency shenanigans. That causes them to queue up globally across all workflows without any caps on number of waiting jobs.

The only major limitation is that if this runner fails, these jobs won't use any other runner. That was considered an acceptable risk to unblock us until we had a better solution.

[liry]

Well, this is possible only for cases, where you need to serialize only a single job.
Once you have a workflow where multiple jobs need to finish in the given order before new workflow is started (e.g. job-A: deploy to test env, job-B: run tests against the env), then it's not very well usable.

[OliverHenrichs]

This is exactly our use case.
Have you perchance found a way to elegantly solve this in GitHub @liry ?

[AvivPeledTalon]

My team also needs this feature.

[ghost]

My team also needs this.

[MitchellPaff]

My team also need this feature - mainly for queueing mulitple actions each with Terraform plan/deploy steps which cannot be run at same time due to a lock on terraform state.

[galz-cyera]

A must have for my team

[yershalom]

Really hard to run terrafrom/terragrunt at scale without it

[badbort]

Really needing this to be implemented.

I am currently using https://github.com/ben-z/gh-action-mutex for now, but this does pollute the git history.

[dumganhar]

Need this feature too.

[bob-ashforth-sp]

GitHub, I'll add my team's need for this feature as well. I see a LONG list of replies like this, with no response from GitHub. (I'm in DevOps.)

I'm going to put in a GitHub support ticket referencing this request to see if there are any plans to implement it.

Our use cases is that I have created a highly parallelized building and deploying four distinct services in a monorepo. In this workflow, the update_service action is called by the jobs for each of those services, AND we have a very active CI/CD environment which can be processing several merges almost simultaneously.

While I am discussing the use of the current concurrency model (cancel all pending workflow runs except the latest), until and unless that gets the green light our workflow runs will continue to risk colliding and corrupting the shared deployment environment with competing service updates.

[gnuman]

Thanks for the feature request, we need this feature

[dannysauer]

I voted on the issue, but the flood of plus one comments seems pretty impressive too. So, "hey, I want multiple overlapping jobs to queue and run serially instead of terminating" as well. I'm pretty sure this is possible, because one job will queue and multiple jobs waiting on runners will queue. So all the necessary logic seems like it's essentially present somewhere. 😅

[fetwar]

The more I try use GitHub actions for advanced use cases the more I stumble onto community conversations like this.

• seemingly simple to implement
• large number of users requesting the feature
• support ticket/s opened
• users frequently trying to get any word from GitHub staff through multiple different methods

Yet, years later and still... nothing... Not so much as a reply with whether the feature is planned, on the roadmap at all or if this is something we should just learn to live with.

[Felixoid]

Actually, in this case, the silence is a very clear answer.

It looks to me like Actions isn't the first class citizen, and it's not planned to change the state any soon.

[oferqm]

+1

[jeliasson]

Stop it. This thread don't need the spam that +1 invokes.

[LiteBrite82]

@jeliasson is correct, please remember to use the upvote button or an emoji instead of creating a new thread to agree with a comment. It keeps the discussion clean.

Thank You community!!!

[nebuk89]

Hey, sorry the Actions team has been quiet on this. We have added this to the list of paper cuts we are hoping to address in the next 12 months :) I don't have a timeframe for this one, but I want to say we know it's a pain point and do have it on our list <3

[shellscape]

Sadly that reply is meaningless vapor without any action whatsoever.

[bob-ashforth-sp]

Good to see a reply, @nebuk89 . This is really a LOT more than a paper cut; it's a fact that many workflows are incrementally changing/updating things like databases, and MUST run in sequence. The current design presumes that later workflows are just an addition to the same codebase, and so are expected to produce the same result whether or not an intervening run is cancelled.
I hope that you can spread this perspective to everyone involved in prioritizing work.
I and probably everyone who has chimed in here have been some combination of shocked, annoyed, and even angry that our requests have not resulted in getting this problem fixed. I really hope that you can help!

[nebuk89]

Thank you for keeping an eye and replying. Sorry I don't have solid updates yet.
Most of our time at the moment is going on a re-architecture to improve availability (we are going to get some blog posts out soon to provide more clarity on why Actions has been quite for the last 6-9 months). This is still on our list and the engaged requests of all kind (shocked, annoyed and angry!) do help me keep this higher up our prioritization.

So thank you for taking the time <3 please keep engaging and never stop being passionate about us making Actions the best product it can be.

[Qwuke]

@nebuk89 If it helps for prioritization, this problem has caused correctness or production availability problems in different positions/companies when using GH Actions and is probably the most painful part of using GH Actions out of the box for a large team. I don't think there is any other GH Actions, GH feature in general, or any other non-GH CI/CD software feature gap that has caused me so much pain, stress, and lost hours for something that is reasonably expected to be configurable in tooling. There is probably even more impact than what is ascribed here because large teams may not even know this is where they need to go to flag why their CI steps decided to skip unexpectedly, which even if it causes down time, may only come up as a problem intermittently. Additionally, users appear to talk about this problem across several different community discussions that have not yet been consolidated, each with hundreds of upvotes.

Thanks for the transparency and letting us know the situation.

[github-actions[bot]]

💬 Your Product Feedback Has Been Submitted 🎉

Thank you for taking the time to share your insights with us! Your feedback is invaluable as we build a better GitHub experience for all our users.

Here's what you can expect moving forward ⏩

• Your input will be carefully reviewed and cataloged by members of our product teams. 
  • Due to the high volume of submissions, we may not always be able to provide individual responses.
  • Rest assured, your feedback will help chart our course for product improvements.
• Other users may engage with your post, sharing their own perspectives or experiences. 
• GitHub staff may reach out for further clarification or insight. 
  • We may 'Answer' your discussion if there is a current solution, workaround, or roadmap/changelog post related to the feedback.

Where to look to see what's shipping 👀

• Read the Changelog for real-time updates on the latest GitHub features, enhancements, and calls for feedback.
• Explore our Product Roadmap, which details upcoming major releases and initiatives.

What you can do in the meantime 💻

• Upvote and comment on other user feedback Discussions that resonate with you.
• Add more information at any point! Useful details include: use cases, relevant labels, desired outcomes, and any accompanying screenshots.

As a member of the GitHub community, your participation is essential. While we can't promise that every suggestion will be implemented, we want to emphasize that your feedback is instrumental in guiding our decisions and priorities.

Thank you once again for your contribution to making GitHub even better! We're grateful for your ongoing support and collaboration in shaping the future of our platform. ⭐

[hschimpf]

Hi all! I have a more complex desired behavior:

We have a step that deploys with terraform. We want that if a running job already reached that step, the other ones should wait instead of killing it. Ex:

• Job A reaches, nothing in-progress, starts
• Job B reaches, finds A in-progress but not at deployment step, kills A and starts
• Job C reaches, finds B in-progress and already at deployment step, waits for B to finish
• Job D reaches, finds B in-progress, but also C in-progress (waiting), should kill C since it not reached deployment step, waits for B to finish
• Job B finishes
• Job D continues
• Job D finishes

Deployment succeeded, version B deployed and then replaced with version D

[iusildra]

@hschimpf what about merge queues ? For a TF deploy it seems better to me to run the full deploy for each commits

[tinazhouhui]

This is exactly what we are looking for, @hschimpf were you successful in implementing something similar?

@iusildra I agree that merge queue can do this, especially if we tweak the merge limits, but it also means that all the changes are stuck on the merge branch rather than in main

[ryeleo]

Per original request: my team needs something like cancel-in-queue: false directive to enable queuing workflow runs that cannot be concurrent.

cancel-in-queue: false a new command to disable cancelling queuing jobs

Motivating Example

I have a 'deploy' GitHub Workflow that takes 600 seconds to run a deployment.
I have set the concurrency directive up with group: prod-deployment and cancel-in-progress: false
I have 3 pull requests (PRs: PR1, PR2, PR3).
All 3 of these PRs are merged within a 30 second time window:

1. PR1 merges at 12:00:00
2. PR2 merges at 12:00:15
3. PR3 merges at 12:00:30

Requirements

1. My GitHub Workflow must run for each PR.
2. My GitHub Workflow cannot be run concurrently.

Expected Behavior

1. PR1 is deployed,
2. PR2 is deployed, then
3. PR3 is deployed.

Documented Behavior

1. PR1 is deployed,
2. PR2 is cancelled, then
3. PR3 is deployed

[gladykov]

I can understand why it is so difficult to implement. There are many bad jobs out there, which run when they should not. So current behavior prevents them from overwhelming Github servers. Especially if you have heavy jobs running after every commit.

But at the same time there are people out there who are willing to throw money at you just for using your CPU. So this is clearly revenue possibility for you.

So why it was not implemented already for paid clients, is funny mystery.

For people who will cry , that this unfair - do you imagine Github allowing to queue jobs faster than they can execute for free?

[Anonymous-Coward]

That can be easily addressed. You can have explicitly defined queues and set limits on queue sizes and you can add rate limits. This would discard excessive runs of badly behaving jobs.

You can implement something like this with what's already in place, but it's tedious and more resource consuming than if Github would do it. So it can't be that difficult. Only, it won't appear out of nothing. Somebody needs to implement it, and apparently MS isn't willing to pay for it.

[fosskers]

There is a workaround if you have self-hosted runners.

1. Give one of your runners a unique label. In our case it's cpp as the C++ builds are very heavy and dominate the machine's resources.
2. Give your heavy jobs this extra label:

  my-job:
    runs-on: [self-hosted, linux, cpp]

    concurrency:
      group: ${{ github.workflow }}-${{ github.ref }}
      cancel-in-progress: true
  # ... etc. ...

Runners themselves have no maximum queue length, so all the heavy jobs will wait their turn properly. Note that there is a 24 hour time limit for such queued jobs.

[eddieatbrumaline]

Hah, this is pretty brilliant. I may have to resort to this 🤦‍♂️

[OctavioBR]

Found that softprops/turnstyle helps to workaround this limitation. Instead of relying on jobs with concurrency groups to begin, first I run that action that holds it while there are other workflows still running.

[Felixoid]

It can work for cases when runners are free.

And for huge compiling hosts, it's a money black hole.

[OctavioBR]

We set it on a single action job "queue" that runs-on cheap/free runners then the expensive jobs have needs: queue.

[nebuk89]

Ok y'all 🤫 we have shipped but not publicly announced that we have shipped

concurrency:
  group: demo-queue
  queue: max

If anyone wants to try it out, have a go :D and let me know any initial thoughts

[amfage]

I may be misunderstanding the way this is supposed to work, but using the new queue: max concurrency, when I merge two PRs in short succession, the GitHub Action that runs on master only runs once. I would expect it to run once for each PR.

Edited to add: if I leave a bit of time between submitting PRs, maybe a minute, then the master branch action is triggered twice, once for each PR.

[jasongin]

@amfage, There's not enough information provided to confirm.. but from the screenshots it looks like you are using merge queues, which may group multiple PRs together and merge them at the same time as one merge commit. So that could explain why your on-push action is only triggered once when multiple PRs are merged. I think it's unrelated to use of queue: max.

[Shereef]

This works great for our purpose thanks!

[loliver]

Have had this going for ~10 days now and it's worked flawlessly, thank you so much.

[lkraav]

Thanks, I see it made it to official docs https://docs.github.com/en/actions/how-tos/write-workflows/choose-when-workflows-run/control-workflow-concurrency

Supplies clarity on "max: Up to 100 jobs or workflow runs can be pending in the concurrency group. When the queue is full, any additional jobs or workflow runs are canceled."

[qoomon]

Awesome improvement. The only thing that's missing for our use case is an option to queue specific jobs however ensure execution order stays the same as the workflows where startet, all other jobs before should be able to run in any order because the don't have any interference.

[Soju06]

Using queue: max in production since it shipped. Hit a failure mode twice in one day where jobs in the group stop being dispatched entirely — pending, runner_name=null, empty steps, while nothing in the group is in progress. 194 and 139 minutes.

A run holding one of those jobs also could not be cancelled: the request was accepted and then ignored for 40+ minutes, while a run with no jobs cancelled in 8 seconds.

Both cleared on their own with no config change. Wrote it up with job IDs and timings here: https://github.com/orgs/community/discussions/206259

@nebuk89 since you shipped this one — is this a known edge in the queue implementation?

[jasongin]

I looked at telemetry for the reported run and found there were other jobs sharing the concurrency group that caused the delay. And the cancellation was prevented by an always() condition. Details at https://github.com/orgs/community/discussions/206259#discussioncomment-18224179