[Feature] Have the ability to set workers count on a per project basis

[surecloud-rwattie]

Would it be feasible to make the workers count be configurable on a per project basis?

My use case is that I'm have a project as a dependency which does various setup tasks. The main thing it does is stores login state and then creates some data via a few api calls. Our application uses Auth0 for authentication and because of this there are rate limits on the number of logins within a minute for the same user.

I currently have my workers set to 10 on my laptop, eventually when this goes into CI/CD it'll be running many more. I login once per project and we have 18 projects defined (19 including the setup project). 10 workers in parallel is a good number for the setup project because the length of time it takes to do the login and the api calls means that I never get rate limited.

Obviously setting the workers to 10 does not make my tests scalable in CI/CD but at the same time, setting the workers to a much greater number also does not help as I'll then be rate limited with the authentication.

By being able to specify a global number of workers and then override this number on a per project basis would be very handy. The alternative is to move everything back into a global-setup file which I don't really want to do if I don't have to.

[kLjubomir]

We would also like this feature, reason: our projects are separated by regions and users have session limitations, so logging in with the same user twice as would inadvertently happen if on the global level workers>1 would fail some other tests, we can't control the test data so we need to use the given test users, ideally by specifying inside each project worker:1, but current workaround is that we are running "concurrency" by executing X containers concurrently in Jenkins where X is the amount of regions in test.

[np-xavier]

We would also like this feature! We see projects as composable 'slices' of config so supporting more options (especially workers) would be great.

[gskierk]

If you have a project that connects to a single Android device, it seems to be the best option to set workers to 1 to execute tests one by one. It would be great if we could configure it on the project level.

[HendrikJan]

This feature would be very useful to us.

We have some tests that write to the database and these tests should never run at the same time. They should run with workers = 1.

Other tests don't write to (but do read from) the database. The more of them running together, the better. For performance.

[kasperkow]

Upvote. I would like to create one repository for the whole application. The project would not only be other browsers but a real part of apps. But those parts have completely different structures/specifics. Two workers are max in one project, but I can double it in the other.

[HendrikJan]

I would like this even more on a file level:

test.describe.configure({ workers: 1 });

That would make it possible to have some destructive tests in a file marked with workers: 1 to be used in any project and be confident that it will never run together with another test.

[MaksimMedvedev]

that would really be useful for us, we have a small amount of test files with tests which check a documents signing behaviour and should run sequentially, other tests may run in parallel. Specifying workers count per test file would really help

[gottschd]

I would like this even more on a file level:

test.describe.configure({ workers: 1 });

That would make it possible to have some destructive tests in a file marked with workers: 1 to be used in any project and be confident that it will never run together with another test.

that would really be useful for us, we have a small amount of test files with tests which check a documents signing behaviour and should run sequentially, other tests may run in parallel. Specifying workers count per test file would really help

Same for us.
Some Tests (Testfiles) are independent/randomized so they are not disturbing each other.
However, the reality is that some tests must run only on one (the same) worker sequentially.
Maybe another wording would be: "Pinned to worker#1".

Also such a simple line is easier to communicate/teach in a team.

[t-mish]

Upvote

[samixchoumi]

Up vote

[VadimMentorQA]

Up vote

[vadimJemJanuary]

Up

[mi4i0]

Up

[sriram-kothapalli]

Up vote

[RozekNaGicie]

Up

[himaz-incentivio]

up vote!

[D-jazic]

up

[TikouWeb]

definitely UP 😃

[cheruvian]

I would like this even more on a file level:

test.describe.configure({ workers: 1 });

That would make it possible to have some destructive tests in a file marked with workers: 1 to be used in any project and be confident that it will never run together with another test.

that would really be useful for us, we have a small amount of test files with tests which check a documents signing behaviour and should run sequentially, other tests may run in parallel. Specifying workers count per test file would really help

Same for us. Some Tests (Testfiles) are independent/randomized so they are not disturbing each other. However, the reality is that some tests must run only on one (the same) worker sequentially. Maybe another wording would be: "Pinned to worker#1".

Also such a simple line is easier to communicate/teach in a team.

FYI I think you may be able to get away with

setup.describe.configure({ mode: `serial` });

[deviationist]

Up

[galiavasylieva]

up

[LeviticusMB]

Another option that would work well for us would be to be able to specify a (dynamic; see #32365) limit on worker fixtures themselves, so that a worker fixture is only ever used on N number of workers at the same time (limiting the number of concurrent Android emulators, VM:s or whatever).

But this might just make things unnecessarily complicated. A per-project worker config would be good enough.

[HShishmanyan]

We also have a case in which workers per project will be super beneficial.
We wanted to set two projects:

• P1 - order dependent tests. Those are actions that have strict executions order. - set up with 1 worker.
• P2 - order independent tests. - set up with more workers .

Currently, we have no workaround at all for this problem.

[samixchoumi]

Hello, for information I made a solution (with GitLab but may be similar with GitHub) using script to launch wanted tagged feature file to be run with 1 worker while other use X worker. A bit tricky but it work fine, and you can add multiple tag to it 👌🏻

…

On Wed, Sep 4, 2024 at 08:42 Hagop Shishmanyan ***@***.***> wrote: We also have a case in which workers per project will be super beneficial. We wanted to set two projects: - P1 - order dependent tests. Those are actions that have strict executions order. - set up with 1 worker. - P2 - order independent tests. - set up with more workers . Currently, we have no workaround at all for this problem. — Reply to this email directly, view it on GitHub <#21970 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AFJFCGBRLTCKOVNVYGLERNLZU2TTXAVCNFSM6AAAAAAWHBEV26VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMRYGA2DAMBXGI> . You are receiving this because you commented.Message ID: ***@***.***>

[MatthewIceland]

I would like this even more on a file level:

test.describe.configure({ workers: 1 });

That would make it possible to have some destructive tests in a file marked with workers: 1 to be used in any project and be confident that it will never run together with another test.

This would be really helpful for us

[ksvvandana]

Would it be feasible to make the workers count be configurable on a per project basis?

My use case is that I'm have a project as a dependency which does various setup tasks. The main thing it does is stores login state and then creates some data via a few api calls. Our application uses Auth0 for authentication and because of this there are rate limits on the number of logins within a minute for the same user.

I currently have my workers set to 10 on my laptop, eventually when this goes into CI/CD it'll be running many more. I login once per project and we have 18 projects defined (19 including the setup project). 10 workers in parallel is a good number for the setup project because the length of time it takes to do the login and the api calls means that I never get rate limited.

Obviously setting the workers to 10 does not make my tests scalable in CI/CD but at the same time, setting the workers to a much greater number also does not help as I'll then be rate limited with the authentication.

By being able to specify a global number of workers and then override this number on a per project basis would be very handy. The alternative is to move everything back into a global-setup file which I don't really want to do if I don't have to.

Curious on how you are authenticating per project, is it through dependency?

[ksvvandana]

I would like this even more on a file level:

test.describe.configure({ workers: 1 });

That would make it possible to have some destructive tests in a file marked with workers: 1 to be used in any project and be confident that it will never run together with another test.

Upvote upvote!! This will reduce 10-15 dependency files for us. Very helpful. Any update?

[fanrita]

Would be really useful to have this config per project e. g. in playwright.config.ts. E.g if we have several envs and on one of the we want to run tests with workers=1, while on other projects we can run tests in parallel.
{
name: 'webkit',
workers: 1,
use: { ...devices['Desktop Safari'] },
}

[shifttt]

upvote

[lzanchin]

Upvote

[guido-visser]

Upvote

[mkgreenventory]

up

[chermdev]

up

[sahilanguralla]

up

[apis3445]

I would like this even more on a file level:

test.describe.configure({ workers: 1 });
That would make it possible to have some destructive tests in a file marked with workers: 1 to be used in any project and be confident that it will never run together with another test.

This will be very useful I have some restrictions where I only need one test at the same time and sometimes that I can run some in parallel

[rmisantos]

Upvote

[nickolay-smyk]

up

[jatinmakhija]

up

[mihaipoenaru]

Same! My use case is that separate files do some modifications that affect the database, however some modules of our app are interconnected so modifying stuff in one will affect others. I can't run the tests with just 1 worker because we have a separate "readonly" test suite that can be fully parallel (it checks that the page looks ok, that the access is correct, that the fields are mapped correctly to the tables etc).

It would also be fantastic if I could set the worker per file, if the tests in the file itself are parallelizable, but more than 1 file can't run at the same time.

[Zanzibarbarian]

Big +1 to this feature request

[muradium]

Just because of lack of this feature I run separate test runs and have to do a lot of extra work to combine all the test results and coverage reports into one. Looking forward to it 👍

[berengamble]

Exactly the same case as @muradium for me.

[developerdanwu]

This would be very useful for me!