[BUG]: Tests are marked as executed on windows despite being set to run on linux

[AdamOakman]

Describe the bug
Tests are marked in Playwright portal as executed on windows despite being set to run on linux in playwright.service.config.ts

To Reproduce
Steps to reproduce the behavior:

1. Execute any spec file on Playwright Testing Service with Linux as desired OS to run browsers on
2. See error

Expected behavior
Tests should be executed on Linux and information in Playwright portal should confirm it correctly

Screenshots

Additional context
playwright.service.config.ts

import { getServiceConfig, ServiceOS } from '@azure/microsoft-playwright-testing';
import { defineConfig } from '@playwright/test';
import config from './playwright.config';
import dotenv from 'dotenv';

dotenv.config({ override: true });

/* Learn more about service configuration at https://aka.ms/mpt/config */
export default defineConfig(
    config,
    getServiceConfig(config, {
        useCloudHostedBrowsers: true,
        serviceAuthType: 'ACCESS_TOKEN',
        exposeNetwork: '<loopback>',
        timeout: 30000,
        os: ServiceOS.LINUX
    }),
    {
        workers: 25,
        reporter: [
            ['@azure/microsoft-playwright-testing/reporter'],
            ['list']
        ]
    }
);

[vvs11]

Hi @AdamOakman
Thanks for writing to us. This is a bug in our dashboard. It is showing the OS where Playwright client is run and not the OS where browsers are hosted. We are working on this and will fix it soon.
If you select Linux in your playwright.service.config.ts file, the tests run on Linux.

[abranj19]

Thanks @AdamOakman , we will come back with ETA for this.

[marcusNumminen]

Hi!
I faced the same issue with version 1.0.0-beta.4. My screenshot tests is indicating that the test is running on windows even if I have set os: ServiceOS.LINUX in the playwright.service.config, I use the {platform} in the snapshotPathTemplate in the playwright.config.ts and the generated screenshots have win32 in the names

[puagarwa]

Hi @marcusNumminen, We were showing the client machine OS type and not the cloud browser OS info. To avoid confusion, we have removed showing this field in portal in last update. We have an item in backlog to show cloud browser OS info in future.

[marcusNumminen]

But just to be clear here. Do the test run in Windows or Linux? My screenshot testing failed due to that I don't have screenshots for Windows (but I have for Linux). In my playwright config i use something like this: snapshotPathTemplate: '{snapshotDir}/{arg}-{platform}{ext}',
Like I said my test where failing when I ran the test in Playwright Testing Service due to missing (win) files and playwright created new ones for windows. So this is indicating to me that the test are ran in Windows.

In my opinion you should not remove the the OS type. I think you should have them both there, i.e. both OS type and Browser type.

[puagarwa]

HI @marcusNumminen , Cloud Browser OS is defined by value of field os in getServiceConfig and irrespective of your client machine OS.

In your case, value of {platform} field might be client OS value, if you want this value of cloud browser OS then i suggest you to do some plumbing to have this value same as value passed in getServiceConfig to match cloud browser OS.

As I mentioned currently, we were earlier showing client machine OS which was confusing for customers hence we have removed it. We are evaluating if we could show both client OS as well as Cloud Browser OS for more clarity.

[marcusNumminen]

Hi!
I now realize that I might made the wrong assumptions on how the playwright testing service works. I kind of thought that it was more or less (Windows or Linux) VMs that had the browser installed on. I now realize that this is probably not the case :) My friend that showed this to me didn't really explained this to me :)

[puagarwa]

HI @marcusNumminen, Service has isolated cloud browsers hosted on linux and windows contains and we connect your browser request depend upon your config ask. If you can tell us more about your scenario then we could clarify exact question. https://learn.microsoft.com/en-us/azure/playwright-testing/overview-what-is-microsoft-playwright-testing

[vvs11]

@marcusNumminen We recently did a session on the Playwright YouTube community, you can check it out to understand more about the service.
https://www.youtube.com/watch?v=W-2KKXEegyM&pp=ygUcbWljcm9zb2Z0IHBsYXl3cmlnaHQgdGVzdGluZw%3D%3D