I have two self-hosted azure pipeline agents. Both have the same hardware to my knowledge and same windows 10 image.
However, when we try to run UI tests on these machines with auto-logon, one of the machines works fine, however on the other machine the UI tests fail. On close examination we discovered that the screen resolution on the failing machine was 640x480. I tried using the ScreenResolutionUtility task to set an alternate resolution. One the first machine we discovered many options of resolution but on the failing machine it only shows 640x480 32 bits 60 Hz.
If I remote into that failing machine or connect a monitor it does display the content at a much higher resolution. Not sure what I can do to fix it.
UPDATE
The Agent is indeed running in interactive mode and not as service
The screen resolution utility task's log
2020-10-20T04:36:21.2741354Z ##[section]Starting: Setting optimal resolution
2020-10-20T04:36:21.3127362Z ==============================================================================
2020-10-20T04:36:21.3128428Z Task : Screen Resolution Utility
2020-10-20T04:36:21.3129441Z Description : Set the screen resolution of the agent machine. Useful when running UI tests.
2020-10-20T04:36:21.3130381Z Version : 1.0.2
2020-10-20T04:36:21.3131031Z Author : Microsoft Corporation
2020-10-20T04:36:21.3132048Z Help : [More Information](https://go.microsoft.com/fwlink/?linkid=865425)
2020-10-20T04:36:21.3133210Z ==============================================================================
2020-10-20T04:36:23.9224027Z Attempting to change display settings to 640 by 480, 32 bit, 60 hertz
2020-10-20T04:36:23.9313420Z Display settings changed successfully.
2020-10-20T04:36:23.9912069Z ##[section]Finishing: Setting optimal resolution
In order to change the screen resolution on the private agent, the agent needs an interactive session. Configure the agent to run interactively, instead of as a service.
You could check this document Interactive vs. service for some more details.
If you run that agent interactively, please share the build log of the task Screen Resolution Utility in your question.