Browser headful mode at Linux for web scraping

Question

https://swappie.com/en/iphone/ we managed to bypass the Cloudflare protection page if we use a proxy and a browser in the headful mode (both with Chrome headless in Selenium and Chromium in Playwright). But on the Linux server we can run browser only as headless.
Do you have any ideas how to break thru ?

Answer 1

There's a Linux tool called Xvfb, which creates a virtual display so that you don't have to use Chrome's headless mode when running browser tests in it. Various frameworks have a option to use it.

Here's a sample SeleniumBase script that bypasses the Cloudflare Turnstile page on a popular site:

"""SB Manager using UC Mode for evading bot-detection."""
from seleniumbase import SB

with SB(uc=True, test=True, xvfb=True) as sb:
    sb.driver.uc_open_with_reconnect("https://top.gg/", 5)
    if not sb.is_text_visible("Discord Bots", "h1"):
        sb.driver.uc_open_with_reconnect("https://top.gg/", 5)
    sb.assert_text("Discord Bots", "h1", timeout=3)
    sb.highlight("h1", loops=3)
    sb.set_messenger_theme(location="top_center")
    sb.post_message("Selenium wasn't detected!", duration=3)

It also works in Linux (assuming that the IP Address you have wasn't blocked already).