Parse page in detach mode still getting detected as a bot

Question 1

I'm trying to parse some site which use bot detection algorithms. I use following Chrome Options

chrome_options = Options()
chrome_options.add_argument("start-maximized")
chrome_options.add_experimental_option("detach", True)
chrome_options.add_experimental_option("excludeSwitches", ["enable-automation"])
chrome_options.add_experimental_option('useAutomationExtension', False)
chrome_options.add_argument("--disable-blink-features")
chrome_options.add_argument('--start-maximized')
chrome_options.add_argument('window-size=1920,1080')
chrome_options.add_argument("--disable-blink-features=AutomationControlled")
driver = webdriver.Chrome(executable_path='chromedriver.exe', chrome_options=chrome_options)
driver.get(url)

I was also patched chromedriver.exe remove cdc_ pattern.

When browser window starts and goes to site, site detect it as bot and prevent loading contents. But when browser goes in detach mode and i manually press "Refresh" button, site contents is loading successfully. Simulating press "f5" also don't work, site still block contents loading.

Question 2

detach (Chrome-specific capabilities)

Name:           detach 
Type:           boolean 
Default:        false 
Description:    If false, Chrome will be quit when ChromeDriver is killed, 
                regardless of whether the session is quit. If true, Chrome will only 
                be quit if the session is quit (or closed). Note, if true, and the 
                session is not quit, ChromeDriver cannot clean up the temporary user 
                data directory that the running Chrome instance is using.

As per the description, the browser goes in detach mode but the session is not quit, ChromeDriver cannot clean up the temporary user data directory that the running google-chrome instance is using.

Hence, ChromeDriver initiated Chrome Browser still gets detected.

DebanjanB · Answer 1 · 2021-11-24T10:55:59

detach (Chrome-specific capabilities)

Name:           detach 
Type:           boolean 
Default:        false 
Description:    If false, Chrome will be quit when ChromeDriver is killed, 
                regardless of whether the session is quit. If true, Chrome will only 
                be quit if the session is quit (or closed). Note, if true, and the 
                session is not quit, ChromeDriver cannot clean up the temporary user 
                data directory that the running Chrome instance is using.

As per the description, the browser goes in detach mode but the session is not quit, ChromeDriver cannot clean up the temporary user data directory that the running google-chrome instance is using.

Hence, ChromeDriver initiated Chrome Browser still gets detected.

Parse page in detach mode still getting detected as a bot

detach (Chrome-specific capabilities)

In other languages

This page is in other languages

Popular in the category