Most of the more sophisticated anti-scraping protections, such as Recaptcha V3, take advantage of browser fingerprinting. Every browser has a unique fingerprint, so detecting an unmasked headless Chrome is trivial. Stealth mode overrides the attributes specified for headless mode so as to be harder to distinguish from normal Chrome. 

Stealth mode comes with Apify SDK, and you can also turn it on in the apify/webscraper and apify/puppeteer-scraper. Let's explore how to use it in our SDK. 

First, you need to turn it on in the PuppeteerCrawler. You only need to set a few properties.

const crawler = new Apify.PuppeteerCrawler({
    requestList: <yourRequestList>,
    launchPuppeteerOptions: {
        headless: true,
        stealth: true,
        useChrome: true,
    },
    handlePageFunction: async ({ request, page, puppeteerPool }) => {
        // you are on the page now with the stealth mode on
    },
});
await crawler.run();

Stealth mode should not break anything, but some of the tricks work differently with different anti-scraping protections. You should try which combination works best for you for a particular blocking system. The default is for the Google Recaptcha V3 score. In most cases you usually want to use everything since there is a no difference in performance between using one or all properties.

You can  specify which tricks you want to apply by passing the stealthOptions configuration. For example, the code to turn on only the web driver hiding trick looks as follows. 

const crawler = new Apify.PuppeteerCrawler({
    requestList,
    launchPuppeteerOptions: {
        headless: true,
        stealth: true,
        useChrome: true,
        stealthOptions: {
            addPlugins: false,
            emulateWindowFrame: false,
            emulateWebGL: false,
            emulateConsoleDebug: false,
            addLanguage: false,
            hideWebDriver: true,
            hackPermissions: false,
            mockChrome: false,
            mockChromeInIframe: false,
            mockDeviceMemory: false,
        },
    },
    handlePageFunction: async ({ request, page, puppeteerPool }) => {
        // you are on the page now with the stealth mode on
    },
});
await crawler.run();

All options are set true by default, but you can turn off anything that you don't need or want.

You might also want to use stealth mode in the single browser instance using Apify.launchPuppeteer. To achieve this, you need to do the following.

const puppeteerOptions = {
    headless: true,
    stealth: true,
    useChrome: true,
};
const browser = await Apify.launchPuppeteer(puppeteerOptions);

To turn off some hiding tricks in Apify.launchPuppeteer you can do the following.

const puppeteerOptions = {
    headless: false,
    stealth: true,
    useChrome: true,
    stealthOptions: {
        addPlugins: false,
        emulateWindowFrame: false,
        emulateWebGL: false,
        emulateConsoleDebug: false,
        addLanguage: false,
        hideWebDriver: true,
        hackPermissions: false,
        mockChrome: false,
        mockChromeInIframe: false,
        mockDeviceMemory: false,
    },
};
const browser = await Apify.launchPuppeteer(puppeteerOptions);


If you come across a website that still blocks your headless Chrome with stealth mode turned on, please share your feedback with us and we will try our best to improve it.

Did this answer your question?