Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Iridium is causing false ad detection on YouTube #928

Closed
ParticleCore opened this issue Feb 13, 2024 · 69 comments
Closed

Iridium is causing false ad detection on YouTube #928

ParticleCore opened this issue Feb 13, 2024 · 69 comments

Comments

@ParticleCore
Copy link
Owner

ParticleCore commented Feb 13, 2024

A Chrome user reported YouTube detecting the Iridium ad manager even if it is disabled:

image

Sets off Youtube's adblock detector, even with the ad-blocking features disabled. I've used this extension for years and I hate to leave a negative review but it cannot function right now

Iridium does not have any code modifications when ad manager is disabled, it is built as a non-running rule logic:

if (!isInVideoAdsAllowed) {

if (!isAdTaggedProductsAllowed) {

if (!isAdMastheadAllowed || !isAdHomeFeedAllowed) {

if (!isAdHomeFeedAllowed) {

if (!isAdSearchFeedAllowed) {

if (!isAdVideoFeedAllowed) {

As a result it is not possible for YouTube to detect Iridium's ad manager even when it is disabled since in that scenario there is no code modification that could trigger detection.

On top of that, ever since I resumed work on Iridium, YouTube has never detected the ad manager once.

What is likely happening is the user has a third party adblocker which he thinks is disabled on YouTube, but it is actually still affecting it.

See #919 as a confirmed example of that situation.

At this time this is not an actionable item, hopefully the user can follow up here so we can figure out what is happening.

@JoshS-dev
Copy link

Yeah, no. This person is me, and I know it is Iridium because the moment I uninstall Iridium from Chrome, YouTube starts working again. The only adblocker I have is Ublock, and it is evidentially not causing it because as said, once Iridium is uninstalled things are working fine. And notably things have worked fine for years between Ublock and Iridium until this recent update.

I don't know, what code or console things am I supposed to be showing here?

@ParticleCore
Copy link
Owner Author

ParticleCore commented Feb 13, 2024

Thanks for letting me know about that. There is a chance that your issue might reveal a possible incompatibility between both extensions, I am very interested in finding out what that might be.

Unfortunately I have been using both for at least 3 months, and I have not yet stumbled on an issue like yours, the closest issue I faced so far is the one I encountered in #919 on Firefox mainly.

As a result I also am kind of clueless here as to what you can look for, there wouldn't be any logs on the console but you can surely check if you have the time! Let me know if something shows up.

Other than that I can only ask if did you try testing with Iridium enabled and uBlock disabled in the Chrome extension settings?

Like this:
image

I did experience in the past YouTube detecting the adblocker with uBlock enabled (and disabled as per #919) without Iridium even being installed (fresh browser profile running only uBlock), that is why my initial assumption being that this is an issue being caused by, or contributed by uBlock.

And the fact that the Iridium extension is no-op when ads are allowed raises the suspicion of this issue being related to another extension even further, even if it just makes it more evident as a result.

@ParticleCore
Copy link
Owner Author

ParticleCore commented Feb 13, 2024

Just an update on this topic;

YouTube has multiple different ways to detect an adblocker. These can be grouped into 3 types:

  • Page ad block detection
  • Video ad block detection
  • Generic ad block detection

For each of these they employ a few different ways to detect ad blocking, some of which are still unknown to me.

However, I am able to confirm one method they have been using for years, but never used it to act on it.

YouTube tries to load the following script:
https://static.doubleclick.net/instream/ad_status.js

The contents of which is a simple window level flag:
window.google_ad_status = 1;

This is then used in code in a couple of locations, example:

https://www.youtube.com/s/desktop/a4aa4e18/jsbin/desktop_polymer.vflset/desktop_polymer.js

    function REc() {
        if (!H("disable_biscotti_fetch_for_ad_blocker_detection") && !H("disable_biscotti_fetch_entirely_for_all_web_clients") && A4a()) {
            var a = dl("PLAYER_VARS", {});
            if ("1" != Hb.get(a, "privembed", !1) && !G4a(a)) {
                var b = function() {
                    QEc = !0;
                    "google_ad_status"in window ? cl("DCLKSTAT", 1) : cl("DCLKSTAT", 2)
                };
                try {
                    Mz("//static.doubleclick.net/instream/ad_status.js", b)
                } catch (c) {}
                PEc.push(yi.addLowPriorityJob(function() {
                    if (!(QEc || "google_ad_status"in window)) {
                        try {
                            n4a("//static.doubleclick.net/instream/ad_status.js", b)
                        } catch (c) {}
                        QEc = !0;
                        cl("DCLKSTAT", 3)
                    }
                }, 5E3))
            }
        }
    }

The way this works is by simply using the same ad serving host (doubleclick.net) to load this script, any adblockers not excluding specifically this file and using a generic static.doubleclick.net blocking rule, will inadvertedly prevent the flag from being enabled, thus resulting in a detection. Or this might be to avoid biting CPM due to page robots/crawlers.

The result of this operation is then stored in yt.config_.DCLKSTAT

Another method they have in place if enabled is the player event based detection in which the player dispatches the onAbnormalityDetected event and triggers the detection:

(...)
H("enable_async_ab_enf") && this.playerEvents_.set("onAbnormalityDetected", this.onAdBlockingDetected_);
(...)
        onAdBlockingDetected_: function() {
            var a = qpb(this.playerData);
            a && (KF(5),
            this.playerData = Object.assign({}, this.playerData, {
                playabilityStatus: Object.assign({}, this.playerData.playabilityStatus, {
                    status: "ERROR",
                    errorScreen: {
                        enforcementMessageViewModel: a
                    }
                })
            }))
        },

As to how the event is dispatched is not known to me since there isn't any evident call in code, but I suspect this would be triggered by injecting a deferred loading script that invokes the call, similar to the previous technique.

Likely this method is linked to https://support.google.com/admanager/answer/6147120

In tandem with:

https://www.youtube.com/s/player/5e928255/player_ias.vflset/en_US/base.js

    g.k.Ch = function(a, b) {
        if (!this.j.has(a))
            return {};
        if ("seek" === b) {
            b = this.J.U().L("html5_dai_enable_active_view_creating_completed_adblock");
            b = void 0 === b ? !1 : b;
            var c = Cr(Lu).cI(a, {});
            c ? Kt(c) : b && (a = Cr(Lu).iF(null, es(), !1, a),
            a.gk = 3,
            Pja([a]));
            return {}
        }
        b = meb(b);
        if (null === b)
            return {};
        var d = this.J.ud()
          , e = d && Mbb(d);
        if (!e)
            return {};
        I1(this.Da.get()) ? (d = this.J.getPresentingPlayerType(!0),
        d = null == (c = this.J.getVideoData(d)) ? void 0 : c.isAd()) : d = this.J.U().L("enable_server_stitched_dai") ? 2 === this.J.getPresentingPlayerType(!0) : 2 === d.getPlayerType();
        if (!d)
            return {};
        c = {
            opt_adElement: e,
            opt_fullscreen: this.Ia.get().isFullscreen()
        };
        return Pka(b, a, c)
    }

Or by testing script injected values behind the ab_det_sc_inj_val (adblock detect script injected values) experiment flag:

    function lqb() {
        try {
            var a = Apa("ab_det_sc_inj_val");
            1 === a.length && "" === a[0] && (a = ["json=JSON.parse(text);for(k of[\"playerAds\",\"adPlacements\",\"adSlots\"])"]);
            for (var b = document.querySelectorAll("script"), c = k(b), d = c.next(); !d.done; d = c.next())
                for (var e = d.value, h = k(a), l = h.next(); !l.done; l = h.next()) {
                    var m = l.value;
                    b = void 0;
                    if (null == (b = e.textContent) ? 0 : b.includes(m))
                        return 0
                }
        } catch (p) {
            return 4
        }
        return 1
    }

This basically checks if there is a script injected in the page that matches a specific string pattern. I assume this might be a method used by an adblocker extension, meaning that specific extension could have the script injected and not doing anything, and it would still trigger the YouTube detection by presence alone.

This assumption is proven true given that's exactly a snippet from one of the methods that were attempted before in uBlock Origin:
https://www.reddit.com/r/uBlockOrigin/comments/163cy47/comment/jyjj4ck/
uBlockOrigin/uAssets#19550 (comment)

These are other methods I could find:

ab_det_apb_b possibly adblock detect adblocplus block
ab_det_el_h possibly adblock detect element hiding
ab_det_fet_wr unknown
ab_det_fet_wr_en unknown
ab_det_gen_re unknown

During this time I haven't been able to encounter a detection method implemented by YouTube that might target specifically the method Iridium uses in its ad manager.

Whereas typical adblockers usually employ the following methods; Object override, Script blocking, and DOM manipulation
Iridium goes a bit further than that and actually disables the ads at the source code before they are processed and rendered:

const adPlacements = Util.getSingleObjectByKey(args, "adPlacements");
const adSlots = Util.getSingleObjectByKey(args, "adSlots");
const playerAds = Util.getSingleObjectByKey(args, "playerAds");
if (adPlacements?.length) {
adPlacements.fill({});
}
if (adSlots?.length) {
adSlots.length = 0;
}
if (playerAds?.length) {
playerAds.length = 0;
}

I am still unable to believe that Iridium can be the reason for the detection, I would be more inclined to believe that Iridium allows issues with third-party adblockers to become more evident which may lead to unintended detection as a result. However, it would also be very valid to consider the blame on the third-party adblocker side if by disabling the third-party adblocker and keeping Iridium enabled the issue does not occur.

Hopefully progress can be done on this, and if there is something I can do on my side to help avoid this issue I'll be more than happy to try and find it.

@JoshS-dev
Copy link

JoshS-dev commented Feb 14, 2024

Unfortunately, even though I understand code I understand very little of web code so I cannot really corroborate with any of the code snippets. But I did try to go into the extensions and disable ublock origin and kept iridium, and yes the adblocking message on youtube still occurred.

I'd at least provide console errors if even relevant, but I get a ton of console errors because of my existing extensions regardless of Iridium even being there so it would likely be a waste of time.

@ParticleCore
Copy link
Owner Author

ParticleCore commented Feb 14, 2024

Thanks for the update. So there are more extensions installed, would it be possible to turn all off except Iridium, refresh YouTube and see if it keeps detecting? Since I am unable to replicate this on my side my goal is to ensure the problem happens on your end only with Iridium running and nothing else.

Also could you tell me what are the other extensions you're running? I can try installing them all on my side and test it as well.

Other than that could you run the following in the browser console (Ctrl+J) and let me know what it returns

$("#base-js").src and yt.config_.PLAYER_JS_URL

It should show something like this:

https://www.youtube.com/s/desktop/2144b1c0/jsbin/desktop_polymer.vflset/desktop_polymer.js

/s/player/2b39c590/player_ias.vflset/en_US/base.js

If you have different files I will try to inject the ones you're being served on my end to see if I can make it happen to me.

@JoshS-dev
Copy link

Yep I got pretty much the exact same responses from those two commands.

As for extensions, these are the ones that I'd imagine even remotely affect YouTube adblocking.

hyde — hide the YouTube video player controls
Dark Reader (It better not lol)
Privacy Badger
Return YouTube Dislike
Volume Master
and Ublock and Iridium.

@ParticleCore
Copy link
Owner Author

Thanks, I was really counting on the files being different in hopes of some new code YouTube might have added that would explain this, but seems like this isn't the case.

I will install all of those and test things on my side as well.

@ParticleCore
Copy link
Owner Author

Forgot to ask, did you try turning off all extensions except Iridum to see if the issue still persists?

Privacy Badger, for example, does interfere with YouTube Ads and has been documented as one extension that triggered YouTube detection: https://www.reddit.com/r/youtube/comments/176apzt/finally_they_just_reached_me_who_is_a/

@ParticleCore
Copy link
Owner Author

ParticleCore commented Feb 14, 2024

Going back to the detection methods I mentioned before, the most common one is such that is being affected by Privacy Badger:

image

image

And this is confirmed by entering the following command in the console on a YouTube page:
window.google_ad_status which returns undefined when it should return 1 if it wasn't affected

So for sure Privacy Badger is affecting ads in YouTube as well, and can even trigger their most basic and oldest detection method.

@JoshS-dev
Copy link

Sorry yeah, I just tried turning off all the extensions now. I kept on my password manager and Session Buddy because I dont want to risk losing any of their data, but with just them and Iridium it still caught the adblock detector.

Also, it can be inferred from that, but turning off Privacy Badger also did not fix it.

@ParticleCore
Copy link
Owner Author

ParticleCore commented Feb 14, 2024

I see, I am now running all of the extensions you listed but I have yet to encounter the detection.

When you mentioned in #928 (comment) "pretty much the exact same responses" did you mean literally the exact same ones or close to it?

Because for the most part the links will be pretty much identical, except for their version IDs. For example, the version id for this one

https://www.youtube.com/s/desktop/2144b1c0/jsbin/desktop_polymer.vflset/desktop_polymer.js

is in the middle 2144b1c0

and for this one /s/player/2b39c590/player_ias.vflset/en_US/base.js is close to the beggining 2b39c590

Are yours using the same id or a different one? If any of them are different let me know what version each one it is respectively so I can try loading them on my side too.

It could be that you're being served with new versions of their scripts that I have yet to receive so the issue has not happened to me yet.

@JoshS-dev
Copy link

Ohhh, I was just looking for a similar looking response.

For the base-js I got 2144b1c0, so the same.

And for player_js_url I got 2b39c590, also the same.

@ParticleCore
Copy link
Owner Author

Thank you, I have officially run out of ideas now and will see if I can make it happen on my side using the same extensions you have.

You mentioned you also had session buddy and a password manager, would it be possible to say which one if that is ok with you?

And is there any other extension you might have installed that I could add to my current test profile?

@ParticleCore
Copy link
Owner Author

I almost forgot, could you attach your Iridium settings so I load the exact same config as yours?

In the Iridium settings you can export your current settings into a .json file, you can just copy the content here or attach it, whichever you prefer.

image

@JoshS-dev
Copy link

Sorry, not comfortable to say which password manager.

And I don't think any other extensions would be too relevant, other extensions in question would be things like BetterTTV and Chrome Tab Counter which do not affect the YouTube page at all. That's pretty much all thats left, extensions for particular websites and extensions that need to be called on.

@ParticleCore
Copy link
Owner Author

That is perfectly understandable. Regarding the Iridium settings would you be willing to share them?

@JoshS-dev
Copy link

Oh I didn't see that message.
Iridium2.0.3.json

@JoshS-dev
Copy link

Hmmmm those are not my usual settings. When they were exported they must have changed to default... youtube wasnt detecting the adblocker for like, 2 minutes. But now even when changing back to default it doesnt work again so I am very confused.

@JoshS-dev
Copy link

JoshS-dev commented Feb 14, 2024

Perhaps its something with my settings, changed from default I:

Disable default video focus
Turn on auto-play
Disable scroll volume
Allow info cards
End screen hidden on

If I had to guess, and I don't know your code, my guess is that may be autoplay.

From this I am pretty sure it is SOMETHING to do with the settings. Because Iridium works for short periods when I switch back and forth from default to my usual settings.

@ParticleCore
Copy link
Owner Author

ParticleCore commented Feb 14, 2024

Thanks for the detailed updates, I am really glad the issue got fixed in a way. The autoplay feature does in fact mess with some player internals so it is possible it might have something to do with it. I will now focus on testing these settings based on what you shared.

Sorry about all this trouble, and I really appreciate your help in trying to figure this one out.

Edit: I am assuming that with the default settings the issue stops happening, am I correct? I ask this because I am a bit confused about this part

works for short periods when I switch back and forth from default to my usual settings

@ParticleCore
Copy link
Owner Author

I will be using the default settings except for these changes:

Autoplay: Enabled
Scroll volume: Disabled
Info cards: Enabled
End screen hidden: Enabled
Video focus: Disabled

image

Let me know if there is any other setting I might need to change to replicate your configuration.

@JoshS-dev
Copy link

Yeah those are the right settings.

And yeah sorry for the confusing wording, on the default settings, after a ctrl-f5 refresh there seems to be no problems, but with those settings that I described, the adblock detector shows up. It was only going in-and-out because I was switching back and forth and perhaps the settings didnt all apply between a normal f5 refresh.

@ParticleCore
Copy link
Owner Author

I am afraid I have yet to make it happen to me.

  • I have been testing with all the listed extensions enabled, as well as with only Iridium enabled
  • I am testing while logged in
  • I tried with the autoplay upnext enabled and disabled (the button on the yt player)
  • I navigated forward and backwards multiple times
  • Left videos paused for a long time and then resumed
  • Switched country to USA, CA, AUS, UK
  • And the ad manager settings are all default

image

I am unsure of what else to try at this moment

@ParticleCore
Copy link
Owner Author

One consideration to test still is updating from the previous store version 0.2.5 to 2.0.3 and see if any old settings are carried over and affect the extension functionality.

If this is the case then uninstalling and installing the extension should clear the issue.

I will test this today.

@JoshS-dev
Copy link

Assuming that means 'remove from chrome' then 'add to chrome' in the chrome web store - I did already try that. Is there another way to fully uninstall and reinstall?

@ParticleCore
Copy link
Owner Author

Thanks for letting me know, and no, what you did was pretty much what needed to be done.

This is quite the puzzling situation, I have been playing videos with a configuration like yours on Chrome since yesterday and not once did the detection got triggered.

I wonder if this is a region-like situation, which is not the same thing as just changing the location in YouTube.

I understand you might not prefer to say it, but I have tested this through VPN in USA and Canada. Do any of these cover your location? If not, and if you're ok with saying it, which other country should I try?

@JoshS-dev
Copy link

Yep, those cover me.

@ParticleCore
Copy link
Owner Author

One thing I haven't confirmed yet because I was assuming it was Chrome all along is what browser are you using? Is it the latest Chrome or is it some other chromium based browser?

@JoshS-dev
Copy link

It is Chrome. I can only assume it is latest Chrome, doesnt it update automatically?

@ParticleCore
Copy link
Owner Author

Thank you so much for going through this trouble, I am glad to finally have something out of this.

So I am understanding that if you only comment the 5th and leave everything else untouched then the issue still takes place, and the same happens if you only comment the first one?

@axxes8
Copy link

axxes8 commented Feb 18, 2024

@ParticleCore
That is correct.

@ParticleCore
Copy link
Owner Author

If you comment the 5th just like before, and only comment this section of the 1st one does the issue still happen?

if (temp?.constructor === Object) {
listeners.forEach((listener) => listener?.(temp));
}

This will let us know if the issue in that override is specifically with just having the override in place, or with the features making use of that override.

@axxes8
Copy link

axxes8 commented Feb 18, 2024

With the if statement commented out the video plays.

@ParticleCore
Copy link
Owner Author

That's good to hear, that would be a tricky one to work around.

There are multiple features that make use of that override, something as simple as the extension button all the way to the ad manager.

Could you do a similar round of tests as before, but this time you only comment each one of these entries one by one and test the issue after each one until the issue stops happening?

OverrideJsonParse.onParseListener(

Remember to uncomment this: #928 (comment)

So in summary:

  • Do not comment anything in the 1st one
  • Only comment the 5th one like before
  • And one by one comment each entry of OverrideJsonParse.onParseListener(, reload the extension and see if the issue still happens, if it still happens then comment another entry of OverrideJsonParse.onParseListener(, reload the extension and see if the issue still happens, and so on until it stops happening

Let me know after which one did the issue finally stop.

@axxes8
Copy link

axxes8 commented Feb 19, 2024

Having the following lines commented out lets the video play:

1322
1457
1507

@ParticleCore
Copy link
Owner Author

So even if just one of those is not commented then the issue shows up.

It is weird that the last one (loudness normalization) is triggering this problem, the other two not so much.

According to your findings, then the following will not cause any detection:

  • Leave 5th commented
  • Enable video ads in Iridium settings
    image
  • Enable info cards in Iridium settings
    image
  • Enable loudness normalization in Iridium settings
    image

Can you confirm with the above changes the detection no longer happens?

@axxes8
Copy link

axxes8 commented Feb 19, 2024

@ParticleCore
That is correct. With those settings toggled the video will play.

Any other combination of those three settings and the detection is triggered.

@ParticleCore
Copy link
Owner Author

Perfectly understood.

I have made a new version with some changes on the 5th you commented to see if maybe the detection does not happen because of that one, could you replace the current Iridium with this one attached?

The only thing you have to do is replace the Iridium files, reload the extension in Chrome, make sure the video ads, info cards, and loudness normalization are enabled in the settings, and test if it still gets detected in YouTube.

iridium2.zip

@axxes8
Copy link

axxes8 commented Feb 19, 2024

With the new files and the settings enabled, the detection is triggered.

@ParticleCore
Copy link
Owner Author

And if you enable autoplay in the settings does the detection stop?
image

@axxes8
Copy link

axxes8 commented Feb 19, 2024

No. The detection is still triggered

@ParticleCore
Copy link
Owner Author

Are you able to comment this portion and test again?

if (data?.args) {
    setLoaded(data, data?.loaded);
}

Should be in lines 423-425

@axxes8
Copy link

axxes8 commented Feb 19, 2024

Still detected.

@ParticleCore
Copy link
Owner Author

Then the issue is with the config override as well.
Thanks for helping me out with all of this. I will try to find a way to work around these issues and see what I can come up with to fix this.

@ParticleCore ParticleCore changed the title Chrome user reports ad manager being detected by YouTube Iridium is causing false ad detection on YouTube Feb 19, 2024
@ParticleCore
Copy link
Owner Author

ParticleCore commented Feb 20, 2024

@axxes8 Could you try testing with the version 3 attached to this reply?

Iridium3.zip

I have completely removed the dependency on JSON.parse override (that was the 1st one here), I had already implemented other methods that made this one redundant so there was no longer a need to use that method.

The 5th one was also redesigned to make use of a different method that hopefully is no longer affecting unintended code.

A couple other overrides that were being used in a wide range were updated to be more limited in hopes of avoiding any false positive detections in the future even though they weren't causing any now.

Let me know if the issue keeps showing up with this new version 3.

@axxes8
Copy link

axxes8 commented Feb 20, 2024

@ParticleCore
So far no issues. I'll use it tonight while I'm doing my nightly watching and let you know if I encounter anything.

@ParticleCore
Copy link
Owner Author

I will be pushing the current changes to the new version in hopes it will also work for other users, if the issue still persists I will see what I can do about it in the next version.

@axxes8
Copy link

axxes8 commented Feb 21, 2024

I've had no issues. Everything seems to be working as intended. I even restored my previous settings and it still worked.

@ParticleCore
Copy link
Owner Author

Thanks for the update, I will then go ahead and close this one. The new version should become available within the next few days on the Chrome web store.

@ParticleCore
Copy link
Owner Author

ParticleCore commented Feb 21, 2024

PS: Remember to switch to the chrome webstore version once the 2.0.4 is out so it won't stay using the unpacked extension by mistake.

@ParticleCore
Copy link
Owner Author

@JoshS-dev the new version 2.0.4 should be publicly available for update on the chrome webstore, if you are still interested in it could you try and see if the issue is also resolved for you?

@ParticleCore ParticleCore reopened this Feb 21, 2024
@JoshS-dev
Copy link

Yep I was testing it today! While I haven't watched as much YouTube as usual today, this seems to be fixed! I haven't had it happen once yet :) Ill change that rating! I hated to leave it cause I've used this extension for years even as it kind of fell into a bit of disrepair haha. So happy to see that it is apparently fixed!

@ParticleCore
Copy link
Owner Author

ParticleCore commented Feb 22, 2024

Don't worry about the rating, I was far more concerned about finding out what your issue was and how I could resolve it. Thanks for letting me know it appears to be fixed for you as well, I will then close this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants