Blocking YouTube Video ads with Electron Ad-Blocker
sgebr01 opened this issue · comments
I have been able to block most site ads with the original filter list, but for some reason this filter list does not work on YouTube Video Ads. What can I do to make it work on YouTube Video Ads? Is there a recommended filter list for this use case?
Hi @sgebr01,
Could you please share how you are using/initializing the adblocker? This library does not contain filter lists per se, but does allow you to download community-driven lists with default presets for convenience. Not blocking ads on YouTube could either be an issue with the lists or with the adblocker library itself. If you can share the exact way you are creating the adblocker in your project I can try to reproduce.
Thanks for responding, here is the code that I'm using that involves the ad-blocker.
ElectronBlocker.fromPrebuiltAdsAndTracking(fetch).then((blocker) => { blocker.enableBlockingInSession(session.defaultSession); });
I got this from the documentation, but really struggled to understand how this adblocker works since the documentation didn't really seem to cover most of it for me. This does block banner ads on every website I visit through my electron app, but it doesn't block Video ads on YouTube. My entire code in my main JS file is this.
const { app, BrowserWindow, session } = require('electron')
const path = require('path')
const { ElectronBlocker, fullLists, FiltersEngine } = require('@cliqz/adblocker-electron')
const fetch = require('cross-fetch') // required 'fetch'
const fs = require('fs');
const { BlockList } = require('net');
let mainWindow
async function createWindow() {
// Create the browser window.
mainWindow = new BrowserWindow({
darkTheme: true,
icon: 'Logo.png',
center: 'true',
resizable: 'true',
movable: 'true',
minimizable: 'true',
maximizable: 'true',
closable: 'true',
focusable: 'true',
fullscreen: 'false',
frame: 'true',
width: '800px',
height: '800px',
titleBarOverlay: 'color',
webPreferences: {
preload: path.join(__dirname, 'preload.js'),
}
})
if (mainWindow.webContents.session === undefined) {
throw new Error('defaultSession is undefined');
}
mainWindow.loadURL('https://youtube.com')
mainWindow.setTitle("YouTube")
mainWindow.setBackgroundColor('#181818')
ElectronBlocker.fromPrebuiltAdsAndTracking(fetch).then((blocker) => {
blocker.enableBlockingInSession(session.defaultSession);
});
mainWindow.webContents.on("did-finish-load", () => {
mainWindow.show();
})
mainWindow.on("page-title-updated", event => {
event.preventDefault()
})
mainWindow.on('closed', function () {
mainWindow = null
})
}
app.on('ready', createWindow)
app.on('window-all-closed', function () {
app.quit()
})
Additionally, there are times when even the banners on YouTube don't get blocked - but I think this can easily be solved by adding another filter list.
I could reproduce. I seems like part of the adblocker features are not working on Electron anymore, it might have been caused by a regression with recent updates. To be honest I am not actively using Electron in any project at the moment and it has always been challenging to keep the adblocker working on this platform in the past. I will try to find some time to investigate.
OK, so as of now - what should I do? Can I install a previous past version and that will work?
Let me check if I can find an older version that works.
For example v13 works, maybe a more recent version would also work.
OK, I will try that out and let you know - additionally, I had one more question as to using multiple filter lists. I saw another closed post that used this, but when I tried that method, it did not work, none of the filter lists did their job. Here is the link - #1901 What I've tried to do my multiple filter lists is this - is this the right way?
ElectronBlocker.fromPrebuiltAdsAndTracking(fetch).then((blocker) => {
blocker.enableBlockingInSession(session.defaultSession);
});
const easylist= await ElectronBlocker.fromLists(fetch, [
'https://easylist.to/easylist/easylist.txt'
]);
easylist.enableBlockingInSession(session.defaultSession)
const easyprivacy = await ElectronBlocker.fromLists(fetch, [
'https://easylist.to/easylist/easyprivacy.txt'
]);
easyprivacy.enableBlockingInSession(session.defaultSession)
No this will not work, at the moment you can only use one blocker per session. The only way would be to use fromLists(...) and give all the lists you need in one go. Are you trying to selectively enable part of the lists in a session based on some conditions?
There is something fishy with the adblocker in electron, I have been able to make it work with v17 (latest release) but it never works on the first start. If I cache the engine on disk then restart then it works, it is unclear to me why that is happening.
I'm just trying to load all of these filter lists at once, to have a bigger database of filters. I have also noticed that the video ads are blocked at times, I'm not sure if that has anything to do with this, but it does work from time to time when I run it - even though I didn't cache the engine.
It seems that for some reason the preload script is either not registered as it should be, or does not manage to send messages to the main process. I am not sure why that is. Since that does not happen consistently I have tried to let some time with a setTimeout without success.
On your example above, you need to enable the blocker before you navigate to YouTube:
await ElectronBlocker.fromPrebuiltAdsAndTracking(fetch).then((blocker) => {
blocker.enableBlockingInSession(session.defaultSession);
});
mainWindow.loadURL('https://youtube.com')
mainWindow.setTitle("YouTube")
mainWindow.setBackgroundColor('#181818')
OK, and this should work?
I'm just trying to load all of these filter lists at once, to have a bigger database of filters. I have also noticed that the video ads are blocked at times, I'm not sure if that has anything to do with this, but it does work from time to time when I run it - even though I didn't cache the engine.
Try to call fromLists only once with multiple list in the array then, it should work.
OK, and this should work?
There is still this issue where the preload script does not consistently work. But with your current code the adblocker will never block on initial page load.
Using the array seems to work - here is what I have done -
const blocker = await ElectronBlocker.fromLists(fetch, [
'https://easylist.to/easylist/easylist.txt',
'https://easylist.to/easylist/easyprivacy.txt',
]);
blocker.enableBlockingInSession(session.defaultSession)
mainWindow.loadURL('https://youtube.com')
Also, which is v13, my current version is only 1.23.6 using npm.
Also, which is v13, my current version is only 1.23.6 using npm.
v13 should be the version of Electron you are using.
My bad, I thought you were talking about the version of the ad blocker. I've tried using v13.0.0 of electron to no success. Could the version of electron-packager and builder make a difference? This is with the latest version of the ad-blocker.
Sure - my main.js is this
const { app, BrowserWindow, session } = require('electron')
const path = require('path')
const { ElectronBlocker, fullLists, FiltersEngine } = require('@cliqz/adblocker-electron')
const fetch = require('cross-fetch') // required 'fetch'
const fs = require('fs');
const { BlockList } = require('net');
let mainWindow
async function createWindow() {
// Create the browser window.
mainWindow = new BrowserWindow({
darkTheme: true,
icon: 'AdExtractLogo.png',
center: 'true',
resizable: 'true',
movable: 'true',
minimizable: 'true',
maximizable: 'true',
closable: 'true',
focusable: 'true',
fullscreen: 'false',
frame: 'true',
width: '800px',
height: '800px',
titleBarOverlay: 'color',
webPreferences: {
preload: path.join(__dirname, 'preload.js'),
}
})
if (mainWindow.webContents.session === undefined) {
throw new Error('defaultSession is undefined');
}
ElectronBlocker.fromPrebuiltAdsAndTracking(fetch).then((blocker) => {
blocker.enableBlockingInSession(session.defaultSession);
});
mainWindow.loadURL('https://youtube.com')
mainWindow.setTitle("YouTube")
mainWindow.setBackgroundColor('#181818')
mainWindow.webContents.on("did-finish-load", () => {
mainWindow.show();
})
mainWindow.on("page-title-updated", event => {
event.preventDefault()
})
mainWindow.on('closed', function () {
mainWindow = null
})
}
app.on('ready', createWindow)
app.on('window-all-closed', function () {
app.quit()
})
This is my preload.js
window.addEventListener('DOMContentLoaded', () => {
const replaceText = (selector, text) => {
const element = document.getElementById(selector)
if (element) element.innerText = text
}
for (const type of ['chrome', 'node', 'electron']) {
replaceText(`${type}-version`, process.versions[type])
}
})
And my package.json
{
"name": "youtube-desktop",
"ProductName": "YouTube",
"version": "1.0",
"description": "YouTube App for Desktops",
"main": "main.js",
"scripts": {
"start": "electron .",
"osx": "electron-packager . --overwrite --platform=darwin --arch=x64 --icon=./icon/youtube.icns --prune=true --out=./dist ",
"windows": "electron-packager . --overwrite --platform=win32 --arch=ia32 --icon=./icon/youtube.icns --prune=true --out=./dist ",
"linux": "electron-packager . --overwrite --platform=linux --icon=./icon/youtube.icns --prune=true --out=./dist "
},
"dependencies": {
"@cliqz/adblocker-electron": "^1.23.6",
"electron": "^13.0.0",
"electron-builder": "^22.14.13",
"electron-packager": "^15.4.0",
"tslint-config-standard": "^9.0.0"
}
}
Additionally these are the packages that are installed
+-- @cliqz/adblocker-electron@1.23.6
+-- electron-builder@22.14.13
+-- electron-packager@15.4.0
+-- electron@13.0.0
`-- tslint-config-standard@9.0.0
I can see that a lot of requests are getting blocked, but the ad and its corresponding ad banner is not getting blocked.
Have you been able to find out why? If you are busy since it's a weekday, no worries - I appreciate your help so far.
Hey @sgebr01, sorry I did not get the time to investigate further but I will try to find the time. It's definitely something that needs fixing.
No worries, go at your own pace.
Hello @sgebr01,
I would just like to chime in and say that it does work for me. I am using Electron 16.
with the following configuration. Hopefully this helps.
Also note the use of async and await to wait for the fetch to complete.
import { ElectronBlocker, fullLists } from '@cliqz/adblocker-electron';
import { fetch } from 'cross-fetch';
import { session } from 'electron';
import { readFileSync, writeFileSync } from "original-fs";
module.exports = async function enableAdBlocker(){
var adBlocker = await ElectronBlocker.fromLists(
fetch,
fullLists,
{
enableCompression: true
},
{
path: `engine.bin`,
read: async (...args) => readFileSync(...args),
write: async (...args) => writeFileSync(...args),
}
);
adBlocker.enableBlockingInSession(session.defaultSession);
}Also make sure you are using the latest version: "@cliqz/adblocker-electron": "^1.23.6"
Thanks for chiming in @kylegundersen. Out of curiosity, does it also work when you start your Electron app immediately after deleting the on-disk cache? (engine.bin)
(sorry for closing/opening, wrong button :))
@remusao yea it does. Actually I had the same issue a week or two ago. Upgrading to the latest version is what fixed the YouTube Ad issue for me, because of the filter updates in the lists.
Sure, I will try this out later today, and get back to you if it works.
@kylegundersen I am still seeing some video ads, here is my code - however, I am guessing that some are getting blocked since I can see in my terminal [4468:0225/185105.745:ERROR:ssl_client_socket_impl.cc(983)] handshake failed; returned -1, SSL error code 1, net_error -100
Main.js Code
const { ElectronBlocker, fullLists } = require('@cliqz/adblocker-electron');
const { fetch } = require('cross-fetch');
const { session, BrowserWindow, app } = require('electron');
const { readFileSync, writeFileSync } = require("original-fs");
const path = require('path');
let mainWindow
module.exports = async function enableAdBlocker(){
var adBlocker = await ElectronBlocker.fromLists(
fetch,
fullLists,
{
enableCompression: true
},
{
path: `engine.bin`,
read: async (...args) => readFileSync(...args),
write: async (...args) => writeFileSync(...args),
}
);
adBlocker.enableBlockingInSession(session.defaultSession);
}
async function createWindow () {
// Create the browser window.
mainWindow = new BrowserWindow({
width: 800,
height: 600,
titleBarStyle: 'hidden',
darkTheme: true,
icon: 'icon/youtube.ico',
webPreferences: {
preload: path.join(__dirname, 'preload.js'),
}
})
if (mainWindow.webContents.session === undefined) {
throw new Error('defaultSession is undefined');
}
mainWindow.loadURL('https://youtube.com')
mainWindow.setTitle("Youtube")
mainWindow.setBackgroundColor('#181818')
mainWindow.webContents.on("did-finish-load", () => {
mainWindow.show();
})
mainWindow.webContents.on("did-fail-load", () => {
if(confirm("Youtube could not be loaded. Retry?")) {
mainWindow.reload()
}
else {
mainWindow.close()
}})
mainWindow.on("page-title-updated", event => {
event.preventDefault()
})
mainWindow.on('closed', function () {
mainWindow = null
})
}
app.on('ready', createWindow)
app.on('window-all-closed', function () {
app.quit()
})
Hey @sgebr01, I was using commonJS to load it from a separate file, so you can remove the module.exports = . Also you need to trigger that function enableAdBlocker() because I am not seeing that in the code above. Let me know how it goes, good luck.
My bad, I'm trying that out now - also thank you for your help. Are the error codes I showed you in my previous comment showing that the video is getting blocked?
I tried it, and I'm still getting video ads.
Here is my updated code.
const { ElectronBlocker, fullLists } = require('@cliqz/adblocker-electron');
const { fetch } = require('cross-fetch');
const { session, BrowserWindow, app } = require('electron');
const { readFileSync, writeFileSync } = require("original-fs");
const path = require('path');
let mainWindow
async function enableAdBlocker(){
var adBlocker = await ElectronBlocker.fromLists(
fetch,
fullLists,
{
enableCompression: true
},
{
path: `engine.bin`,
read: async (...args) => readFileSync(...args),
write: async (...args) => writeFileSync(...args),
}
);
adBlocker.enableBlockingInSession(session.defaultSession);
}
enableAdBlocker();
async function createWindow () {
// Create the browser window.
mainWindow = new BrowserWindow({
width: 800,
height: 600,
titleBarStyle: 'hidden',
darkTheme: true,
icon: 'icon/youtube.ico',
webPreferences: {
preload: path.join(__dirname, 'preload.js'),
}
})
if (mainWindow.webContents.session === undefined) {
throw new Error('defaultSession is undefined');
}
mainWindow.loadURL('https://youtube.com')
mainWindow.setTitle("Youtube")
mainWindow.setBackgroundColor('#181818')
mainWindow.webContents.on("did-finish-load", () => {
mainWindow.show();
})
mainWindow.webContents.on("did-fail-load", () => {
if(confirm("Youtube could not be loaded. Retry?")) {
mainWindow.reload()
}
else {
mainWindow.close()
}})
mainWindow.on("page-title-updated", event => {
event.preventDefault()
})
mainWindow.on('closed', function () {
mainWindow = null
})
}
app.on('ready', createWindow)
app.on('window-all-closed', function () {
app.quit()
})
Sorry to hear that @sgebr01. What version of Electron and Cliqz are you running? Also I would try adding await and putting the enableAdBlocker function inside the createWindow function as it fires when everything in Electron is ready.
BTW I tested it locally with the code you provided above and it worked as expected for me once I moved the enableAdBlocker function.
Here was my package.json dependencies.
"dependencies": {
"@cliqz/adblocker-electron": "^1.23.6",
"cross-fetch": "^3.1.5"
},
"devDependencies": {
"electron": "^16.0.6"
}I tried this out as well, it works on some videos, but on others it doesn't, Is it 100% effective for you? I replaced my dependencies and installed the exact versions with npm, afterwards, I moved the enableAdBlocker function inside my create window function as well. I'm guessing this is still an issue with the ad-blocker?
@sgebr01 Yea it has been 100% effective for me. When upgrading dependencies did you then delete the bin file?
That might be the issue, I think I need to do that.
Yea its my bad, I probably should have removed it from the sample, its just a simple way to cache filters for consecutive startups.
Still didn't work, after deleting that file.
Maybe there is a preferred place to put the enableAdBlocker function? What is the updated code that you used? I put it into the createWindow function, I'm not sure if placing it towards the end or the beginning will have an effect?
AD-blocker YouTube.zip
Here is my simple sample I made from the code you provided.
OK, I tried this, it looks like it is working, but I think I will need to test it more. I'll try it tomorrow, and get back to you. Thank You for the Help!
Looks like it's working, I'm not sure why I didn't it work for me before. I am guessing that this should work on webview elements as well?
Edit - Testing it out - it does.
@sgebr01 Awesome, glad you got it all to work. Please feel free to close this issue.😄
Sure, I'm just wondering why it didn't work with the previous versions - thanks though for the help, I really appreciate it!