Giters

warren-bank / HLS-Proxy

Node.js server to proxy HLS video streams

Home Page:http://webcast-reloaded.surge.sh/proxy.html

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

feature request: more secured video url source

siberkolosis opened this issue · comments

commented

hope HLS-Proxy have this feature.
i have implemented to website. people can easy to decode base64 to view url resource.

thank you.

commented

hmm..
this could be done if I was to add a new hook function: rewrite

for example:

  hlsd -v 3 --port 8080 --hooks "/path/to/hooks.js"

where:

  // hooks.js

  const Cryptr = require('cryptr')
  const cryptr = new Cryptr('my secret symmetric AES encryption key')

  const encrypted_url_prefix = 'http://encrypted.example.com/'

  const encrypt_url = (cleartext_url) => {
    const encrypted_url = (cleartext_url)
      ? encrypted_url_prefix + cryptr.encrypt(cleartext_url)
      : ''

    return encrypted_url
  }

  const decrypt_url = (encrypted_url) => {
    let cleartext_url = encrypted_url

    if (encrypted_url && encrypted_url.startsWith(encrypted_url_prefix)) {
      encrypted_url = encrypted_url.substring(encrypted_url_prefix.length, encrypted_url.length)

      cleartext_url = cryptr.decrypt(encrypted_url)
    }

    return cleartext_url
  }

  module.exports = {
    "redirect": (cleartext_url) => encrypt_url(cleartext_url),
    "rewrite":  (encrypted_url) => decrypt_url(encrypted_url)
  }
  • redirect already allows you to write a function that modifies URLs extracted from manifest files before they are base64 encoded to load through the proxy
    • ex: obfuscate URLs with encryption
  • rewrite would give you the opportunity to write a function that modifies base64 decoded URLs requested through the proxy before they are requested from an origin server
    • ex: decrypt URLs so the proxy can request the actual URLs
commented

sorry i mean .
proxy_url='https://mysite.com';
video_url="https://cph-msl.akamaized.net/hls/live/2000341/test/master.m3u8";
file_extension='.m3u8';
hls_proxy_url="${proxy_url}/"$(echo -n "$video_url" | base64 --wrap=0)"$file_extension";
echo $hls_proxy_url;

will generate. https://mysite.com/aHR0cHM6Ly9jcGgtbXNsLmFrYW1haXplZC5uZXQvaGxzL2xpdmUvMjAwMDM0MS90ZXN0L21hc3Rlci5tM3U4.m3u8

aHR0cHM6Ly9jcGgtbXNsLmFrYW1haXplZC5uZXQvaGxzL2xpdmUvMjAwMDM0MS90ZXN0L21hc3Rlci5tM3U4 can easily decoded back to https://cph-msl.akamaized.net/hls/live/2000341/test/master.m3u8

i dont want people can see video url source.
including the URL inside master m3u8 playlist.

commented

v3.4.3 adds support for this hook function, as previously described.

this recipe is verbatim from my last comment. It works perfectly.. to obfuscate all URLs.

To be clear, if a visitor to your website was to base64 decode the data being passed between video player client and hls-proxy.. they'd only see encrypted URLs. Without the symmetric key used by your hook functions (redirect to encrypt, and rewrite to decrypt).. it's completely meaningless.

commented

yes. it works perfect. another one that I really need.

proxy_url='https://mysite.com/';
video_url="https://cph-msl.akamaized.net/hls/live/2000341/test/master.m3u8";
file_extension='.m3u8';
hls_proxy_url="${proxy_url}/"$(echo -n "$video_url" | base64 --wrap=0)"$file_extension";
echo $hls_proxy_url;

how to encrypt result echo $hls_proxy_url
because this generated url is clear base64 which I pasted in my site's web player input.

is good if hlsd have built in paramater or command to generate ecrypted URL to HLS Proxy .

commented

the simplest way would be to use the same hook function from the command-line:

proxy_url='https://mysite.com/'
video_url="https://cph-msl.akamaized.net/hls/live/2000341/test/master.m3u8"
file_extension='.m3u8'
hls_proxy_url="${proxy_url}/"$(echo -n "$video_url" | base64 --wrap=0)"$file_extension"

# hooks_js='https://mysite.com//aHR0cHM6Ly9jcGgtbXNsLmFrYW1haXplZC5uZXQvaGxzL2xpdmUvMjAwMDM0MS90ZXN0L21hc3Rlci5tM3U4.m3u8'

hooks_js='/path/to/hooks'
hls_proxy_url=$(node -e "const hooks = require('${hooks_js}'); process.stdout.write(hooks.rewrite('${hls_proxy_url}'));")
echo "$hls_proxy_url"
commented

cool.. thank you very much....

commented

got this error..

node:internal/crypto/cipher:122
this[kHandle].initiv(cipher, credential, iv, authTagLength);
^

TypeError: Invalid initialization vector
at Decipheriv.createCipherBase (node:internal/crypto/cipher:122:19)
at Decipheriv.createCipherWithIV (node:internal/crypto/cipher:141:3)
at new Decipheriv (node:internal/crypto/cipher:295:3)
at Object.createDecipheriv (node:crypto:149:10)
at Cryptr.decrypt (/home/jmpr/node_modules/cryptr/index.js:72:33)
at decrypt_url (/home/jmpr/hls/hook.js:20:30)
at Object.rewrite (/home/jmpr/hls/hook.js:28:36)
at [eval]:1:77
at Script.runInThisContext (node:vm:123:12)
at Object.runInThisContext (node:vm:299:38) {
code: 'ERR_CRYPTO_INVALID_IV'
}

Node.js v18.17.1

it works if I use a different url to the one in hooks.js .
but echo "$hls_proxy_url" still base64
still showing aHR0cHM6Ly9jcGgtbXNsLmFrYW1haXplZC5uZXQvaGxzL2xpdmUvMjAwMDM0MS90ZXN0L21hc3Rlci5tM3U4

proxy_url='https://mysite.com/'
video_url="https://cph-msl.akamaized.net/hls/live/2000341/test/master.m3u8"
file_extension='.m3u8'
hls_proxy_url="${proxy_url}/"$(echo -n "$video_url" | base64 --wrap=0)"$file_extension"

hooks_js='/home/jmpr/hls/hook.js'
hls_proxy_url=$(node -e "const hooks = require('${hooks_js}'); process.stdout.write(hooks.rewrite('${hls_proxy_url}'));")
echo "$hls_proxy_url"

https://mysite.com//aHR0cHM6Ly9jcGgtbXNsLmFrYW1haXplZC5uZXQvaGxzL2xpdmUvMjAwMDM0MS90ZXN0L21hc3Rlci5tM3U4.m3u8

commented

yeah.. that's probaby because my code was incorrect.. the right idea.. but rushed.. and I messed up. sorry.

here's how it should go:

hooks_js='/path/to/hooks'
cleartext_video_url='https://cph-msl.akamaized.net/hls/live/2000341/test/master.m3u8'
encrypted_video_url=$(node -e "const hooks = require('${hooks_js}'); process.stdout.write(hooks.redirect('${cleartext_video_url}'));")

echo "$encrypted_video_url"
# http://encrypted.example.com/f7b1c95b2eb8ff8681833ecb443ac73039e0f31937971f5874c58adff80d1f310b77ce3ac2e3a31f02bfa9ac11a16566e93f07b7f4f5aace133a28d1dfad9d93f570354c45b857478608bd706ced98438565116c53bddfea725d501361ab8067e072d320fe9e32d05d78b3c890cef9243df6e292b9a192c017089ff2d7d6ce7da7f075fa59f3764b161a4c4d8cc03cdeffd06b915da78546cb1412a3b75c6e

proxy_url='https://mysite.com'
video_url="$encrypted_video_url"
file_extension='.m3u8'
hls_proxy_url="${proxy_url}/"$(echo -n "$video_url" | base64 --wrap=0)"$file_extension"

echo "$hls_proxy_url"
# https://mysite.com/aHR0cDovL2VuY3J5cHRlZC5leGFtcGxlLmNvbS9mN2IxYzk1YjJlYjhmZjg2ODE4MzNlY2I0NDNhYzczMDM5ZTBmMzE5Mzc5NzFmNTg3NGM1OGFkZmY4MGQxZjMxMGI3N2NlM2FjMmUzYTMxZjAyYmZhOWFjMTFhMTY1NjZlOTNmMDdiN2Y0ZjVhYWNlMTMzYTI4ZDFkZmFkOWQ5M2Y1NzAzNTRjNDViODU3NDc4NjA4YmQ3MDZjZWQ5ODQzODU2NTExNmM1M2JkZGZlYTcyNWQ1MDEzNjFhYjgwNjdlMDcyZDMyMGZlOWUzMmQwNWQ3OGIzYzg5MGNlZjkyNDNkZjZlMjkyYjlhMTkyYzAxNzA4OWZmMmQ3ZDZjZTdkYTdmMDc1ZmE1OWYzNzY0YjE2MWE0YzRkOGNjMDNjZGVmZmQwNmI5MTVkYTc4NTQ2Y2IxNDEyYTNiNzVjNmU=.m3u8

tested with: node v16.14.0

commented

and..
just to be clear..
what happens when this is requested by a client..
is that the proxy will do:

  const encoded_url = 'aHR0cDovL2VuY3J5cHRlZC5leGFtcGxlLmNvbS9mN2IxYzk1YjJlYjhmZjg2ODE4MzNlY2I0NDNhYzczMDM5ZTBmMzE5Mzc5NzFmNTg3NGM1OGFkZmY4MGQxZjMxMGI3N2NlM2FjMmUzYTMxZjAyYmZhOWFjMTFhMTY1NjZlOTNmMDdiN2Y0ZjVhYWNlMTMzYTI4ZDFkZmFkOWQ5M2Y1NzAzNTRjNDViODU3NDc4NjA4YmQ3MDZjZWQ5ODQzODU2NTExNmM1M2JkZGZlYTcyNWQ1MDEzNjFhYjgwNjdlMDcyZDMyMGZlOWUzMmQwNWQ3OGIzYzg5MGNlZjkyNDNkZjZlMjkyYjlhMTkyYzAxNzA4OWZmMmQ3ZDZjZTdkYTdmMDc1ZmE1OWYzNzY0YjE2MWE0YzRkOGNjMDNjZGVmZmQwNmI5MTVkYTc4NTQ2Y2IxNDEyYTNiNzVjNmU='
  const decoded_url = atob(encoded_url)

  console.log(decoded_url)
  // http://encrypted.example.com/f7b1c95b2eb8ff8681833ecb443ac73039e0f31937971f5874c58adff80d1f310b77ce3ac2e3a31f02bfa9ac11a16566e93f07b7f4f5aace133a28d1dfad9d93f570354c45b857478608bd706ced98438565116c53bddfea725d501361ab8067e072d320fe9e32d05d78b3c890cef9243df6e292b9a192c017089ff2d7d6ce7da7f075fa59f3764b161a4c4d8cc03cdeffd06b915da78546cb1412a3b75c6e

  const cleartext_url = hooks.rewrite(decoded_url)

  console.log(cleartext_url)
  // https://cph-msl.akamaized.net/hls/live/2000341/test/master.m3u8

  // request from origin server, modify URLs in manifest, serve..
  // conditionally prefetch some media segments..
commented

very perfect. thank you for taking the time....

commented

not work if source url have path like this. without hooks.js , its work.

hls/mykr/index.m3u8?hdnts=st=1700725590exp=1700772383acl=/hls/ipfmix/*id=f655d1f39e42355f15fcaef898bf7af3data=UmFXQzJ1RExhaGtXb29zajE4UTVCSHlzeEhTdUhnN292V0tKdHFLWmM5UEpFQUp1MXB2OUFjJTJCazVYTEdkakdKhmac=bbd303937d081f9a7721929d56ed3e9baaafc4dcc777e06711d91fc29c62b777&hdnts2=st=1700725590exp=1700772383acl=/hls/ipfmix/*id=f655d1f39e42355f15fcaef898bf7af3data=UmFXQzJ1RExhaGtXb29zajE4UTVCSHlzeEhTdUhnN292V0tKdHFLWmM5UEpFQUp1MXB2OUFjJTJCazVYTEdkakdKhmac=bbd303937d081f9a7721929d56ed3e9baaafc4dcc777e06711d91fc29c62b777

commented

I assume you truncated the URL and just didn't include the protocol and host and some of its path in this issue, but you did include them when using the proxy.

Here is a test:

#!/usr/bin/env bash

original_url='https://cph-msl.akamaized.net/foo/bar/baz/hls/mykr/index.m3u8?hdnts=st=1700725590~exp=1700772383~acl=/hls/ipfmix/*~id=f655d1f39e42355f15fcaef898bf7af3~data=UmFXQzJ1RExhaGtXb29zajE4UTVCSHlzeEhTdUhnN292V0tKdHFLWmM5UEpFQUp1MXB2OUFjJTJCazVYTEdkakdK~hmac=bbd303937d081f9a7721929d56ed3e9baaafc4dcc777e06711d91fc29c62b777&hdnts2=st=1700725590~exp=1700772383~acl=/hls/ipfmix/*~id=f655d1f39e42355f15fcaef898bf7af3~data=UmFXQzJ1RExhaGtXb29zajE4UTVCSHlzeEhTdUhnN292V0tKdHFLWmM5UEpFQUp1MXB2OUFjJTJCazVYTEdkakdK~hmac=bbd303937d081f9a7721929d56ed3e9baaafc4dcc777e06711d91fc29c62b777'

hooks_js='/path/to/hooks'
encrypted_url=$(node -e "const hooks = require('${hooks_js}'); process.stdout.write(hooks.redirect('${original_url}'));")
cleartext_url=$(node -e "const hooks = require('${hooks_js}'); process.stdout.write(hooks.rewrite('${encrypted_url}'));")

if [ ! "$original_url" == "$encrypted_url" ];then
  echo 'encryption: OK'
else
  echo 'encryption: FAIL'
fi


if [ "$original_url" == "$cleartext_url" ];then
  echo 'roundtrip: OK'
else
  echo 'roundtrip: FAIL'
fi

What exactly is the issue?
My test results are:

encryption: OK
roundtrip: OK
commented

I added a test bash script to the "recipe" for what we've been discussing..
so it's easy to run, and easy to test (just edit the bash script to configure your own video urls, etc).

commented

Doing a little Sunday morning housekeeping.. and marked this issue as closed.

If you can find a video URL to plug into the test script that results in a failing result,
then please let me know.. and I'll take another look.

commented

i try from another source video url. its work.
not work if using hooks.js for any video or vod for motogp.com you can try with any video on motogp.com just change mpd extension to m3u8.

commented

you were correct that:

  • motogp.com hosts videos and although the video player on its site uses DASH streams.. you can simply change the file extension from .mpd to .m3u8.. and the server response will change to an HLS stream

you were incorrect that:

  • you said it doesn't play in HLS Proxy.. I just tested a stream and it played properly

I believe your issue is that:

  • the lifespan of each video stream URL is very short
  • after about a minute, the server response will change to 403
  • so it's best to automate the process: capture URL, change to HLS format, (optionally) encrypt, encode for proxy, send to video player
commented

observations:

  • the video URL I tested included: 
    index.m3u8?hdnts=st=1701116003~exp=1701116395
  • I'm assuming that these are 2 timestamps: start and expiry 
      console.log(`start: ${new Date(1701116003 * 1000)}`)
  // start: Mon Nov 27 2023 12:13:23 GMT-0800 (Pacific Standard Time)

  console.log(`expiry: ${new Date(1701116395 * 1000)}`)
  // expiry: Mon Nov 27 2023 12:19:55 GMT-0800 (Pacific Standard Time)

  console.log(`lifespan (seconds): ${Math.floor((new Date(1701116395 * 1000) - new Date(1701116003 * 1000)) / 1000)}`)
  // lifespan (seconds): 392

  console.log(`lifespan (minutes): ${392/60}`)
  // lifespan (minutes): 6.5
commented

yes it play properly when not using hooks.js to encrypt url. video dont play if using hooks.js to ecrypts url. it happen only on motogp.com

i setup hls proxy behind nginx
with this command on server
hlsd --hooks "hooks.js" --tls --port 8081 --manifest-extension "m3u8" --segment-extension "pict

generate inside playlist look like this.
#EXTM3U
#EXT-X-VERSION:3
#EXT-X-TARGETDURATION:6
#EXT-X-MEDIA-SEQUENCE:3806738
#EXTINF:6.000000,
https://mysite.com/aHR0cHM6Ly9zdHYuYmlvL2JkYzMxZmZmMWQwYjgzMTMxY2VjY2RmYjg0YWIxM2M0OTlmODY0M2ZhNDY2MDg3OTkwZWNkNmE2ODY5YzUzMTgwZWM0Y2EzZDNhYmNkMDQwZTM4YjJlN2Q5NjY5MmNiZThkNzQ3YTU3MDE1YTE3ZTZhYTdiZTU5YWMxZDdmOTA5NTAzMDBiOTgwMmMwNTQ5ZGZhZDIwZDM1YjI1NjY3YWUzNTNlMWY0OTlkMjBlZTFkOTI4MjdhZmJiYmNiMjM4NjcwNWE4M2NiMWNhYmY4ZjY1MDdjNGQ2ODMyYjdjMzA5OTdiMzVjNDk2ODhlMjExODYwMzRjY2NjZWRmNWI0MTk5ODAxNWZiMTcwYmYwODhlZmJhZmMyMGVkMWRjZGRhNmMyZTk1ZmE2YjFhZDUzM2U4NWYzZmQxNDk4YTc3OWU5YTlhY2E0MzUwNzAzOGMxM2MwZGZmODA3MWQ3MjUxM2M1M2Y5NTc1YQ==.pict
#EXTINF:6.000000,
https://mysite.com/aHR0cHM6Ly9zdHYuYmlvLzQxOGIwOWMxOWViZjAwMjdjZGJhNmIzM2U3ZGVlZGQxN2Q3MDM0Zjk1NmQ0ZWZhNjdhNTQ3M2Y3NTE4NGRlNmYwNjZmNTM0ZmIwYmFkN2NjOTI4ZTA1MWJhYmE0MTc0ZDUwNjE0NTYxNTgzNzUxNTY4ZTFlZjE4N2Y1NjA1ZjQxNWFkZWNjMjEwZWYzOWY4NWI1ZTllM2FjNzFiZjk4MmJiMmQ3Mzg2NzUxZGQxMmZmZGNlNGJmYjk0NGNkOTg5MTg1YjUyN2Q2M2JjMzk1N2QzNmNiZWY2N2JkNmQ3Y2QxNjg0ZTJiM2YyNmQyNDMzYzBmMWVkZWExMDQ3NzM3YzUzYzNmNTc2NGZlMzc4OGQ3OGJiMTVlMjQwMGE0OWVhZDU1NDljZjk3Y2UyMGJmNTk3N2M3ODJkYjc3ODNjOGVhYzdmMmY1MzE2ZmRiODkwODNjNDk3ZmFlNmNkOGY4NDkxNjA3ZDlkZA==.pict
#EXTINF:6.000000,
https://mysite.com/aHR0cHM6Ly9zdHYuYmlvL2U5NzFkYjU0ZmI0NTk0MWExYzEwOTRlNWM0ZTc0OGMxMWVjMjRjOGJiZTViYTU2ODM1ZTI2ZjNjNzllMmNmMTZhOWI4MWFjMGZjNzE3MGZjYmVjNTA3MmVkNzlkNWY5YzliYmExNzYyMWY2MzNkNDA1ZWExYzhiZGMxNzE3Yzc4NjYxODJiMmVhNjU4MjA2NGQzZjU1NTk4OWFmMDhiZWFmM2MyNTNkMGViY2QxYzZlZDViOTliMDhkYzQ0ZGJmNzA3ZDYxNTk5NWZmYmFmNTc2M2QxZjRmNmY0MWI5NTZhODVhZjhkMjg4Y2Y2OGY2MmJjNzE1ZGMwMTVkYjhiZWRmNWVhZTQwOWNlYzM4YzE1NjM1ODZlNDVhMTczNWFiOTUzNGIwNWZjY2ExNzlmMzljNzZlYjkxNzdmNWYyNzdhODIxODEwMTUwNmZmZDg3MTE5YTQwYzUxNWYwOTRjZjAzZjc2NzZlMw==.pict

there's something else I want
i want to set url domain inside playlist
changed from mysite.com to https://images-blogger-opensocial.googleusercontent.com/gadgets/proxy?refresh=2592000&container=focus&gadget=a&url=https://mysite.com
so will be look like this

#EXTM3U
#EXT-X-VERSION:3
#EXT-X-TARGETDURATION:6
#EXT-X-MEDIA-SEQUENCE:3806738
#EXTINF:6.000000,
https://images-blogger-opensocial.googleusercontent.com/gadgets/proxy?refresh=2592000&container=focus&gadget=a&url=https://mysite.com /aHR0cHM6Ly9zdHYuYmlvL2U5NzFkYjU0ZmI0NTk0MWExYzEwOTRlNWM0ZTc0OGMxMWVjMjRjOGJiZTViYTU2ODM1ZTI2ZjNjNzllMmNmMTZhOWI4MWFjMGZjNzE3MGZjYmVjNTA3MmVkNzlkNWY5YzliYmExNzYyMWY2MzNkNDA1ZWExYzhiZGMxNzE3Yzc4NjYxODJiMmVhNjU4MjA2NGQzZjU1NTk4OWFmMDhiZWFmM2MyNTNkMGViY2QxYzZlZDViOTliMDhkYzQ0ZGJmNzA3ZDYxNTk5NWZmYmFmNTc2M2QxZjRmNmY0MWI5NTZhODVhZjhkMjg4Y2Y2OGY2MmJjNzE1ZGMwMTVkYjhiZWRmNWVhZTQwOWNlYzM4YzE1NjM1ODZlNDVhMTczNWFiOTUzNGIwNWZjY2ExNzlmMzljNzZlYjkxNzdmNWYyNzdhODIxODEwMTUwNmZmZDg3MTE5YTQwYzUxNWYwOTRjZjAzZjc2NzZlMw==.pict

how to do this? already try adding --host option in hlsd parameter. dont work. just get a messed up url inside playlist.

commented

a few quick notes:

  • try these cli options: 
    hlsd --port 8080 --host "mysite.com:443"--hooks "hooks.js" --segment-extension "jpg"
  • when you ran this, was "hooks.js" in your current working directory?
    • the reason I ask, is that your URLs aren't encrypted, as they should be.. by redirect
  • regarding redirecting all HLS Proxy URLs through a 3rd-party web service,
    by including them as a querystring parameter to a different URL..
    • see the following update
    • refer to its example

related update:

  • v3.4.4 adds a new hooks function: redirect_final(url)
    • input: a URL that refers back to the proxy and contains a base64 encoded link.. which was constructed for inclusion in a modified manifest
    • output: a new URL that makes whatever changes you need
commented

when you ran this, was "hooks.js" in your current working directory?
yes. i try use https://cph-msl.akamaized.net/hls/live/2000341/test/master.m3u8, it working.
not work when use m3u8 vod from motogp.com, for motogp live event streaming, I haven't tried it. MotoGP live event has ended now.
are you already test use motogp.com with hooks.js url encrypts?

thanks for new hooks function: redirect_final(url).
works perfect. but not work when i add parameter --prefetch --cache-timeout 3600 --max-segments 10

commented

I just played around for a few minutes with videos hosted on motogp.com, found that:

  • it won't play the HLS stream when the encoded HLS Proxy URL does include a Referer
  • but it plays fine without one
    • this is the opposite of how most servers treat this request header.. but that's how it goes
    • leave it out, and you'll be fine
  • as expected, using the hooks to obfuscate URLs has no effect..
    • since it doesn't change the proxied URLs.. that are requested from the origin server
commented

module.exports = {
"redirect_final": (url) => https://images-blogger-opensocial.googleusercontent.com/gadgets/proxy?refresh=4&container=focus&gadget=a&url=${encodeURIComponent(url)}
}

not work. if i add parameter --prefetch --cache-timeout 3600 --max-segments 10

using google proxy. i can get good connection in my country. and prevent blocking my domain on their dns.

commented

this is interesting..

test.sh:

#!/usr/bin/env bash

url='https://cdn1vodmcrdornacom.akamaized.net/112759/index.m3u8?hdnts=st=1701327912~exp=1701328374~acl=/112759/...'
ref='https://www.motogp.com/en/videos/2023/11/29/mir-talks-crew-chief-switch-im-really-happy-with-the-decision/485682?playlistId=425969'
agt='Chrome/100'

curl -s -I "$url"

curl -s -I --referer "$ref" "$url"

curl -s -I --user-agent "$agt" --referer "$ref" "$url"

and logging its output:

./test.sh >test.log 2>&1

the log file doesn't confirm my earlier observation:

HTTP/1.1 200 OK
Server: nginx/1.22.1
X-Amz-Cf-Pop: LAX53-P4
X-Amz-Cf-Id: RLKG4rWDo4p04-2WLCkoHQWpBDQgEiAaIcBUCU3kgAIrjIK6-0cjyA==
Expires: Thu, 30 Nov 2023 07:07:18 GMT
Cache-Control: max-age=0, no-cache, no-store
Pragma: no-cache
Date: Thu, 30 Nov 2023 07:07:18 GMT
Connection: keep-alive
Akamai-Mon-Iucid-Del: 1291036
Alt-Svc: h3-Q050=":443"; ma=93600,quic=":443"; ma=93600; v="46,43"
Content-Type: application/x-mpegURL
Content-Length: 8547

HTTP/1.1 200 OK
Server: nginx/1.22.1
X-Amz-Cf-Pop: LAX53-P4
X-Amz-Cf-Id: uskDWS1BqGQ-2qPFaSfj57usSzR5WzWlmvGVkwGEecfMKrxZW1RMbw==
Expires: Thu, 30 Nov 2023 07:07:18 GMT
Cache-Control: max-age=0, no-cache, no-store
Pragma: no-cache
Date: Thu, 30 Nov 2023 07:07:18 GMT
Connection: keep-alive
Akamai-Mon-Iucid-Del: 1291036
Alt-Svc: h3-Q050=":443"; ma=93600,quic=":443"; ma=93600; v="46,43"
Content-Type: application/x-mpegURL
Content-Length: 8547

HTTP/1.1 200 OK
Server: nginx/1.22.1
X-Amz-Cf-Pop: LAX53-P4
X-Amz-Cf-Id: _QvY-2CCw1lR814kAGqrsOJ2ZUmFfaoib4IvRzWZvfOg_MoKSMCD5A==
Expires: Thu, 30 Nov 2023 07:07:19 GMT
Cache-Control: max-age=0, no-cache, no-store
Pragma: no-cache
Date: Thu, 30 Nov 2023 07:07:19 GMT
Connection: keep-alive
Akamai-Mon-Iucid-Del: 1291036
Alt-Svc: h3-Q050=":443"; ma=93600,quic=":443"; ma=93600; v="46,43"
Content-Type: application/x-mpegURL
Content-Length: 8547

which makes me think that when HLS Proxy is using the rewrite hook to decrypt..
the referer URL isn't getting extracted from the encoded URL properly..
TBD

commented

I'm not familiar with that google proxy..
so I just quickly tested it..
with the URL for CBS News master manifest..
and the response was:

<html><head><meta http-equiv="content-type" content="text/html; charset=utf-8"/><title>Sorry...</title><style> body { font-family: verdana, arial, sans-serif; background-color: #fff; color: #000; }</style></head><body><div><table><tr><td><b><font face=sans-serif size=10><font color=#4285f4>G</font><font color=#ea4335>o</font><font color=#fbbc05>o</font><font color=#4285f4>g</font><font color=#34a853>l</font><font color=#ea4335>e</font></font></b></td><td style="text-align: left; vertical-align: bottom; padding-bottom: 15px; width: 50%"><div style="border-bottom: 1px solid #dfdfdf;">Sorry...</div></td></tr></table></div><div style="margin-left: 4em;"><h1>We're sorry...</h1><p>... but your computer or network may be sending automated queries. To protect our users, we can't process your request right now.</p></div><div style="margin-left: 4em;">See <a href="https://support.google.com/websearch/answer/86640">Google Help</a> for more information.<br/><br/></div><div style="text-align: center; border-top: 1px solid #dfdfdf;"><a href="https://www.google.com">Google Home</a></div></body></html>
commented

..but that would explain why it breaks prefetch
..if HLS Proxy isn't allowed to request these URLs directly
..you may need to configure some additional request headers (on the command line)

commented

tested it using curl. its working just use random referrer -H 'Referer: https://www.nodfgdf.vom' or anything

commented

ok.. the Google proxy does require a Referer request header..
it looks like any value will do

ex:

#!/usr/bin/env bash

url='https://images-blogger-opensocial.googleusercontent.com/gadgets/proxy?refresh=4&container=focus&gadget=a&url=https%3A%2F%2Fwww.cbsnews.com%2Fcommon%2Fvideo%2Fcbsn_header_prod.m3u8'
ref='https://www.google.com/'

curl -s -I "$url"
# 403

curl -s -I --referer "$ref" "$url"
# 200
commented

same time 😃

so..
you can set a default Referer header in HLS Proxy with the option:

--referer 'http://example.com/'

..and that should allow prefetch to work with this Google proxy

or.. after I've figured out what's going on with encryption/decryption..
combined with passing a referer URL along with video URL in the base64 encoded value..
and fix it..
then that should work too

commented

yes, its working now. thank you.

i am using cloudflare cdn with custom ts extension name. , and then using google proxy. its works perfect.

just need optimize nginx conf. already tested 8k viewer live streaming. my server only use 1gbps.
the problem is cloudflare need first request to get cached. but, with many viewer first request ts from m3u8 in same time. many get cache-status MISS. miss status that mean fresh content before cf cached content. my cf rule only cached for ts file.
my trick is i restarted nginx every 10 or 5 second using loop in terminal. to stop ,many cloudflare connection because hit my 1gbps my server connection .
i think if few person is first request. and then all viewer can delayed. it should all viewer will hit cached content on cloudflare.
or may 1gbps server it's not enough.

commented

v3.4.5 fixes the issue having to do with combining (video + referer urls) with (encryption/decryption).

the order of events needed correction.

the way it works now (correctly) is:

  • after proxying a manifest and extracting the urls it contains, foreach..
    • call: redirect()
    • append: |${referer}
    • base64 encode
    • prepend: protocol, host, path
    • call: redirect_final()
  • when urls are requested from the proxy, foreach..
    • base64 decode
    • extract: referer
    • call: rewrite()
commented

encryption/decryption avoid stranger people use my proxy..

sorry, can you help write hooks.js, combining with google proxy.

commented
  1. copy this
  2. edit hooks.js
    • add to module.exports: 
       "redirect_final": (url) => `https://images-blogger-opensocial.googleusercontent.com/gadgets/proxy?refresh=4&container=focus&gadget=a&url=${encodeURIComponent(url)}`
commented

already updated to v3.4.5, i still no success to play video hosted on motogp.com with hooks.js encryption.