nergalex / f5-bot-selenium

initial

Unidentified Filching Object | OVNI

Contents

11. UDF lab Add-on

Local environment

• Install Python
• Install PyCharm community edition

• Open PyCharm

  • Create a new project Get from VCS and copy paste this github repository URI
  • Attach a Python interpreter

• Download here same Chrome driver as your Chrome browser chrome://settings/help
• OPTION: for other browser, follow this guide
• Copy downloaded chromedriver(.exe) file in ./_files/chromedriver.exe of your project
• For MacOS only, allow chromedriver as described here:

xattr -d com.apple.quarantine chromedriver

• Open file requirements.txt
• Click on Install requirements

• Choose ONLY those packages ton install: selenium, requests

• Install 2captcha-python package: at the bottom left of the window, open a Terminal

PC: pip3.exe install 2captcha-python==1.1.0
Mac: pip3 install 2captcha-python==1.1.0

Remote environment

• Deploy the blueprint Advanced WAF Demo v16 + LCC, ML, Device ID+ and IBD / version of 30/09/2021 and after
• Start the deployment Advanced WAF Demo v16 + LCC, ML, Device ID+ and IBD
• Get Hackazon URI: Components >> BIG-IP >> Details >> Access methods >> IBD Demo with Hackazon as backend >> External
• Get Hackazon user credential (login and passowrd) : Components >> LAMP-server >> Documentation >> Access methods >> Description >> <login>/<password>
• Open the e-mail received from Cloud Services and connect to IBD portal

Test 1 - Human

Task

• Open a web browser
• Paste Hackazon URI and append PATH /user/login
• Try to login using the user credential and solving the CAPTCHA

Result

• You have been redirected to /account

Clean

• Logout

Google recaptcha API key

A Google Recaptcha account has been already created. A site key and a secret key was added in Application's code.

Task

• Use 2captcha user guide to find the site key
• You do not need to read after step 3

Note

• For trainer: statistics are available here

Test 2 - bot - bypass CAPTCHA, viva 2CATPCHA!

Task

• In PyCharm, open website11.py
• Click on Structure

• Open setUp function

• Set global variables

  • Note: CAPTCHA_API_KEY is 2CAPTCHA API key and NOT the the Google recaptcha site-key

URI = "https://{{your_hackazon_uri}}/user/login"
LOGIN_USER = "test_user"
LOGIN_PASSWORD = "123456"
CAPTCHA_API_KEY = "{{ask_your_trainer}}"

• For Mac user, set local variable

PATH = "./_files/chromedriver"

• Click on the left column, as described in the picture below, to suspend the script during his future execution

• Go to the end of the file and click on the green triangle

• Choose debug mode

• The script launch a Web browser controlled by Selenium
• Just for your understanding, when the script is suspended, locate the element in the Web browser's dev tool windows >> Elements tab
• Execute next action by clinking on "go to cursor"

• When element solveGRecaptcha function is suspended, check that site_key variable as the same value as the one you found in the previous exercise
• When element loginbtn is suspended, add a Filter on transaction to catch it: Chrome >> DevTool >> Network >> filter method:POST
• "Go to cursor" and then see the POST request: note the g-recaptcha-response info in the form request

Result

• Bot succeed to login and he is redirected to /account

Clean

• In PyCharm, stop the script by clicking on the stop button i.e. a red square

---

Configure Integrated Bot Defense

More details in Integrated Bot Defense admin guide

Task

• View detected human and bot transaction in Cloud Services >> IBD >> Dashboard
• Get BIG-IP credential: UDF >> `Components >> BIG-IP >> Details >> Credentials`
• Connect to BIG-IP: UDF >> `Components >> BIG-IP >> Access >> TMUI`
• Open IBD iApp: iApps >> `App Services >> Hackazon_IBD >> Reconfigure`

• Set parameters:

  • Entry points | JS Injection Paths in specific Webpages only: /user/login

  • Endpoints | Paths to be Routed to Shape >>

    • Host HOST_of_your_hackazon_URI
    • Endpoint /user/login
    • Method POST
    • Mitigation Action block

  • Block Data:

<h1 class='page-title'>Demo | Blocked by Shape | Demo</h1>

• Click on Finished to reconfigure
• Connect to BIG-IP using SSH: UDF >> `Components >> BIG-IP >> Access >> Web shell`
• Drop all connection:

tmsh del sys conn all
tmsh show sys conn cs-server-addr 10.1.10.52

Test 3 - bot - mitigated

Task

• In PyCharm, run a test in debug mode, as done in step Test 2
• When element loginbtn is suspended, add a Filter on transaction to catch it: Chrome >> DevTool >> Network >> filter method:POST
• Click on preserve log
• "Go to cursor" and then see the POST request: note all the parameters, in the form, used to forward signal data to IBD
• Run a test again in NO debug mode:

Results

• See page Demo | Blocked by Shape | Demo
• View mitigated bot transaction in Cloud Services >> IBD >> Dashboard
• Filter on Application >> hackazon-ibd

Cross-Origin Resource Sharing (CORS)

A web site could have an authentication API endpoint hosted in another domain. In this case, web browser make a "preflight" request to the server hosting the cross-origin resource, in order to check that the server will permit the actual request. In that preflight, the browser sends headers that indicate the HTTP method and headers that will be used in the actual request.

To handle this use case:

• Attach IBD to 2 VS:

  1. VS of the landing web site
  2. VS of the authentication API endpoint web site

• Collect IBD header names in a same domain transaction
• Update the iRule iRule_CORS.tcl with header names and iapp_name (i.e. Application Service name)

Reference:

• CORS
• Access-Control-Allow-Headers

Cross-Origin Communication

A web site could delegate its authentication mechanism to a 3rd party. In the rendered web page, an iframe - managed by the 3rd party hosted in another domain - display a CAPTCHA.

Normally, scripts on different pages are allowed to access each other if and only if the pages they originate from share the same protocol, port number, and host (also known as the "same-origin policy"). In a Cross-Origin use case, the window.postMessage() method safely enables cross-origin communication between Window objects; e.g., between a page and a pop-up that it spawned, or between a page and an iframe embedded within it.

The objective to handle this use case is to:

1. Add a listener EventTarget.addEventListener() on the 3rd party page, that will get useful info on CAPTCHA and send back info to the source caller using window.postMessage()
2. On the main page, Add a listener EventTarget.addEventListener() to catch the response back and write it in the page, for example in the 3rd party's iframe attribute
3. On the main page, call the listener to do action using window.postMessage()

Configuration for step 1:

• Create a profiles >> Content >> HTML >> rule captcha-delivery_postMessage

  • Match settings >> Match Tag Name: body
  • Action settings >> HTML to Append: <script> {{ copy paste ./files/iframe_cors_bypass-listener.js }} </script>

• Create a profiles >> Content >> HTML captcha-delivery

  • Content settings: text/html text/xhtml
  • HTML rules: captcha-delivery_postMessage

• Create a VS to listen on 3rd party domain and attach HTML profile captcha-delivery

Configuration for step 2 and 3: view website7.py as an example

As you can see, bot configuration is splitted in 2 (Selenium and a reverse-proxy). If you want to have a all in one solution, use Hackium