GOES site changed from ENTER to Check Here If Human Tick Box
mickrussom opened this issue · comments
Was working perfect, GOES changed the website today.
2016-08-01 19:20:51,944 Couldn't convert output: Fatal error: Unable to find terms acceptance button from phantomJS script into a valid date.
Traceback (most recent call last):
File "/home/mick/globalentry/5446_SFO/ge-checker-cron.py", line 70, in main
new_apt = datetime.strptime(script_output, '%B %d, %Y')
File "/usr/lib/python2.7/_strptime.py", line 332, in _strptime
(data_string, format))
ValueError: time data 'Fatal error: Unable to find terms acceptance button' does not match format '%B %d, %Y'
2016-08-01 19:24:34,026 Running GE cron with arguments: {'no_email': False, 'configfile': '/home/mick/globalentry/5446_SFO/config.json', 'notify_osx': False, 'use_gmail': True}
2016-08-01 19:24:38,024 Couldn't convert output: Fatal error: Unable to find terms acceptance button from phantomJS script into a valid date.
Traceback (most recent call last):
File "/home/mick/globalentry/5446_SFO/ge-checker-cron.py", line 70, in main
new_apt = datetime.strptime(script_output, '%B %d, %Y')
File "/usr/lib/python2.7/_strptime.py", line 332, in _strptime
(data_string, format))
Can confirm this issue as well. Looks like the checkbox onclick does "javascript: submitHome();" I'm not sure how to tell phantomjs to do this but maybe this is all that's needed? Unless they actually included one of those delay checkers in which case you might need to include a small delay before sending the command to appear more human.
Update, added the fix to my fork and created a pull request. Hopefully someone else can also verify it works!
@jack3k3 I tried your new version - I got this error .
ge-checker-cron.py --use-gmail
08/02/2016 08:55:07 AM Running GE cron with arguments: {'no_email': False, 'configfile': './config.json', 'notify_osx': False, 'use_gmail': True}
08/02/2016 08:55:12 AM Couldn't convert output: Fatal error: Unable to find Manage Appointment button from phantomJS script into a valid date.
Verified the fix was in - it was only that one file, correct?
diff ge-cancellation-checker.phantom.js ../ge-cancellation-checker_broke/ge-cancellation-checker.phantom.js
diff ge-cancellation-checker.phantom.js ../ge-cancellation-checker_broke/ge-cancellation-checker.phantom.js
submitHome();
console.log('Bypassing human check...');
function fireClick(el) {
var ev = document.createEvent("MouseEvents");
ev.initEvent("click", true, true);
el.dispatchEvent(ev);
}
var $acceptTermsBtn = document.querySelector('a[href="/goes/HomePagePreAction.do"]');
if (!$acceptTermsBtn) {
return window.callPhantom('fatal-error', 'Unable to find terms acceptance button');
}
fireClick($acceptTermsBtn);
console.log('Accepting terms...');
@mickrussom hmm very strange, I verified the change worked on my machine and gave me the latest date on GOES this morning, however when I made a new fork I did manually apply the change so it's possible I messed up copying it over. Looking at the code it seems fine to me (unfortunately I'm at work so can't check the original ATM).. I'll verify when I'm home at lunch.
@jack3k3 Thanks for the update. I tried downloading the file, git clone and even copying from the project page @ jack3k3/ge-cancellation-checker - I think I have the right files in place. I'll be standing by to test again and will poke at this a bit in the meantime. TY, TIA
Hi @jack3k3, thank you so much for updating your fork and posting here to let everyone know. We really appreciate everyone's time and effort they put into this.
I can confirm it is successfully working for me!
Please wait...
On GOES login page...
Logging in...
Bypassing human check...
Error on page: TypeError: undefined is not an object (evaluating 'document.document.ApplicationActionForm')
phantomjs://code/ge-cancellation-checker.phantom.js:54 in onError
Entering appointment management...
Entering rescheduling selection page...
Choosing Location: [redacted]
Next available appointment is at: November 10, 2016
Edit: Commands used on Debian
wget https://github.com/jack3k3/ge-cancellation-checker/archive/master.zip
mv ge-cancellation-checker-master ge-cancellation-checker-master1
unzip master.zip
cd ge-cancellation-checker-master1
cp config.json ../ge-cancellation-checker-master/
@jack3k3 @0rgan1c I just ran the same test using a config.json that worked before the site break -
(Ubuntu 16.04 phantomjs 2.1.1+dfsg-1 )
mick@pytivo:~/globalentry/jack3k3$ wget https://github.com/jack3k3/ge-cancellation-checker/archive/master.zip
--2016-08-02 11:57:45-- https://github.com/jack3k3/ge-cancellation-checker/archive/master.zip
Location: https://codeload.github.com/jack3k3/ge-cancellation-checker/zip/master [following]
Saving to: ‘master.zip’
mick@pytivo:~/globalentry/jack3k3$ unzip master.zip
Archive: master.zip
6a4f186
creating: ge-cancellation-checker-master/
extracting: ge-cancellation-checker-master/.gitignore
inflating: ge-cancellation-checker-master/CONTRIBUTING.md
inflating: ge-cancellation-checker-master/Dockerfile
inflating: ge-cancellation-checker-master/README.md
inflating: ge-cancellation-checker-master/config.json.example
inflating: ge-cancellation-checker-master/ge-cancellation-checker.phantom.js
inflating: ge-cancellation-checker-master/ge-checker-cron.py
mick@pytivo:~/globalentry/jack3k3$ cd ge-cancellation-checker-master/
mick@pytivo:~/globalentry/jack3k3/ge-cancellation-checker-master$ cp /home/mick/globalentry/5446_SFO/config.json .
mick@pytivo:~/globalentry/jack3k3/ge-cancellation-checker-master$ ls
config.json config.json.example CONTRIBUTING.md Dockerfile ge-cancellation-checker.phantom.js ge-checker-cron.py README.md
mick@pytivo:~/globalentry/jack3k3/ge-cancellation-checker-master$ ./ge-checker-cron.py --use-gmail
08/02/2016 12:00:30 PM Running GE cron with arguments: {'no_email': False, 'configfile': './config.json', 'notify_osx': False, 'use_gmail': True}
08/02/2016 12:00:34 PM Couldn't convert output: Fatal error: Unable to find Manage Appointment button from phantomJS script into a valid date.
config.json
{
"current_interview_date_str": "August 4, 2016",
"logfile": "./gecheck.log",
"email_from": "mickr@gmail.com",
"email_to": ["mickr@gmail.com"],
"username": "mickr",
"password": "_________",
"gmail_password": "_________________",
"use_gmail": "True",
"enrollment_location_id": "5446",
"init_url": "https://goes-app.cbp.dhs.gov/main/goes"
}
@jack3k3 @0rgan1c I just re-ran this test using docker and it works. Seems there is an issue with phantomjs 2.1.1 and the fixes @jack3k3 put in. The docker build indicates phantomjs 1.9.7
@mickrussom oh interesting, does it work fine if you run the phantomjs script directly? I think something like this: phantomjs --ssl-protocol=any ge-cancellation-checker.phantom.js
Just for notes, I am successfully using phantomjs 2.1.1 and Python 2.7
@jack3k3 I just tried to run that "phantomjs --ssl-protocol=any ge-cancellation-checker.phantom.js" - and it gives just this output: "Fatal error: Unable to find Manage Appointment button "
@jack3k3 same machine running docker, and the check goes through.
@mickrussom unfortunately at the moment I don't have a clue what's going on beyond some problem with your phantomjs 2.1.1 install? If you want to try you could go to the phantomjs site and download the phantomjs binary for your linux machine and place it in the folder with the scripts and try ./phantomjs --ssl-protocol=any ge-cancellation-checker.phantom.js
I used this method successfully already so it should work
@mickrussom here are my suggestions for troubleshooting. This is what I had done when I encountered issues in the past:
1.) Start completely from scratch (make a new directory)
2.) Extract master.zip
3.) Copy the sample config to config.json (do not modify it yet/erase all values)
4.) Run the script, and it will produce an error saying what is missing
5.) Add information ONE line at a time, save, then run the script
6.) Repeat this until your your config.json is complete
The reason I suggest this is because I had errors in mine that were not obvious based off the error messages I was receiving. If something gets inputted incorrectly along the way the error message may change and then you can pinpoint your issue.
@jack3k3 @0rgan1c I downloaded the phantomjs 2.1.1 binary from the mater site and ran the cron script with the path to the new binary embedded. Works like a charm now! So ubuntu 16.04's phantomjs package is not working for whatever reason.
@jack3k3 I also noticed that between earlier and whatever you did to fix the issue that the script will now run truly headlessly. before I had to run this in a terminal which could display to X.
Thanks guys, merged @jack3k3's fix. Surprised they didn't add a proper captcha.