limit scraping service doesn't work when setting default region when not us-east-1

[Injae Kwak]

Hi Team,

I've had the aphelion stack run for > 1 week and it seems like it is not able to scrape limits data (and the dashboard shows "No data returned".

Looking at the log file, it looks like it's throwing an exception when trying to connect to an endpoint that doesn't exist: support has a single endpoint in us-east-1 .

Below are my configs. Will try to redeploy using us-east-1 for AWS_DEFAULT_REGION in my docker compose file and post result back here.

bash-4.2# tail /var/log/cron.log 
    caught_exception)
  File "/usr/local/lib/python3.4/site-packages/botocore/retryhandler.py", line 277, in _should_retry
    return self._checker(attempt_number, response, caught_exception)
  File "/usr/local/lib/python3.4/site-packages/botocore/retryhandler.py", line 317, in __call__
    caught_exception)
  File "/usr/local/lib/python3.4/site-packages/botocore/retryhandler.py", line 223, in __call__
    attempt_number, caught_exception)
  File "/usr/local/lib/python3.4/site-packages/botocore/retryhandler.py", line 359, in _check_caught_exception
    raise caught_exception
botocore.exceptions.EndpointConnectionError: Could not connect to the endpoint URL: "https://support.ap-southeast-2.amazonaws.com/"

I'm using a custom web docker image with SSL conf disabled as per PR16

[ec2-user@aphelion ~]$ cat docker-compose.yml 
version: '3'
services:
  aphelion-service:
    image: finraos/aphelion-service
    environment:
       - ASSUMED_ROLE_NAME=get-cloudwatchrole
       - ACCOUNT_ID_LIST=<REDACTED>, <REDACTED>, <REDACTED>, <REDACTED>
       - ASSUMED_ROLE_SESSION_NAME=limit_dashboard
       - REGIONS=us-east-1, us-west-2, ap-southeast-2
       - AWS_DEFAULT_REGION=ap-southeast-2
       - REPORT_FILE_NAME=limits.csv
       - CRON_REFRESH=* 2 * * *
       - CRON_LIMITS=* 1 * * *
  dashboard-service:
    image: finraos/aphelion-dashboard-service
  web:
    image: <REDACTED>.dkr.ecr.ap-southeast-2.amazonaws.com/aphelion
    ports:
       - 8443
    restart: always
    links:
       - dashboard-service
       - aphelion-service

[Daniel Koo]

Thanks @saltysoup, testing us-east-1 as default would be great. We will also take a deeper look and follow up.

[Injae Kwak]

hi @kood1 , just tested this by change my docker compose file and can confirm that switching the default region to us-east-1, made the service work properly with limit data available on the dashboard.

[Daniel Koo]

Thanks for testing @saltysoup! We will take a look and determine what changes are needed.