Help adding Athena service?

[bruno pini]

I managed to create a separate JS repository to be able to query Athena (using the v3 of the AWS SDK).

I'm not a javascript developer and I'm having trouble importing this new library into Google Apps Script. Could you guys help me out with any tips or reference materials?

Or maybe help me out incorporating my new functions into this brilliant and life-saving repo?

Here's what I came up with:

const { AthenaClient, StartQueryExecutionCommand, GetQueryExecutionCommand, GetQueryResultsCommand } = require('@aws-sdk/client-athena');

async function runAthenaQuery(athenaClient, database, query, outputLocation) {

  const queryExecutionParams = {
    QueryString: query,
    QueryExecutionContext: {
      Database: database,
    },
    ResultConfiguration: {
      OutputLocation: outputLocation,
    },
  };

  const startQueryExecutionCommand = new StartQueryExecutionCommand(queryExecutionParams);

  try {
    const queryExecutionResponse = await athenaClient.send(startQueryExecutionCommand);
    const queryExecutionId = queryExecutionResponse.QueryExecutionId;
    console.log("Query submitted. Execution ID:", queryExecutionId);
    return queryExecutionId;
  } catch (error) {
    console.error("Error executing Athena query:", error);
    throw error;
  }
}

function getQueryResultsWithPolling(athenaClient, queryId) {
  return new Promise((resolve, reject) => {
    const interval = setInterval(() => {
      const queryExecutionParams = { QueryExecutionId: queryId };
      const getQueryExecutionCommand = new GetQueryExecutionCommand(queryExecutionParams);

      athenaClient.send(getQueryExecutionCommand)
        .then((response) => {
          const status = response.QueryExecution.Status.State;
          console.log(`Query execution status: ${status}`);

          if (status === 'SUCCEEDED') {
            clearInterval(interval);

            const getQueryResultsParams = { QueryExecutionId: queryId };
            const getQueryResultsCommand = new GetQueryResultsCommand(getQueryResultsParams);

            return athenaClient.send(getQueryResultsCommand);
          } else if (status === 'FAILED' || status === 'CANCELLED') {
            clearInterval(interval);
            reject(new Error(`Query execution failed or cancelled. Status: ${status}`));
          }
        })
        .then((resultResponse) => {
          resolve(resultResponse);
        })
        .catch((error) => {
          clearInterval(interval);
          reject(error);
        });
    }, 2000);
  });
}

function executeAthenaQuery(database, query, outputLocation, awsRegion, awsAccessKeyId, awsSecretAccessKey) {
  const awsCredentials = {
    accessKeyId: awsAccessKeyId,
    secretAccessKey: awsSecretAccessKey,
  };
  
  const athena = new AthenaClient({
    region: awsRegion,
    credentials: awsCredentials,
  });

  return new Promise(async (resolve, reject) => {
    try {
      const queryId = await runAthenaQuery(athena, database, query, outputLocation, awsRegion, awsAccessKeyId, awsSecretAccessKey);
      console.log('Query ID:', queryId);

      const queryResults = await getQueryResultsWithPolling(athena, queryId);
      console.log('Query Results:', queryResults);

      resolve(queryResults);
    } catch (error) {
      console.error('Error:', error);
      reject(error);
    }
  });
}

Thanks a ton

[Daniel Caspi]

Hi @brunopini, see the README. You can add more services, e.g.

npm run sdk --sdk=ses,s3,ec2,lambda,dynamodb && npm run build. I'm not sure what the service name is for athena, but maybe it's just athena. If you only wanted that service, you could add it that way.

As for the actual commands from the SDK, you would have to test it for that individual service and consult the AWS SDK documentation.

[bruno pini]

@dxdc thanks for the quick reply and pardon my ignorance here -- as mentioned, I'm not really familiar with javascript.

I've read the docs and explored all of your code, but failed to understand the steps.

• Where do I npm run? Locally after cloning your repo?
• How do I then import the new code into Google Apps Script as a library?
  -- I ask this because I tried bundling my own code and adding it to a GAS project, but keep getting a ReferenceError: executeAthenaQuery is not defined.

As for the commands, the code I shared above is executing succesfully, although using V3. I'm sure I can adapt back to V2.

If it's not asking too much, could you provide more info even if only an overview so I can do my own research?

[bruno pini]

Updates:

I managed to clone this repo and followed your instructions to access the athena service, but I'm struggling to resolve this error:

> aws-sdk-google-apps@1.0.0 sdk
> echo "Building SDK for AWS Services: $npm_config_sdk" && node aws-sdk-js/dist-tools/browser-builder.js $npm_config_sdk > src-sdk/aws-sdk.js

"Building SDK for AWS Services: $npm_config_sdk" 
(node:13968) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023.

Please migrate your code to use AWS SDK for JavaScript (v3).
For more information, check the migration guide at https://a.co/7PzMCcy
(Use `node --trace-warnings ...` to show where the warning was created)
C:\Users\Bruno\AWS\aws-sdk-google-apps\aws-sdk-js\dist-tools\service-collector.js:142
    throw new Error('Incorrectly formatted service names');
    ^

Error: Incorrectly formatted service names
    at ServiceCollector (C:\Users\Bruno\AWS\aws-sdk-google-apps\aws-sdk-js\dist-tools\service-collector.js:142:11)
    at module.exports (C:\Users\Bruno\AWS\aws-sdk-google-apps\aws-sdk-js\dist-tools\transform.js:32:17)
    at nr (C:\Users\Bruno\AWS\aws-sdk-google-apps\aws-sdk-js\node_modules\module-deps\index.js:301:23)
    at C:\Users\Bruno\AWS\aws-sdk-google-apps\aws-sdk-js\node_modules\module-deps\index.js:283:21
    at onfile (C:\Users\Bruno\AWS\aws-sdk-google-apps\aws-sdk-js\node_modules\resolve\lib\async.js:155:21)
    at onex (C:\Users\Bruno\AWS\aws-sdk-google-apps\aws-sdk-js\node_modules\resolve\lib\async.js:210:32)
    at C:\Users\Bruno\AWS\aws-sdk-google-apps\aws-sdk-js\node_modules\resolve\lib\async.js:22:20
    at FSReqCallback.oncomplete (node:fs:209:5)

Node.js v18.16.1

I'm positive it is not a typo on the service name, because running with the default services, I still get the error.

Any idea why?

Also: any plans on updating to V3?

Thanks again

[bruno pini]

(not sure if I needed to @ you for notifications to apply)
@dxdc

[Daniel Caspi]

I don't think your V3 code will port to V2. I don't have plans to revisit V3, when I looked before, it seemed more complicated.

Which command was failing for you at the terminal did you run? You didn't paste what you typed in the terminal. I don't think I've tested it with Node 18 though. Maybe try Node 16? It may also be a path issue since I haven't tested this on Windows.

I built the SDK for you as follows, though not sure if you need all of these services. It adds a little bit of bloat, but it's still pretty small.

npm run sdk --sdk=ses,s3,ec2,lambda,dynamodb,athena && npm run build

See the advanced deployment instructions here: https://github.com/dxdc/aws-sdk-google-apps#create-your-own-library

Just use this version of AwsSdk.js instead of the one in the dist folder.

AwsSdk.js.zip