webhooks example
nick-youngblut opened this issue · comments
It would be very helpful to include webhooks example, especially since https://github.com/Asana/devrel-examples/tree/master/python/webhooks is out-of-date and does not function at least with python=3.9 and asana=4.0.10
It would be especially helpful to include an example of deploying the python webhook app on AWS or GCP. This would have the added advantage of not requiring ngrok, as in https://github.com/Asana/devrel-examples/tree/master/python/webhooks.
Hi @nick-youngblut, thank you for the feedback. I brought this up with my team and we've added it to our todo.
@jv-asana I've started to re-write https://github.com/Asana/devrel-examples/tree/master/python/webhooks that works with asana=4.0.10. I can share it here, if I can get it working
It would also be helpful to provide a full working example of create_webhook, since the example is not actually a working example (e.g., body = asana.WebhooksBody({"param1": "value1", "param2": "value2",})).
The example is especially confusing when comparing it to the code at devrel-examples:
class CreateWebhookThread(threading.Thread):
def run(self):
# Note that if you want to attach arbitrary information (like the target project) to the webhook at creation time, you can have it
# pass in URL parameters to the callback function
webhook = client.webhooks.create(resource=project, target="https://{0}.ngrok.io/receive-webhook?project={1}".format(ngrok_subdomain, project))
create_thread = CreateWebhookThread()
def get_all_webhooks():
webhooks = list(client.webhooks.get_all(workspace=os.environ["ASANA_WORKSPACE"]))
app.logger.info("All webhooks for this pat: \n" + str(webhooks))
return webhooks
@app.route("/create_webhook", methods=["GET"])
def create_hook():
global create_thread
# First, try to get existing webhooks
webhooks = get_all_webhooks()
if len(webhooks) != 0:
return "Hooks already created: " + str(webhooks)
# Should guard webhook variable. Ah well.
create_thread.start()
return """<html>
<head>
<meta http-equiv=\"refresh\" content=\"10;url=/all_webhooks\" />
</head>
<body>
<p>Attempting to create hook (may take up to 10s) Redirecting in 10s to <a href=/all_webhooks>/all_webhooks</a> to inspect.</p>
</body>"""
Do I need to use class CreateWebhookThread(threading.Thread) for create_webhook, or was that only necessary when using client.webhooks.create()?
Hi @nick-youngblut,
Thanks for attempting to re-write our sample webhooks code to support asana=4.0.10.
We understand that the sample code won't work out of the box since it shows: {"param1": "value1", "param2": "value2",} -> This is something tricky that we are trying to solve. Essentially, all of our sample code is auto generated and we need to edit our generator template to extract the examples out for POST and PUT requests. This is why you will see that the sample codes for POST and PUT requests are {"param1": "value1", "param2": "value2",}.
In the mean time, when you see {"param1": "value1", "param2": "value2",} you can look at our developer documentation and look at the available request body keys and substitute param1 and param2 etc... with that key and for value add the value of that property's type.
EX: If you see:
body = asana.WebhooksBody({"param1": "value1", "param2": "value2",}) # WebhooksBody | The webhook workspace and target.You can do:
body = asana.WebhooksBody({"resource": "123", "target": "https://example.com/target",}) # WebhooksBody | The webhook workspace and target.QUESTION: Do I need to use class CreateWebhookThread(threading.Thread) for create_webhook, or was that only necessary when using client.webhooks.create()?
ANSWER: Technically it's not needed even in the original example code. I don't have context on this since the original author is no longer at Asana. I think the reason it was added is so that the 10 second redirect can have a head start while there is a thread that executes the webhook handshake.
Also, I did a quick re-write of the original example here:
import asana, sys, os, json, logging, signal, threading, hmac, hashlib
from asana.rest import ApiException
from flask import Flask, request, make_response
"""
Procedure for using this script to log live webhooks:
* Create a new PAT - we're going to use ngrok on prod Asana, and don't want to give it long-term middleman access
* https://app.asana.com/-/developer_console
* Set this PAT in the environment variable TEMP_PAT
* export TEMP_PAT={pat}
* Set the workspace in the environment variable ASANA_WORKSPACE. This is required for webhooks.get_all
* export ASANA_WORKSPACE={workspace_id}
* Set the project id in the environment variable ASANA_PROJECT
* export ASANA_PROJECT={project_id}
* Run `ngrok http 8090`. This will block, so do this in a separate terminal window.
* Copy the subdomain, e.g. e91dadc7
* Run this script with these positional args:
* First arg: ngrok subdomain
* Second arg: ngrok port (e.g. 8090)
* Visit localhost:8090/all_webhooks in your browser to see your hooks (which don't yet exist)
and some useful links - like one to create a webhook
* Make changes in Asana and see the logs from the returned webhooks.
* Don't forget to deauthorize your temp PAT when you're done.
"""
# Check and set our environment variables
pat = None
if 'TEMP_PAT' in os.environ:
pat = os.environ['TEMP_PAT']
else:
print("No value for TEMP_PAT in env")
quit()
workspace = None
if 'ASANA_WORKSPACE' in os.environ:
workspace = os.environ['ASANA_WORKSPACE']
else:
print("No value for ASANA_WORKSPACE in env")
quit()
project = None
if 'ASANA_PROJECT' in os.environ:
project = os.environ['ASANA_PROJECT']
else:
print("No value for ASANA_PROJECT in env")
quit()
# Configure python-asana client
configuration = asana.Configuration()
configuration.access_token = os.environ["ASANA_PERSONAL_ACCESS_TOKEN"]
api_client = asana.ApiClient(configuration)
# Create a webhook instance
webhook_instance = asana.WebhooksApi(api_client)
app = Flask('Webhook inspector')
app.logger.setLevel(logging.INFO)
ngrok_subdomain = sys.argv[1]
# We have to create the webhook in a separate thread, because webhook_instance.create_webhook
# will block until the handshake is _complete_, but the handshake cannot be completed
# unless we can asynchronously respond in receive_webhook.
# If running a server in a server container like gunicorn, a separate process
# instance of this script can respond async.
class CreateWebhookThread(threading.Thread):
def run(self):
# Note that if you want to attach arbitrary information (like the target project) to the webhook at creation time, you can have it
# pass in URL parameters to the callback function
body = asana.WebhooksBody({"resource": project, "target": f"https://{ngrok_subdomain}.ngrok-free.app/receive-webhook?project={project}"})
try:
# Establish a webhook
webhook_instance.create_webhook(body)
return
except ApiException as e:
app.logger.warning("Exception when calling WebhooksApi->create_webhook: %s\n" % e)
create_thread = CreateWebhookThread()
def get_all_webhooks():
workspace = os.environ["ASANA_WORKSPACE"]
webhooks = list(webhook_instance.get_webhooks(workspace).to_dict()["data"])
app.logger.info("All webhooks for this pat: \n" + str(webhooks))
return webhooks
@app.route("/create_webhook", methods=["GET"])
def create_hook():
global create_thread
# First, try to get existing webhooks
webhooks = get_all_webhooks()
if len(webhooks) != 0:
return "Hooks already created: " + str(webhooks)
# Should guard webhook variable. Ah well.
create_thread.start()
return """<html>
<head>
<meta http-equiv=\"refresh\" content=\"10;url=/all_webhooks\" />
</head>
<body>
<p>Attempting to create hook (may take up to 10s) Redirecting in 10s to <a href=/all_webhooks>/all_webhooks</a> to inspect.</p>
</body>"""
@app.route("/all_webhooks", methods=["GET"])
def show_all_webhooks():
return """<p>""" + str(get_all_webhooks()) + """</p><br />
<a href=\"/create_webhook\">create_webhook</a><br />
<a href=\"/remove_all_webhooks\">remove_all_webhooks</a>"""
@app.route("/remove_all_webhooks", methods=["GET"])
def teardown():
retries = 5
while retries > 0:
webhooks = get_all_webhooks()
if len(webhooks) == 0:
return "No webhooks"
for hook in webhooks:
try:
webhook_instance.delete_webhook(hook[u"gid"])
return "Deleted " + str(hook[u"gid"])
except ApiException as e:
print(f"Caught error: {str(e)}")
retries -= 1
print(f"Retries {str(retries)}")
return ":( Not deleted. The webhook will die naturally in 7 days of failed delivery. :("
# Save a global variable for the secret from the handshake.
# This is crude, and the secrets will vary _per webhook_ so we can't make
# more than one webhook with this app, so your implementation should do
# something smarter.
hook_secret = None
@app.route("/receive-webhook", methods=["POST"])
def receive_webhook():
global hook_secret
app.logger.info("Headers: \n" + str(request.headers));
app.logger.info("Body: \n" + str(request.data));
if "X-Hook-Secret" in request.headers:
if hook_secret is not None:
app.logger.warning("Second handshake request received. This could be an attacker trying to set up a new secret. Ignoring.")
else:
# Respond to the handshake request :)
app.logger.info("New webhook")
response = make_response("", 200)
# Save the secret for later to verify incoming webhooks
hook_secret = request.headers["X-Hook-Secret"]
response.headers["X-Hook-Secret"] = request.headers["X-Hook-Secret"]
return response
elif "X-Hook-Signature" in request.headers:
# Compare the signature sent by Asana's API with one calculated locally.
# These should match since we now share the same secret as what Asana has stored.
signature = hmac.new(hook_secret.encode('ascii', 'ignore'),
msg=str(request.data).encode('utf-8'), digestmod=hashlib.sha256).hexdigest()
if not hmac.compare_digest(signature,
request.headers["X-Hook-Signature"]):
app.logger.warning("Calculated digest does not match digest from API. This event is not trusted.")
return ""
contents = json.loads(request.data)
app.logger.info("Received payload of %s events", len(contents["events"]))
return ""
else:
raise KeyError
def signal_handler(signal, frame):
print('You pressed Ctrl+C! Removing webhooks...')
teardown()
sys.exit(0)
signal.signal(signal.SIGINT, signal_handler)
app.run(port=int(sys.argv[2]), debug=True, threaded=True)This example should be able to establish a webhook. You can follow the instructions from the old sample code on how to run this. However, the webhook handshake validation is not right I believe the original example code did not properly validate the signature as well hence you will see "Calculated digest does not match digest from API. This event is not trusted." in the console.
I'll leave it up to you to see if you can figure out how to generate a matching signature. Essentially, this is the part you'll want to focus on:
elif "X-Hook-Signature" in request.headers:
# Compare the signature sent by Asana's API with one calculated locally.
# These should match since we now share the same secret as what Asana has stored.
signature = hmac.new(hook_secret.encode('ascii', 'ignore'),
msg=str(request.data).encode('utf-8'), digestmod=hashlib.sha256).hexdigest()
if not hmac.compare_digest(signature,
request.headers["X-Hook-Signature"]):
app.logger.warning("Calculated digest does not match digest from API. This event is not trusted.")
return ""Our node webhook example does properly validate the signature. You can use this to help with troubleshooting.
Thanks @jv-asana for all of the help!
In the mean time, when you see {"param1": "value1", "param2": "value2",} you can look at our developer documentation and look at the available request body keys and substitute
Yeah, it was hard to figure out how for format the parameters, as you show in your example code:
asana.WebhooksBody({"resource": project, "target": f"https://{ngrok_subdomain}.ngrok-free.app/receive-webhook?project={project}"})
In regards to the instructions at the top of your example code:
- Run
ngrok http 8090. This will block, so do this in a separate terminal window.- Copy the subdomain, e.g. e91dadc7
I don't see a subdomain that looks like e91dadc7 in the ngrok output:
Session Status online
Session Expires 1 hour, 59 minutes
Terms of Service https://ngrok.com/tos
Version 3.3.4
Region United States (California) (us-cal-1)
Latency 9ms
Web Interface http://127.0.0.1:4040
Forwarding https://a259-2600-1700-3681-ca90-41e4-3c7b-ad6d-6ebe.ngrok.io -> http://localhost:8090
Connections ttl opn rt1 rt5 p50 p90
0 0 0.00 0.00 0.00 0.00
It appears that the docs should be changed to e.g., Copy the subdomain a259-2600-1700-3681-ca90-41e4-3c7b-ad6d-6ebe, or maybe it depends a lot on the ngrok install.
One other small note on your example code: you seem to have circumvented TEMP_PAT in favor of ASANA_PERSONAL_ACCESS_TOKEN, but the docs at the top of the code still state:
* Set this PAT in the environment variable TEMP_PAT
* export TEMP_PAT={pat}
Thanks so much for the example code. I believe that I am correctly running the updated python flask app, but when I run Postman (as shown in the example video), I always get the following return:
{
"id": "W123456",
"url": "https://api.example.com/webhooks/listener-for-payment-gateway",
"events": [
"payment.created",
"payment.updated",
"transaction.created",
"transaction.updated",
"refund.created",
"refund.updated"
],
"created_at": "2023-01-01T12:00:00Z"
}
This generic answer is likely a result of me not setting up Postman correctly, but I find it odd that I get a valid response instead of a 404 error (or other error). Any suggestions would be appreciated.
Hi @nick-youngblut,
Yes the subdomain looks like a259-2600-1700-3681-ca90-41e4-3c7b-ad6d-6ebe I think the original author shortened it or maybe long ago they had short domains. Also, my bad. I had a token stored in ASANA_PERSONAL_ACCESS_TOKEN in my environment variable and changed it so it should be TEMP_PAT from the example. Also, note the ngrok in the script shows .ngrok.io suffix. The free version with an account sign up will give you a .ngrok-free.app suffix (at least that's what I have noticed on my side) so if yours is .ngrok.io you can modify my version of the script to reflect that.
I am not sure what error you are experiencing with webhooks through postman but I recommend my approach that I explained here in our developer forum for starting with webhooks.
@jv-asana the Postman setup as show in the example video is very glib, but the setup is critical to getting the working example running (as described in the docs). Some Postman setup items not explained:
{{baseUrl}}in the POST requestHeaderssetupAuthorizationssetupEnvironmentssetup (if needed)