thomasoide / nicar2024-weather

nicar2024-weather

Overview

• A bit about us and the NYT Weather Data Team (also: We're hiring!)
• Monitoring real-time warnings
• Finding and using historical data

Monitoring Real Time Warnings

The Weather Service API

• Documentation: https://www.weather.gov/documentation/services-web-api
• Click on "Specification"
• Building a URL:
  • Base endpoint: https://api.weather.gov/alerts/active
  • We want actual warnings, not tests: ?status=actual
  • Area? Let's say Maryland, Virginia and D.C. You can get fancier here, but states are easy: &area=MD,DC,VA
  • Code. This is the warning type. Tornado warning, tornado watch, etc. List is here.
  • Limit. &limit=500
• But what about monitoring it ... for free!

The Bot Tooling We Use

• Github Actions
• Makefiles
• Slack

Setup for this class

1. Sign into Github (or quickly make an account if you haven't already)
2. Go to github.com/jkeefe/nicar2024-weather
3. Chose the "Fork" button
4. Note that the owner is now you. Click "Create fork"
5. After a minute, you will have a new screen. Note that your name is up at the top! This is your copy. You can use this now or just watch and return to it later. (If you see jkeefe instead of your name, you're on the wrong screen. Go find your copy in your github account.)
6. Now click the green "<> Code" button and, after you do, the "Codespaces" tab under it.
7. Click "Create Codespace on Main"

Take a look around!

• File list on the left side
• Coding happens in the big window
• There's a terminal window at the bottom.

How to save your work

This is an ephemperal instance! The instance will live in your account for a few days, but unless you take active steps, it will disappear. Which is good! You need to actively save your code back to the repo to make sure you have it.

• Save all of the files you want to commit to your code
• Maybe even close them to make sure!
• Click the github source-control button
• Enter a commit message, like "edited readme"
• Use the blue dropdown arrow
• Pick "Commit & Sync"
• You will be warned that there are no changes staged, and do you want to stage and all of your changes. Say "Yes"

When you're done for the day

Running computers cost money. You get 60 hours free every month and 15 gigabytes of storage. But let's not waste those free hours.

• Go to (github.com/codespaces)[https://github.com/codespaces]
• Pick the three dots next to the Active codespace.
• Chose "Stop codespace"
• If you forget, don't worry: It'll shut down automatically after 30 minutes. But why waste that?
• Go back to the main tab, and you'll see it's gone
• Can restart

Weather Warnings Code

• Open the warnings folder
• Look at the Makefile in that folder
• Open the Terminal
• cd warnings
• make clean
• make download

Don't do make warnings yet. Let's look at what this does!

• okay, when John says so, do make warnings

Try each of these and then run make warnings again.

• Add a full description: uncomment line 87 (delete the //)
• Add a map link, using geojson.io! uncomment lines 89-92
• Ignore warnings we've seen before: uncomment the others ...
  • line 57
  • lines 72-79
  • line 94
  • line 98

Making it work for Slack

You need to make a Slack app (it's easier than that sounds) and get a "bot token."

The only catch is that depending on your existing Slack setup, you may need to get an administrator to approve the creation of an app. The good thing is that you are only requesting the ability to chat:write, which is simply posting into a channel.

Also? This works in a free Slack workspace. So you can do this all on your own if you want; just make a new workspace at slack.com.

OK, here's what to do:

• Do steps 1, 2 and 3 in this Slack app quickstart
• In step 2, "Requesting Scopes" you just need the chat:write scope and it will only work in channels where the bot is invited.
• The thing you want is the "Bot User OAuth Token" which always starts oxob-. That's the token.
• Locally, you can enter the following to test it directly into your Slack:

export SLACK_TOKEN=[your_token]

For example:

export SLACK_TOKEN=xoxb-123-456-abc-zyz

Now try make slack!

Running as a Github Action

Github actions allow you to run your code in the cloud really easily.

The driver of any github action is a yaml file in the .github/workflows directory of a repo, like this one.

In short, here's what our warnings Github Action does:

• It starts running according to a cron statement ([here every 10 minutes(https://github.com/jkeefe/nicar2024-weather/blob/39ae476058f19021be90a70dbc59b60cef120fd5/.github/workflows/warnings.yml#L5C1-L5C103)])
• Spins up a computer running ubuntu.
• Checks out this repo
• Loads node.js and installs packages (or pulls them from a cache if nothing has changed).
• Reads a SLACK_TOKEN secret
• Runs make all just like we did in the terminal
• Commits the new data to the repo (saving our seen.json for next run)

To get this working, you need to do two key things:

Let the Action write back to the repo

• Settings > Actions > General > Workflow Permissions > Read and Write permissions > SAVE
• Don't forget to click "Save!"

Let the Action know your Slack Token

• Settings > Secrets and varialbes > Actions > Repository Secrets > New Repository Secret

• Enter SLACK_TOKEN in the top box
• Paste your "Bot User OAuth Token" which always starts oxob- into the larger box

Then ... run your action:

• Actions > warnings > Run workflow dropdown > Run workflow button

• Click the "warnings" label next to the yellow dot to watch it in action

To automate it ...

• Uncomment these two lines in the warnings.yaml file
• Adjust the cron syntax if necessary (here's a good site for that)
• Commit your changes and push to Github.

Historical Data

• Look at threadex/Makefile
• Open the Terminal
• cd ../threadex
• make initialize
• Optionally to generate station ids: make stations
• make scrape

ACIS documentation: http://www.rcc-acis.org/docs_webservices.html

Query builder: https://builder.rcc-acis.org/