Convert and export GitHub Issues and Pull Requests to JSON and CSV from a Terminal or Node.js app.
- 1. Overview
- 2. Installation
- 3. Usage
- 4. API
- 4.1.
grc.authenticate({token, type, key}): void
- 4.2.
grc.getAll({owner, repo}): Promise
- 4.3.
grc.issues.getForRepo({owner, repo}): Promise
- 4.4.
grc.logger
- 4.5.
grc.options
- 4.6.
grc.pullRequests.getForRepo({owner, repo}): Promise
- 4.7.
grc.save({data, dest}): Promise
- 4.8.
grc.toCsv({data=[]}): Promise
- 4.1.
- 5. Version
- 6. Contributing
- 7. License
github-resource-converter
(alias grc
) exports your GitHub and GitHub Enterprise repositories' Issues and Pull Requests to CSV
and JSON
file formats. It's helpful whenever you need to:
-
Use spreadsheets to analyze, modify, print, or summarize large amounts of data with Pivot Tables and other important financial or statistical operations
Example:
# Convert all GitHub issues to CSV: $ github-resource-converter \ --owner foo \ --repo bar # => Saved "foo-bar-issues-export.2018-03-20T02_11_04_356Z.csv".
-
Share data with other tools like GitLab and JIRA.
Example:
# Convert and save all GitHub Enterprise # Pull Requests as JSON (using the grc alias): $ grc \ --owner gregswindle \ --repo github-resource-converter \ --resource-type pr \ --dest './docs/reports/export.json' # Saved "docs/reports/gregswindle-github-resource-converter-pr-export.2018-03-20T02_18_33_682Z.json".
- Required:
github-resource-converter
is written in JavaScript (CommonJS) for Node.js version
which must be installed prior to use. Node.js requires npm, which is used for installing dependencies. (npm installs with Node.js.)
-
Recommended: To avoid rate-limiting, you should create a personal access token and save your personal access token.
-
MacOS and Unix
$ mkdir -p /usr/local/etc/github-resource-center/envvars/ $ touch /usr/local/etc/github-resource-center/envvars/.env $ echo "GITHUB_ACCESS_TOKEN="{your-personal-access-token-value}" > \ /usr/local/etc/github-resource-center/envvars/.env
-
Windows
> md -p C:\usr\local\etc\github-resource-center\envvars\ > touch C:\usr\local\etc\github-resource-center\envvars\.env > echo "GITHUB_ACCESS_TOKEN="{your-personal-access-token-value}" >> C:\usr\local\etc\github-resource-center\envvars\.env
-
# Install globally to execute from a Terminal/command-line
$ npm i -g github-resource-converter
# Install as a dependency within a Node.js app
$ npm i --save github-resource-converter
The following examples assume that
github-resource-converter
is installed globally and invoked from a Terminal (command-line interface)
--dest export.[csv|json]
You can convert GitHub (Enterprise) Issues and Pull Requests into two file formats:
CSV
andJSON
.
- ![CSV]CSV is the default format.
- JSON formatting requires a
--dest
value with a.json
file extension.
--resource-type, -t
Convert and export GitHub (Enterprise) by
--resource-type
:issues
,pull_requests
, or both (all
).
-
issues
is the defaultresource-type
. -
prs
require a--resource-type
or-t
value ofpr
prs
pull_request
pull_requests
-
--resource-type all
will export allissues
andprs
into a single file.
Filtering is currently unavailable.
If you're interested in CONTRIBUTING to features like filters--e.g., only select "open" issues--we're happily accepting pull requests!
-
CSV
GitHub:
# GET https://api.github.com/rrepos/:owner/:repo/issues $ github-resource-converter \ --owner gregswindle \ --repo github-resource-converter \ --dest './docs/reports/export.csv'
GitHub Enterprise:
# GET https://api.ecorp.com/api/v3/repos/:owner/:repo/issues $ grc --base-url https://api.ecorp.com/api/v3 \ --owner evilcorp \ --repo ecoin
-
JSON
GitHub:
# GET https://api.github.com/repos/:owner/:repo/issues $ github-resource-converter \ --owner gregswindle \ --repo github-resource-converter \ --dest './docs/reports/export.json'
GitHub Enterprise:
# GET https://api.ecorp.com/api/v3/repos/:owner/:repo/issues $ grc --base-url https://api.ecorp.com/api/v3 \ --owner evilcorp \ --repo ecoin --dest ./export.json
-
CSV
GitHub:
# GET https://api.github.com/repos/:owner/:repo/pulls $ github-resource-converter \ --owner gregswindle \ --repo github-resource-converter \ --resource-type pr
GitHub Enterprise:
# GET https://api.ecorp.com/api/v3/repos/:owner/:repo/pulls $ grc --base-url https://api.ecorp.com/api/v3 \ --owner evilcorp \ --repo ecoin \ --resource-type pr
-
JSON
GitHub:
# GET https://api.github.com/repos/:owner/:repo/pulls $ github-resource-converter \ --owner gregswindle \ --repo github-resource-converter \ --resource-type pr --dest './export.json'
GitHub Enterprise:
# GET https://api.ecorp.com/api/v3/repos/:owner/:repo/pulls $ grc --base-url https://api.ecorp.com/api/v3 \ --owner evilcorp \ --repo ecoin \ --resource-type pr --dest './export.json'
-
CSV
GitHub:
# GET https://api.github.com/repos/:owner/:repo/pulls $ github-resource-converter \ --owner gregswindle \ --repo github-resource-converter \ --resource-type all
GitHub Enterprise:
# GET https://api.ecorp.com/api/v3/repos/:owner/:repo/pulls $ grc --base-url https://api.ecorp.com/api/v3 \ --owner evilcorp \ --repo ecoin \ --resource-type all
-
JSON
GitHub:
# GET https://api.github.com/repos/:owner/:repo/pulls $ github-resource-converter \ --owner gregswindle \ --repo github-resource-converter \ --resource-type all --dest './export.json'
GitHub Enterprise:
# GET https://api.ecorp.com/api/v3/repos/:owner/:repo/pulls $ grc --base-url https://api.ecorp.com/api/v3 \ --owner evilcorp \ --repo ecoin \ --resource-type all --dest './export.json'
--owner, -o
Required. The GitHub account name or organization name.
--repo, -r
Required. The name of the GitHub (or GitHub Enterprise) repository.
--base-url
The GitHub REST API v3 URL origin, or a GitHub Enterprise URL origin and path-prefix.
Default value: https://api.github.com
--dest, -d
The destination path and file name of the CSV.
Default value: ./export.csv --no-auto-filename
Disable automatic file naming.
Default value: false --resource-type, -t
Declares whether to convert and export Issues, Pull Requests, or All.
Default value: issues Valid values: all, issue, issues, pr, prs, pull_request, pull_requests
Errors are written to the console (stdout
) as JSON:
# Attempt to fetch issues from a repository that doesn't exist
$ grc --owner example --repo error
[2018-03-20T02:31:24.737Z] ERROR: github-resource-converter/48219 on User.router.home: {"message":"Not Found","documentation_url":"https://developer.github.com/v3/issues/#list-issues-for-a-repository"}
HttpError: {"message":"Not Found","documentation_url":"https://developer.github.com/v3/issues/#list-issues-for-a-repository"}
at response.text.then.message (/p/a/t/h/github-resource-converter/node_modules/@octokit/rest/lib/request/request.js:56:19)
at <anonymous>
at process._tickCallback (internal/process/next_tick.js:188:7)
The --help
flag displays all options:
$ grc --help
Convert and export GitHub resources--Issues and Pull Requests--to CSV and JSON.
Usage
$ grc [options] [info]
$ github-resource-converter [options] [info]
Options
--base-url The GitHub REST API v3 URL origin, or a
GitHub Enterprise URL origin and path-prefix.
[Default: 'https://api.github.com']
--dest, -d The CSV's destination path and file name.
[Default: './resources.csv']
--no-auto-filename Disable automatic file naming.
[Default: false]
--owner, -o The GitHub account name or organization name.
--repo, -r The name of the GitHub (or GitHub enterprise)
repository.
--resource-type, -t "issues", "prs", or "all".
[Default: 'issues']
Info
--help Show this dialog.
--version Display the installed semantic version.
Examples
$ grc --owner github --repo hub
// => Exported CSV to /path/of/cwd/issues.csv.
$ grc --owner github --repo hub -dest './reports/issues/YYYY-MM-DD.csv'
// => Exported CSV to /path/to/reports/issues/YYYY-MM-DD.csv.
$ grc --owner example --repo error
// =>
[2018-03-19T08:04:06.596Z] ERROR: github-resource-converter/00000 on localhost: Cannot destructure property `data` of 'undefined' or 'null'.
TypeError: Cannot destructure property `data` of 'undefined' or 'null'.
at paginate (/p/a/t/h/github-resource-converter/lib/base-resource-converter.js:39:16)
at <anonymous>
at process._tickCallback (internal/process/next_tick.js:188:7
Use the --version
flag to see which version you have installed:
$ github-resource-converter --version
# => 1.1.17
Test the
github-resource-converter (grc)
API in your Web browser .
Most GitHub API calls don't require authentication. Rules of thumb:
- If you can see the information by visiting the site without being logged in, you don't have to be authenticated to retrieve the same information through the API.
- If you want to change data, you have to be authenticated.
octokit/rest.js. (2018). GitHub. Retrieved 21 March 2018, from https://github.com/octokit/rest.js#authentication
Name | Type | Description | Notes |
---|---|---|---|
key | String | ||
token | String | ||
type | Enum | basic , oauth , oauth-key-secret , token , and integration |
// Token (https://github.com/settings/tokens)
grc.authenticate({
token: 'secrettoken123',
type: 'token'
})
Retrieve all open and closed Issues and Pull Requests from a GitHub project.
/repos/:owner/:repo/issues
/repos/:owner/:repo/pulls
grc.getAll
combines the results ofgrc.issues.getForRepo
andgrc.pullRequests.getForRepo
into a singleArray<object>
.For more information, see
grc.issues.getForRepo
andgrc.pullRequests.getForRepo
below.
Retrieve all open and closed issues for a repository with this proxy method for octokit.issues.getForRepo
.
/repos/:owner/:repo/issues
-
async/await:
const result = await grc.issues.getForRepo({ owner: 'gregswindle', repo: 'eslint-plugin-crc' })
-
Promise:
grc.issues .getForRepo({ owner: 'gregswindle', repo: 'eslint-plugin-crc' }) .then(result => {}) .catch(err => {})
A trentm/node-bunyan
logger instance that uses the LONG
thlorenz/bunyan-format
writeable
stream for output.
Logger {
domain: null,
_events: {},
_eventsCount: 0,
_maxListeners: undefined,
_level: 30,
streams:
[{
type: 'stream',
stream: BunyanFormatWritable {
},
closeOnExit: false,
level: 30,
raw: false
}],
serializers: null,
src: false,
fields: {
name: 'github-resource-converter',
hostname: 'Gregorys-MBP-2.fios-router.home',
pid: 76698
},
debug(): void,
error(): void,
fatal(): void,
info(): void,
trace(): void,
warn(): void
}
Contains default values for api
, cli
, and meta
data.
Toggle view of grc.options.
{
"api": {
"baseUrl": "https://api.github.com",
"headers": {
"Accept":
"application/vnd.github.v3+json, application/vnd.github.symmetra-preview+json",
"user-agent": "gregswindle/github-resource-converter v1.1.17"
},
"owner": null,
"repo": null,
"requestMedia": "application/vnd.github.v3+json",
"timeout": 0
},
"cli": {
"flags": {
"baseUrl": {
"default": "https://api.github.com",
"type": "string"
},
"dest": {
"alias": "d",
"default": "./export.csv",
"type": "string"
},
"no-auto-filename": {
"default": false,
"type": "boolean"
},
"owner": {
"alias": "o",
"type": "string"
},
"repo": {
"alias": "r",
"type": "string"
},
"resource-type": {
"alias": "t",
"default": "issues",
"type": "string"
}
}
},
"meta": {
"input": [],
"flags": {
"noAutoFilename": false,
"baseUrl": "https://api.github.com",
"dest": "./export.csv",
"d": "./export.csv",
"resourceType": "issues",
"t": "issues"
},
"pkg": {
"name": "github-resource-converter",
"description":
"Convert and export GitHub resources--Issues and Pull Requests--to CSV and JSON.",
"version": "1.1.17",
"author": {
"name": "Greg Swindle",
"email": "greg@swindle.net",
"url": "https://github.com/gregswindle"
},
"bin": {
"grc": "lib/cli.js"
},
"bugs": {
"url": "https://github.com/gregswindle/github-resource-converter/issues"
},
"commitplease": {
"style": "angular",
"types": [
"build",
"chore",
"ci",
"docs",
"feat",
"fix",
"perf",
"refactor",
"revert",
"style",
"test"
],
"scope": "\\S+.*"
},
"contributors": [],
"dependencies": {
"@octokit/rest": "15.2.6",
"bunyan": "1.8.12",
"bunyan-format": "0.2.1",
"dotenv": "5.0.1",
"fs-extra": "5.0.0",
"insight": "0.10.1",
"jsonexport": "2.0.11",
"lodash": "4.17.5",
"meow": "4.0.0"
},
"devDependencies": {
"@semantic-release/changelog": "2.0.1",
"@semantic-release/git": "4.0.1",
"@semantic-release/npm": "3.2.4",
"ajv": "6.4.0",
"ajv-keywords": "3.1.0",
"codacy-coverage": "2.1.1",
"commitplease": "3.2.0",
"coveralls": "3.0.0",
"eslint": "4.19.1",
"eslint-config-prettier": "^2.4.0",
"eslint-config-scanjs": "1.0.0-beta4",
"eslint-config-standard": "11.0.0",
"eslint-config-xo": "0.20.1",
"eslint-plugin-import": "2.10.0",
"eslint-plugin-jsdoc": "3.5.0",
"eslint-plugin-json": "1.2.0",
"eslint-plugin-no-unsafe-innerhtml": "1.0.16",
"eslint-plugin-node": "6.0.1",
"eslint-plugin-prettier": "^2.2.0",
"eslint-plugin-promise": "3.7.0",
"eslint-plugin-security": "1.4.0",
"eslint-plugin-standard": "3.0.1",
"eslint-plugin-unicorn": "4.0.2",
"eslint-plugin-xss": "0.1.9",
"fixpack": "2.3.1",
"husky": "^0.14.3",
"jest": "22.4.3",
"jest-runner-eslint": "0.4.0",
"lec": "^1.0.1",
"lint-staged": "7.0.3",
"markdown-magic": "0.1.21",
"markdown-magic-dependency-table": "1.3.2",
"markdown-magic-install-command": "1.3.1",
"markdown-magic-package-scripts": "1.2.1",
"nsp": "^3.2.1",
"prettier": "1.11.1",
"semantic-release": "15.1.5"
},
"engines": {
"node": ">=8.0.0"
},
"eslintIgnore": ["lib/__tests__/coverage/**"],
"files": ["lib"],
"homepage":
"https://github.com/gregswindle/github-resource-converter/#readme",
"jest": {
"automock": false,
"collectCoverage": true,
"coverageDirectory": "lib/__tests__/coverage",
"coverageThreshold": {
"global": {
"branches": 100,
"functions": 100,
"lines": 100,
"statements": 100
}
},
"moduleDirectories": ["node_modules", "lib"],
"testMatch": [
"<rootDir>/lib/__tests__/**/*.test.js",
"<rootDir>/lib/__tests__/*.test.js"
],
"watchman": false
},
"jest-runner-eslint": {
"cliOptions": {
"config": ".github/config/jest-runner-eslint.config.json",
"fix": true
}
},
"keywords": [
"QA",
"convert",
"converter",
"converter",
"csv",
"export",
"github",
"google sheets",
"issue",
"json",
"pr",
"pull request",
"pull-request",
"quality assurance",
"quality-assurance",
"reporting",
"reports",
"save",
"sheets",
"spreadsheets",
"testing"
],
"license": "MIT",
"lint-staged": {
"*.js": ["npm run lint:js", "git add"],
"*.json": ["npm run lint:json", "git add"],
"*.md": ["npm run lint:md", "npm run docs", "git add"]
},
"main": "lib/index.js",
"prettier": {
"semi": false,
"singleQuote": true
},
"release": {
"generateNotes": {
"preset": "angular"
},
"prepare": [
"@semantic-release/changelog",
"@semantic-release/npm",
"@semantic-release/git"
],
"verifyConditions": ["@semantic-release/npm", "@semantic-release/git"]
},
"repository": {
"type": "git",
"url":
"git+https://github.com/gregswindle/github-resource-converter.git"
},
"scripts": {
"docs": "npm run docs:toc",
"docs:toc":
"md-magic --config '.github/config/markdown.config.js' --path '**/*.md' --ignore 'node_modules'",
"lint":
"npm run lint:js && npm run lint:json && npm run lint:manifest && npm run lint:md",
"lint:js":
"node_modules/.bin/eslint -c .github/config/.eslintrc.yml --ext .js . --fix",
"lint:json":
"prettier ./**/*.json --ignore-path '.github/config/.prettierignore' --write",
"lint:manifest": "fixpack",
"lint:md": "prettier ./**/*.md -prose-wrap always --write",
"posttest:ci:coverage:codacy":
"cat ./lib/__tests__/coverage/lcov.info | codacy-coverage",
"precommit": "lint-staged",
"prepare": "npm run lint && npm run security",
"prepublish":
"lec lib/cli.js -c LF && npm run security && semantic-release -d",
"prepublishOnly": "npm run prepare",
"pretest": "npm run lint",
"security": "npm run security:nsp:scan",
"security:nsp:scan": "nsp check",
"test": "jest",
"test:config": "jest --showConfig",
"test:watch": "jest ./lib/__tests__/*.test.js --watch",
"test:watch:all": "jest ./lib/__tests__/*.test.js --watchAll"
},
"readme": "ERROR: No README data found!",
"_id": "github-resource-converter@1.1.17"
},
"help":
"\n Convert and export GitHub resources--Issues and Pull Requests--to CSV and JSON.\n\n Usage\n\n $ grc [options] [info]\n $ github-resource-converter [options] [info]\n\n Options\n --base-url The GitHub REST API v3 URL origin, or a\n GitHub Enterprise URL origin and path-prefix.\n [Default: 'https://api.github.com']\n --dest, -d The CSV's destination path and file name.\n [Default: './resources.csv']\n --no-auto-filename Don't append an ISO 8601-like timestamp to the\n output file.\n [Default: false]\n --owner, -o The GitHub account name or organization name.\n --repo, -r The name of the GitHub (or GitHub enterprise)\n repository.\n --resource-type, -t \"issues\", \"prs\", or \"all\".\n [Default: 'issues']\n\n Info\n\n --help Show this dialog.\n --version Display the installed semantic version.\n\n Examples\n\n $ grc --owner github --repo hub\n // => Exported CSV to /path/of/cwd/issues.csv.\n\n $ grc --owner github --repo hub -dest './reports/issues/YYYY-MM-DD.csv'\n // => Exported CSV to /path/to/reports/issues/YYYY-MM-DD.csv.\n\n $ grc --owner example --repo error\n // =>\n [2018-03-19T08:04:06.596Z] ERROR: github-resource-converter/00000 on localhost: Cannot destructure property `data` of 'undefined' or 'null'.\n TypeError: Cannot destructure property `data` of 'undefined' or 'null'.\n at paginate (/p/a/t/h/github-resource-converter/lib/base-resource-converter.js:39:16)\n at <anonymous>\n at process._tickCallback (internal/process/next_tick.js:188:7)\n"
}
}
Retrieve an array of all open and closed pull requests for a GitHub or GitHub Enterprise repository with this proxy method for
octokit.pullRequests.getAll
.
/repos/:owner/:repo/pulls
-
async/await:
const grc = require('github-resource-coverter') const getAllPullRequests = async (params) = { try { const WHITESPACE = 2 const prs = await grc.pullRequests.getForRepo(params) grc.logger.info(JSON.stringify(prs, null, WHITESPACE)) } catch (err) { grc.logger.error(err) } } await getAllPullRequests({ owner: 'democracy-ia', repo: 'govinfo-link-js' })
-
Promises:
const grc = require('github-resource-coverter') grc.pullRequests .getForRepo({ owner: 'democracy-ia', repo: 'govinfo-link-js' }) .then(prs => { const WHITESPACE = 2 grc.logger.info(JSON.stringify(prs, null, WHITESPACE)) }) .catch(err => { grc.logger.error(err) })
Export a collection of Issues or Pull Requests to your local filesystem.
Field | Type | Description |
---|---|---|
data | JSON | object | |
dest | string |
Default value:
Allowed values:
|
-
async/await:
// Save as JSON const result = await grc.save({ data, dest: './export.json' })
-
Promise:
// Save as CSV grc .save({ data, dest: './export.csv' }) .then(result => {}) .catch((err = {}))
Converts (deeply) nested JSON into CSV format, returning a Promise<string>
.
Field | Type | Description |
---|---|---|
data | JSON | object |
-
async/await:
const result = await grc.toCsv(data)
-
Promise:
grc .toCsv(data) .then(result => {}) .catch((err = {}))
We welcome contributions with GitHub issues and pull requests.
Before embarking on a significant change, please adhere to the following guidelines:
-
Create an issue—e.g., a defect ("bug") report or a feature request—to propose changes.
Exceptions:
If you're working on documentation and fixing something simple like a typo or an easy bug, go ahead and make a pull request.
-
Follow the CONTRIBUTING guidelines.
Why:
Standards and guidelines make communication easier. If you're willing and able to program—or want to learn how— following the guidelines will increase the likelihood of adding your changes to the software product.
-
Why:
It's more fun when everybody's friendly and respectful.
-
Make a pull request when you're ready for other to review your changes (or you get stuck somewhere).
PR novices:
🙋 Never created a pull request? No problem. 🆓 Take this free online training . (It even covers most of the conventions in the CONTRIBUTING guidelines!)
MIT © Greg Swindle.
Read the NOTICE for all third-party software that github-resource-converter
uses.