Giters

nwtgck / rich-piping-server

Rich Piping Server

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

basic-authenticationopenid-connectpiping-server

rich-piping-server

Node CI

Rich Piping Server

Usage

Prepare config.yaml as follows.

version: '1'
config_for: rich_piping_server

# optional
basic_auth_users:
  - username: user1
    password: pass1234

# optional
allow_paths:
  # Allow transfer over "/0s6twklxkrcfs1u", not "/0s6twklxkrcfs1u/mypath"
  - /0s6twklxkrcfs1u
  # Allow transfer over the regular expression below
  - regexp: ^/[abcd]+.*$
  # Simple at /mytop1/. Show version at /mytop1/version. Show help at /mytop1/help. Allow transfer /mytop1/mypath, /mytop1/hoge,....
  - index: /mytop1
  # Create multiple "index".
  - index: /mytop2

# Respond a fake nginx 500 down page when path not allowed
rejection: fake_nginx_down

# Close socket when path not allowed
#rejection: socket_close

# Respond a fake nginx 500 down with version
#rejection:
#  fake_nginx_down:
#    nginx_version: 99.9.9

Run the server as follows. Hot reload of config is available.

npx nwtgck/rich-piping-server --config-path=config.yaml

Here are some example results of the server with the config.

  • transferable: curl -u user1:pass1234 http://localhost:8080/0s6twklxkrcfs1u
  • transferable: curl -u user1:pass1234 -T- http://localhost:8080/0s6twklxkrcfs1u
  • transferable: curl -u user1:pass1234 http://localhost:8080/aabbaaccba
  • transferable: curl -u user1:pass1234 http://localhost:8080/b
  • Web UI because of "index": curl -u user1:pass1234 http://localhost:8080/mytop1/
  • version because of "index": curl -u user1:pass1234 http://localhost:8080/mytop1/version
  • help because of "index": curl -u user1:pass1234 http://localhost:8080/mytop1/help
  • transferable because of "index": curl -u user1:pass1234 http://localhost:8080/mytop1/mypath
  • Web UI because of "index": curl -u user1:pass1234 http://localhost:8080/mytop2/
  • reject because path is not allowed: curl -u user1:pass1234 http://localhost:8080/
  • reject because of no basic auth: curl http://localhost:8080/0s6twklxkrcfs1u

Tags

These tags are available in config.

  • !env MY_VALUE1
  • !concat [ "hello", !env "MY_VALUE1" ]
  • !json_decode "true"
  • !unrecommended_js "return new Date().getMonth() < 5"

Here is an example.

...

basic_auth_users:
  - username: !env "USERNAME1"
    password: !env "PASSWORD1"
...

!unrecommended_js is not recommended to use because this behavior highly depends on the underlying runtime and the behavior may change.

OpenID Connect

This is an experimental feature and it may have breaking changes without config version update.

version: '1'
config_for: rich_piping_server

# OpenID Connect is experimental
experimental_openid_connect: true

# optional
openid_connect:
  issuer_url: https://example.us.auth0.com
  client_id: !env "OIDC_CLIENT_ID"
  client_secret: !env "OIDC_CLIENT_SECRET"
  redirect:
    # Rich Piping Server callback URL
    uri: https://my.rich.piping.server/callback
    path: /callback
  allow_userinfos:
    - sub: auth0|0123456789abcdef01234567
    - email: johnsmith@example.com
    - email: alice@example.com
      require_verification: false
  # Session ID is generated after authentication successful and user in "allow_userinfos"
  # Shutting down Rich Piping Server revokes all sessions for now
  session:
    cookie:
      name: my_session_id
      http_only: true
    # optional (useful especially for command line tools to get session ID)
    forward:
      # A CLI may server an ephemeral HTTP server on :65106 and open https://my.rich.piping.server/?my_session_forward_url=http://localhost:65106
      # The opened browser will POST http://localhost:65106 with `{ "session_id": "..." }` after logged in.
      query_param_name: my_session_forward_url
      allow_url_regexp: ^http://localhost:\d+.*$
    # optional
    custom_http_header: X-My-Session-ID
    age_seconds: 86400
  # optional
  log:
    # optional
    userinfo:
      sub: false
      email: false

# Close socket when path not allowed
rejection: socket_close

What is session forwarding for?

The config has .openid_connect.session.forward. It is useful for CLI to get session ID.

Example CLI to get session ID in Node.js 
const http = require("http");

(async () => {
  const richPipingServerUrl = "https://my.rich.piping.server";
  const sessionId = await getSessionId(richPipingServerUrl);
  console.log("sessionId:", sessionId);
  // (you can use session ID now save to ~/.config/... or something)

  // Example to access the Rich Piping Server
  const res = await fetch(`${richPipingServerUrl}/version`, {
    headers: { "Cookie": `my_session_id=${sessionId}` }
  });
  console.log("Underlying Piping Server version:", await res.text());
})();

// Open default browser and get session ID
function getSessionId(richPipingServerUrl) {
  return new Promise((resolve, reject) => {
    const server = http.createServer((req, res) => {
      if (req.method === "OPTIONS") {
        res.writeHead(200, {
          "Access-Control-Allow-Origin": "*",
          "Access-Control-Allow-Methods": "GET, POST, OPTIONS",
          "Access-Control-Allow-Headers": "Content-Type",
          // Private Network Access preflights: https://developer.chrome.com/blog/private-network-access-preflight/
          ...(req.headers["access-control-request-private-network"] === "true" ? {
            "Access-Control-Allow-Private-Network": "true",
          }: {}),
          "Access-Control-Max-Age": 86400,
          "Content-Length": 0
        });
        res.end();
        return;
      }
      if (req.method === "POST") {
        let body = "";
        req.on('data', (chunk) => {
          body += chunk;
        });
        req.on('end', () => {
          res.writeHead(200, {
            "Access-Control-Allow-Origin": "*",
          });
          res.end();
          try {
            const sessionId = JSON.parse(body).session_id;
            resolve(sessionId);
          } catch (err) {
            reject(err);
          }
          server.close();
        });
        req.on("error", (err) => {
          server.close();
          reject(err);
        });
      }
    });
    server.listen(0, () => {
      // This ephemeral server is session forward URL
      const sessionForwardUrl = `http://localhost:${server.address().port}`;
      const serverUrl = new URL(richPipingServerUrl);
      serverUrl.searchParams.set("my_session_forward_url", sessionForwardUrl);
      // Open the browser
      // NOTE: This is only for macOS. Use other command for Windows, Linux
      require("child_process").execSync(`open ${serverUrl.href}`);
      // Use `npm install open` and `open(serverUrl.href)`
    });
  });
}

Run on Docker

Prepare ./config.yaml and run as follows on Docker.

docker run -p 8181:8080 -v $PWD/config.yaml:/config.yaml nwtgck/rich-piping-server --config-path=/config.yaml

The server runs on http://localhost:8181.

Config examples

Config examples are found in the tests:

rich-piping-server/test/config-v1.test.ts

Lines 42 to 459 in a2cd491

it("should transfer when all path allowed", async () => {
// language=yaml
configRef.set(readConfigV1AndNormalize(`
version: "1"
config_for: rich_piping_server
rejection: socket_close
`));
await shouldTransfer({path: "/mypath1"});
});
it("should transfer with regular expression", async () => {
// language=yaml
configRef.set(readConfigV1AndNormalize(`
version: "1"
config_for: rich_piping_server
allow_paths:
- regexp: "^/[a-c]+"
rejection: socket_close
`));
await shouldTransfer({path: "/aabbcc"});
await shouldTransfer({path: "/abchoge"});
await shouldNotTransferAndSocketClosed({path: "/hoge"});
});
it("should transfer at only allowed path", async () => {
// language=yaml
configRef.set(readConfigV1AndNormalize(`
version: "1"
config_for: rich_piping_server
allow_paths:
- /myallowedpath1
rejection: socket_close
`));
await shouldTransfer({path: "/myallowedpath1" });
await shouldNotTransferAndSocketClosed({path: "/mypath1"});
await shouldNotTransferAndSocketClosed({path: "/myallowedpath1/path1"});
});
context("index", () => {
it("should create a new index", async () => {
// language=yaml
configRef.set(readConfigV1AndNormalize(`
version: "1"
config_for: rich_piping_server
allow_paths:
- index: /myindex1
rejection: socket_close
`));
await shouldTransfer({path: "/myindex1/path1" });
// Should respond simple Web UI
{
const res = await requestWithoutKeepAlive(`${pipingUrl}/myindex1`);
assert((await res.body.text()).includes("Piping"));
}
// Should respond version
{
const res = await requestWithoutKeepAlive(`${pipingUrl}/myindex1/version`);
assert.strictEqual((await res.body.text()).trim(), pipingVersion.VERSION);
}
});
it("should create multiple indexes", async () => {
// language=yaml
configRef.set(readConfigV1AndNormalize(`
version: "1"
config_for: rich_piping_server
allow_paths:
- index: /myindex1
- index: /myindex2
rejection: socket_close
`));
await shouldTransfer({path: "/myindex1/path1" });
// Should respond simple Web UI
{
const res = await requestWithoutKeepAlive(`${pipingUrl}/myindex1`);
assert((await res.body.text()).includes("Piping"));
}
// Should respond version
{
const res = await requestWithoutKeepAlive(`${pipingUrl}/myindex1/version`);
assert.strictEqual((await res.body.text()).trim(), pipingVersion.VERSION);
}
await shouldTransfer({path: "/myindex2/path1" });
// Should respond simple Web UI
{
const res = await requestWithoutKeepAlive(`${pipingUrl}/myindex2`);
assert((await res.body.text()).includes("Piping"));
}
// Should respond version
{
const res = await requestWithoutKeepAlive(`${pipingUrl}/myindex2/version`);
assert.strictEqual((await res.body.text()).trim(), pipingVersion.VERSION);
}
});
});
it("should reject with Nginx error page", async () => {
// language=yaml
configRef.set(readConfigV1AndNormalize(`
version: "1"
config_for: rich_piping_server
allow_paths:
- /myallowedpath1
rejection: fake_nginx_down
`));
await shouldTransfer({path: "/myallowedpath1" });
// Get request promise
const res = await requestWithoutKeepAlive(`${pipingUrl}/mypath1`);
assert.strictEqual(res.statusCode, 500);
assert.strictEqual(res.headers.server, "nginx/1.17.8");
});
it("should reject with Nginx error page with Nginx version", async () => {
// language=yaml
configRef.set(readConfigV1AndNormalize(`
version: "1"
config_for: rich_piping_server
allow_paths:
- /myallowedpath1
rejection:
fake_nginx_down:
nginx_version: 99.9.9
`));
await shouldTransfer({path: "/myallowedpath1" });
// Get request promise
const res = await requestWithoutKeepAlive(`${pipingUrl}/mypath1`);
assert.strictEqual(res.statusCode, 500);
assert.strictEqual(res.headers.server, "nginx/99.9.9");
});
it("should transfer with basic auth", async () => {
// language=yaml
configRef.set(readConfigV1AndNormalize(`
version: "1"
config_for: rich_piping_server
basic_auth_users:
- username: user1
password: pass1234
allow_paths:
- /myallowedpath1
rejection: socket_close
`));
await shouldNotTransferAndSocketClosed({path: "/mypath1"});
await shouldTransfer({
path: "/myallowedpath1",
headers: {
"Authorization": `Basic ${Buffer.from("user1:pass1234").toString("base64")}`,
},
});
});
context("custom tag", () => {
it("should resolve !env tag", async () => {
assert.strictEqual(process.env["MY_BASIC_PASSWORD"], undefined);
process.env["MY_BASIC_PASSWORD"] = "my_secret_password";
// language=yaml
configRef.set(readConfigV1AndNormalize(`
version: "1"
config_for: rich_piping_server
basic_auth_users:
- username: user1
password: !env "MY_BASIC_PASSWORD"
rejection: socket_close
`));
assert.strictEqual(configRef.get()!.basic_auth_users![0].password, process.env["MY_BASIC_PASSWORD"]);
delete process.env["MY_BASIC_PASSWORD"];
});
it("should resolve !concat tag", async () => {
// language=yaml
configRef.set(readConfigV1AndNormalize(`
version: "1"
config_for: rich_piping_server
basic_auth_users:
- username: user1
password: !concat ["my", "secret", "pass", "word"]
rejection: socket_close
`));
assert.strictEqual(configRef.get()!.basic_auth_users![0].password, ["my", "secret", "pass", "word"].join(""));
});
it("should resolve !json_decode tag", async () => {
// language=yaml
configRef.set(readConfigV1AndNormalize(`
version: "1"
config_for: rich_piping_server
basic_auth_users:
- username: user1
password: !json_decode '"mypassword"'
rejection: socket_close
`));
assert.strictEqual(configRef.get()!.basic_auth_users![0].password, "mypassword");
});
it("should resolve !unrecommended_js tag", async () => {
// language=yaml
configRef.set(readConfigV1AndNormalize(`
version: "1"
config_for: rich_piping_server
basic_auth_users:
- username: user1
password: !unrecommended_js |
return "mypasswordfromjavascript"
rejection: socket_close
`));
assert.strictEqual(configRef.get()!.basic_auth_users![0].password, "mypasswordfromjavascript");
});
it("should resolve nested tags", async () => {
assert.strictEqual(process.env["MY_USER_NAME_PREFIX"], undefined);
process.env["MY_USER_NAME_PREFIX"] = "myuserprefix_";
// language=yaml
configRef.set(readConfigV1AndNormalize(`
version: "1"
config_for: rich_piping_server
basic_auth_users:
- username: !concat [!env "MY_USER_NAME_PREFIX", 1234]
password: dummy
rejection: socket_close
`));
assert.strictEqual(configRef.get()!.basic_auth_users![0].username, "myuserprefix_1234");
delete process.env["MY_USER_NAME_PREFIX"];
});
});
context("OpenID Connect", () => {
it("should transfer", async () => {
const clientId = "myclientid";
const clientSecret = "thisissecret";
const issuerPort = await getPort();
const issuerUrl = `http://localhost:${issuerPort}`;
const providerServer = await serveOpenIdProvider({
port: issuerPort,
clientId,
clientSecret,
redirectUri: `${pipingUrl}/my_callback`,
});
const sessionCookieName = "my_session_id"
// language=yaml
configRef.set(readConfigV1AndNormalize(`
version: "1"
config_for: rich_piping_server
experimental_openid_connect: true
openid_connect:
issuer_url: ${issuerUrl}
client_id: ${clientId}
client_secret: ${clientSecret}
redirect:
uri: ${pipingUrl}/my_callback
path: /my_callback
allow_userinfos:
- sub: user001
session:
cookie:
name: ${sessionCookieName}
http_only: true
age_seconds: 60
rejection: socket_close
`));
const cookieJar = new CookieJar();
const axiosClient = axiosCookieJarSupport.wrapper(axios.create({ jar: cookieJar }));
const res1 = await axiosClient.get(`${pipingUrl}/my_first_visit`);
assert(res1.request.res.responseUrl.startsWith(`${issuerUrl}/interaction/`));
// NOTE: login should be "user001", any password is OK
const res2 = await axiosClient.post(`${res1.request.res.responseUrl}/login`, "login=user001&password=dummypass");
assert(res2.request.res.responseUrl.startsWith(`${issuerUrl}/interaction/`));
const res3 = await axiosClient.post(`${res2.request.res.responseUrl}/confirm`);
assert(res3.request.res.responseUrl.startsWith(`${pipingUrl}/my_callback?code=`));
const cookie = cookieJar.toJSON().cookies.find(c => c.key === sessionCookieName)!;
assert.strictEqual(cookie.domain, "localhost");
assert.strictEqual(cookie.httpOnly, true);
// HTML redirect included
assert(res3.data.includes(`content="0;/my_first_visit"`));
await shouldTransfer({
path: "/mypath",
headers: {
"Cookie": `${sessionCookieName}=${cookie.value}`,
},
});
providerServer.close();
});
it("should respond session forward page", async () => {
const clientId = "myclientid";
const clientSecret = "thisissecret";
const issuerPort = await getPort();
const issuerUrl = `http://localhost:${issuerPort}`;
const providerServer = await serveOpenIdProvider({
port: issuerPort,
clientId,
clientSecret,
redirectUri: `${pipingUrl}/my_callback`,
});
const sessionCookieName = "my_session_id"
// language=yaml
configRef.set(readConfigV1AndNormalize(`
version: "1"
config_for: rich_piping_server
experimental_openid_connect: true
openid_connect:
issuer_url: ${issuerUrl}
client_id: ${clientId}
client_secret: ${clientSecret}
redirect:
uri: ${pipingUrl}/my_callback
path: /my_callback
allow_userinfos:
- sub: user001
session:
forward:
query_param_name: my_session_forward_url
allow_url_regexp: (http://dummy_session_forward_url1)|(http://dummy_session_forward_url2)
cookie:
name: ${sessionCookieName}
http_only: true
age_seconds: 60
rejection: socket_close
`));
const cookieJar = new CookieJar();
const axiosClient = axiosCookieJarSupport.wrapper(axios.create({ jar: cookieJar }));
const res1 = await axiosClient.get(`${pipingUrl}?my_session_forward_url=http://dummy_session_forward_url1`);
assert(res1.request.res.responseUrl.startsWith(`${issuerUrl}/interaction/`));
// NOTE: login should be "user001", any password is OK
const res2 = await axiosClient.post(`${res1.request.res.responseUrl}/login`, "login=user001&password=dummypass");
assert(res2.request.res.responseUrl.startsWith(`${issuerUrl}/interaction/`));
const res3 = await axiosClient.post(`${res2.request.res.responseUrl}/confirm`);
assert(res3.request.res.responseUrl.startsWith(`${pipingUrl}/my_callback?code=`));
const cookie = cookieJar.toJSON().cookies.find(c => c.key === sessionCookieName)!;
assert.strictEqual(cookie.domain, "localhost");
assert.strictEqual(cookie.httpOnly, true);
assert(res3.data.includes(`<html>`) && res3.data.includes("</html>"));
assert(res3.data.includes(`<script>`) && res3.data.includes("</script>"));
assert(res3.data.includes(`sessionForwardUrl = "http://dummy_session_forward_url1"`));
assert(res3.data.includes(`window.close()`));
// Immediately forward page responded after logged in
{
const res = await axiosClient.get(`${pipingUrl}?my_session_forward_url=http://dummy_session_forward_url2`);
assert(res.data.includes(`<html>`) && res.data.includes("</html>"));
assert(res.data.includes(`<script>`) && res.data.includes("</script>"));
assert(res.data.includes(`sessionForwardUrl = "http://dummy_session_forward_url2"`));
assert(res.data.includes(`window.close()`));
}
// URL not in "allow_url_regexp" should be rejected
try {
await axiosClient.get(`${pipingUrl}?my_session_forward_url=http://should_be_invalid_session_forward_url`);
} catch (err) {
const axiosError = err as AxiosError;
assert.strictEqual(axiosError.response!.status, 400);
assert.strictEqual(axiosError.response!.data, "session forward URL is not allowed\n");
}
providerServer.close();
});
it("should parse log config", async () => {
// language=yaml
configRef.set(readConfigV1AndNormalize(`
version: "1"
config_for: rich_piping_server
experimental_openid_connect: true
openid_connect:
issuer_url: https://dummyissue
client_id: myclientid
client_secret: thisissecret
redirect:
uri: https://dummyredirecturi/my_callback
path: /my_callback
allow_userinfos: [ ]
session:
cookie:
name: dummycookiename
http_only: true
age_seconds: 60
log:
userinfo:
sub: true
email: false
rejection: socket_close
`));
assert.strictEqual(configRef.get()?.openid_connect?.log?.userinfo?.sub, true);
assert.strictEqual(configRef.get()?.openid_connect?.log?.userinfo?.email, false);
});
});

Migration from legacy config

The command below prints new config.

rich-piping-server --config-path=./config.yaml migrate-config

New Rich Piping Server supports the legacy config schema without migration.

Options

rich-piping-server [command]

Commands:
  rich-piping-server migrate-config  Print migrated config

Options:
  --help                             Show help                         [boolean]
  --version                          Show version number               [boolean]
  --host                             Bind address (e.g. 127.0.0.1, ::1) [string]
  --http-port                        Port of HTTP server         [default: 8080]
  --enable-https                     Enable HTTPS     [boolean] [default: false]
  --https-port                       Port of HTTPS server               [number]
  --key-path                         Private key path                   [string]
  --crt-path                         Certification path                 [string]
  --env-path                         .env file path                     [string]
  --config-path, --config-yaml-path  Config YAML path        [string] [required]
  --debug-config                     Print normalized config as JSON (all env!
                                     and other tangs are evaluated)
                                                      [boolean] [default: false]

Example configs are found in
https://github.com/nwtgck/rich-piping-server#readme

Relation to Piping Server

Rich Piping Server uses internally Piping Server as a library:

rich-piping-server/src/rich-piping-server.ts

Line 3 in 7e687bf

import {Server as PipingServer} from "piping-server";

Transfer logic is completely the same as the original Piping Server.

About

Rich Piping Server

basic-authenticationopenid-connectpiping-server

License:MIT License

Languages

Language:TypeScript 99.4%Language:Dockerfile 0.6%