appropriate use-case

Question

appropriate use-case

travisghansen opened this issue 2 years ago · comments

Travis Glenn Hansen commented 2 years ago

Would this library be appropriate for the following operations:

upsert a routing table
replace a rule in the routing table created in step 1
add (preferably in an idempotent fashion, no duplicates) ip rules to force specific traffic to use routing table in step 1

# create table
# test table exists
# exit 0   = exists
# exit 255 = not exists
ip route show table cilium
echo 20 cilium >> /etc/iproute2/rt_tables

# upsert routes to the table
# note that linux does a hash-based 3/5 tuple algorithm
# https://serverfault.com/questions/696675/multipath-routing-in-post-3-6-kernels
ip route replace default table cilium \
    nexthop via 172.28.4.130 weight 1 \
    nexthop via 172.28.4.131 weight 1

# add rule(s)
# test if the rule is already present? do not create duplicates
ip rule add from 172.28.42.0/24 lookup cilium
...

Thanks!

Alba Mendez · Answer 1 · Sat Mar 26 2022 02:37:58 GMT+0800 (China Standard Time)

yes, these actions (except creating the routing table) use rtnetlink, and rtnetlink is carred by netlink.
so you can use this library to access the API.
.
what's more, the rtnetlink API in particular is specifically exposed in a high level way, complete with typescript bindings.
check out RtNetlinkSocket.
however, I see that the NEWRULE, GETRULE, etc. operations are not implemented.

Alba Mendez · Answer 2 · Sat Mar 26 2022 06:58:39 GMT+0800 (China Standard Time)

NEWRULE / GETRULE / DELRULE operations implemented in e978e06, I'll release when I find time

Travis Glenn Hansen · Answer 3 · Sat Mar 26 2022 11:40:29 GMT+0800 (China Standard Time)

Awesome! I’ll take a peek.

Alba Mendez · Answer 4 · Sat Mar 26 2022 17:36:23 GMT+0800 (China Standard Time)

in general you should be able to do anything ip can do, except maybe netns. the problem is that documentation on rtnetlink is super scarce. strace, kernel source code and iproute2 source code are your friends.

Travis Glenn Hansen · Answer 5 · Sat Mar 26 2022 23:51:06 GMT+0800 (China Standard Time)

Attempting to use getRules results in this in my environment:

Error: Unexpected length (got 1, expected 2)
    at checkLength (/home/thansen/Projects/github/travisghansen/metallb-node-route-agent/node_modules/netlink/dist/structs.js:241:11)
    at Object.getU16 (/home/thansen/Projects/github/travisghansen/metallb-node-route-agent/node_modules/netlink/dist/structs.js:251:44)
    at Object.21 (/home/thansen/Projects/github/travisghansen/metallb-node-route-agent/node_modules/netlink/dist/rt/gen_structs.js:341:52)
    at /home/thansen/Projects/github/travisghansen/metallb-node-route-agent/node_modules/netlink/dist/structs.js:362:27
    at parseAttributes (/home/thansen/Projects/github/travisghansen/metallb-node-route-agent/node_modules/netlink/dist/structs.js:232:9)
    at Object.getObject (/home/thansen/Projects/github/travisghansen/metallb-node-route-agent/node_modules/netlink/dist/structs.js:359:5)
    at Object.parseRouteAttrs (/home/thansen/Projects/github/travisghansen/metallb-node-route-agent/node_modules/netlink/dist/rt/gen_structs.js:320:20)
    at Object.parseRouteMessage (/home/thansen/Projects/github/travisghansen/metallb-node-route-agent/node_modules/netlink/dist/rt/structs.js:115:22)
    at parseMessage (/home/thansen/Projects/github/travisghansen/metallb-node-route-agent/node_modules/netlink/dist/rt/structs.js:186:23)
    at /home/thansen/Projects/github/travisghansen/metallb-node-route-agent/node_modules/netlink/dist/rt/rt.js:75:56

Alba Mendez · Answer 6 · Mon Mar 28 2022 04:11:56 GMT+0800 (China Standard Time)

sorry, it seems the manpage had incorrect information about NEWRULE, GETRULE, DELRULE:

Add, delete, or retrieve a routing rule. Carries a struct rtmsg.

Which is either incorrect or outdated. According to kernel source they carry a fib_rule_hdr + attrs, defined here, not an rtmsg.

Therefore adding support for these operations requires more work, particularly transcribing the types in that header into types/rt.ts. I'll revert the commit. PRs are welcome, otherwise maybe I'll do it myself if I find some time.

Alba Mendez · Answer 7 · Mon Mar 28 2022 07:20:52 GMT+0800 (China Standard Time)

in the end I did it myself, since it was a short header. I've also released v0.2.1

Travis Glenn Hansen · Answer 8 · Mon Mar 28 2022 12:07:18 GMT+0800 (China Standard Time)

Awesome! Both issues have cleared up for me now. This is all a bit lower-level than I tend to mess with so thanks for the patience answering some questions :)

How would I go about turning this: https://netlink.alba.sh/docs/interfaces/rt_gen_structs.RouteAttrs.html#multipath into their respective objects?

For example I've got this logged:

{
  kind: 'route',
  data: {
    family: 2,
    dstLen: 0,
    srcLen: 0,
    tos: 0,
    table: 20,
    protocol: 'BOOT',
    scope: 'UNIVERSE',
    type: 'UNICAST',
    flags: {}
  },
  attrs: {
    table: 20,
    multipath: <Buffer 10 00 00 63 02 00 00 00 08 00 05 00 ac 1d 00 01 10 00 00 63 02 00 00 00 08 00 05 00 ac 1d 00 03>
  }
}

which corresponds to this (what I'm currently using to get the feature):

ip -j -d route show table 20  | jq .
[
  {
    "type": "unicast",
    "dst": "default",
    "protocol": "boot",
    "scope": "global",
    "flags": [],
    "nexthops": [
      {
        "gateway": "172.29.0.1",
        "dev": "wlp0s20f3",
        "weight": 100,
        "flags": []
      },
      {
        "gateway": "172.29.0.3",
        "dev": "wlp0s20f3",
        "weight": 100,
        "flags": []
      }
    ]
  }
]

I'm unclear how I turn that multipath buffer into these: https://netlink.alba.sh/docs/modules/rt_structs.html#RouteNextHop

Also, out of curiosity, is it possible to receive a stream of events when new rules/routes are added/deleted/modified? If so is there an example floating around I can look at?

Thanks!

Alba Mendez · Answer 9 · Wed Apr 06 2022 06:14:26 GMT+0800 (China Standard Time)

To answer your first question: if I remember correctly, multipath isn't automatically parsed by the API because it's a "strange" type: a struct with a 'length' field (RouteNextHop), which IIUC is followed by RouteAttrs* if there's excess space. It's unusual to find stuff like that in Netlink, so the parsing system doesn't support it.

It's very poorly documented, but the code to parse it would be something like:

function parseMultipath(input) {
  const result = []
  while (input.length) {
    // Parse struct at the start
    const data = parseRouteNextHop( input.slice(0, __LENGTH_RouteNextHop) )
    // Parse excess data as attrs
    if (input.length < data.len)
      throw Error(`invalid rtnexthop length ${data.len}`)
    const attrs = parseRouteAttrs(input.slice(__LENGTH_RouteNextHop, data.len))
    // Remove consumed data from input (plus alignment)
    input = input.slice(data.len + (-data.len & 3))
    // Add to result
    delete data.len // not useful to caller
    result.push({ data, attrs })
  }
  return result
}

By the way, as you can see, weight is reported as data.hops + 1 according to iproute2's source code.

(*): in fact only gateway is currently allowed there AFAIK.

Travis Glenn Hansen · Answer 10 · Wed Apr 06 2022 08:08:34 GMT+0800 (China Standard Time)

Got it. Thanks for the help!

Alba Mendez · Answer 11 · Wed Apr 06 2022 08:36:58 GMT+0800 (China Standard Time)

To answer the second question:

Also, out of curiosity, is it possible to receive a stream of events when new rules/routes are added/deleted/modified? If so is there an example floating around I can look at?

Yes, this was one of Netlink's objectives. In Netlink, both parties send can messages of a certain type. Sending NEWROUTE indicates a request to create a route, and receiving NEWROUTE from the kernel is a notification that a new route was added.

Messages may be (1) unsolicited, or (2) received in response to a previous request. For the latter, the message is returned from the promise as you know. Unsolicited messages are emitted as message events. But in order to receive unsolicited messages from a Netlink API, you usually have to subscribe to a multicast group. Rtnetlink has many multicast groups.

You can find an example here (beware this is an old version of the README, the latest version is a bit different since I'm retouching the API for the next version). Make sure you're using at least v0.2.2 since I fixed a little bug regarding unsolicited messages.

Travis Glenn Hansen · Answer 12 · Wed Apr 06 2022 08:52:46 GMT+0800 (China Standard Time)

Awesome! I’ll give it a try. Thanks again for the answers!

Travis Glenn Hansen · Answer 13 · Thu Apr 07 2022 10:53:06 GMT+0800 (China Standard Time)

Yup, got the multicast groups going just fine locally with v0.2.2!