SIP003 - A simplified plugin design for shadowsocks
madeye opened this issue · comments
As discussed in #26, it's dirty to hack original shadowsocks protocol for additional transport features.
A proposal from @falseen , @Artoria2e5, and @anonymous-contributor is that we may support Pluggable Transport from Tor. However, I found PT seems too heavy for shadowsocks. As we never try to or plan to support distributed architecture, I propose a simplified design instead.
SIP003: A simplified plugin design for shadowsocks
- Architecture Overview
Dislike the socks5 proxy design in PT, every SIP003 plugin works like a tunnel (or called local port forwarding). This design aims to avoid per-connection arguments in PT, leading to much easier implementation.
+------------+ +---------------------------+
| SS Client +-- Local Loopback --+ Plugin Client (Tunnel) +--+
+------------+ +---------------------------+ |
|
Public Internet (Obfuscated/Transformed traffic) ==> |
|
+------------+ +---------------------------+ |
| SS Server +-- Local Loopback --+ Plugin Server (Tunnel) +--+
+------------+ +---------------------------+
- Life cycle of a plugin
Very similar to PT, the plugin client/server is started as child process of shadowsocks client/server.
If any error happens, the child process of plugin should exit with a error code. Then, the parent process of shadowsocks stops as well (SIGCHLD).
When a shadowsocks client/server is stopped by user, the child process of plugin will also be terminated.
- Passing arguments to a plugin
A plugin accepts arguments through environment variables.
a. Four MUST-HAVE environment variables are SS_REMOTE_HOST, SS_REMOTE_PORT, SS_LOCAL_HOST and SS_LOCAL_PORT. SS_REMOTE_HOST and SS_REMOTE_PORT are the hostname and port of the remote plugin service. SS_LOCAL_HOST and SS_LOCAL_PORT are the hostname and port of the local shadowsocks or plugin service.
b. One OPTIONAL environment variable is 'SS_PLUGIN_OPTIONS'. If a plugin requires additional arguments, like path to a config file, these arguments can be passed as extra options in a formatted string. An example is 'obfs=http;obfs-host=www.baidu.com', where semicolons, equal signs and backslashes MUST be escaped with a backslash.
- Compatibility with PT
For all the plugins from Tor projects, there are two possible ways to support them. 1) We can fork these plugins and modify them to support SIP003, e.g. obfs4-tunnel. 2) Implement a adapter of PT as SIP003 plugin.
- Licenses of plugins
As all plugin services should run in a separate process, they can pick any license they like. There is no GPL restrictions for any plugin providers.
- Restrictions
a. Plugin over plugin is NOT supported. Only one plugin can be enabled when a shadowsocks service is started. If you really need this feature, implement a plugin-over-plugin transport as a SIP003 plugin.
b. Only TCP traffic is forwarded. For now, there is no plan to support UDP traffic forwarding.
- Example projects
- A SIP003 plugin for traffic obfuscating: simple-obfs.
- A shadowsocks implementation based on SIP003: madeye/shadowsocks-libev.
- Command line example
On the server:
ss-server --plugin obfs-server --plugin-opts "obfs=http"On the client:
ss-local -c config.json --plugin obfs-local --plugin-opts "obfs=http;obfs-host=www.baidu.com"
What about --plugin 'obfs-server --obfs http', --plugin 'obfs-local --obfs http --obfs-host www.baidu.com'?
The plugin here is the name of the executable. Maybe rename obfs-local to obfs-http-local and handle the mode in the plugin would be clearer.
Since all plugins are processes already, I wonder if just letting systemd do the job is more KISS?
@nfjinjing Here we need a cross platform solution. Systemd looks not a good idea.
I mean --plugin command_string. So that we don't need --plugin-args. Since the executable is just $0.
@Mygod It looks doable. Actually no spec for the command line, let me refine it later.
I still prefer the PT one just because we can use it right now, and the saved overhead can be easily eaten by obfuscation itself.
The new draft and PT just differs in the level of integration.
And personally speaking, if user want to use obfuscation, then they are already trading bandwidth and latency for extra security.(Not data security, but channel security)
A really good obfuscation will (and should) add extra bandwidth usage (to obfuscating packet size) and extra latency (to obfuscating packet sequence).
in that case, no matter how hard we try our best to make the integration lightweight, the overhead will increase hugely.
The draft maybe good for certain case, just like the http hack, as a performance hack, but I just
want to clarify that, obfuscation (at least, good one) will introduce obvious overhead to make our effort just meaningless.
So for user who want maxium bandwidth, go plain ss and spend more time finetuning their server.(Encryption of ss won't really be a problem, for the most common use-case behind GFW)
If there is some really strong deep packet filter, such obfuscation may bring some usability, but never expect high performance.
(And in that case, more obvious fix would be load balancing to multiple ss server)
@anonymous-contributor I agree with you about the "personally speaking" part.
I think you are aiming for two different things here.
I believe @madeye introduces obfuscation to boost performance. #26 was meant to boost performance (i.e. bandwidth) (see #26 (comment)). #26 is about HTTP obfuscation because HTTP/HTTPS traffic is the most common (i.e. least suspicious) traffic.
Other people including @anonymous-contributor is wishing to use obfuscation to hide from your ISP that you use Shadowsocks or Tor, i.e. making detection computationally expensive (the so-called channel security).
And currently SIP003 is a balance of both. It wishes to minimize performance overhead while making the platform easily extensible and compatible with Tor PT so that we will have something to move onto if the cheaper approach is no longer beneficial (which is why we call #26 a dirty hack).
There's an obvious use case for chaining of plugins: ss -> obfs4 -> simple-obfs / fteproxy. This will increase the priority of QoS in some ISPs, confuse the hell out of gfw, and still be modular enough to deploy. This only works on the condition that gfw will not interfere with fake http requests. I know, strange. So we might keep the option of enabling multiple plugins all at the parent process open?
And additionally, we are currently adding obfuscation mainly to gain additional performance since Shadowsocks is designed to be indistinguishable with random traffic. We wish to get additional bandwidth and get prioritized by pretending it's something else (like a normal HTTP/TLS connection).
And mentioning lightweight, it's better to have a benchmark to determine if it's worthy.
I'd like to compare the same obfuscation method using standalone mode vs the local interface mode.
Same server, same obfuscation, only different plugin implementation.
If the difference is less than, for example 10%, for average speed, using PT seems more reasonable.
I'm quite in favour of this draft mainly because it doesn't touch the core protocol of shadowsocks!
With this plugin approach, we should expect more contributions to the shadowsocks project since it means everyone can write their green-field plugins (no need to touch existing code bases)
The communication of shadowsocks service and plugin service is not very clear to me. Does it mean the local shadowsocks client process would pass the datagram to plugin and retrieve them back, then finally sends it to the remote shadowsocks service?
@librehat This proposal traces its roots to my mention of Tor's Pluggable Transport protocol in #26. You are welcome to read Tor's PT spec for more idea on this.
In short, both the server and the client speaks and hears through an instance of the transport program. The transport is responsible for communication. Think KCPTUN.
Thanks for all the feedbacks!
@anonymous-contributor Yes, I think reusing PT is also important to us. However, it's quite a lot effort to implement it for all our implementations. My suggestion is to build a adapter to PT based on SIP003. This adapter would be very easy to write as you can reuse all the golang source code from Tor project. After that, all the SIP003 based shadowsocks implementations would also benefit from PT.
@nfjinjing There may be real cases for plugin chains, although ss-obfs4-simpleobfs is not necessary in my opinion. An easy way to do this is also implement SIP003 in your plugin. It means a SIP003 plugin could fork other plugins as a chain. However, shadowsocks itself should only start, maintain and talk to one plugin.
@librehat Every SIP003 plugin works like a tunnel, like KCPTUN, stunnel and ssh -L. It also helps to simplify the development of a plugin that a programmer only needs to focus on his own transport protocol with his familiar language and techniques.
This proposal aims to provide a easy way to integrate third-party transport protocols with shadowsocks infrastructure. With this, I hope we can end all the debates about forking and license issues in the future.
I suggest all the contributors try to add SIP003 support following the example. I believe you will find it's really easy to implement (It takes me four hours to write and debug the full example).
Looks good. It seems without a tunnel software doing anything, a simple wrapper script that passes environment variables to command line arguments can make it complaint with this spec.
-
Can you clarify the meaning of four environment variables and how the plugin is supposed to use them? IIUC the plugin should always listen on
SS_LOCAL_HOST:SS_LOCAL_PORTand tunnel whatever data it receives toSS_REMOTE_HOST:SS_REMOTE_PORT, no matter whether the plugin runs on client or server. -
Is there a side channel for the plugin to talk to ss process, other than exit-code? How is the plugin supposed the handle errors like "port already bind"?
-
I don't think the handshake procedure in socks protocol (as used in PT) would cause any performance penalty on the whole proxying workflow. The RTT is single-digit millisecond on local loopback, while over 100 ms on the internet. It is true that the flexibility in socks handshake is not necessary in this case, but on the other hand it looks not wise to 'reinvent a wheel`.
-
On the server,
SS_LOCAL_HOST:SS_LOCAL_PORTis the address of thess-serverlistening on the loopback, andSS_REMOTE_HOST:SS_REMOTE_PORTis the address of theplugin-serverlistening on the outbound interface. On the client,SS_LOCAL_HOST:SS_LOCAL_PORTis the address of theplugin-locallistening on the loopback, andSS_REMOTE_HOST:SS_REMOTE_PORTis the address thatplugin-localwill connect to. -
No side channel here. The error log of the plugin will be output to the parent process' stdout/stderr or a log file of the plugin. Actually here I assume shadowsocks isn't aware of any error code of the plugins.
-
Yes, I agree that there is no performance issue for per-connection arguments. That's why I encourage to implement an adapter for PT instead of directly adding it to shadowsocks: the adapter wouldn't hurt the performance at all. At the same time, this approach would simplify the development of both shadowsocks and plugins.
No side channel here. The error log of the plugin will be output to the parent process' stdout/stderr or a log file of the plugin. Actually here I assume shadowsocks isn't aware of any error code of the plugins.
Then what is the benefit from launching the plugin process by shadowsocks? How about just 1) bringing up the plugin in advance, and 2) asking shadowsocks to talk to SS_LOCAL_HOST:SS_LOCAL_PORT? It could be another program for doing these 2 steps automatically and smoothly, and handling errors from both parties.
@v2ray I think it's possible. Actually, in that case, shadowsocks itself works like a plugin. It'd be a better way to integrate all protocols in one framework (maybe it's just what v2ray is doing? 😄 )
However, as a SIP, I still prefer the design proposed here.
Yes. That is exactly what V2Ray is trying to achieve.
@madeye The extra work is almost to nothing if using standalone mode Tor PT.
Managed mode needs extra work, but still less than the draft, which is just a SS variant of Tor PT.
(Which we need PT style communication infrastructure for both ss and plugins)
No idea why it's needed to re-invent the wheel, just for so-called lightweight communication?
(And I already expressed the doubt if it's really the bottleneck)
Using standalone mode parent process just exec the obfsproxy, redirecting its output as log, passing random port as SS<->obfsproxy communication, passing real destination port for obfsproxy <-> server.
In that case, the only difference between your draft is, we are using the random local port to run ss.
Instead of lo interface.
From this respect of view, it's even easier to implement than the draft.
No wrapper, just extra configuration parameters.
Even no extra environment variants.
In this case, I can even submit a pull-request in several days for implement cmdline parameters.
(Json parameter will only be added if we settled how the plugin system works.)
-
I think if standalone mode is available (seems not in obfs4), we should take advantage of it to implement our plugin. That's why I forked obfs4 and made it work in standalone mode: https://github.com/madeye/obfs4-tunnel
-
Any pull request is welcome. If we can integrate obfproxy very easily with a simple wrapper, that should be great!
Hey, have we figured out how to properly pass server-returned args to the client yet? There has to be some stdout capturing and passing.
@Artoria2e5 Answer 2 in #28 (comment)
If this is going stable (for real), I think we should remove the "[Draft]" in the title.
How to use it on the mobile device?
@madeye Are you currently working on porting shadowsocks/kcptun to SIP003?
Hmm. Now that this plugin system isn't optional, how should we implement/change #27?
@Mygod BTW, I don't think shadowsocks-android and shadowsocks-windows should directly follow the implementation of shadowsocks-libev for plugins, as they are already implemented as a proxy chain. It means we can continue to use the current approach to start/stop kcptun in shadowsocks-android. The only thing we need to change is the URL schema for plugins.
What do you mean then?
I mean they can control plugins in their own way. For example, all the plugins in shadowsocks-android need to copy the binary from other package to solve permission issue of passing file descriptors between processes.
This kind of behavior is actually not defined in SIP003.
I suggest all the contributors try to add SIP003 support following the example. I believe you will find it's really easy to implement (It takes me four hours to write and debug the full example).
I just spent 3 hours on Android client and I haven't even started to build the platform. 💢
UPDATE: 3.5 hours later I can finally start to build the platform.
Maybe we should remove obfs-local from PLUGIN_OPTIONS?
SS_PLUGIN_OPTIONS format: A ;-separated key-value list, the first item can be plugin ID. It can be ignored when passing to plugin and should never be ignored when used in SS URIs.
When to use options and when to create new plugins: If most options can remain the same configuration after changing this option, you should use options, otherwise create a new plugin.
I use ss-libev 1.6.2 on openwrt, and only start ss-redir, If not use obfs, it will work, but add the 2 lines about plugin, it doesn't work
我是在openwert上使用的ss-libev 1.6.2，并且/etc/init.d/shadowsocks里只启动ss-redir。如果没有添加plugin那两行，就可以正常运行，但是添加之后就不能运行了。麻烦帮我看一下
server:
ss-server -s 0.0.0.0 -p 233 -k ******* -m chacha20 -t 20 --fast-open --plugin obfs-server --plugin-opts "obfs=http"
client:
{
"server": "...",
"server_port": 1080,
"password": "password",
"method": "chacha20",
"timeout": 20,
"plugin": "obfs-local",
"plugin-opts": "obfs=http;obfs-host=www.baidu.com",
"fast-open": true
}
plugin-opts ---> plugin_opts
still not working
{
"server": "153.125.234.177",
"server_port": 31671,
"password": "YM3g7jSXnb5w9NTv",
"method": "chacha20",
"plugin": "obfs-local",
"plugin_opts": "obfs=http;obfs-host=www.baidu.com",
"timeout": 20,
"fast-open": true
}
@rampageX
Actually now I'm thinking that maybe we should have taken @anonymous-contributor's approach and use PT directly. Advantages:
- No wrappers needed;
- A new plugin for shadowsocks can also be contributed to Tor community which have more intelligent people assessing new plugins;
- Therefore Tor community can be attracted and hopefully our shadowsocks project can be assessed by more people.
To be honest, I don't think our current shadowsocks protocol is by any means secure or GFW-proof. It works only because GFW really kinda sucks at censoring for now (and we suck less). We need to see for the future.
The aim of this proposal is to simplify the development of a plugin for shadowsocks, and also reduce the effort to add plugin support into our current implementations. IMO, it's almost impossible to support PT in shadowsocks due to our limited resource.
Actually, if there is any volunteer to implement PT for us. I'm glad to merge every PR from him... 😃
shadowsocks/shadowsocks-libev#1156
Finally I got my spare time, after KSP and several PS4 games
Maybe 13 days is somewhat long term :)
At least, we can use Tor PT standalone mode now.
And it's quite easy to implement in fact.
While it's still ugly since we need to co-exist with SS_* environment one, I hope after I implement SOCKS5 proxy sopport, we can move to full managed mode.
Then we should be able to unify all plugins to full Tor PT spec.
Thanks
Time to plan for UDP forwarding? Perhaps using 0x20 as a flag for UDP in ATYP i.e. using UDP over TCP?
@Mygod What about let UDP bypass for now? UDP over TCP expects very poor performance.
Implemented and published via https://github.com/shadowsocks/shadowsocks-libev/releases/tag/v3.0.3
I am not sure if I (already) understand this all, but I am wondering if this architecture can be used to let Shadowsocks and NetGuard (I am the author) operate together? If this is the wrong place to discuss this and there is interest in this, I am happy to create a new issue to discuss about this.
@M66B It looks NetGuard works as a firewall? If so, it'd be not related. This issue is a proposal for plugin architecture of shadowsocks protocol.
@madeye yes, NetGuard is a VPN based firewall.
I was thinking like this, but maybe I am thinking wrong and/or there is a better way:
+------------+ +---------------------------+
| SS Client +-- Local Loopback --+ NetGuard firewall +--+
+------------+ +---------------------------+ |
|
Public Internet ==> |
|
+------------+ +---------------------------+ |
| SS Server +-- Local Loopback --+ <empty> +--+
+------------+ +---------------------------+
@M66B If it works as a TCP firewall, I think it'd be doable.
What you need is only to implement a simple TCP tunnel feature for your app and follow the instruction here: https://github.com/shadowsocks/shadowsocks-android/blob/master/plugin/README.md