Init script issue on CM-12
cjeanneret opened this issue · comments
Seems CM 12 (lollipop) has some issue with orWall. At least, the init-script doesn't seem to work as expected:
- iptables doesn't seem to be initialized properly
- orbot apparently is locked out unless we deactivate orwall, and re-activate it
Obviously, this is bad.
i'ts about busybox
when not in /system/bin/ but in system/xbin/ orwall not working
ps. or generally with links to busybox dont know which one use orwall
try remap patch to those links
pleas fix it ethack
Heya!
I'll have a look at busybox level, though I'm not sure it's used in orwall (maybe some path in the script shebang, but I don't think it's used anywhere else).
Should be sorted out by the end of the month, I'm quiet busy those days…
its not about busybox binary execution by orwall but symlinks (those for netd,route,iptables etc. ) linked to busybox (not in system /bin but /xbix) and result of some differences in syntax used on toolbox vs. busybox
having this problem too. installing busybox to /system/bin didnt help. disabling and then re-enabling orwall on boot seems to make it work tho. the notice about iptables still stays tho
how did you install busybox to /system/bin?
via terminal emulator "busybox --install -s /system/bin"
did you remove busybox executables and old symlinks from /system/xbin/ ?
i use temasek mod of cm12 its very tight bound with bb changing bb place
will result in system crash
i got also symlinks to toolbox in /system/bin (so there are two sets of
symlinks for example when u execute
/system/xbin/chmod is resulting in /system/xbin/busybox chmod" and second
"/system/bin/chmod resulting in /system/bin/toolbox chmod"
busybox swiss army horror :)
i just used the busybox app which lets u choose the directory to install to. not sure if it works though.
also when i was updating cyanogenmod to 12, i remember seeing a notice about how my system may have lost root. i think the notice was there while i was in recovery? its when i was updating cm. and every time i told it to try and retrieve root again, the phone would shut off. i just assumed it worked. im pretty sure my phone is still rooted, as many root apps are still working. its just busybox not working for some reason? another app i use, 'permissions pro' isnt able to open now. it says 'busybox not found on your system'. so im guessing theres just a problem with busybox and cm12.
ive installed the latest cm12 nightly, and the latest busybox. all other apps which require busybox seem to be working correctly now. orwall however still doesnt run the initscript though..
I didn't have the time required in order to debug this part.
I'm not that sure busybox has anything to do with the init-script though… I need some hours in order to check how the init process is working on lollipop (CM12), maybe there are some unmet requirements…
Thank you for your patience.
NO its not about CM12
the BUG is related with those somehow
- toolbox
- busybox
- dir /system/bin
- dir /system/xbin
- inid.d support
- enabling init.d
why? because i got the same "ip ... initialized" error while:
- no busybox & init.d enabled
- installing busybox in /system/xbin & init.d enabled
regardless - CM11 / CM12 or STOCK
with 2 different mobiles : SM-N9005 and C6903 (Sony Xperia Z1)
i looked at orwall code and didnt find "any why?"
first thought it was about SU acces granting durning boot / on qualcomm devices to initialize script
im familiary with coding but logging is not my area on linux and got no time to lern now so tell me in simple way how to read log from orwall / logcat dmsg ? to see whats wrong
if someone does not understand what I wrote:
orwall is not working when u enable init.d support
ok now funny part :)
i got c6903 (z1) mobile on pure STOCK (14.4.A.0.157)
- rooted
- installed supersu 2.46 (latest stable)
- granted su to orwall
- try to set in orwall fdroid acces
that i got from su logs :
now as an end user i need to learn iptables chains etc ?
it would be for me faster to lern that and build own rules rather use orwall
4 first !!!
ATTACH TO APPLICATION an appropriate IPTABLES BINARY or DO CHECK VERSION and use TODO list when u coding to to prevent this type of errors !!!
beside that i think it could be one more thing:
PROBLEM LYING IN KERNEL :) depending on module compilation ?
maybe this will clear little about dependiences of iptables vs. busybox symlinks :)
Ok i manually added chain accounting_OUT/IN
Now Orwall rerouting traffic to orbot but other apk are not blocked
Where initialization(add of those chains) and rules to block all traffic are taking place in code? are they excluded after Whitnes chain is added?
If I understand clearly No initi.d support then rules to block all traffic are applied after every reboot? Or installing and first run of orwall ?
After manual execution
sh /system/etc/init.d/91firewall
iptables: Chain already exists.
ip6tables v1.4.20: can't initialize ip6tables table `nat': Table does not exist (do you need to insmod?)
Perhaps ip6tables or your kernel needs to be upgraded.
All traffic blocks even rule in chain exist to rerouting to orbot
Message in orwall 'iptables not initialized' vanished
I'll send iptables rules little later as I take some sleep ! :)
Nothing to do with some missing iptables binary, nor TODO in app code, nor iptables version.
It seems your device doesn't handle "accounting" chain, i.e. your system doesn't create it at boot time.
This accouting_OUT chain was created by the system on all my tested devices (nexus4 with different CM, nexus7 with SlimKat, and stock Slate 7). This chain is used in order to provide network stats…
So, a "todo", as you seem to want one, would be: detect the presence of accounting_OUT and accouting_IN, and add specific accounting rules iff there are such chains.
Regarding ip6tables, there's maybe an issue there, but this shouldn't prevent the init script to fail — proof: the warning message is orWall vanished.
So, there are two problems here:
1- lack of some system chains, preventing some rules application
2- initd support showing some differences on CM12 (CM11 was working fine on my side — still works with SlimKat).
First one might be corrected with a proper chain detection — not easy, but neither hard. Just annoying.
Second one will prove to be a lot more annoying, especially for the "why doesn't it kick of, motherf**ker?!" part.
No initd support means orwall locks rules when it starts, meaning there's a time, right after device startup, where no iptables stuff is set, and thus leaks might occur.
yes I came to this conclusion last night
for
- c6903 (z1) on kitikat (4.4.4) enought is to create chains accounting_OUT/IN
& enable init.d support eg. by script or mannualy
init.d is executed in sysinit by run-parts /system/etc/init.d
SO BUSYBOX (symlink to run-parts is in /system/bin) -> access to all users ! - but for CM12 its different run-parts ( busybox symlink ) is in /system/xbin -> access to current user
so still dont know if its about su permission for all users (execution of run-parts / bussybox symlink) or su access granted durning boot. ill check that and I will tell you later
btw. its good idea to add to apk binaries (iptables) for eg. execution syntax differences like orbot got its own iptables (u can choose one in case something fails)
does orwall add whitness chain only durning post by script or also by executing apk ?
its the same sytuation described by other issue that i need enable script and reboot twice
- install orwall
- run orwall apk
- enable script (cp to init.d )
- reboot
- run orwall apk
(orwall after second run displays message "ip tables ...not initialized" - reboot
- orwall is OK
like script itselfs is not adding whitness chain but apk is doing that so it needs to be run twice
// everything is about adding chains to iptables because rules are adding (by script?)
OK, so I just dug a bit in my code (didn't open it for months, literally), and here's some more information:
- accounting_OUT is added back when orWall initializes iptables OUTPUT chain
- this chain is removed when we deactivate orWall
- accounting_OUT is in fact a custom chain, used as "buffer" before sending packets to bw_OUTPUT, which is in turn used by the system in order to get network usage.
Yeah, a bit tricky. I forget I did that. Anyway, the accounting_OUT should be present, as it's created by orWall when it get initialized…
What might happen: commands are pushed to some queue, maybe there are some issues at this level, some commands being executed before a "mandatory" one… But that's dubious.
Regarding busybox on CM12: just a small remark: at least on my devices, the CM images are "nightly" builds, meaning they aren't even RC, thus the busybox stuff might as well be changed/corrected later by CM team. I'll try to contact them in order to see what might be done on their side.
Good news: seems to be working with the latest CM12 nightly: https://gist.github.com/ahf/ec18b83c3a5c40c30ba3
Will give it a try.
Still not working :(
Boot sequence as recorded:
W/iptables( 218): type=1400 audit(0.0:6): avc: denied { create } for scontext=u:r:sysinit:s0 tcontext=u:r:sysinit:s0 tclass=rawip_socket permissive=0
W/iptables( 218): type=1400 audit(0.0:7): avc: denied { create } for scontext=u:r:sysinit:s0 tcontext=u:r:sysinit:s0 tclass=rawip_socket permissive=0
W/iptables( 220): type=1400 audit(0.0:8): avc: denied { create } for scontext=u:r:sysinit:s0 tcontext=u:r:sysinit:s0 tclass=rawip_socket permissive=0
W/iptables( 220): type=1400 audit(0.0:9): avc: denied { create } for scontext=u:r:sysinit:s0 tcontext=u:r:sysinit:s0 tclass=rawip_socket permissive=0
W/iptables( 221): type=1400 audit(0.0:10): avc: denied { create } for scontext=u:r:sysinit:s0 tcontext=u:r:sysinit:s0 tclass=rawip_socket permissive=0
W/iptables( 222): type=1400 audit(0.0:11): avc: denied { create } for scontext=u:r:sysinit:s0 tcontext=u:r:sysinit:s0 tclass=rawip_socket permissive=0
W/iptables( 222): type=1400 audit(0.0:12): avc: denied { create } for scontext=u:r:sysinit:s0 tcontext=u:r:sysinit:s0 tclass=rawip_socket permissive=0
W/iptables( 223): type=1400 audit(0.0:13): avc: denied { create } for scontext=u:r:sysinit:s0 tcontext=u:r:sysinit:s0 tclass=rawip_socket permissive=0
W/iptables( 223): type=1400 audit(0.0:14): avc: denied { create } for scontext=u:r:sysinit:s0 tcontext=u:r:sysinit:s0 tclass=rawip_socket permissive=0
W/iptables( 224): type=1400 audit(0.0:15): avc: denied { create } for scontext=u:r:sysinit:s0 tcontext=u:r:sysinit:s0 tclass=rawip_socket permissive=0
W/iptables( 225): type=1400 audit(0.0:16): avc: denied { create } for scontext=u:r:sysinit:s0 tcontext=u:r:sysinit:s0 tclass=rawip_socket permissive=0
W/iptables( 225): type=1400 audit(0.0:17): avc: denied { create } for scontext=u:r:sysinit:s0 tcontext=u:r:sysinit:s0 tclass=rawip_socket permissive=0
W/iptables( 226): type=1400 audit(0.0:18): avc: denied { create } for scontext=u:r:sysinit:s0 tcontext=u:r:sysinit:s0 tclass=rawip_socket permissive=0
W/iptables( 226): type=1400 audit(0.0:19): avc: denied { create } for scontext=u:r:sysinit:s0 tcontext=u:r:sysinit:s0 tclass=rawip_socket permissive=0
W/iptables( 227): type=1400 audit(0.0:20): avc: denied { create } for scontext=u:r:sysinit:s0 tcontext=u:r:sysinit:s0 tclass=rawip_socket permissive=0
W/iptables( 227): type=1400 audit(0.0:21): avc: denied { create } for scontext=u:r:sysinit:s0 tcontext=u:r:sysinit:s0 tclass=rawip_socket permissive=0
V/NatController( 250): runCmd(/system/bin/iptables -F natctrl_FORWARD) res=0
V/NatController( 250): runCmd(/system/bin/iptables -A natctrl_FORWARD -j DROP) res=0
V/NatController( 250): runCmd(/system/bin/iptables -t nat -F natctrl_nat_POSTROUTING) res=0
V/NatController( 250): runCmd(/system/bin/iptables -F natctrl_tether_counters) res=1
V/NatController( 250): runCmd(/system/bin/iptables -X natctrl_tether_counters) res=1
V/NatController( 250): runCmd(/system/bin/iptables -N natctrl_tether_counters) res=0
V/NatController( 250): runCmd(/system/bin/iptables -t mangle -A natctrl_mangle_FORWARD -p tcp --tcp-flags SYN SYN -j TCPMSS --clamp-mss-to-pmtu) res=0
V/NatController( 1729): runCmd(/system/bin/iptables -F natctrl_FORWARD) res=0
V/NatController( 1729): runCmd(/system/bin/iptables -A natctrl_FORWARD -j DROP) res=0
V/NatController( 1729): runCmd(/system/bin/iptables -t nat -F natctrl_nat_POSTROUTING) res=0
V/NatController( 1729): runCmd(/system/bin/iptables -F natctrl_tether_counters) res=0
V/NatController( 1729): runCmd(/system/bin/iptables -X natctrl_tether_counters) res=0
V/NatController( 1729): runCmd(/system/bin/iptables -N natctrl_tether_counters) res=0
V/NatController( 1729): runCmd(/system/bin/iptables -t mangle -A natctrl_mangle_FORWARD -p tcp --tcp-flags SYN SYN -j TCPMSS --clamp-mss-to-pmtu) res=0
D/IPTables: ( 3583): Comments are supported
The SELinux part is a bit weird, I first thought it was from my script, but it seems it's something else…
Also, I'm wondering if the "dual-init-script" part isn't messing around: now, the init-script is pushed in /etc/init.d/91firewall and /data/local/userinit.d/91firewall (if the directory exists).
I'll try to:
- add some "log" calls in the init-script in order to get some outputs (via adb logcat)
- add a test in the init-script so that it won't start twice (prevents useless errors)
- maybe sleep one second or two before applying the iptables rules in order to see what happens
It's a really, really annoying issue, as we don't have easy way to understand and see what's happening. I suspect "something" in the system does some iptables stuff as well, after orwall init-script.
I think I managed to fix it by using SuperSU to change the SELinux context during the boot-up. I made a patch here:
https://github.com/ahf/orWall/commit/0823e12f0afbc7ff2de039c53ec007d4e04f0f7f
Would be nice if someone would be up for testing it. I tested it on my Moto-E running CM-12, but I haven't had a chance to test on Android 4.x. It requires an SELinux aware version of SuperSU, but this version of the init script is MUCH easier to debug this than the old init script.
Let me know if there's anything that needs changing before I'll make a PR :-)
Forgot to add in the commit message that I'm also adding a the --wait as (-w) argument to all iptables call. This will make iptables wait for the xtables lock to be released instead of just failing the command with an error exit code.
Here's some info from some earlier debugging where I enabled -w to the iptables commands:
https://gist.github.com/ahf/f9fc93641636fe22f188
And this is how it looks after enabling the SELinux context switch:
wow. That's what we can call "debugging" :).
I'll see if there's a way to do it as follow:
- set SELinux context at script level (chcon [script])
- check if this still works on android 4.x
Setting script context will allow orwall to check if SELinux is enabled, and run the appropriate chcon command — else, we might need to create two init-script, one with SELinux context (like the one you proposed), and the second, old way. Relaying on a third party binary isn't really good :(.
Anyway, thanks a lot for your time and details, that will be very helpful!
Cheers,
C.
Closing it as it seems to be related to SELinux, not busybox path or anything else. You might want to follow #79 instead.