Moved ~/.local/share/steam. Ran steam. It deleted everything on system owned by user.
keyvin opened this issue · comments
Edit: Please stop posting stupid image memes or unhelpful messages. This interferes with Valve's ability to sift through the noise and see if anyone can figure out what triggers it.
This may not be a common problem because I change all sorts of configuration about my system. The script in question does something in a really, really stupid way, but it probably doesn't trigger the fail scenario for every system because...
Original Bug:
I am not sure what happened. I moved the folder in the title to a drive mounted under /media/user/BLAH and symlinked /home/user/.local/steam to the new location.
I launched steam. It did not launch, it offered to let me browse, and still could not find it when I pointed to the new location. Steam crashed. I restarted it.
It re-installed itself and everything looked great. Until I looked and saw that steam had apparently deleted everything owned by my user recursively from the root directory. Including my 3tb external drive I back everything up to that was mounted under /media.
Everything important, for the most part, was in the cloud. It is a huge hassle, but it is not a disaster. If there is the chance that moving your steam folder can result in recursively deleting everything in the directory tree you should probably just throw up an error instead of trying to point to other stuff. Or you know, allow the user to pick an install directory initially like on windows.
My system is ubuntu 14.04, and the drive I moved it to was ntfs if its worth anything.
I am impressed how calm you stay about this. This is terrible. I just lost my home directory. All i did was start steam.sh with STEAM_DEBUG=1. I will investigate this and report back.
edit1: I suspect steam.sh got some bugs(does not check own variables) and when it tried to do scary things it crapped himself.
Line 468: rm -rf "$STEAMROOT/"*
edit2: It gets better. Seems on windows Steam is overeager too! https://support.steampowered.com/kb_article.php?ref=9609-OBMP-2526 (The warning part is interesting. Because everybody reads this before uninstalling...)
I agree, that line minimally requires an exists and not null check for $STEAMROOT
#scary!
As an ex programmer, that really makes me chuckle. Can I at least get an apology from whoever committed that comment without adding a fix?
This also happened to me a few weeks ago, my entire home was deleted by the steam.sh script.
introduced here Sep 10, 2013 https://github.com/indrora/steam_latest/commit/21cc14158c171f5912b04b83abf41205eb804b31 line 359
rm -rf "$STEAMROOT/"* could be evaluated as rm -rf "/"* if $STEAMROOT is empty
but what exactly caused this? i've symlinked ~/.local/share/steam to, so i am a bit afraid to start steam :/
pythoneer,
I believe the issue starts on line 19:
# figure out the absolute path to the script being run a bit
# non-obvious, the ${0%/*} pulls the path out of $0, cd's into the
# specified directory, then uses $PWD to figure out where that
# directory lives - and all this in a subshell, so we don't affect
# $PWD
STEAMROOT="$(cd "${0%/*}" && echo $PWD)"
STEAMDATA="$STEAMROOT"
This probably returns as empty which mean: rm -rf "$STEAMROOT/"* is the same ass rm -rf "/"*.
TcM1911,
that's my guess, too.
@keyvin @d00fy :
Line proceeded by # Scary! comment is in reset_steam() function, which is executed if and only if steam.sh is invoked with --reset as first argument.
Did any of you deliberately invoked that script with that option? If yes, why did you do it? What were you trying to achieve?
Removing user data is obviously wrong, no doubt about it. But if this happens only when user requests certain action, scope of that issue is somewhat limited.
Yeah, they kinda need a readlink in there.
STEAMROOT=$(readlink -nf "${0%/*}")
@minio Not "only if". reset_steam is also invoked by removing your .steam directory, since that sets INITIAL_LAUNCH
@minio A script accidentally running rm -rf /* is unacceptable in any scenario.
it is like bumblebee all over again!
MrMEEE/bumblebee-Old-and-abbandoned#123
I encountered Steam behaviour like with «--reset» for several times:
when I added «--reset» AND when I didn't. So — not «only if», can
confirm this, even having not deleted ~/.steam/ dir, too.
wonder what the code path is to hit that rm without --reset
Can confirm; I have Steam bounded in an SELinux context ("Steam") and SELinux spits out:
Context violation: process /home/indrora/.local/share/Steam/ubuntu12_32/steam is only allowed in context steam_context, attempted to remove /boot/efi/grub/efistub
Ooops. I'll write a patch and PR it 🍺
Does anybody have reliable repro steps for this? I can easily add the checks for STEAMROOT being empty, but I would also like to fix the root cause if possible.
It will definitely fail if you run steam.sh as bash steam.sh. I don't know if that's the cause in this case. In terms of root cause, I would say you should use set -e, set -u, and similar options in order to make the script less likely to silently ignore errors.
Using ${STEAMROOT:?} instead of $STEAMROOT would have helped, too.
(For those not familiar, ${FOO:?} is identical to $FOO except that it errors out automatically if $FOO is empty or unset.)
Which is the same thing that would have happened had the unnecessary '/*' not been there anyway, it would have errored out. Its not necessary because the rm was set to recursive already...
if [ ! -z "${STEAMROOT}" ]; then
rm -rf "${STEAMROOT}"
fi
Here is a patch which enables set -e, set -u, and a few others, and then fixes up all the places where undefined variables are expected to be (as far as I can tell): https://gist.github.com/rcxdude/1f6257e0a965147a462c
@rcxdude Come on. We are on github here. Do that in a PR please! Do not post whole patch files into issues... 👎
@mablae There is no code on this repo, there is nothing to send in a PR against. A gist wouldn't have gone astray though 😄
@rcxdude: Please link to a Gist.
@johnv-valve regardless of tracking down the cause of this, this rm line must be protected from future accidental gremlins due to the severity of the fail scenario.
@dannyfallon Oh, sorry then...
@sdt16 I am sorry. Thought the code was on this repo ... Didn't wanted to be a "jerk" :)
Does it happen only if you move ~/.local/share/steam ? #scary! :s
The idiomatic way to do this in Bash is to use default variables as in ,"${var:-/sane/path}" or "${var:?}" as was already mentioned. While using set -u or similar could have prevented this mistake, it's lazy and considered bad style.
If steam really wants to act like a package manager it should only delete files created by itself.
@soren121 +1
Shell scripts can also have tests, see shunit2. I wish I used it instead of making mini/naive one myself.
it's 2015; I think we can do better than complex badly written shell scripts.
rm is dangerous; you should never use rm ${macro}/
@d00fy Steam is essentially a package manager for your games.
@ju2wheels your snippet has a spurious 'D' in it, 'STEAMROOT' versus 'STEAMDROOT' -- so ${STEAMDROOT} will be empty and that code is going to end up starting at the current directory and doing a recursive delete from there.
Could be almost as bad depending what the current directory is. Let's hope nobody copy/pastes that snippet for actual use.
@mpnordland I think that's what he said.
I hope no one is running this as root
@d00fy I'm going to make the argument that while Steam is certainly at fault, you should definitely be using off site backups for this exact reason.
I hope no one is running this as root
On a typical desktop Linux system that would only make it marginally worse - reinstalling the OS is less of a problem than losing all your data from $HOME
@john-valve You are more than welcome to any log files that a data recovery service can get from the drive, as long as you give me a copy of everything. I do not have a large drive to undelete to, so the data that wasn't in the cloud is. Mad props for #Scary though
@nicatronTg Yes, I should need to have a complete back up rotation of dailies, four weeklies, and 12 monthlies so that I can feel safe and secure running steam for linux.
@DanielGibson Sure, as long as you didn't mount other partitions rw somewhere else
@carlosmcevilly even if they did it would be better than the existing code thats there ;-) , it will just fail and not take the world with it. But.... but... i fixed my typo ;-) , thx.
Hmm.. when I was copying .steam from an older installation to a newer installation of Kubuntu, I saw in stdout messages like "running rm -rf " when I was running it (after the rm -rf it installed Steam again with the 200MB download). Though of course I didn't lose any data.
@keyvin This is somewhat OT but to recovery your data, immediately stop using the external drive and run something like PhotoRec or TestDisk or if the drive has important files you can even use a program like ddrescue to copy the drive. If you haven't used the drive since the data loss then most of your data should be recoverable.
Note there is also protection against this within GNU rm itself. I.E. this would have protected the / dir at least:
rm -rf "$STEAMROOT/" && mkdir "$STEAMROOT"
But it's best leave out the / in any case as it's redundant
rm -rf "$STEAMROOT" && mkdir "$STEAMROOT"
Bash-isms are the problem here:
(cd `dirname $0`; STEAMROOT=`pwd`)
If dirname and pwd aren't in your "$PATH" then fuck you.
My condolences, @keyvin. I lost all my data like this in 1998 due to a SuSE Linux installation script having exactly this bug in it as well. Back then I lost my Linux, my Windows, and all my data and I only learned what backups are really for from that incident...
@TcM1911 Determining the path of the script being invoked, in a fully portable (w.r.t. platforms) and completely reliable manner, even in the face of symlinks, is a tricky problem and all too easy to get wrong. The only bullet proof way of doing it, of which I'm aware, is provided below (assumes bash); could be prepended to any script needing this functionality.
#!/bin/bash
script_path () {
local scr_path=""
local dir_path=""
local sym_path=""
# get (in a portable manner) the absolute path of the current script
scr_path=$(cd -P -- "$(dirname -- "$0")" && pwd -P) && scr_path=$scr_path/$(basename -- "$0")
# if the path is a symlink resolve it recursively
while [ -h $scr_path ]; do
# 1) cd to directory of the symlink
# 2) cd to the directory of where the symlink points
# 3) get the pwd
# 4) append the basename
dir_path=$(dirname -- "$scr_path")
sym_path=$(readlink $scr_path)
scr_path=$(cd $dir_path && cd $(dirname -- "$sym_path") && pwd)/$(basename -- "$sym_path")
done
echo $scr_path
}
script_dir=$(dirname -- "$(script_path)")
@sindresorhus I'd disagree with using trash instead of rm because having to rely on node just adds complexity to systems that wouldn't otherwise need node.
It's also worth mentioning that most of the linux distros I've used recently have the --preserve-root flag already available...
I'd disagree with using trash instead of rm because having to rely on node just adds complexity to systems that wouldn't otherwise need node.
I know people disagree with that. That's fine. I was linking to the safeguard rm section.
I've moved my directory and symlinked it too, and I've thankfully not had such issues before. Regardless, as someone mentioned before, I'd strongly suggest adding test cases against such major bugs.
This big companies are so in the need of engineers that would even hire my grandma, and then shit happens.
In this context, how can i remove steam from my Linux without using steam?
As @d00fy said, it's amazing how calm you were about this, @keyvin. A great bug report it was too. Embarrassing as bugs like these are, it's a lot easier to take it on with reports like this one.
As others have pointed out too, this is an excellent bug for advocating using some BASH-strictness, I like to use this:
set -euo pipefail
This bug is also a good case for advocating TDD with shunit2. Make the steam script modular so that parts of it can be (unit) tested, write a test for this bug which fails and then fix it. We all do this in other "real" programming languages and there's no reason why we can't do this in BASH too. Hooking it up with CI servers like Jenkins is no problem either.
Tangentially related:
I assume all the people that moved this folder are running out of disk space/want to stuff the files elsewhere. Like myself. This is what I did on Windows just a couple days ago, I'd hope that Steam on Linux supports the same (hidden) option: http://www.rockpapershotgun.com/2012/09/11/finally-an-in-built-way-to-choose-steam-install-locations/
Basically:
- Close Steam
- Run steam -dev
- Click on the new 'Console' tab
- Enter 'install_folder_ui' (you even get tab completion)
- Add a new path, say .. /media/steam
- Close steam, restart without -dev
Now you can pick the location during installation ("This game? Install to my SSD. That? Ah, large slow disk is fine"). You can also move games between locations (from $path1/SteamApps/common to $path2/SteamApps/common and that will 'just work' in nearly all cases).
So, maybe Valve should make that more accessible in the UI, bypassing the need of users to mess with the default install path?
Thanks for reminding me to always check shell scripts before executing them
@darklajid changing the destination can already be done without using -dev, console, and install_folder_ui.
steam -> settings -> downloads -> steam library folders
@synapse84 Whoa - and here I kept that bookmark around, treasuring my knowledge of hidden options. :) Thanks a lot!
Amazingly scary.
Stupidest bug I've seen in years.
git blame on that, who did the commit?
Valve is retarded on so many levels, they completely screw up how libraries are linked up in some vain attempt to make Linux a kind of Ubuntu ABI stable distro. Now this.. The fact that steam is not using chroot and Linux kernel namespace support to protect against buggy/nefarious applications is idiotic security practice.
Further, what is Valve doing for a proper review and audit system for patchsets and continues integration, github comments, is that it.. F* may as well write software via Stasi-book (Facebook) wall posts lol..
I just wanted, on behalf of whole internet, to tell that the whole world could learn a thing or two from you
@keyvin.
The "tone" used to describe the issue is informative and polite, raging, straight to the point.
well done!
🍻
Classic Daily WTF: http://thedailywtf.com/articles/Bourne-Into-Oblivion
This is Bumblebee all over again.
@stryju Part of being a programmer now is being deluged in a sea of github repositories pretending that the software contained within works. Only after actually trying to use these projects, maybe even to the point where you start relying on them, do you realise half are garbage. After twenty or so such disappointments it's only natural then to remain calm in the face of yet another software project failure.
@nuisanceofcats while this might be true, I'm still impressed how calm and polite @keyvin remained.
I went through a fair shareof github issues and this one caught my eye 😄
@bumrang You got this open source thing all wrong. There is no room for civil or mature feedback. You should try emulate Linus like @xtraeme is doing and be abusive to anybody whose work displeases you. Because clearly people don't make mistakes, their work is an embodyment of who they are. You either are good developer or your not, there is no growth.
While using set -u or similar could have prevented this mistake, it's lazy and considered bad style.
It may not a good idea to knowingly write code that relies on set -u, but it helps prevent accidents exactly like the one this thread is about. It doesn't break any sane code. Turning it on is opting into some much-needed sanity checks.
@nuisanceofcats exactly! 🍻
Seriously, though... lets try to keep things civil. There can be any of a bunch of reasons for this having slipped by. It's especially obvious with how Valve works (joining and leaving projects almost at whim). The code could have been put in place by someone who's not even at Valve any more and folks picking it up just went with it because it was working. It's something that you could easily miss if you're not looking for it and didn't experience an issue with it. Anyone using Steam for Linux and following best practices and inspecting scripts before running them could have found this bug ages ago (myself included; I'm not saying I'm not equally to blame).
Think about how long it took for this to get a ticket. Everybody missed it for over a year.
@skybert set -e and set -u are recognized as poor coding practice. From bash-hackers.org:
set -e causes untested non-zero exit statuses to be fatal. It is a debugging feature intended for use only during development and should not be used in production code, especially init scripts and other high-availability scripts. Do not be tempted to think of this as "error handling"; it's not, it's just a way to find the place you've forgotten to put error handling. Think of it as akin to "use strict" in Perl or "throws" in C++: tough love that makes you write better code. Many guides recommend avoiding it entirely because of the apparently-complex rules for when non-zero statuses cause the script to abort. Conversely, large software projects with experienced coders may recommend or even mandate its use. Because it provides no notification of the location of the error, it's more useful combined with set -x or the DEBUG trap and other Bash debug features, and both flags are normally better set on the command line rather than within the script itself. Most of this also applies to the ERR trap, though I've seen it used in a few places in shells that lack pipefail or PIPESTATUS. The ERR trap is not POSIX, while set -e is. failglob is another Bash feature that falls into this category (mainly useful for debugging). The set -e feature generates more questions and false bug reports on the Bash mailing list than all other features combined! Please do not rely upon set -e for logic in scripts. If you still refuse to take this advice, make sure you understand exactly how it works. See: Why doesn't set -e (or set -o errexit, or trap ERR) do what I expected? and http://www.fvue.nl/wiki/Bash:_Error_handling
Correct me if I'm wrong, but AFAIK "$0" expands to the name the script has been invoked as, not the absolute path. (open a shell and try echo $0)
I guess this adds to the problem of not checking for unset variables...
$0 is useful for programs like busybox, that act differently depending on by what name they have been called.
Using it for filepath-magic OTOH opens yet another can of worms, as soon as there are symlinks involved: The regex can expand to a path that doesn't necessarily exist.
tl;dr: don't use $0 in paths.
@markgraf: $0 is the command the script was called with, for example emacs, or /usr/bin/emacs, or ../../bin/emacs. It's really just the first "word" of the command line. If you just want the name of the executable, use basename $0, if you want the full path, use readlink -f $0 (edit: except if $0 is a symlink, then readlink -f will return the full path to the symlink target).
actually $0 can be anything or nothing at all. See help exec in bash.
Everything is impermanent and transient. Especially bits on disk. No use crying over flipped bits.
I also don't want to get angry about it because steam for linux was a counter-thrust agaisnt microsoft turning windows into a walled garden with the app store. I didn't pay for steam for linux, and I appreciate that Valve is taking the time to make a proper port so we aren't all stuck running steam through wine.
If my root directory had to be sacrificed to appease the god of bugs, so be it. It'd be cool if I could get a special badge and an email apology from whoever added the #Scary! comment. That still makes me chuckle.
Came across a similar problem a few weeks ago when Steam started deleting files for unclear reasons; I killed the process as soon as I saw that. I first thought it was in its own dir, as I discovered that some games had been uninstalled but the maths didn't really add up. Now that I see this issue, I really wonder if I have lost important stuff (like backups of my work). I sure hope not but I'm pretty scared now.
Anyway, I haven't really understood from the long list of previous comments: is the issue fixed or not?
@markgraf, no not always, as it depends on invocation.
jordi@penyagolosa:~$ cat /tmp/meh
#!/bin/sh
echo "This is $0: $0"
echo "This is basename
jordi@penyagolosa:~$ /tmp/meh
This is $0: /tmp/meh
This is basename $0: meh
Priceless...
comment before it goes hot.
Wow.
I think it is time to move Steam to a separate user and/or chroot, just in case.
@korobochka I think that its best to do that for all games. I also only run Skype from a Windows VirtualBox. Its hardly as secure as Qubes but I think that proprietary apps that do a lot of stuff should be isolated in at least some minor way.
This issue has been open for 2 days without any response, fix, or apology from Valve? What the actual f**k?
Fire in the hole
It's like Sierra Half-Life 1 installer all over again. The installer had a default target of C:\Sierra\Half-Life. If you changed this to C:\Games\Half-Life and later you uninstalled Half-Life 1, the uninstaller removed C:\Games completely. Great achievement for Valve to bring such behavior back.
Just a hint: Did you ever check the amazing quality and documentation of Valve's Linux dedicated servers? It is horrific, I spent so many hours debugging this stuff.
Oh by the way: Valve is an US company deploying an automatic software installer onto your free software operating system. Would also be a great place for NSA to access your system.
@stratumnine As far as we know this has only affected one person and that person is chill about it. There's no need to get aggressive on behalf of these hypothetical people who it also messed up. You should learn from @keyvin, chillest brother alive.
Remember kids, before you rage in github think: "What would @keyvin do."
Praise be to lord keyvin.
PS Valve should provide offerings to keyvin.
This might become even worst than bumblebee... it can potentially erase any network volume that you have mounted at your filesystem. Imagine if you have your NAS drive mounted...
@nuisanceofcats no this happened to other people, too e.g. @onodera-punpun, @d00fy and other people reporting this on reddit r/linux
but nonetheless you are right about the tone especially these "w00t" ppl posting stupid pictures and posting nothing than bumblebee references which helped no one cuz there are so kewl.
@nuisanceofcats It has not affected one person, read the second comment for instance. I myself might have been affected by this as well. I lost tons of files for no apparent reason a couple of days ago. I am still investigating what happened and just found out about this.
I need to comment on this, because of yes. I use steam and omg!
Thanks for report this "scary bug"
Another genius.
Github is not a forum, if your comment is not related to the technical issue please stop posing and take it here: http://www.reddit.com/r/linux/comments/2sjjr3/warning_to_steam_users_dont_try_to_move_your/
but nonetheless you are right about the tone especially these "w00t" ppl posting stupid pictures and posting nothing than bumblebee references which helped no one cuz there are so kewl.
@pythoneer Yes, I'm kewl, what's your problem?
Bumblebee and this are not just bugs, they are gems. You can reference them in beginner's books where you say "don't do this, because that's what happens". I see nothing wrong with having some fun about it, especially if that causes people to remember it better.
I am doing exactly the same except that instead of using the Steam root folder as symlink, I use ~/.local/share/Steam/SteapApps and nothing got removed after the upgrade.
I think we should consider what @keyvin (praise be) would want before making any more comments on this bug. nuisanceofcats - first order keyvinite
@alexander-yakushev for fun posts and showing others how kewl you are pls visit other places like @ju2wheels mentioned. If you like to help in a serious way e.g. sharing experiences with this bug or reproducible behavior, feel free to post everything you know about. but this style of post is not helping but complicates to read through the post for ppl. who work with this information to track down and solve the problem.
@ju2wheels That code would seem to fix it. I don't see why a * was added to the rm command. If it wasn't there it would at least have errored out and preserved the files for those in OP's position.
On the other hand, the check should be made for the entire set of reset commands.
I would rather not have Steam installed until this is fixed. Is there an official procedure to uninstall Steam manually under Linux, like there is for Windows?
I'll just mention safe-rm - that's what I found after searching for a possibility to avoid such misfortune in vendor shell scripts altogether. It's a wrapper for /bin/rm that checks the given path against global and user specific blacklists. Sounds like a good way to tackle the problem at its root.