Changes to homebrew-cask installation behaviour

Question

Changes to homebrew-cask installation behaviour

vitorgalvao opened this issue 9 years ago · comments

Changes to homebrew-cask installation behaviour

Depending on how long you’ve been following the development of homebrew-cask (henceforth HBC), you might’ve come across this great rant¹. It was the turning point in making the link stanza (precursor to app) a requirement to every cask. Prior to that, HBC tried hard to be smart about what to link, but we eventually concluded being smart doesn’t work — there are too many edge cases.

The end of linking, and the beginning of moving (reasoning)

We reached another such point with linking. What works for CLI apps does not work for GUI apps, and by sticking close to the homebrew model we’re bending apps to do what they’re not expecting, and only suffering from it. Issues we cannot fix continue to pop up, and copy as merely a preference or something to be done occasionally can’t solve them. We cannot, in addition to the inconsistency with pkgs being installed instead of linked, add yet one extra inconsistency where even apps won’t have a predictable behaviour. To put it bluntly, linking sucks for GUI apps. It doesn’t work and will never not be broken, as we can’t control what other developers do.

Linking isn’t even needed, if you think about it. What separation of concerns are we worried about? CLI tools should have the separation homebrew provides because some of their apps (like newer versions of OS-level tools) provide critical features that could break the entire system if overridden. That is not the case at all with GUI apps; the only apps we have that could break the system on install (pkgs) are already given free range. If you use HBC, there’s not even any reason to install a GUI app any other way, so why would that separation matter?

GUI apps aren’t the only suffering from the current behaviour, even. Other artifacts suffer from similar issues and some even required the introduction of a different kind of linking.

In addition, while having different versions of the same CLI app is many times desirable, that is a far from the norm with GUI apps. Having an upgrade that works on every app, even ones that auto-update, also becomes nonsensical.

We should emulate regular usage (i.e. what people do without HBC), instead of trying to perpetuate a new paradigm that is basically contradicting expectations. This, however, does not mean losing customisation options when it makes sense (like --appdir).

Every workaround we build is forcing apps to fit our model, and our model is bad. We’re building workaround on top of workaround, adding more code and work for something that will never work correctly.

From The Cathedral and the Bazaar: “you often don't really understand the problem until after the first time you implement a solution. The second time, maybe you know enough to do it right. So if you want to get it right, be ready to start over at least once”.

As with other changes in the past, we now understand this problem a bit better and will adjust our solution accordingly.

The end of linking, and the beginning of moving (technical)

First steps towards new behaviour:

Every artifact stanza except binary will move its target to the appropriate location, instead of being sym/hardlinked.
apps will be moved to /Applications by default.
:target will continue to work, but rename the artifact itself instead of a link.
The Caskroom (/opt/homebrew-cask/Caskroom/ by default) still needs to exist.
Versioned subdirectories of the Caskroom won’t exist anymore, though.

Explanations:

Explained in the reasonings above why most artifacts will be moved instead of linked. As for binaries being the exception, it’s simple: not only is it expected, it’s advantageous, recommended, and sometimes necessary. Many times a binary is linked from inside an app bundle and cannot simply be moved/copied from there and continue to work. In these cases, symlinking is the recommended (by the developer) course of action anyway, so we stay in line with the idea of emulating regular usage. It also makes it work similarly to other CLI tools, like installs from homebrew.
Self-explanatory. It’s what most users expect, and would be our default anyway.
Self-explanatory.
It will be needed to keep a few things:
1. The originals from binary, when they’re standalone.
2. Other things like uninstall scripts that come with some pkgs.
Explained in the reasonings above.

New stanzas/keys/flags to enhance these changes

auto_updates. Similarly to stage_only, it should be used when a value of true is needed. It means “the artifact this cask uses auto-updates, so even if the cask file updates, leave this alone when upgrading others”. This does not mean upgrade in general is no longer a desirable feature — it is, but for casks who do not do it themselves.
:required_location. Example: app 'FontForge.app', :required_location => '/Applications'. It means the artifact needs to be in a certain location to function correctly and as such will be moved there even if you have a different default location.
--force-appdir. Takes no arguments and forcibly overrides required_location.

Pseudo-code for :required_location, considering an app:

if (--appdir is unset)
  move_to /Applications
else
  if (cask has :require_location) and (required_location_target != wherever_appdir_is_set_to)
    print_warning "This app only works if installed in required_location_target, so it will be moved there"
    move_to required_location_target
  else
    move_to wherever_appdir_is_set_to
end

Doesn’t this mean a bunch of stuff might break?

In the short term, yes, it does, but it is the right decision. We understand we have a bunch of users that are using homebrew-cask daily, and ideally we could try to make this a smooth transition like in the past. However, we’re admittedly pre-alpha and setting transition strategies is an inglorious job (everything will be discarded in the end, after all) that puts on hold even the smallest of changes indefinitely. It is incredibly time consuming, has variable results (we still get a bunch of issues pertaining to it), and (above all) kills momentum. Furthermore, this is a big change and there’s really no good way to make this seamlessly.

In the end, we want to continue to move forward and improve HBC. Being afraid of changes in a pre-alpha would mean stagnation.

Are there any other big changes planned?

There are ideas, yes, but no exact plans yet. Those ideas still need to be discussed and most are concerned with the creation and maintenance of casks, not the usage of HBC.

¹ If you haven’t, I highly recommend it as piece of HBC’s history.

Aditya Dalal · Answer 1 · Wed Aug 12 2015 23:46:56 GMT+0800 (China Standard Time)

Seems like a useful change, and might be timely as well - I plan to do a clean install with 10.11, so breaking changes are not a huge issue.

So does the auto_updates stanza remove the need for explicit versioned urls, or is there still a preference for versioned, shasum'd formulas?

Would an example of a formula where auto_updates: true then be an app like Spotify, as it manages its own updates (ie, any forumas that use version :latest & sha256 :no_check)

Vítor Galvão · Answer 2 · Thu Aug 13 2015 00:02:33 GMT+0800 (China Standard Time)

and might be timely as well

Well, depending on how long the implementation (and release) takes.

So does the auto_updates stanza remove the need for explicit versioned urls, or is there still a preference for versioned, shasum'd formulas?

For now, preference should be left with versioned urls. The overlap of apps that auto-update and apps that have unversioned urls isn’t clear (auto_update will make it so), so this is a discussion for later. The security provided by always having a version is desirable.

as it manages its own updates (ie, any forumas that use version :latest & sha256 :no_check)

This is incorrect. A cask having an unversioned url (i.e. it almost exclusively will use version :latest) is not guaranteed to auto-update, or vice-versa. The two are completely unrelated.

Sean Zhu · Answer 3 · Thu Aug 13 2015 00:05:32 GMT+0800 (China Standard Time)

Is there a milestone, task list, thread, or wiki page where people can go check how much of this has been implemented and/or released?

Vítor Galvão · Answer 4 · Thu Aug 13 2015 00:08:50 GMT+0800 (China Standard Time)

Is there a milestone, task list, thread, or wiki page where people can go check how much of this has been implemented and/or released?

Not currently. Things should be released as issues as time goes on. It might actually take a few weeks for development of this to start, but this is the type of thing that should be announce early.

scribblemaniac · Answer 5 · Thu Aug 13 2015 01:30:30 GMT+0800 (China Standard Time)

👍
Everything sounds good to me, I'm glad to hear we are moving forward in this direction for the most part.

If apps come in a folder with a bunch of other things (readmes, licenses, plugin directories, etc.) will those be moved to /Applciations as well? What about if they are required to be in the same directory as the application?

Should auto_update casks be maintained from version to version?

If an application downloads the latest update when necessary, but does not install it for the user, should it be considered auto_update? I know that Filezilla does something like this (or it used to, I haven't checked recently).

Vítor Galvão · Answer 6 · Thu Aug 13 2015 01:43:27 GMT+0800 (China Standard Time)

If apps come in a folder with a bunch of other things (readmes, licenses, plugin directories, etc.) will those be moved to /Applciations as well?

No. See point 4 of The end of linking, and the beginning of moving (technical) in the top post. Only what we already link will be moved, it’s one to one.

What about if they are required to be in the same directory as the application?

Than we use suite to move the directory. We’re already covered, there, and some casks were already changed, even.

Should auto_update casks be maintained from version to version?

Yes, so anyone installing it for the first time gets the latest version. Many (most?) of these have discoverable appcasts, though, which makes the job easier.

If an application downloads the latest update when necessary, but does not install it for the user, should it be considered auto_update?

No. Only the ones that can do it seamlessly should, because otherwise we can still provide a better experience.

Sean Zhu · Answer 7 · Thu Aug 13 2015 01:50:15 GMT+0800 (China Standard Time)

If apps come in a folder with a bunch of other things (readmes, licenses, plugin directories, etc.) will those be moved to /Applciations as well? What about if they are required to be in the same directory as the application?

(The following is my own preference.) I don't like folders in the Applications folder unless it really is to organize a bunch of apps (Microsoft Office).

Suggestions for apps like Audacity and Eclipse that have a required folder structure:

Place folder contents in /Library/Application Support/Audacity, and place an app symlink in /Applications/Audacity.app -> /Library/Application Support/Audacity/Audacity.app
Place folder contents in /Library/Application Support/net.sourceforge.audacity, and place an app symlink in /Applications/Audacity.app -> /Library/Application Support/net.sourceforge.audacity/Audacity.app (bc sometimes apps have the same name)
Place folder contents in /Applications/Audacity, and place an app symlink in /Applications/Audacity.app -> /Applications/Audacity/Audacity.app, and chflags hidden /Applications/Audacity (probably not a good idea bc hidden files)

Suggestions for apps with non-code supporting files:

Place supporting files in /Library/Application Support/SomeApp (more consistent with the above)
Place supporting files in SomeApp.app/Support, SomeApp.app/Extras, SomeApp.app/, etc. (more consistent with Mac expectation that trashing the app will pretty much uninstall it)
Discard supporting files
Leave supporting files in Caskroom

Vítor Galvão · Answer 8 · Thu Aug 13 2015 01:55:41 GMT+0800 (China Standard Time)

I like those ideas, @szhu. However, I’ll also say “definitely not for now”, and the first paragraph of the top post explains why. Those could introduce issues we’re not foreseeing, and this change is in part meant to fix things, not introduce new possible points of failure.

It doesn’t mean we can’t revisit those in the future, naturally, but for now the takeaway from this post should be emulate regular usage. Let’s nail that first, and only then see if being smart can pay off.

Sean Zhu · Answer 9 · Thu Aug 13 2015 02:09:00 GMT+0800 (China Standard Time)

Okay. I'm assuming that means the following:

For apps that require a folder structure, move the entire folder to /Applications or leave it in the Caskroom
For apps that provide a .app with some documentation, discard the documentation or leave it in Caskroom

Vítor Galvão · Answer 10 · Thu Aug 13 2015 02:19:46 GMT+0800 (China Standard Time)

For apps that require a folder structure, move the entire folder to /Applications or leave it in the Caskroom

Move it, using suite.

For apps that provide a .app with some documentation, discard the documentation or leave it in Caskroom

Leave it in the caskroom.

Both those points were addressed in the previous post.

Mark Huber · Answer 11 · Thu Aug 13 2015 10:11:19 GMT+0800 (China Standard Time)

You reference /Applications , currently all of my casks have defaulted to ~/Applications . Is this change also changing that behavior of utilizing a folder local to the user's home directory? (sorry if this was covered elsewhere)

scribblemaniac · Answer 12 · Thu Aug 13 2015 10:16:22 GMT+0800 (China Standard Time)

Yes, it is changing that behavior. You will still be able to use ~/Applications if you want by using by using the --appdir flag.

Terry Nycum · Answer 13 · Thu Aug 13 2015 12:12:52 GMT+0800 (China Standard Time)

Was there an open discussion of the pro's and con's of this somewhere that I've missed?

Vítor Galvão · Answer 14 · Thu Aug 13 2015 18:30:58 GMT+0800 (China Standard Time)

@markhuber:

sorry if this was covered elsewhere

It was all covered in the top post. @scribblemaniac’s explanation is correct.

@tjnycum Yes, there were multiple, including #3083 and #3888. I have a slight feeling from your last comment and this one you might be under the impression this might be a matter of user preference and voting. It’s not.

This is about fixing a broken model, and I’ve spent many months thinking about this solution and managing homebrew-cask always with that in mind, to understand how it’d affect it. Keeping the old system is unsustainable.

Every workaround we build is forcing apps to fit our model, and our model is bad. We’re building workaround on top of workaround, adding more code and work for something that will never work correctly.

Every time there’s a new problem our workarounds fail to address we have to discuss how to implement a new one. Those are becoming harder and harder, and many of them are impossible to solve without breaking experience. All of them are solved immediately by emulating regular usage.

As mentioned in the top paragraph of the top post, being smart failed, and as mentioned in a previous post, let’s first get things to work, and think about being smart later.

Võ Anh Duy · Answer 15 · Fri Aug 14 2015 02:01:17 GMT+0800 (China Standard Time)

👍 i want this for years

Todd Eddy · Answer 16 · Fri Aug 14 2015 20:09:45 GMT+0800 (China Standard Time)

After just quickly scanning 👍 on the auto_update. Right now I just don't install apps that I know do automatic updating assuming it would break things. Also the moving of apps I can see as a much needed change as I've had issues with some apps being linked which result in me again not using cask but installing it the "Old Fashioned Way".

This big changes can be annoying in the short term but it's good to figure these things out while the project is still relatively new than wait 10 years and have to deal with everything breaking

Ibraheem Zaman · Answer 17 · Mon Aug 17 2015 13:24:02 GMT+0800 (China Standard Time)

The most important benefit of linking in my opinion was that the old versions did not need to be replaced in that workflow, and thus the user could easily downgrade back to a previous version simply by changing the link (or just use the previous version directly to check some previous functionality). The ability to downgrade to a previous state is one of the most important benefits of a package manager in my opinion.

Applications that depend on being installed at a hardcoded path are exhibiting bad behavior. There doesn't seem to be too many of them listed at #12858, and we shouldn't need to change the whole structure just to accommodate these.

If we want to support the use of the upgrading options provided by the applications themselves, then (lacking a copy-on-write filesystem) we could have application versioning disabled by default while having a configuration property to enable it.

Another issue I discovered with the linking was that the dock seems to resolve symlinks and store the original source path, which forces me to update the items manually. I hope this issue can be fixed in some way, but if not then I can live with it.

If there are some other major issues with linking, then please elaborate.

As for the auto_updates stanza, I am fine with it in general for applications that auto-update by default, but please have an option for ignoring it for those who do not want auto-updates. When using a package manager I like to manage my updates, and I disabled auto-updating and update prompts for all applications before discovering the lack of an upgrade command in Homebrew Cask.

Sean Zhu · Answer 18 · Mon Aug 17 2015 13:33:28 GMT+0800 (China Standard Time)

There are actually numerous issues stemming from linking.

Many apps will prompt the user, if not installed in a standard location, to move it there.
Many apps support in-app upgrading, which will result in an inconsistent experience when used in conjunction with brew cask upgrade.
Many apps are already not versioned (:latest)
Having multiple versions of the same app causes issues with launching the app, i.e., file associations and apps that open other apps. If the app is identified by id, it's not clear which version of the app should be launched, and the arbitrary choice can be confusing for the user. If the app is identified by path, that identifier will break when old versions are uninstalled. This is an issue with not only Dock but basically most other launcher apps too.
If you want a backup of an older version of the app, you probably don't expect to use both at the same time. The proper way to do this is probably to backup the install media (dmg/zip) instead of keeping the live app around. This file already exists in /Library/Caches/Homebrew!

Ibraheem Zaman · Answer 19 · Mon Aug 17 2015 14:06:01 GMT+0800 (China Standard Time)

@szhu:

Many apps will prompt the user, if not installed in a standard location, to move it there.

There is a suppress_move_to_applications method to handle this issue.

Many apps support in-app upgrading, which will result in an inconsistent experience when used in conjunction with brew cask upgrade.

If I install an application from Homebrew Cask, I don't expect to use the in-app updater. I have disabled all these updaters in my applications. However, if we actually want to support this behaviour, then I have suggested using an option to disable versioning.

Many apps are already not versioned (:latest)

Perhaps we can extract the version information from these in some other way? Otherwise, I guess we're forced to drop versioning in these cases as per the current behaviour.

Having multiple versions of the same app causes issues with launching the app, i.e., file associations and apps that open other apps. If the app is identified by id, it's not clear which version of the app should be launched, and the arbitrary choice can be confusing for the user. If the app is identified by path, that identifier will break when old versions are uninstalled. This is an issue with not only Dock but basically most other launcher apps too.

I don't see this issue with my Calibre associations, which I upgraded a few days ago. Files associated with it open the latest version. Although upon opening the "Open With" submenu it does list multiple versions. Again, it would be nice if there was some way to have it only look within /Applications, but I can live with it.

If you want a backup of an older version of the app, you probably don't expect to use both at the same time. The proper way to do this is probably to backup the install media (dmg/zip) instead of keeping the live app around. This file already exists in /Library/Caches/Homebrew!

Good point. However, there isn't a simple way to have Homebrew Cask reinstall a previous version short of manually editing the cask file. Also, this is how Homebrew mainline itself is implemented, and it would be best to keep conformity with it as far as possible.

Sean Zhu · Answer 20 · Mon Aug 17 2015 14:11:57 GMT+0800 (China Standard Time)

I'm on mobile but there's a comment somewhere about how they tried to be smart about installing, but realized that was an unmaintainable amount of overhead, and that the most sensible way to install is in the same way a normal, non-cask user would.

Also, having a way to extract/install old install media would be a great idea (but probably for later, unless you want to make it!).

Also, this is how Homebrew mainline itself is implemented, and it would be best to keep conformity with it as far as possible.

From what I glean, cask uses Homebrew because it's good infrastructure, not because the user experience should be the same. It's probably better to keep conformity with the non-cask app experience.

Ibraheem Zaman · Answer 21 · Mon Aug 17 2015 14:55:24 GMT+0800 (China Standard Time)

I'm on mobile but there's a comment somewhere about how they tried to be smart about installing, but realized that was an unmaintainable amount of overhead, and that the most sensible way to install is in the same way a normal, non-cask user would.

In that case I would like to know about these issues. The only issue mentioned in the original post here was of the hardcoded location expectation from a few applications.

Also, having a way to extract/install old install media would be a great idea (but probably for later, unless you want to make it!).

If we introduce a generic downgrade feature, then users would probably expect it to be able to work across versions which weren't installed before as well, but that is problematic to implement in the Homebrew/Cask Git structure, where updates to version and installation instructions are not kept separate.

I guess we could implement a "revert" feature that only reverts to versions already present in the cache, and reinstall it using the cask instructions at the revision at which it was originally installed. If this is actually implemented or planned for implementation, then I withdraw my objections to removing the linking structure. I am not familiar with Ruby programming, so I would probably not be able to implement this feature myself.

From what I glean, cask uses Homebrew because it's good infrastructure, not because the user experience should be the same. It's probably better to keep conformity with the non-cask app experience.

The FAQ on this very issue suggests the opposite:

The reason we implement this project on top of Homebrew was based on a belief that their methodology for managing applications has a lot of merit. We'd prefer to try and work things so that we can keep ourselves Homebrewy both in implementation and idioms. Trying to manage all of ~/Applications would move the project more towards a standalone system, which would mean reimplementing a lot of the Homebrew stuff we lean on now.

Ibraheem Zaman · Answer 22 · Mon Aug 17 2015 15:07:45 GMT+0800 (China Standard Time)

Also, this is a somewhat separate issue, but I always thought it strange that it seems like Homebrew Cask uses a hardcoded global path (/opt/homebrew-cask) to store the Casks regardless of the actual location of Homebrew itself, so even a local Homebrew installation would create global Casks (I haven't tested this myself, but this is my assumption lacking documentation to the contrary). It would probably have been best for it to use some location inside the Homebrew installation itself (unless explicitly configured otherwise), or at any rate not use a global location if Homebrew itself is not in one. The default value for the appdir option could probably also have been inferred from this.

Caleb Xu · Answer 23 · Mon Aug 17 2015 22:29:53 GMT+0800 (China Standard Time)

The reason we're moving away from linking and towards copying/moving instead is also due to an inherent difference in the nature of OS X apps and binary/CLI tools.

Within a Linux/Unix/BSD environment, it is commonplace to link binaries into */bin directories.
- Binaries are built against libraries, which reside in a preset location (/lib, /usr/lib, etc.).
- App bundles are somewhat self-contained. Sometimes, they also contain config/preference data (although that is a bad idea, IMO)
It is not uncommon to use multiple versions of some CLI tools, especially if there are breaking/compatibility changes.
- A CLI tool will almost never update itself. There are a few newer ones, particularly package managers themselves, that can also update themselves (e.g. npm, pip) - in fact, self-upgrading CLI tools often break things for Homebrew - a red flag that perhaps we shouldn't handle app bundles, many of which have some sort of update functionaly or contain persist data within the bundle, the same way Homebrew handles CLI tools.
- Different versions of CLI tools will often be named differently from other versions, especially when managed by Homebrew (e.g. gcc vs gcc-51). Some app bundles also follow this type of a scheme, in terms of adopting a new name with a new major release.

The only issue mentioned in the original post here was of the hardcoded location expectation from a few applications.

Some apps will throw dialogs to move to /Applications, like you said, but there are some that will simply refuse to run.

If we introduce a generic downgrade feature, then users would probably expect it to be able to work across versions which weren't installed before as well, but that is problematic to implement in the Homebrew/Cask Git structure, where updates to version and installation instructions are not kept separate.

This could get very messy. It's why Homebrew dropped the feature (brew versions has been deprecated in favor of homebrew-versions). I think it wouldn't be as big of an issue with casks, simply because building software is a more fragile process than downloading/installing app bundles. Nevertheless, it's still a headache to implement at this point, and if we ever add it, it would be much later in the roadmap for this project.

Also, this is how Homebrew mainline itself is implemented, and it would be best to keep conformity with it as far as possible.

From what I glean, cask uses Homebrew because it's good infrastructure, not because the user experience should be the same. It's probably better to keep conformity with the non-cask app experience.

In the past, we've tried to keep with consistent user experience with Homebrew. However, we've found that it simply doesn't work as well as we want to, because OS X apps just can't be handled the same way as CLI binaries. Imagine using an apple peeler to peel an orange - it doesn't get the job done (unless you wanted orange zest in the first place). Thus, we're willing to compromise a bit on UX conformity with Homebrew in order to make the system more functional and less buggy overall.

The FAQ on this very issue suggests the opposite:

This has yet to be updated.

Vítor Galvão · Answer 24 · Tue Aug 18 2015 03:10:52 GMT+0800 (China Standard Time)

@1zaman I’m on a phone (won’t have access to a computer for the next week), so forgive any bluntness. @szhu and @alebcay addressed your points, but I’d like to add something quick.

As I’ve mentioned in the top post, we’re not like other package managers. Any point you justify with “I expect that of other package managers” is irrelevant: we tried those, and they failed. Following a pattern that breaks for us simply because others do so is nonsensical.

Furthermore, all your solutions are nothing more than more workarounds, which is exactly the problem, as mentioned in the top post, so they don’t solve anything.

Lastly, you link to an issue that has few examples, and ask for more cases. I want to be perfectly clear that I see you were perfectly cordial when making your case and I appreciate that and want to be just as polite when I say your points are mostly due to lack of context and experience with HBC. Those few examples are part of a list still being made, and part of a much larger issue. We’re also not going to link to every specific issue that lead to every specific reasoning — the top post is verbose as it is. I’ll say again this is not a vote. We tried other solutions before, and this new course was decided after years of dealing with HBC and seeig what works and doesn’t.

Discussion is encouraged and appreciated, and we will give context for the reasonings, but if you want specific examples, those would take a while to search for just to get them to you simply to justify in detail what we already know and explained in broad strokes. That would be very time consuming and frankly useless, so if you want specifics, you’ll have to do the searching. I hope you understand we can’t be doing that for every user, or we’d do nothing else. We will explain reasonings and link to the appropriate contexts, but if conclusions have been reached through experience with many separate issues throughout the years, you can’t reasonably expect any of us to drop everything and do your reasearch to satisfy your doubts, when you just got here, and every issue is public and searchable.

We’re trying to make the project better, and this decision is just one more towards that goal. It’s a conclusion arrived at from being here everyday seeing things break and work, and striving for less of the former and more of the latter.

Ibraheem Zaman · Answer 25 · Tue Aug 18 2015 14:02:19 GMT+0800 (China Standard Time)

@alebcay:

Thanks for explaining the structural differences between OS X GUI apps and CLI tools. You describe three differences:

Binaries are built against libraries, while app bundles are self-contained.
Sometimes, app bundles store configuration data inside themselves.
App bundles provide self-updating functionalities.

The first difference does not seem to be relevant to versioning.

As you mention, keeping configuration data inside app bundles seems to be a bad practice, and would get overwritten by Homebrew Cask upgrades in the new schema as well.

The self-updating issue seems to be the relevant issue here. These don't seem to be problematic for the most part, but they do mess up the versioning. As I have mentioned before, we can have versioning as an optional feature for those who don't want to use the application updaters.

Ibraheem Zaman · Answer 26 · Tue Aug 18 2015 14:02:25 GMT+0800 (China Standard Time)

@vitorgalvao:

I acknowledge that I am not aware of the full context. I merely noted that there didn't seem to be too many examples compiled of the main issue that you noted in your original post. If you assert that there are a significant number of applications that break if not located at a hardcoded path, then I'll take your word for it.

I think that when making such a large structural change, the reasoning for it should be documented as comprehensively as possible. You may have had lots of isolated discussions on this issue at various points, but there is some point to bringing these together (not necessarily in this issue) so that everyone (not just the maintainers and people associated with the project from the start) can see the full context. There might be alternative solutions or enhancements provided as a result of a comprehensive discussion on this.

I did look through some old issues on this subject that were linked from the discussion on the upgrade feature before commenting here, but you yourself took the opposite stance on those. I can try looking further, but I can never be sure that my research is comprehensive without some official compilation to cross-reference.

I didn't present any solution yet, other than keeping the status quo as an option.

PS: Sorry to disturb your vacation 😉 Please feel free to not respond until you return.

Vítor Galvão · Answer 27 · Tue Aug 18 2015 18:03:14 GMT+0800 (China Standard Time)

There might be alternative solutions or enhancements provided as a result of a comprehensive discussion on this.

Here’s the thing: there were discussions. This issue is discussed and settled. If we restart the discussion everytime someone who wasn’t there asks to have the conversation again, we’ll never do anything.

I did look through some old issues on this subject that were linked from the discussion on the upgrade feature before commenting here, but you yourself took the opposite stance on those.

That is a good sign, I hope, I did really think hard about this.

PS: Sorry to disturb your vacation 😉 Please feel free to not respond until you return.

Thank you. Going well so far. A good day to you.

Caleb Xu · Answer 28 · Tue Aug 18 2015 21:45:34 GMT+0800 (China Standard Time)

The first difference does not seem to be relevant to versioning.

But it is! This often means that app bundles are built against old or outdated libraries, where as the dependencies of CLI tools can be updated without having to update the actual tool itself. App bundles also sometimes stick their stuff in weird places while CLI tools are pretty standard with their behavior - I've found that some non-native apps are the worst offenders, placing items in random folders that I would never think to check. It's not so much an issue with versioning directly, but rather unnecessary overhead if a maintainer has to keep up with all of the files in order to stage the upgrade process via homebrew-cask, especially some of these apps offer somewhat reliable systems for upgrading themselves.

As you mention, keeping configuration data inside app bundles seems to be a bad practice, and would get overwritten by Homebrew Cask upgrades in the new schema as well.

You're right about that. Perhaps some sort of app bundle "snapshots" could assist in identifying config/persistent data within the bundles: take snapshot of bundle right after install, and take snapshot of bundle right before upgrade, and then perform the upgrade and re-apply the changes from between the snapshots. I have no idea how effective that would be, nor how much code we would have to write.

Regardless of how you look at it, this project has been practically stagnant for the past few months up to now in terms of core development, and the original direction we were moving in was simply a huge ball of workarounds that somehow worked. But it has become pretty clear that the approach we've used in the past isn't "cost-effective" to maintainers, nor is it the best for end-users either. There were too many corner cases to accomodate for, and so rewriting the core to handle corner case apps more elegantly is a priority right now.

I'm sorry I couldn't find any more old issues that are relevant to this discussion, as you've asked for, but here's one last megalist of everything that's been referred to in this issue so far, for your reference - I imagine they might have gotten lost in the shuffle. They cover the default installation process, default installation location, etc. - many decisions/discussions that are the basis of the decision in the recent changes. There's a lot of back and forth discussion on pretty much every decision that has had been made in the past, so you may find that somebody else can explain the rationale for a decision much better than I can.

Zhiming Wang · Answer 29 · Fri Aug 21 2015 04:05:19 GMT+0800 (China Standard Time)

I wholeheartedly welcome this change. From a user's perspective I've had to deal with explicit or subtle breakage due to linking on multiple occasions, and I have a personal installer script that swaps some applications into /Applications (browsers, Tunnelblick, etc.). I add to the script every time I'm bitten, and the script is growing. Not fun.

@1zaman Please don't just talk from your experience. "I don't have problems so you shouldn't make changes" isn't a good attitude. I apologize for exaggerating; I know you were asking about what exactly the problems are. However, if you allow me to talk from my experience, I'll tell you that problems abound. And I'm not even a maintainer, who oversees all the headaches.

If I install an application from Homebrew Cask, I don't expect to use the in-app updater. I have disabled all these updaters in my applications.

Please excuse me, but I think this sentence along sets you apart from normal users. I call that paranoid. Homebrew doesn't even support upgrade at the moment (and not any time soon, as far as I know), I just can't picture how you do your upgrades. At any rate, you're not the norm, unless I'm completely carried away by my own workflows.

Zhiming Wang · Answer 30 · Fri Aug 21 2015 04:05:22 GMT+0800 (China Standard Time)

I would like to add to the thread by pointing out that not only some app bundles need to live in a certain location in order to work. There are other nonstandard artifacts, like Mail plugins, that won't work if linked; and as OS X tightens up its security across releases, we should expect to see more of those (or maybe the hackish things would stop working altogether, who knows).

Example: the QuoteFix Mail plugin. Last time I fixed it I had to use an ugly stageonly + copy in postflight hack to make it work.

Should we also have an issue (similar to #12858) to track those nonstandard casks that should benefit greatly from plans in this issue?

Dieter De Meyer · Answer 31 · Fri Aug 21 2015 04:38:11 GMT+0800 (China Standard Time)

@zmwangx would you mind sharing your script that describes some of the workarounds needed for non working linked apps ?
I'm currently refactoring my provisioning setup to use homebrew casks but I'm not sure that was a good idea with what I'm reading here...

Zhiming Wang · Answer 32 · Fri Aug 21 2015 04:41:27 GMT+0800 (China Standard Time)

What I currently use: https://github.com/zmwangx/dotfiles/blob/7c13466cd0ccfb705fb1f147fbd9d70a58432122/cask/special. It doesn't solve every problem though, just things I would do on a clean install. And it would break uninstalling to some extent, but hopefully you wouldn't need to uninstall those critical apps.

Vítor Galvão · Answer 33 · Fri Aug 21 2015 05:46:21 GMT+0800 (China Standard Time)

Should we also have an issue (similar to #12858) to track those nonstandard casks that should benefit greatly from plans in this issue?

Comment in the issue instead, and I’ll add to and adapt the main post when I have the chance.

Philip · Answer 34 · Fri Aug 28 2015 20:13:18 GMT+0800 (China Standard Time)

so, for us that already have N applications installed through cask, how will the upgrade and relocation happen? Will Cask automatically move the apps there after some brew cask update in the future? Will i have to remove all and reinstall for it to automatically happen? will there be any command to make this happen?

Also, for those of us with export HOMEBREW_CASK_OPTS="--appdir=/Applications" already on their shell profile, will this conflict in any way with the new mode? should i remove it ?

Vítor Galvão · Answer 35 · Fri Aug 28 2015 20:18:04 GMT+0800 (China Standard Time)

@maccouch All those questions are answered in the top post.

There will be no automatic transition for the new system, and there will be no conflict by having that line in your shell profile.

Kasper Peulen · Answer 36 · Mon Sep 07 2015 02:39:27 GMT+0800 (China Standard Time)

hi all, just learning about this great project today

would it may be wise to wait for this update to be done ? or doesn't it really matter ?

Vítor Galvão · Answer 37 · Mon Sep 07 2015 02:41:27 GMT+0800 (China Standard Time)

@kasperpeulen We have no time-frame set for completion, so it’s really up to you.

Eduard Rozenberg · Answer 38 · Sat Sep 19 2015 21:36:09 GMT+0800 (China Standard Time)

This change to moving apps is fantastic news. I avoided using casks because I never liked the linking behavior and inconsistency of having a mix of apps and links in my /Applications. And superficially because I hated seeing icons in Applications with the symlink symbol :). Can't wait for this change, will start using casks as soon as it happens. Thanks!

Jason Karns · Answer 39 · Tue Sep 22 2015 01:14:08 GMT+0800 (China Standard Time)

I have concerns with the behavior of required_location.

As the owner of my machine, I should have final say in where applications are installed; not the formula author. There is clearly potential conflict between required_location and appdir. As the pseudo-code above does, it is helpful to print a warning when those values are in conflict. However, final say ought to be determined by the machine owner, not the author of the formulas (who may be errant, if well-intentioned). I would propose that if required_location conflicts with appdir, a warning message still be emitted as suggested here. But appdir should be the respected value, not required_location. The user then becomes responsible for any issues with installation.

The following scenarios are possible:

formula indicates required_location but does not, in fact, actually need to be in said location (be it benign ignorance or accidental configuration, or anything really)
formula indicates required_location but app actually functions just fine in a specific appdir (known by the user)
formula indicates required_location for certain uses of the app, but user does not need said uses, and app works fine for given subset of uses from appdir
formula indicates required_location but user is attempting to determine if app actually functions in appdir anyway

Josh Hagins · Answer 40 · Tue Sep 22 2015 01:20:09 GMT+0800 (China Standard Time)

@jasonkarns I think required_location should be respected over appdir by default, but we could provide a --force-appdir flag that overrides this decision. I envision lots of bug reports about faulty installations otherwise.

Jason Karns · Answer 41 · Tue Sep 22 2015 01:24:23 GMT+0800 (China Standard Time)

I envision lots of bug reports about faulty installations otherwise.

I can definitely see that.

The biggest thing, I think, is that users should have final say somehow. If that's with Yet Another Flag™, so be it. As long as it's possible for the user to override the formula author, I'm happy.

Vítor Galvão · Answer 42 · Tue Sep 22 2015 08:03:32 GMT+0800 (China Standard Time)

Added --force-appdir to the top post.

E14 · Answer 43 · Tue Sep 22 2015 23:36:38 GMT+0800 (China Standard Time)

I'm curious, what would happen if I was to submit an application that is called "Mail" or "Pages", etc. would brew-cask overwrite the existing App? How does it decide whether an App is pre-existing or not? Does it even do that or do you completely rely on peer-review?

Vítor Galvão · Answer 44 · Tue Sep 22 2015 23:38:48 GMT+0800 (China Standard Time)

The logic course of action there is to not overwrite what already exists, and print an error.

E14 · Answer 45 · Wed Sep 23 2015 00:02:36 GMT+0800 (China Standard Time)

then how do upgrades work? At some point you have to overwrite an existing path

Vítor Galvão · Answer 46 · Wed Sep 23 2015 00:35:46 GMT+0800 (China Standard Time)

To upgrade, you must install first. This is why they are different verbs. When you install, it can be assumed nothing with the same name exists — if it does, we know something is wrong, and should abort. However, when you upgrade, if something exists with the same name it is expected, and we can proceed.

Furthermore, we should always trash and never rm, so none of these are irreversible.

Zhiming Wang · Answer 47 · Wed Sep 23 2015 01:09:16 GMT+0800 (China Standard Time)

Furthermore, we should always trash and never rm, so none of these are irreversible.

Well, I think there should be an option to remove the trashed files after a successful upgrade. Many of us have good back up policies, and we'll be clearing out the trashes anyway. In my opinion homebrew-cask should automate things as much as possible.

Vítor Galvão · Answer 48 · Wed Sep 23 2015 01:15:44 GMT+0800 (China Standard Time)

Many of us have good back up policies

I’d wager most don’t.

and we'll be clearing out the trashes anyway.

What’s the problem then? No harm done. The difference there is you’re the one cleaning out the trash, not us.

In my opinion homebrew-cask should automate things as much as possible.

It should. Above that, however, it shouldn’t screw up your setup.

In all, a few MBs for an updated app in the trash is way better than the hassle of losing an important app or having to go get your backup. Also, if I recall correctly, this is what apps that auto-update themselves do, so that even goes in line with the new concept of emulating regular usage.

Zhiming Wang · Answer 49 · Wed Sep 23 2015 01:44:49 GMT+0800 (China Standard Time)

Also, if I recall correctly, this is what apps that auto-update themselves do, so that even goes in line with the new concept of emulating regular usage.

Yes it is. Okay it makes sense.

Eduard Rozenberg · Answer 50 · Wed Sep 23 2015 03:41:37 GMT+0800 (China Standard Time)

Trashing sounds fine. It's the same as what MacUpdate does. MacUpdate is a good model generally - it does not ever try to update system apps such as Mail.app, and any updated apps send the old version to the Trash.

Bryce Glover · Answer 51 · Thu Sep 24 2015 01:15:50 GMT+0800 (China Standard Time)

Should we think about seeing if Homebrew Cask could use a command-line interface to run application auto-updaters for casks whose auto_update stanza is set to true if such a CLI is provided by said auto-updater? And should we ask more auto-updating applications to provide CLIs to their auto-updaters?

Vítor Galvão · Answer 52 · Thu Sep 24 2015 01:29:27 GMT+0800 (China Standard Time)

No. Not anywhere in the near future, at least.

Bryce Glover · Answer 53 · Fri Sep 25 2015 02:34:51 GMT+0800 (China Standard Time)

@vitorgalvao: Guess I'll put that on the wishlist, then — where is that, by the way, or would I just mention my feature request in 'Issue #13969: Add :auto_updates stanza for Casks'?

Vítor Galvão · Answer 54 · Fri Sep 25 2015 05:03:31 GMT+0800 (China Standard Time)

@RandomDSdevel There isn’t any. To be clear, what I mean by that is that it is out of the question for now. You can mention that again a few months after this installation behaviour is implemented and we actually see how it worked in the wild, but to be honest I don’t see that feature ever happening.

For one, it’s two much work for little to no gain. We’d need to support each application individually and it’d be mostly useless: if the application can auto update itself, let it, that is the whole point. Furthermore, asking developers to support CLI ways of updating their GUI apps is a losing proposition: I bet it is not going to happen. Not in any meaningful number.

Bryce Glover · Answer 55 · Sun Sep 27 2015 02:05:06 GMT+0800 (China Standard Time)

@vitorgalvao: Ah, I see. Such a feature would be convenient, though…sigh.

Nathan Youngman · Answer 56 · Thu Oct 01 2015 12:52:40 GMT+0800 (China Standard Time)

As someone who does expect in-app auto-update to work (and hasn't gone around disabling it), I'm really looking forward to this change. Over half a dozen apps installed via HBC exist not only in ~/Applications as symlinks, but also in my /Applications folder. Likely due to auto-updates deciding that's where they belong. It's a mess, and then there are bizarre errors like this:

From a user experience perspective, I think it's very important to arrange applications the way the regular installers do, making cask upgrade and in-app upgrades interchangeable. In the following screenshot, a Lightrooom 6 upgrade left me with "Lightroom 5" (must confirm whether their installer renamed it, but I believe so).

As for concerns of overwriting Mail.app, HBC couldn't do so if it tried, at least as of El Capitan. System Integrity Protection protects the Apple apps listed in /System/Library/Sandbox/rootless.conf. (Of course, not overwriting apps managed by the App Store could still be a valid concern).

Important 12/Aug/2015 update: Homebrew-cask will change its behaviour from linking apps to moving them. See issue #13201 for details.

One final remark. Could we please update the README to use more precise wording ("in the future"). I misread it: "as of 12/Aug/2015, Homebrew-cask will..." And then I was sad that it wasn't the case.

Nathan Youngman · Answer 57 · Thu Oct 01 2015 13:06:41 GMT+0800 (China Standard Time)

I believe @1zaman was looking for some evidence of the issues this change will help resolve.

Here's another one. I only have one copy of Things installed, but Things is confused because there is a symlink in ~/Applications and the actual copy in /latest. I don't want to know what happens if I click Delete "Extra" Copy.

Chen · Answer 58 · Mon Oct 05 2015 16:51:49 GMT+0800 (China Standard Time)

So when will this change happen?

Vítor Galvão · Answer 59 · Mon Oct 05 2015 20:26:24 GMT+0800 (China Standard Time)

When it’s done. There’s really no other answer, there.

Mathias Fredriksson · Answer 60 · Tue Oct 06 2015 23:11:41 GMT+0800 (China Standard Time)

I'm happy to see this change is finally happening, I've been using cask for a long time but the application management has always bothered me. Good work on this proposal/documentation of how it's going to be implemented, exactly what I've been hoping for! 👍

Kevin Suttle · Answer 61 · Wed Oct 07 2015 04:03:39 GMT+0800 (China Standard Time)

I'm interested in how this is going to work in a situation where a user has already installed an app from the Mac App Store. Will an overwrite prompt be given?

Mathias Fredriksson · Answer 62 · Wed Oct 07 2015 04:09:17 GMT+0800 (China Standard Time)

@kevinSuttle I believe that was addressed in the previous comments. Since the app was not installed beforehand, and the file exists, a warning/error will be given and no overwriting will take place.

Kevin Suttle · Answer 63 · Wed Oct 07 2015 04:15:28 GMT+0800 (China Standard Time)

Actually, I just tested out Skitch. There was no prompt at all.

Succeeded in installing skitch
Success: 55 Fail: 0

Brewfile
cask 'skitch'
using https://github.com/Homebrew/homebrew-bundle

Kevin Suttle · Answer 64 · Wed Oct 07 2015 04:24:33 GMT+0800 (China Standard Time)

Was able to reproduce with several App Store/free apps.

Evernote
Alfred
etc.

Josh Hagins · Answer 65 · Wed Oct 07 2015 04:26:06 GMT+0800 (China Standard Time)

@kevinSuttle There is no prompt. This is what happens if you already have an app installed:

$ brew cask install skitch
==> Downloading https://cdn1.evernote.com/skitch/mac/release/Skitch-2.7.8.zip
######################################################################## 100.0%
==> It seems there is already an App at '/Applications/Skitch.app'; not linking.
🍺  skitch staged at '/opt/homebrew-cask/Caskroom/skitch/2.7.8' (1863 files, 73M)

No failure is reported because nothing failed. The cask was downloaded, unpacked, and staged. We won't remove an existing non-cask installation.

Kevin Suttle · Answer 66 · Wed Oct 07 2015 04:27:34 GMT+0800 (China Standard Time)

I'm telling you it's happening on my system. Clean install of El Capitan.

I'm running brew bundle, btw. Maybe that's the difference.

Josh Hagins · Answer 67 · Wed Oct 07 2015 04:30:28 GMT+0800 (China Standard Time)

homebrew-bundle is an external plugin, and not officially supported by homebrew-cask. I can't speak for what it does or doesn't do.

Have you double-checked your /Applications/Skitch.app to make sure that it is, indeed, a symlink to the Cask installation? If not, then everything is working as intended.

Josh Hagins · Answer 68 · Wed Oct 07 2015 04:32:17 GMT+0800 (China Standard Time)

In any case, what you are reporting would be a bug with the existing behavior of homebrew-cask. If you determine that we are, in fact, overwriting app bundles with symlinks, please open a separate issue.

Kevin Suttle · Answer 69 · Wed Oct 07 2015 04:33:41 GMT+0800 (China Standard Time)

@jawshooah Help my understand why homebrew-bundle is an external plugin, if it's maintained by the homebrew org on GitHub. cc @MikeMcQuaid

In any event, what's the recommended way to batch install multiple casks?

Nathan Youngman · Answer 70 · Wed Oct 07 2015 04:35:25 GMT+0800 (China Standard Time)

@kevinSuttle See https://github.com/Homebrew/homebrew-bundle for issues with homebrew-bundle

Kevin Suttle · Answer 71 · Wed Oct 07 2015 04:41:57 GMT+0800 (China Standard Time)

Homebrew/homebrew-bundle#112

Josh Hagins · Answer 72 · Wed Oct 07 2015 04:42:25 GMT+0800 (China Standard Time)

@kevinSuttle We are not Homebrew. Homebrew and Homebrew-cask are maintained by completely separate teams. Therefore, homebrew-bundle is external to us.

If you want to install multiple casks, a simple shell script would suffice:

brew cask install \
  skitch \
  evernote \
  alfred \
  ...

Or perhaps:

brew cask install skitch
brew cask install evernote
brew cask install alfred
...

Or even:

casks=(
  skitch
  evernote
  alfred
  ...
)

for cask in "${casks[@]}"; do
  brew cask install "$cask"
done

Kevin Suttle · Answer 73 · Wed Oct 07 2015 04:46:19 GMT+0800 (China Standard Time)

Ok, that makes sense. Thanks @jawshooah.

Andrew Brown · Answer 74 · Wed Oct 07 2015 13:56:00 GMT+0800 (China Standard Time)

Why is it that after I brew cask install github-desktop that my GitHub Desktop.app shows in my ~/Applications folder as a symlinked app but shows in launchpad as an app that isn't symlinked?

Chen · Answer 75 · Wed Oct 07 2015 15:13:19 GMT+0800 (China Standard Time)

@satomusic Oh launchpad don't mind whether it is a symlink or a directory.

Andrew Brown · Answer 76 · Wed Oct 07 2015 15:26:00 GMT+0800 (China Standard Time)

Then can I take that directory from launchpad and apply that to my /Application folder?

Vítor Galvão · Answer 77 · Wed Oct 07 2015 18:18:30 GMT+0800 (China Standard Time)

@satomusic Your problem has nothing to do with this issue. Lets please stay on-topic, it is big enough as it is. Either way, it’ll be irrelevant once this is up, so we won’t work on it.

Furthermore, your last post is an OS X question, not a homebrew-cask question, so you should resort to a website with that specialty, like Ask Different from Stack Exchange.

Sean Zhu · Answer 78 · Wed Oct 07 2015 18:24:14 GMT+0800 (China Standard Time)

@vitorgalvao I recommend you post one more comment that sort of acts as an FAQ and then lock this thread. I think the FAQ should include answers to the following questions (my best-guess answers are in parens):

What is this thread about? (HBC will be changing the way it works, and this is an announcement)
Are these changes in effect yet? (No)
When will these changes be in effect? (Not sure, but not now)
Is there a timeline for that? (No)
Why not? (Because this project is run by volunteers)
How can people help out? (By contributing code)
What if someone has a question regarding this issue? (1. Check that it's actually related to this issue 2. Check that the answer isn't in this thread 3. If neither, open a new issue)

Vítor Galvão · Answer 79 · Wed Oct 07 2015 20:50:21 GMT+0800 (China Standard Time)

@szhu I like that, thank you. Nothing further to add to your FAQ, so we can leave it as is.

I didn’t want to close this at the start because I wanted people to find holes in it, but it seems like the idea as it stands is pretty solid and agreed upon, so yes, lets stop diverging further since comments have been becoming less relevant.

Vítor Galvão · Answer 80 · Fri May 20 2016 04:39:48 GMT+0800 (China Standard Time)

The first and most important part of this change is now very close to being implemented.

If you’re interested in helping test it, head over to #13966 and follow the instructions at the bottom.

Vítor Galvão · Answer 81 · Wed Jun 01 2016 01:19:06 GMT+0800 (China Standard Time)

Getting really close, now, so if you had some things you wanted to do that are dependent on this change, now is the time.

Vítor Galvão · Answer 82 · Wed Jun 01 2016 03:30:21 GMT+0800 (China Standard Time)

And merged.

There are still a few things that need to be done in order to close this issue, but the biggest user-facing change is now present.

Markus Reiter · Answer 83 · Fri Oct 07 2016 21:00:54 GMT+0800 (China Standard Time)

@vitorgalvao, what exactly still needs to happen for this to be closed? The only thing I'm seeing that is not yet implemented is :required_location and --force-appdir, which kind of contradict themselves.

Why would you be able to override a required location? That would mean the location isn't actually required.

Vítor Galvão · Answer 84 · Fri Oct 07 2016 21:11:01 GMT+0800 (China Standard Time)

@reitermarkus Versioned subdirectories of the Caskroom won’t exist anymore is also yet to be implemented.

As for --force-appdir, I do not remember my reasoning there, and I guess it could be stricken (not required_localtion:, though, naturally).

Josh Hagins · Answer 85 · Fri Oct 07 2016 22:00:15 GMT+0800 (China Standard Time)

I suggested --force-appdir in #13201 (comment), mostly because I think the user should have final say in where their stuff is installed, and may have good reason for overriding the developer's stated location requirements.

We should absolutely print a warning when the user overrides required_location though, so they have no grounds to blame us for a broken installation.

Markus Reiter · Answer 86 · Fri Feb 15 2019 23:45:47 GMT+0800 (China Standard Time)

@vitorgalvao, I think we can close this in favour of #12858.

Vítor Galvão · Answer 87 · Sat Feb 16 2019 00:23:03 GMT+0800 (China Standard Time)

@reitermarkus Agreed. At this point other issues should take care of whatever is missing here.