Xcode 6.4 builds deprecated in Travis CI
radarhere opened this issue · comments
As seen at https://travis-ci.org/python-pillow/pillow-wheels/jobs/428587685 -
Running builds with Xcode 6.4 in Travis CI is deprecated and will be removed in January 2019. If Xcode 6.4 is critical to your builds, please contact our support team at support@travis-ci.com to discuss options.
Given #84 and python-pillow/Pillow#3068, this bears discussion.
If we upgrade from Xcode 6.4, then we lose OS X 10.10, as noted #84 (comment)
The macOS version in that grid is the macOS version needed to run the xcode binary, but the xcodes can build for macOS versions prior to that, using the MACOSX_DEPLOYMENT_TARGET environment variable. That gets picked up from the Python build configuration, for Python extensions.
OS X release support
- OS X 10.10 Yosemite was released in 2014.
- The current macOS 10.13 High Sierra was released in 2017.
- The next macOS 10.14 Mojave is due out this (northern hemisphere) autumn.
Wikipedia says of OS X 10.10 Yosemite:
Unsupported as of August 2017, iTunes support terminated as well.
Pillow support
Pillow's platform support says:
In general, we aim to support all current versions of Linux, macOS, and Windows.
Pillow downloads
Here are Pillow's macOS downloads from PyPI for the last 30 days:
| system_name | distro_version | percent | download_count |
|---|---|---|---|
| Darwin | '10.5' | 0.00% | 6 |
| Darwin | '10.6' | 0.01% | 14 |
| Darwin | '10.7' | 0.01% | 9 |
| Darwin | '10.8' | 0.01% | 17 |
| Darwin | '10.9' | 0.14% | 192 |
| Darwin | '10.10' | 0.93% | 1275 |
| Darwin | '10.11' | 3.62% | 4950 |
| Darwin | '10.12' | 12.43% | 17010 |
| Darwin | '10.13' | 79.50% | 108815 |
| Darwin | '10.14' | 3.07% | 4197 |
| Darwin | '10.15' | 0.00% | 4 |
| Darwin | '11.3' | 0.01% | 8 |
| Darwin | None | 0.28% | 384 |
Suggestion
I tried MACOSX_DEPLOYMENT_TARGET last time round but there was a problem due to missing webp, which I think was the reason for upgrading to Xcode 9.2 in the first place: python-pillow/Pillow#3068 (comment). Shall we give that another shot?
If that doesn't work out, I suggest we keep things as they are for the next two Pillow releases (2018-10-01 and 2019-01-01) and then in January 2019 upgrade from Xcode 6.4 (supporting 10.10+) to the next available Xcode 7.4 (supporting 10.11+).
After all, python-pillow/Pillow#3068 was about losing 10.11 by upgrading from Xcode 8 (10.11+) to 9.2 (10.12+).
I have created PR #97 to resolve this, upgrading Xcode to 9.4, with a deployment target of 10.10. To fix webp, I added --disable-sse4.1 as an argument.
An upside of disabling SSE4.1: we can build wheels that can be used on OS X 10.10+, rather than 10.11+.
Is it so that building wheels with SSE4.1 disabled means that Pillow's Mac wheels use WebP without SSE4.1?
Are there any other downsides?
Intel SSE4.1 is the first release of the SSE4 instruction set, first available in Intel Penryn processors.
https://en.wikipedia.org/wiki/SSE4
MacBook Pro, MacBook, iMac and MacBook Air first had Penryn processors in 2008, and Mac mini and Mac mini Server in 2009.
https://en.wikipedia.org/wiki/List_of_Macintosh_models_grouped_by_CPU_type#Penryn
WebP makes quite some use of SSE4.1.
https://chromium.googlesource.com/webm/libwebp/+/master/ChangeLog
Intel believes that SSE4 offers the greatest change to the x86 instruction set in five years and allows the Penryn clock to run at higher frequencies than its Core 2 parents but within the same cool thermal envelope. While this only benefits applications that are able to use SSE4 (like media encoding), the speed improvements are reported to be in the order of 40%.
Both OS X 10.10 and 10.11 are unsupported by Apple. 10.12 is still supported.
Downloads by OS for the same period as above:
| system_name | percent | download_count |
|---|---|---|
| Linux | 86.86% | 2,202,141 |
| Windows | 9.04% | 229,126 |
| Darwin | 4.06% | 103,051 |
| FreeBSD | 0.03% | 674 |
| CYGWIN_NT-10.0 | 0.01% | 150 |
| CYGWIN_NT-6.1 | 0.00% | 54 |
| OpenBSD | 0.00% | 26 |
| SunOS | 0.00% | 19 |
| CYGWIN_NT-6.1-WOW | 0.00% | 17 |
| CYGWIN_NT-10.0-WOW | 0.00% | 15 |
| Total | 2,535,273 |
- 10.10 accounts for 0.93% of 100k Mac downloads
- Mac accounts for 4.06% of 2.5m downloads
- 10.10 accounts for 0.05% of 2.5m downloads
If there are significant performance benefits to building with SSE4.1, I'm not sure if it's worth sacrificing that to support a little-used, obsolete OS version, which we've not been supporting anyway and have had no requests for (unlike 10.11).
People on 10.10 can always build from source, it's not too difficult on Mac.
Thanks for that info.
Do the compile conditions for dependencies in a wheel affect the end user? Renaming a Pillow .whl to .zip and looking inside, I find libwebp .dylib files, so I’m inclined to believe the answer is yes.
However, I find that Xcode 7.3 deploying for 10.11 also experiences the WebP issue - so we can’t simply remove 10.10 in order to keep SSE4.1.
Happy for there to be another solution to this, I just don’t know what it is.
Xcode 8.3 with no specific deploy target passes, but that's no better as it'd mean dropping 10.11.
https://docs.travis-ci.com/user/languages/objective-c/#Supported-Xcode-versions
If nothing else comes up, let's just go with #97 closer to the xcode6.4 removal time.
One question: how does upstream multibuild deal with WebP and Xcode 6.4? Are they not build together?
I have created another PR, #104, to resolve this.
Because I'm concerned about this potentially breaking on the very day that we run the next release, and also to try and avoid dealing with this at the last minute - does anyone have thoughts on #104?
By the way, updated macOS download stats:
| system_name | distro_version | percent | download_count |
|---|---|---|---|
| Darwin | '10.5' | 0.00% | 5 |
| Darwin | '10.6' | 0.01% | 12 |
| Darwin | '10.7' | 0.02% | 41 |
| Darwin | '10.8' | 0.00% | 8 |
| Darwin | '10.9' | 0.10% | 172 |
| Darwin | '10.10' | 0.58% | 1013 |
| Darwin | '10.11' | 3.44% | 5992 |
| Darwin | '10.12' | 9.55% | 16649 |
| Darwin | '10.13' | 35.66% | 62179 |
| Darwin | '10.14' | 50.38% | 87852 |
| Darwin | '10.15' | 0.07% | 119 |
| Darwin | '11.2' | 0.00% | 1 |
| Darwin | '11.3' | 0.00% | 3 |
| Darwin | None | 0.18% | 321 |
I prefer #104, yes
Thanks, I've merged #104.