Installing without Xcode using OSX GCC Installer
beerlington opened this issue · comments
I've been trying to ditch Xcode for the OSX GCC Installer (https://github.com/kennethreitz/osx-gcc-installer) and rb-fsevent is the only standing between me and victory. I uninstalled Xcode and then tried installing rb-fsevent and got the following error. Is this a lost cause or is there potentially a workaround to get it to compile without Xcode?
Installing rb-fsevent (0.4.3) with native extensions /Users/portia/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/site_ruby/1.9.1/rubygems/installer.rb:551:in `rescue in block in build_extensions': ERROR: Failed to build gem native extension. (Gem::Installer::ExtensionBuildError)
/Users/portia/.rvm/rubies/ruby-1.9.2-p290/bin/ruby extconf.rb
creating Makefile
CFLAGS='-isysroot /Developer/SDKs/MacOSX10.7.sdk -mmacosx-version-min=10.7 -mdynamic-no-pic -std=gnu99 -Os -pipe -Wmissing-prototypes -Wreturn-type -Wmissing-braces -Wparentheses -Wswitch -Wunused-function -Wunused-label -Wunused-parameter -Wunused-variable -Wunused-value -Wuninitialized -Wunknown-pragmas -Wshadow -Wfour-char-constants -Wsign-compare -Wnewline-eof -Wconversion -Wshorten-64-to-32 -Wglobal-constructors -pedantic' /usr/bin/clang -isysroot /Developer/SDKs/MacOSX10.7.sdk -mmacosx-version-min=10.7 -mdynamic-no-pic -std=gnu99 -dead_strip -framework CoreServices -o '/Users/portia/.rvm/gems/ruby-1.9.2-p290@sentalis4/gems/rb-fsevent-0.4.3/bin/fsevent_watch' fsevent/fsevent_watch.c
fsevent/fsevent_watch.c:1:10: fatal error: 'stdio.h' file not found
#include
^
1 error generated.
extconf.rb:59:in `': Compilation of fsevent_watch failed (see README) (RuntimeError)
considering that I just ditched the occasionally error prone and hackish extconf.rb build-command-as-we-go method with a rakefile that outright calls xcodebuild on an xcode project... yeah, probably.
in your case, you don't have the lion SDK installed and so it doesn't find those header files. you could switch isysroot to / and it would build i think... the xcode project does something similar by declaring the target SDK as "current macos version" and deployment target to "compiler default". However, the xcode project will require xcodebuild as a commandline tool for taking the xcode project settings and turning them into build commands to be run. i'm not sure if you could install an xcodebuild commandline utility with the rest of xcode.
the pmdoc for the lion osx-gcc-installer suggests that it installs various things that might be what contains xcodebuild for all i know...
if you DO have xcodebuild, then the xcodebuild branch of rb-fsevent should work perfectly.
It appears that I do have xcodebuild at /usr/bin/xcodebuild, but I don't know if that's just leftover from Xcode after I removed it or if it came with the GCC installer. I tried building from your xcodebuild branch and it complained about /Developer not being found. I used xcode-select to switch it the xcode path to / and tried installing rb-fsevent again.
Bundler is currently hung at Using rb-fsevent (0.4.3) from https://github.com/thibaudgg/rb-fsevent.git (at xcodebuild) with native extensions with bash running at about 40% cpu. No errors, but it seems like it's stuck doing something.
It looks like the osx-gcc-installer package includes DeveloperToolsSystemSupport, which does include xcodebuild. I looked at the package payload manually... no way am I installing it and setting up OSX in a virtual machine is painful to say the least.
To get more info, hop into the ext directory and just run "rake".
or, alternatively, go to ext/fsevent_watch and run "xcodebuild build -xcconfig ../rb-fsevent.xcconfig"
I'd like for it to work with this hack osx gcc installer thing, but it does seem that it's... a hack. It guts the xcode metapackage and selectively installs a small portion, some functionality of which is dependent on things that do NOT get installed. When a subset of homebrew and macvim and so forth won't work with it, it doesn't exactly motivate.
I guess it's time to see if virtualbox fixed their EFI implementation. -_-
Let me know if you get any useful output from running xcodebuild manually. I'd be butthurt if I had to abandon writing and testing fsevent_watch seperately as its own xcode project and merging changes in via git-subtree. Before lion it was essentially a given that you had your OS install disk and that xcode was already on it, with updates rarely being more than a few hundred megs... But now with the app store distribution system being such a focus, xcode4.1 is a handful of gigs to obtain. Sigh.
You know... If I could wrap my head around runtime feature detection for this it'd be possible and potentially preferable to bundle a pre-compiled fsevent_watch. That'd remove the need to compile anything at all. Another potential solution would be to only compile fsevent_watch on gem install if the executable isn't found in your PATH, allowing for a pkg install or perhaps a homebrew formula, etc.
I already started splitting out fsevent_watch into its own project since, as it exists now, there's nothing that ties it to ruby. Communication is done via pipe to a completely isolated subprocess. It'd be relatively simple to wrap fsevent_watch in python, nu, perl, bash, etc.
Something to ponder I guess.
Just tried running xcodebuild manually and got the same result as before. It just hands there with no output until I kill it.
I stumbled across autotest-fsevent and was able to install it without a problem. Not sure how similar this is to rb-fsevent, but I thought I would point that out.
...really? That appears to do something similar to the currently released version of rb-fsevent and expects a specific SDK under /Developer, etc. I'll have to set up a VM and install the osx-gcc-installer to really understand that.
I added runtime version detection to fsevent_watch, so a binary compiled on lion should work on snow leopard and leopard, throwing appropriate errors if told to use features unavailable on that OS version. Would you be willing to try an experimental build of rb-fsevent that makes use of a pre-built binary if I made one? I'd certainly appreciate feedback on whether or not it's a good idea.
Extracted fsevent_watch repo: https://github.com/ttilley/fsevent_watch
Runtime detection commit: 1100df6
...I still want to get it working with osx-gcc-installer, but while I can see myself testing in xcode 3.2.6 to 4.2 beta, I worry about osx-gcc-installer breaking something. Brave souls like yourself can hit those problems for a while so that they're ironed out first. ;)
Yea I'd definitely be willing to try out a pre compiled version. Just let me know what to do and I'll test it out.
Sorry for the delay. This gem contains a pre-compiled fsevent_watch binary and removes the section that attempts to compile on install: https://github.com/ttilley/rb-fsevent/blob/pre-compiled-gem-one-off/pkg/rb-fsevent-0.4.3.1pre1.gem?raw=true
Let me know how that works out.
Work great for me, installed via bundler with:
gem 'rb-fsevent', :git => 'git://github.com/ttilley/rb-fsevent.git', :branch => 'pre-compiled-gem-one-off'Worked perfect for me too with @thibaudgg's snippet. Are you going to leave this branch or create a new project for it?
I want to make fsevent_watch a real standalone utility before pulling in any more changes to rb-fsevent. That means making good on the "for future release" promises in the current rb-fsevent readme as well as the standard necessity of having usable --help and --version output (damn, guess I might as well switch to getopt now). I also plan on adding both 'English' and TNetstrings output formats... English for debugging and TNetstrings because even a bash script implementation of a parser would be sane and readable (declaring size of message also removes the need to declare something as a delimiter char so no more abuse of : and \n).
I will not be removing that branch if anyone is making use of it, and you are. You can depend on it not going away, potentially ever. Branches are free, after all. ;)
I'm not going to be doing a proper release until after fsevent_watch is solid and getting feedback from @thibaudgg.
Sent from my iPhone
On Aug 16, 2011, at 7:51 AM, beerlingtonreply@reply.github.com wrote:
Worked perfect for me too with @thibaudgg's snippet. Are you going to leave this branch or create a new project for it?
Reply to this email directly or view it on GitHub:
#20 (comment)
Sounds good to me, thanks @ttilley
Development temporarily stalled by round trip time with apple dev support and the interesting finding that file-level event data is provided on lion even when you don't ask for it... just not the file-level path. So you can apparently receive events with a directory as the path, yet have ItemIsFile set in the flags. To say that this is confusing is a bit of an understatement. In order to properly process an event, you need to know what flags were set when creating the event stream (which isn't an issue, but I want to make sure I don't do something retarded by accident, so I'm waiting for more information from apple dev support).
I'm not ignoring this bug, just waiting to release everything at once. Apple might take some time to get back to me (support said they'd wait a week before poking the engineers again) and I don't see a point in doing a major release until that large, massive, long-standing bug is resolved.
...I am debating updating the interim stand-in repo.
+1 I ran into this today, any update? Given that Xcode 4.2 has removed gcc-4.2 which is needed for compiling Ruby < 1.9.3, we've changed our developer workstation recommendations away from installing Xcode to using the osx-gcc-installer pkg.
If you take a quick look at issue #10 (just the last few posts in the march20->now chaos) you'll see why I've let this project slide... It's fairly discouraging. That's no excuse though!
I'll push out a new gem later tonight as another pre version, and it'll have fsevent_watch already compiled (AND code signed with my mac developer certificate) as a standalone binary for 64bit+32bit intel on 10.6+ (there IS compat code for 10.5, but my please-let-this-work fix attempt for #10 depends on APIs new to 10.6, so I'm not quite sure how to go about handling that).
As an aside, the standalone fsevent_watch source is here (...as an xcode project, sorry): https://github.com/ttilley/fsevent_watch
The issue #10 workaround is documented in a comment here (ifdef'd to 10.6+): https://github.com/ttilley/fsevent_watch/blob/master/fsevent_watch/main.c#L92
@cgriego can you install pre4 and let me know how that works out for you? you can also verify the integrity and signature of the pre-compiled binary for trust purposes:
[13:17:34][bin]$ codesign --verify --verbose ./fsevent_watch
./fsevent_watch: valid on disk
./fsevent_watch: satisfies its Designated Requirement
[13:17:44][bin]$ codesign --display --verbose=9 ./fsevent_watch
Executable=/Users/ttilley/.rvm/gems/ruby-1.9.3-head/gems/rb-fsevent-0.9.0.pre4/bin/fsevent_watch
Identifier=com.teaspoonofinsanity.fsevent_watch
Format=Mach-O universal (i386 x86_64)
CodeDirectory v=20100 size=325 flags=0x0(none) hashes=7+5 location=embedded
Hash type=sha1 size=20
CDHash=045a811866598914147d847b0d8c6eb7bc4035d7
Signature size=4316
Authority=Mac Developer: Travis Tilley (YZ6RPE8C7C)
Authority=Apple Worldwide Developer Relations Certification Authority
Authority=Apple Root CA
Signed Time=Nov 3, 2011 1:02:18 PM
Info.plist entries=2
Sealed Resources=none
Internal requirements count=1 size=148
Just ran into this and installing pre4 worked for me. Thank you!
@ttilley The gem installed fine but when I used it with guard then guard would exit around the time it would start listening to events without any error. I don't know if that's related to the compiled code or an API change.
This is the results of codesign:
[ruby-1.9.2-p290@rb-fsevent-test] $ codesign --verify --verbose ./fsevent_watch
./fsevent_watch: valid on disk
./fsevent_watch: satisfies its Designated Requirement
[ruby-1.9.2-p290@rb-fsevent-test] $ codesign --display --verbose=9 ./fsevent_watch
Executable=/Users/dev/.rvm/gems/ruby-1.9.2-p290@rb-fsevent-test/gems/rb-fsevent-0.9.0.pre4/bin/fsevent_watch
Identifier=com.teaspoonofinsanity.fsevent_watch
Format=Mach-O universal (i386 x86_64)
CodeDirectory v=20100 size=325 flags=0x0(none) hashes=7+5 location=embedded
Hash type=sha1 size=20
CDHash=045a811866598914147d847b0d8c6eb7bc4035d7
Signature size=4316
Authority=Mac Developer: Travis Tilley (YZ6RPE8C7C)
Authority=Apple Worldwide Developer Relations Certification Authority
Authority=Apple Root CA
Signed Time=Nov 3, 2011 12:02:18 PM
Info.plist entries=2
@cgriego can you try running with debug+verbose?
export RUBYOPT="${RUBYOPT} -w -v --debug"
guard
@ttilley Sorry it took so long to get back, here is what I see right before guard exits.
/Users/dev/.rvm/gems/ruby-1.9.2-p290@rb-fsevent-test/gems/guard-0.8.8/lib/guard/interactor.rb:30: warning: instance variable @thread not initialized
Exception `EOFError' at /Users/dev/.rvm/gems/ruby-1.9.2-p290@rb-fsevent-test/gems/rb-fsevent-0.9.0.pre4/lib/rb-fsevent/fsevent.rb:35 - end of file reached
+1 for pre4, works with guard for me. thanks!
Same issue than @cgriego here, pre4 installs just fine but guard stop/crash at the end of its setup phase ...
...can you call fsevent_watch directly and have it work?
hmm actually it does not seems to work either but I am not familliar with the rb-fsevent command line utility:
➜ bin /usr/lib/ruby/user-gems/1.8/gems/rb-fsevent-0.9.0.pre4/bin/fsevent_watch
[1] 64641 killed /usr/lib/ruby/user-gems/1.8/gems/rb-fsevent-0.9.0.pre4/bin/fsevent_watch
I think I should start using a virtual machine for testing. The fsevent_watch binary works just fine on my previous lion install, but I'm on a fresh install of lion as of a week or so ago and that same binary isn't working.
I think I figured it out and lets just say I'm retarded and leave it at that... Give me a few to push up a new build.
\o/ I confirm, it compiles and fsevent_watch is working and gets events from the FS
Guard is also working perfectly.
Thanks ! A fellow developer had the same issue with pre4, I'll ask him to report if it does not work with pre5.
pre5 works for me, everything back to normal. thanks!
@thibaudgg - just a heads up. despite it working for both of us, there was a perfectly valid reason for pre4 not to work for what is probably most people on lion. i'm assuming you also had the apple dev center certificates set up... it was a code signing error where i used the damn dev key instead of production key to sign with. Signed code without a trusted certificate is automatically killed (only on lion) before you can even attach a debugger to see what might be killing it. Oops. -_-
I figure it's worth it to verify whether it's a public release or self-made build (which you can still do from within the gem installed source if you have xcode), but I didn't realize the security system could even act on the binary without an embedded entitlement description... Sans entitlement, signed code is given all the same rights as unsigned code. Figured it would behave the same as a GPG signature or what have you.
Starting with pre5 there's also a significantly more detailed Info.plist embedded directly into the binary with detailed information regarding that particular build (inside __TEXT __info_plist if you want to poke at it with otool or whatever):
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>CFBundleIdentifier</key>
<string>com.teaspoonofinsanity.fsevent_watch</string>
<key>CFBundleExecutable</key>
<string>fsevent_watch</string>
<key>CFBundleName</key>
<string>fsevent_watch</string>
<key>LSMinimumSystemVersion</key>
<string>10.6</string>
<key>CFBundleVersion</key>
<string>0.1.1</string>
<key>FSEWOptimizationLevel</key>
<string>0</string>
<key>FSEWVersionInfoBuilder</key>
<string>ttilley</string>
<key>FSEWVersionInfoString</key>
<string>"@(#)PROGRAM:fsevent_watch PROJECT:fsevent_watch-0.1.1"</string>
<key>BuildMachineOSBuild</key>
<string>11C74</string>
<key>BuildMachineOSVersion</key>
<string>1072</string>
<key>DTCompiler</key>
<string>com.apple.compilers.llvm.clang.1_0</string>
<key>DTPlatformBuild</key>
<string>4D502</string>
<key>DTSDKBuild</key>
<string>11C63</string>
<key>DTSDKName</key>
<string>macosx10.7</string>
<key>DTXcode</key>
<string>0421</string>
<key>DTXcodeBuild</key>
<string>4D502</string>
</dict>
</plist>...I'm not sure if I even mentioned it anywhere, but the fsevent_watch binary also has a --help option, which I figure will be useful for direct use rather than via rb-fsevent:
$ ./bin/fsevent_watch --help
fsevent_watch 0.1.1
Compiled Dec 29 2011 00:20:45
A flexible command-line interface for the FSEvents API
Usage: fsevent_watch [OPTIONS]... [PATHS]...
-h, --help you're looking at it
-V, --version print version number and exit
-s, --since-when=EventID fire historical events since ID
-l, --latency=seconds latency period (default='0.5')
-n, --no-defer enable no-defer latency modifier
-r, --watch-root watch for when the root path has changed
-F, --file-events provide file level event data
-f, --format=name output format (classic, niw,
tnetstring, otnetstring)
Not everything supported by the binary is supported in the ruby code wrapped around it.
@ttilley ok great. Thanks for the heads up!
FYI pre5 works for me on both a Lion machine with Xcode and a Lion machine with GCC Installer.
since sooooo many things just don't work with osx-gcc-installer, nuke it with fire and just get rid of it. seriously.
new official solution is "Command Line Tools for Xcode", distributed by apple as part of the solution for... erm. newer xcode stuff is under NDA still, so just read the release notes if you have access to them. the tools are available now, however.
read here for more info: http://kennethreitz.com/xcode-gcc-and-homebrew.html
since sooooo many things just don't work with osx-gcc-installer, nuke it with fire and just get rid of it. seriously.
That's a bit harsh on the project. It was a godsend for most people.
I'm glad Apple finally did something about the whole situation.
Well, i have upgraded to Xcode4.3 yesterday, and installed the CLI tools as well.
Now i'm trying to execute bundler for octopress, which requires fsevent 0.4.3.1, and i get the same error again. What can i do?
--Well, nevermind. #26 helped.
@simonszu +1. Thanks for the edit and link. That got me going as well, doing the same thing.
@simonszu +1 worked perfect for me. Thanks!