override start_script/extended_start_script with a cuttlefish aware version
marianoguerra opened this issue · comments
hi, I was adapting extended_bin to be aware of cuttlefish and it makes sense to have it in this plugin, look in the overlay section if extended_start_script is set and override the start_script on bin with the cuttlefish aware version.
this would mean that this task should run as a post hook of release as mentioned in #1
That would be great, I'd like to make using cuttlefish with rebar3/relx as simple as possible and with as little manual intervention with overlays and such as possible.
here is the latest version I'm using, it's in a template since some values are project dependent
https://github.com/marianoguerra/rebar3_template_riak_core/blob/master/rebar3_riak_core_extended_bin
how do you think we can handle this?
ignore my last comment about the template since it's just copied, see here: https://github.com/marianoguerra/rebar3_template_riak_core/blob/master/rebar3_riak_core_extended_bin
it's used as a template in the overlay here: https://github.com/marianoguerra/rebar3_template_riak_core/blob/master/rebar3_riak_core_rebar.config.tpl#L35
but I think that part can be in this plugin?
Yea, that'd be cool as part of the plugin.
@marianoguerra I've completely redone the plugin, it handles everything now, let me know if it works for you.
hi, I'm trying to make the little riak core code work against this version but I'm having (at least) one problem :D
releases/VSN/vm.args is not there, but releases/VSN/vm.args.orig is, do you know why/who set's it to vm.args.orig?
I make it work, here is the commit:
marianoguerra/rebar3_template_riak_core@37ea0b8
notes:
- I had to change vm.args to vm.args.orig as I said in the comment above, I don't know if the .orig is a new intended behaviour or something that changed by accident
- all the previous "rebar3 release" are now "rebar3 cuttlefish release"
- I no longer use my custom extended_bin (which is a good thing :)
- riak_ensemble is a dependency that has a nif that used to compile fine, now I had to add an override to make it compile with the latest rebar3 build, don't know if it's new intended behaviour or something that broke
I'll be making it work with rebar3 release in the next release.
If you update your rebar3 to master you shouldn't get the vm.args.orig file anymore with that.
You should have always had to have an override with rebar3, unless something changed with riak_ensemble. Also, I forked it https://github.com/tsloughter/riak_ensemble and got rid of that clock because I'm only supporting 18+ and published to hex https://hex.pm/packages/riak_ensemble
I'm on latest master and still get the vm.args.orig,
you can replicate it with:
mkdir -p ~/.config/rebar3/templates
git clone https://github.com/marianoguerra/rebar3_template_riak_core.git ~/.config/rebar3/templates/rebar3_riak_core
rebar3 new rebar3_riak_core ricor
cd ricor
make release
ls _build/default/rel/ricor/releases/0.1.0
I get:
ricor.boot ricor.rel ricor.script start_clean.boot sys.config.orig vm.args.orig
rebar version:
$ rebar3 report
Rebar3 report
version alpha-4+build.3305.refe3437c1
generated at 2016-02-21T15:56:49+00:00
=================
Please submit this along with your issue at https://github.com/rebar/rebar3/issues (and feel free to edit out private information, if any)
-----------------
Task:
Entered as:
-----------------
Operating System: x86_64-pc-linux-gnu
ERTS: Erlang/OTP 18 [erts-7.0] [source] [64-bit] [smp:4:4] [async-threads:0] [kernel-poll:false]
Root Directory: /usr/lib/erlang
Library directory: /usr/lib/erlang/lib
-----------------
Loaded Applications:
bbmustache: 1.0.4
certifi: 0.3.0
cf: 0.2.1
common_test: 1.11
compiler: 6.0
crypto: 3.6
cth_readable: 1.2.0
dialyzer: 2.8
edoc: 0.7.17
erlware_commons: 0.19.0
eunit: 2.2.10
eunit_formatters: 0.3.1
getopt: 0.8.2
inets: 6.0
kernel: 4.0
providers: 1.6.0
public_key: 1.0
relx: 3.17.0
sasl: 2.5
snmp: 5.2
ssl_verify_hostname: 1.0.5
stdlib: 2.5
syntax_tools: 1.7
tools: 2.8
-----------------
Escript path: /home/mariano/bin/rebar3
Providers:
app_discovery as clean compile compile cover ct deps dialyzer do edoc escriptize eunit help install install_deps list lock new path pkgs release release relup report run shell tar tar tree unlock update upgrade upgrade upgrade version xref
Oh, I think I put the overrides in the wrong order :).
Issue is you shouldn't have {sys_config, "config/sys.config"} or {vm_args, "config/vm.args"} but it won't matter with my latest fix because the cuttlefish release provider forces them to be false even if you have them set.
I tried it again, build seems to be working, I have some questions:
- vm.args is generated from config, so I don't need mine, is that right?
- advanced.config seems to be included in the generated app.config, isn't it?
also, when trying to start it I'm getting a crash that looks like bad command line argument parsing, I get this error:
$ ./_build/default/rel/ricor/bin/ricor console
Exec: /usr/lib/erlang/erts-7.0/bin/erlexec -boot /home/mariano/tmp/ricor/_build/default/rel/ricor/releases/0.1.0/ricor -boot_var ERTS_LIB_DIR /usr/lib/erlang/erts-7.0/../lib -mode embedded -config /home/mariano/tmp/ricor/_build/default/rel/ricor/generated/app.2016.02.23.08.55.22.config -args_file /home/mariano/tmp/ricor/_build/default/rel/ricor/generated/vm.2016.02.23.08.55.22.args -vm_args /home/mariano/tmp/ricor/_build/default/rel/ricor/generated/vm.2016.02.23.08.55.22.args -- console
Root: /home/mariano/tmp/ricor/_build/default/rel/ricor
{error_logger,{{2016,2,23},{8,55,23}},"Error in process ~p with exit value:~n~p~n",[<0.510.0>,{{case_clause,{'EXIT',{function_clause,[{application_controller,'-check_conf/0-fun-0-',[["/home/mariano/tmp/ricor/_build/default/rel/ricor/generated/app.2016.02.23.08.55.22.config","ERL_FULLSWEEP_AFTER","0"],[]],[{file,"application_controller.erl"},{line,1809}]},{lists,foldl,3,[{file,"lists.erl"},{line,1262}]},{application_controller,check_conf,0,[{file,"application_controller.erl"},{line,1808}]},{application_controller,init,2,[{file,"application_controller.erl"},{line,485}]}]}}},[{application_controller,init,2,[{file,"application_controller.erl"},{line,485}]}]}]}
{"could not start kernel pid",application_controller,"{{case_clause,{'EXIT',{function_clause,[{application_controller,'-check_conf/0-fun-0-',[[\"/home/mariano/tmp/ricor/_build/default/rel/ricor/generated/app.2016.02.23.08.55.22.config\",\"ERL_FULLSWEEP_AFTER\",\"0\"],[]],[{file,\"application_controller.erl\"},{line,1809}]},{lists,foldl,3,[{file,\"lists.erl\"},{line,1262}]},{application_controller,check_conf,0,[{file,\"application_controller.erl\"},{line,1808}]},{application_controller,init,2,[{file,\"application_controller.erl\"},{line,485}]}]}}},[{application_controller,init,2,[{file,\"application_controller.erl\"},{line,485}]}]}"}
Crash dump is being written to: -env...done
could not start kernel pid (application_controller) ({{case_clause,{'EXIT',{function_clause,[{application_controller,'-check_conf/0-fun-0-',[["/home/mariano/tmp/ricor/_build/default/rel/ricor/genera
as you can see (or not in that mess :D) it fails because it calls check_conf with this list of config files:
[["/home/mariano/tmp/ricor/_build/default/rel/ricor/generated/app.2016.02.23.08.55.22.config","ERL_FULLSWEEP_AFTER","0"],
clearly "ERL_FULLSWEEP_AFTER" and "0" aren't config files, you can find them in the generated vm.args file, which has this content:
+P 256000
-env ERL_MAX_ETS_TABLES 256000
-env ERL_CRASH_DUMP
-env ERL_FULLSWEEP_AFTER 0
-env ERL_MAX_PORTS 262144
+A 64
-setcookie erlang
-name ricor@127.0.0.1
+K true
+W w
-smp enable
another thing that points at bad command line argument parsing is this line:
Crash dump is being written to: -env...done
the option to dump the files is "-env" which clearly is a case of flags as values, and now that I write this I notice that
-env ERL_CRASH_DUMP
doesn't have a value
as an aside, during schema discovery, do you avoid files like this?
./_build/default/lib/cuttlefish/test/bad_erlang.schema
the ERL_CRASH_DUMP flag comes from _build/default/plugins/cuttlefish/priv/erlang_vm.schema which has this configuration:
{mapping, "erlang.crash_dump", "vm_args.-env ERL_CRASH_DUMP", [
{default, ""},
{datatype, file},
hidden
]}.
the default is the empty string, which when inserted in vm.args it messes the command line argument order (I think)
the solution I think would be to either add a better default to the crash_dump option or more generally, detect options that require a value here https://github.com/basho/cuttlefish/blob/develop/src/cuttlefish_escript.erl#L394 and remove them if the value is the empty string.
what do you think?
PS: adding erlang.crash_dump = erlang_crash.dump to my etc/ricor.conf made it boot
(I'm reusing this issue because it's all part of the same, if you prefer different issues for each let me know)
when running:
rebar3 as dev1 release
I get:
===> Unable to copy from /home/mariano/tmp/ricor/_build/dev1/bin/cuttlefish to /home/mariano/tmp/ricor/_build/dev1/rel/ricor/bin/cuttlefish because of {copy_failed, enoent}
This is what was being talked about yesterday on IRC
should I change it to something else?
should cuttlefish be built and copied for each profile I have?
in case we want to keep the profile on the operations, isn't it safer to copy it from _build/{{profile}}/lib/cuttlefish/cuttlefish?
Ah crap. Yea, I think we need to change this back. @Licenser was right with his patch until we moved to project_plugins which are going to always be in default :)
@tsloughter what's your take on the cuttlefish empty default? should I open an issue on cuttlefish? maybe there's a temporary fix until we get a cuttlefish release without that problem?
I don't know. I just put an entry. But would be nice if the default was erl_crash.dump
put an entry where? manually in the config? or in your plugin?
just asking to know what to do on my projects and templates
In vars.config for overlay_vars, at least for me erlang_vm.schema has {default, "{{crash_dump}}"}
great! thanks for the tip.
when #3 (comment) is solved we can close this issue and #7
@marianoguerra I think it should be resolved with my most recent commit to master.
doing the _checkouts trick and replacing those 3 args for a list seems to make the plugin work fully for me.
latest master works :)