Create a .dat file from an AMPL object after data has been loaded
pjcpjc opened this issue · comments
There is a lot of value in being able to reproduce an amplpy result outside of Python. To do this we need the ability to create a .mod and .dat file consistent with whatever model was being solved from within the Python script. Copying over the .mod string from Python is easy enough, but the .dat file cannot be easily known by a Python programmer if the data was being loaded into the AMPL object via setData calls.
fdabrandao developed AMPL._exportData(datfile) as a pre-release testable feature for this issue available via v0.5.0b1. @wang3160 is testing this feature.
Issue is closeable when there is an AMPL.exportData() on the develop branch. (I.e. I don't care when you upgrade pypi).
@wang3160 were able to reproduce create .dat and .mod files that reproduced Python behavior for diet, metrorail and netflow examples.
However, diet didn't work without manually editing the inf out of the diet.dat file and replacing it with a large number. AMPL._exportData(datfile) should consistently create .dat files that can be solved by AMPL tools, to include data that involves float('inf') which is part of Python. If you need to replace float('inf') with a large number (for example any number bigger than 1e30 should work for Gurobi) during AMPL._exportData(datfile) that's fine.
See the attached notebook for how the diet.dat, diet.mod files can be made.
Thanks for testing this! In a30988b, there is a workaround for this problem with float('inf'). Nevertheless, I have identified other similar issues, which I should only be able to fix during the weekend.
Since you're buggy here only with infinity, here is a bit of a digression on Python infinity (to the best of my knowledge, which I think is expert-level but not omniscient ;)
- float("inf") is built into Python for a reason - and that reason is inequality evaluations. So float("inf"), -float("inf") are totally natural things to pass to a MIP building engine as RHS/LHS constants for constraints or LB, UB bounds for variables. I don't think there is any need to support them as coefficients for constraints or the objective function.
- float("inf") never appears in Python by accident (i.e. 1/0 = infinity isn't native Python)
- float("inf") can be generated "by accident" with pandas (with things like 1/0). This is because pandas values bulk operations that don't throw exceptions whenever reasonably possible. If you don't want to generate infinity this way with pandas they have convenient coding idioms to help you (i.e.
df.a/df.b.replace({ 0 : np.nan }).
Not sure if any of this has any specific bearing to amplpy beyond this specific issue (i.e. the ability to write .dat files even when infinity is part of the model data). But this ticket seemed like the right place for this digression.
Not yet ready for retesting by Opalytics.
Wait, isn't this working now? For example the diet.dat file below seems to be well understood by AMPL tools.
Look at the n_max line, there is an Infinity, which is the appropriate AMPL keyword, no?
set CAT := 'calories','fat','protein','sodium';
set FOOD := 'chicken','fries','hamburger','hotdog','icecream','macaroni','milk','pizza','salad';
param amt := ['icecream','protein']8.0['icecream','fat']10.0['fries','sodium']270.0['fries','calories']380.0['hamburger','fat']26.0['macaroni','sodium']930.0['hotdog','sodium']1800.0['chicken','sodium']1190.0['salad','calories']320.0['icecream','calories']330.0['milk','sodium']125.0['salad','sodium']1230.0['pizza','sodium']820.0['pizza','protein']15.0['pizza','calories']320.0['hamburger','calories']410.0['milk','fat']2.5['salad','protein']31.0['milk','protein']8.0['hotdog','protein']20.0['salad','fat']12.0['hotdog','fat']32.0['chicken','fat']10.0['chicken','protein']32.0['fries','protein']4.0['pizza','fat']12.0['milk','calories']100.0['icecream','sodium']180.0['chicken','calories']420.0['hamburger','sodium']730.0['macaroni','calories']320.0['fries','fat']19.0['hotdog','calories']560.0['macaroni','fat']10.0['macaroni','protein']12.0['hamburger','protein']24.0;
param cost := ['hamburger']2.49['salad']2.49['hotdog']1.5['fries']1.89['macaroni']2.09['chicken']2.89['milk']0.89['icecream']1.59['pizza']1.99;
param n_max := ['protein']Infinity['calories']2200.0['fat']65.0['sodium']1779.0;
param n_min := ['protein']91.0['calories']1800.0['fat']0.0['sodium']0.0;
FYI - this functionality looks like a requirement for demonstrating a use case for pure AMPL notebooks, which is something we are all somewhat excited about.
If you have the latest pre-release installed you can already test the internal implementation of exportData with: ampl._impl.exportData(filename). ampl.exportData(filename) will be calling ampl._impl.exportData(filename) in the future. Currently the only thing missing seems to be indexed sets.
Infinity and -Infinity are recognized as numbers in AMPL .dat files.
This is now looking good with the latest. If anyone disagrees feel free to re-open.