Function toString does not behave the same with hermes turned on
Collin3 opened this issue · comments
I use react-native-qrcode and noticed that after upgrading the React Native v0.60 and turning on hermes, the QR code was broken. We debugged the issue and found that the problem comes when the library tries to stringify this renderCanvas function and then insert it into some HTML to show in the webview.
When calling .toString() on the function we found that we were actually getting
function renderCanvas(a0) { [ native code ] }
instead of getting a string of the actual function.
I realize this is probably not a very common use case, but figured I'd report it in case anyone else is seeing anything similar.
Hermes compiles all source to bytecode before executing it on a phone. As such, we discard source information including the actual text of the code, parameter names, etc. during compilation. This allows us to keep the size of our bytecode file small and ensure that the code is quick to execute.
Due to this, we do not support actually getting the source code using Function.prototype.toString - it's not available in the bytecode file, so we can't print the source.
Ahh, interesting. Thanks for the explanation! I’ll close this issue then.
I don't suppose there'd be any way to inform Hermes that we'd like to hold onto the conventional result of a call toString()?
@cawfree Correct: Hermes does not currently support a way to include the original source in the bytecode bundle.
@cawfree this is an interesting idea. Perhaps we could consider a kind of annotation telling our compiler to preserve the source of a particular function. I can see how obtaining the source in a different way may become very cumbersome.
I think it is doable. The question is what would such an annotation look like. One possibility would be to include "use source" in the beginning of the function (similar to "use strict").
Do you have any ideas or a preference?
You may want to mirror https://github.com/tc39/proposal-function-implementation-hiding and go with something like “show implementation”.
@tmikov Were you able to get somewhere in implementing this? One of our production apps broke after upgrading to hermes because we call toString() on functions and it doesn't return the expected results.
At various points in the DFA, it seems like it would be possible to know that toString() is being called on a function object/prototype, and then only include the source for those precise functions. If that is possible, figuring how to store them in the binary format so the function string usages are friendly to CPU prefetch and don’t disrupt current L1 cache hit rates would be nice to have (tm).
Unfortunately we don't have a use case for this internally, so the priority for implementing it is not very high. I would love to see a PR from community, if anyone is interested I can give detailed pointers about what needs to be done.
I would love to see a PR from community, if anyone is interested I can give detailed pointers about what needs to be done.
Hi, I can work on that. Could you write down all the details?
@lb90 sorry for the delay! Are you still interested?
@lb90 great, I will start filling up the details. We will do it incrementally, it should be a fun project!
Is there someone making some progress about this? I think this case is very important when working with webview. we will injectJavaScript to webview to execute it. If Hermes cannot support this, which means webview is kind of broken.
I don't understand why Hermes team thinks it's not a big problem.
@chj-damon what is preventing you from including the source of the function as a string?
for example, I'm using echarts in webview. it provides an option like this:
{
title: {
text: 'ECharts demo'
},
tooltip: {
show: true,
formatter: (params) => params.name + ': ' + params.value，
},
legend: {
data: ['销量']
},
xAxis: {
data: ['衬衫', '羊毛衫', '雪纺衫', '裤子', '高跟鞋', '袜子']
},
yAxis: {},
series: [
{
name: '销量',
type: 'bar',
data: [5, 20, 36, 10, 10, 20]
}
]
}you see there's a formatter function I declared in the option, when Hermes was enabled, this formatter function will be transformed to formatter: function formatter(params) {[bytecode]} before I inject it to webview. which means formatter will not work.
if formatter is being changed into a string, then having the source code won't help you - it needs to be transmitted as a function.
@ljharb I think the idea may be to eval() the entire string in the WebView, which will then recreate the function. Technically this is not JSON.
@chj-damon I see the appeal of being able to reuse the same JS function in the WebView, but there is a fundamental problem with that approach - there is no guarantee that the WebView and Hermes support the same language features. You are applying one set of Babel transforms to JS for Hermes, but the WebView has a different JS engine and it may need a different set of transforms. So, as soon as you start using it for anything more complex, differences will appear.
I think the idea may be to eval() the entire string
Given that there's no way in the language to guarantee a function is portable (can be toStringed and re-evalled elsewhere, even in the same global environment), that seems like an inherently brittle design that needs rearchitecting.
maybe this problem will be solved if Hermes support some signature, like 'noHermes' which defined in a function, like react-native-reanimated@2
formatted: function(params) {
'noHermes';
return params.name;
}Hermes will ignore this function if it sees 'noHermes' signature.
react-native-reanimated@2 using 'worklet' to define a function that will be running in UI thread.
Hey folks, I want to give an update about this long-hanging issue.
TLDR: Hermes 0.8.1 (available starting from React Native 0.65-rc.3) introduced a special directive "show source" to make toString returning original source code.
As @tmikov and @ljharb already pointed out, injecting JavaScript source code via toString and reevaluating it in WebView or similars seem to be an inherently brittle approach and should be discouraged whenever possible. However, it seems to be a not super uncommon practice by the community.
Our position is that Hermes, whenever possible, shouldn't get in the way of developers' freedoms of doing things and we value generality and consistency. So when a divergence like this has to happen (for good sake of performance), we try to offer an escape hatch.
You can annotate the function that you are passing into libraries with"show source". We hope this can unblock some of you from adopting Hermes in your app.
Interaction w/ the Function Implementation Hiding proposal
The "show source" directive is implemented as if it's part of the stage2 Function Implementation Hiding proposal (thanks @ljharb for bringing it up). We value the ECMAScript/TC39 standardization process and we are thinking of how to upstream or signal our addition to the original proposal.
To exercise our approach, we also implemented the function source part of the original proposal (not the error stack trace part yet). Note that the exact overriding behaviors are considered not finalized.
How Does It Look Like?
You can find the sample code from the function-toString.js test file. The below is a sneak peek of the most basic functionalities that can be expected.
function dflt(x) {};
print(dflt.toString());
// CHECK-NEXT: function dflt(a0) { [bytecode] }
function showSource(x) { 'show source' }
print(showSource.toString());
// CHECK-NEXT: function showSource(x) { 'show source' }
function hideSource(x) { 'hide source' }
print(hideSource.toString());
// CHECK-NEXT: function hideSource() { [native code] }
function sensitive(x) { 'sensitive'; }
print(sensitive.toString());
// CHECK-NEXT: function sensitive() { [native code] } Caveats Regarding Babelified Functions
Note that although this works for all kinds of functions that Hermes natively support (arrow function, async function, generator function), you should be prepared for Babelified result due to the Babel transformers adopted by React Native. Babelified functions often refer to other functions e.g. Babel helpers, Regenerators, that may not be presented from your "reevaluation environment" e.g. Webview. Pay attention to breakages caused by that.
How Does It Work?
Historically, as @avp mentioned:
Hermes compiles all source to bytecode before executing it on a phone. As such, we discard source information including the actual text of the code, parameter names, etc. during compilation. This allows us to keep the size of our bytecode file small and ensure that the code is quick to execute.
Due to this, we do not support actually getting the source code using Function.prototype.toString - it's not available in the bytecode file, so we can't print the source.
Now, the compiler will be looking for "show source" as a hint to preserve the source of a particular function into a new "function source table" in the bytecode file, which maps function ID to StringTable ID, so it can be retrieved from toString. It therefore increase your bytecode file size. Implementation-hiding functions, however, are "marked" by pointing to an empty string, hence are almost free.
TLDR: Hermes 0.8.1 (available starting from React Native 0.65-rc.3) introduced a special directive "show source" to make toString returning original source code.
This is awesome 👍 🙏 . However, when running with React Native 0.67.2, Android and Hermes and injecting code into a react-native-webview, I notice that triggering the app to reload seems to reintroduce the problem despite annotating the function with 'show source':
function myFunctionWithShowSource() {
'show source'
// ...
}
Causes this in the webview console:
Uncaught ReferenceError: bytecode is not defined
at myFunctionWithShowSource (<anonymous>:41:40)
at <anonymous>:41:52
at <anonymous>:57:3
I'm unable to create a minimal reproducible example at the moment; it's possible that the issue could be somewhere in my own setup or implementation. Just leaving this here in case someone else runs into issues with show source.
I have a use case where I need to inject jquery into the webview and hermet is preventing that, even with adding 'show source' inside jquery script I. still get the "error evaluating injected JavaScript" , I have the jQuery script locally stored in a file and I am just importing it trying to inject it to the webview (note: injecting other functions with show source works fine, however, adding specific Regex statements caused the injection to fail for hermes)
any fix for this issue?
TLDR: Hermes 0.8.1 (available starting from React Native 0.65-rc.3) introduced a special directive "show source" to make toString returning original source code.
This is awesome 👍 🙏 . However, when running with React Native 0.67.2, Android and Hermes and injecting code into a react-native-webview, I notice that triggering the app to reload seems to reintroduce the problem despite annotating the function with
'show source':function myFunctionWithShowSource() { 'show source' // ... }Causes this in the webview console:
Uncaught ReferenceError: bytecode is not defined at myFunctionWithShowSource (<anonymous>:41:40) at <anonymous>:41:52 at <anonymous>:57:3I'm unable to create a minimal reproducible example at the moment; it's possible that the issue could be somewhere in my own setup or implementation. Just leaving this here in case someone else runs into issues with
show source.
For those of you wondering, I just tried reproducing this problem on React Native 0.71.3 and it is no longer valid.
My basic reproduction scenario:
const testFunction = () => {
'show source';
Alert.alert('Hello world!!');
}
function App(): JSX.Element {
const isDarkMode = useColorScheme() === 'dark';
const showAlert = useCallback(() => {
console.log(testFunction.toString());
testFunction();
}, []);
....
This implementation logs the following to the console regardless of weather I am in debug or release modes using Hermes engine:
function testFunction() {
'show source';
_reactNative.Alert.alert('Hello world!!');
}
It even works for hot reloading when I change the implementation inside testFunction 🥳
For me, "show source" only works on a archive package in release mode with Archiving like ./gradlew clean && ./gradlew bundleRelease. It won't work longer after a new js bundle is downloaded using react-native-code-push. So the issue also exists. Is there any better solution for that?
@lchenfox out of curiosity: does the new bundle downloaded using react-native-code-push contain JS source?
@lchenfox out of curiosity: does the new bundle downloaded using
react-native-code-pushcontain JS source?
Yes. I fixed a bug using react-native-code-push, then the show source won't take effect in my app. Strangly, it works without downloading a new patch package using react-native-code-push.
@lchenfox Hermes is not intended to run from source in production, we strongly recommend against it.
@lchenfox Hermes is not intended to run from source in production, we strongly recommend against it.
does this mean react-native-code-push means nothing when Hermes enabled?
@lchenfox Hermes is not intended to run from source in production, we strongly recommend against it.
Yeah. Theoretically speaking, however, using react-native-code-push to fix other bugs online shouldn't affect the source that has already been added "show source" before in production. In fact, it indeed makes the formatter function using show source unavailable. If so, show source will be always unavailable after I release each patch package via react-native-code-push?
@lchenfox Hermes is not intended to run from source in production, we strongly recommend against it.
does this mean
react-native-code-pushmeans nothing when Hermes enabled?
react-native-code-push also takes effect, because the online bug was fixed. However, the formatter function using "show source" won't take effect after releasing this patch package. I tried many times, it can be reproduced. the formatter function like:
_tooltipFormatter = (params) => {
'show source';
let htmlStr = '<div>' + params[0].name + '<br/>';
for (let i = 0, l = params.length; i < l; i++) {
const color = params[i].color;
htmlStr += '<span style="margin-right:5px;display:inline-block;width:10px;height:10px;border-radius:5px;background-color:' + color + ';"></span>';
let symbolIndex = params[i].seriesName.indexOf('@');
const hasSymbol = symbolIndex >= 0;
const seriesName = hasSymbol ? params[i].seriesName.substring(0, symbolIndex) : params[i].seriesName;
symbolIndex += 1;
const unit = hasSymbol ? params[i].seriesName.substr(symbolIndex) : '';
let value = '--';
if (unit === '$' || unit === '¥') {
value = unit + (params[i].data !== undefined ? params[i].data : '--');
} else {
value = (params[i].data !== undefined ? params[i].data : '--') + unit;
}
htmlStr += seriesName + '\: ' + value + '<br/>';
}
htmlStr += '</div>';
return htmlStr;
};
tooltip: {
trigger: 'axis',
confine: true,
formatter: this._tooltipFormatter, // Uses `show source`.
backgroundColor: 'white',
borderColor: '#badafa',
borderWidth: 1,
padding: 2,
textStyle: {
color: '#0273f2',
fontSize: 12,
},
}
In case I may make myself unclear. Here is a demo reproduced by npx react-native init chartsdemo --version 0.71.1: App.js🔗
Code snippets
const option = {
......
tooltip: {
show: true,
formatter: function(params) {
"show source"; // Add "show source" in production.
if (Array.isArray(params)) {
return params[0].name + ": " + params[0].data;
}
return params.name + ": " + params.data;
}
......
}
return (
<View style={{ flex: 1, backgroundColor: "white" }}>
<RNEChartsPro height={350} option={option} />
<Text>
Test echarts1
</Text>
</View>
);
Steps
- Archiving a release
apk, then install theapkon a real device. - The
tooltipshows as expected. That is to say,"show source"works in theformatterfunction. - The text shows
Test echarts1normally on the device. - Changing the
Test echarts1above toTest echarts2. - Releasing a new patch package using react-native-code-push
- The text shows
Test echarts2normally on the device. - The
tooltipshows nothing."show source"DO NOT WORK in theformatterfunction.
I don't know how to fix this. Please help. Thanks a lot.
I was banging my head against that why something works in debug, but not in production. I tried implementing 'show source', without luck. Still having the same problem. Any ideas why this is not working in this context
const someObject = {
component: selectedColor => {
'show source';
return <Wuii color={selectedColor} />;
},
id: WUI,
},
@Stophface the first step to identifying the problem is to look at the source that is actually passed to Hermes after being transformed by various tools in the build pipeline.
@tmikov How can I see the source code? Plus: I thought adding 'show source' and then calling .toString() would keep the string representation
@Stophface In order to identify whether this is a problem in Hermes, we need to be able to examine the input given to Hermes. Unfortunately we can't help you debug parts of the build pipeline that happen before Hermes. It is quite possible that the "show source" annotation is stripped before it gets to the Hermes compiler, for example by a minifier.
As a workaround, I added a script that stringifies the needed git module. So the processed module looks like
export const injectionScript = `
/// Functions from the module are here
`;
And I can use injectionScript in my WebView. I would have been happy to use 'show source' if it had worked.
Expo performs very unstable with this feature, it works 1 out of 10 times...
I finally find for the first time it bundles, it does not work. And you can add a console.log to the file that uses 'show source';, let it hot reload, then it works. (on development)