Wrong result for query "Get the top 5 grossing states" in sample colab
lukyanenkomax opened this issue · comments
According to data total value for every row should be calculated as Price Each * Quantity Ordered. In sample colab library summarizes prices but doesn't take in account quantity.
Hey~
Good catch! I agree with your interpretation that the AI should have generated a new feature to measure the "grossing" attribute, and then calculated the result.
This is definitely the main type of failure case with these ai-assistant / code-generation tools, where it misunderstands and gets something "subtly" wrong. This is precisely why we made this tool give you the code back rather than executing the code and telling you the answer
At the quality of current models, we can't blindly trust the code it writes, so; in general, the code that comes back from howto is best thought of as a really good starting place!
Unfortunately, there's not much I can do to improve this, this is somewhat "working as intended" and is reliant on the quality of the underlying model (right now davinci-text-003) so I'm going to close this issue.
If there is anything specific you think I should change (eg. in readmes, or in code) let me know, i welcome any recommendations on how to add clarity!