Assertion failed when accessing non-existing object with const json object
mika1337 opened this issue · comments
Description
When I do a copy of a subpart of a json and then access an element which does not exists an assertion failed is trigger instead of an exception.
Trying to access the non-existing element before seems to prevent the issue (see reproduction steps for explanations).
Reproduction steps
With the following code:
#include <iostream>
#include "nlohmann/json.hpp"
using json = nlohmann::json;
int main()
{
json ex = json::parse(R"( { "foo": { "bar": "value" } } )");
try
{
const json foo = ex["foo"];
const std::string bar = foo["baaar"]; // <= access to non-existing item
}
catch(const std::exception& e)
{
std::cerr << " > exception: " << e.what() << std::endl;
}
return 0;
}
My program gets interrupted with the following message;
$ g++ -Inlohmann/single_include -DJSON_DIAGNOSTICS test.cpp -o a.out
$ ./a.out
a.out: nlohmann/single_include/nlohmann/json.hpp:21449: const value_type& nlohmann::json_abi_diag_v3_11_3::basic_json<ObjectType, ArrayType, StringType, BooleanType, NumberIntegerType, NumberUnsignedType, NumberFloatType, AllocatorType, JSONSerializer, BinaryType, CustomBaseClass>::operator[](const typename nlohmann::json_abi_diag_v3_11_3::basic_json<ObjectType, ArrayType, StringType, BooleanType, NumberIntegerType, NumberUnsignedType, NumberFloatType, AllocatorType, JSONSerializer, BinaryType, CustomBaseClass>::object_t::key_type&) const [with ObjectType = std::map; ArrayType = std::vector; StringType = std::__cxx11::basic_string; BooleanType = bool; NumberIntegerType = long int; NumberUnsignedType = long unsigned int; NumberFloatType = double; AllocatorType = std::allocator; JSONSerializer = nlohmann::json_abi_diag_v3_11_3::adl_serializer; BinaryType = std::vector; CustomBaseClass = void; nlohmann::json_abi_diag_v3_11_3::basic_json<ObjectType, ArrayType, StringType, BooleanType, NumberIntegerType, NumberUnsignedType, NumberFloatType, AllocatorType, JSONSerializer, BinaryType, CustomBaseClass>::const_reference = const nlohmann::json_abi_diag_v3_11_3::basic_json<>&; nlohmann::json_abi_diag_v3_11_3::basic_json<ObjectType, ArrayType, StringType, BooleanType, NumberIntegerType, NumberUnsignedType, NumberFloatType, AllocatorType, JSONSerializer, BinaryType, CustomBaseClass>::value_type = nlohmann::json_abi_diag_v3_11_3::basic_json<>; typename nlohmann::json_abi_diag_v3_11_3::basic_json<ObjectType, ArrayType, StringType, BooleanType, NumberIntegerType, NumberUnsignedType, NumberFloatType, AllocatorType, JSONSerializer, BinaryType, CustomBaseClass>::object_t::key_type = std::__cxx11::basic_string]: Assertion `it != m_data.m_value.object->end()' failed.
Aborted
Now if I add an extra step accessing the non-existing item:
#include <iostream>
#include "nlohmann/json.hpp"
using json = nlohmann::json;
int main()
{
json ex = json::parse(R"( { "foo": { "bar": "value" } } )");
try
{
const std::string bar = ex["foo"]["baaar"]; // <= access to non-existing item
}
catch(const std::exception& e)
{
std::cerr << " > exception 1: " << e.what() << std::endl;
}
try
{
const json foo = ex["foo"];
const std::string bar = foo["baaar"]; // <= access to non-existing item
}
catch(const std::exception& e)
{
std::cerr << " > exception 2: " << e.what() << std::endl;
}
return 0;
}
I got an exception instead of an assertion failed:
$ g++ -Inlohmann/single_include -DJSON_DIAGNOSTICS test.cpp -o a.out
$ ./a.out
> exception 1: [json.exception.type_error.302] (/foo/baaar) type must be string, but is null
> exception 2: [json.exception.type_error.302] (/foo/baaar) type must be string, but is null
Expected vs. actual results
The assertion failed shall not be raised.
Minimal code example
See "Reproduction steps"
Error messages
See "Reproduction steps"
Compiler and operating system
g++ 7.5.0 on Ubuntu 18.04
Library version
3.11.3
Validation
- The bug also occurs if the latest version from the
develop
branch is used. - I can successfully compile and run the unit tests.
The issue is the const. See https://json.nlohmann.me/api/basic_json/operator%5B%5D/#notes
This may be a stupid question but I'll ask it anyway: why not raise an exception instead of using assert() ?
See https://json.nlohmann.me/features/element_access/unchecked_access/ design rationale
It seems to me to be a design choice to have this undefined behavior.
I still don't get why you don't throw an exception in const_reference operator[](const typename object_t::key_type& key) const
when the item is not found.
For the non-const version I understand why you need to insert a null item that can be later filled. But here with the const version you have a way to detect that the item is missing and can throw an exception which would remove the undefined behavior.
Can you tell me more on this design choice ?
For the non-const version I understand why you need to insert a null item that can be later filled. But here with the const version you have a way to detect that the item is missing and can throw an exception which would remove the undefined behavior.
As the caller, you have the ability to detect whether or not the item is there before calling operator[]
. Thus, users that don't need to check, because they've already checked or they know that their data is good and the item is there, don't incur the overhead of checking again in operator[]
. If you have a const
object, and you don't know if the key exists, don't use operator[]
. This is no different than checking that your index is in range before using operator[]
on std::vector
.
There is no operator[](key) const
on std::map
, so this is purely a helper function over what is provided by the standard library.
I have indeed the ability to check for each element presence, but being able to process a whole message with one try/catch
and having a single error management in the catch
would result in far more clear code than testing the existence of each element and having an error management for each one.
The non-const version of operator[]
is robust to a missing element, to me having the same for the const version makes sense (and it also removes an undefined behavior in the library). Also I don't bother the implied small overhead.
Concerning the std::vector
example, to me a list a special case where you can iterate over each element. With dictionaries you have to look for a particular element and nothing guarantees that an element will be present which lead into a lot of sanity check code.
I have indeed the ability to check for each element presence, but being able to process a whole message with one try/catch and having a single error management in the catch would result in far more clear code than testing the existence of each element and having an error management for each one.
If you don't have error management for each one, then attempting to read a non-existing element aborts the whole operation. If that's what you want, then this would give you the same behavior you are seeking from operator[] const
.
auto GetChild(const nlohmann::json &j, const std::string &name)
{
if (!j.contains(name)) { throw "error"; }
return j[name];
}
try
{
const json foo = ex["foo"];
const std::string bar = GetChild(foo, "bar"); // <= access to existing item
const std::string baar = GetChild(foo, "baar"); // <= access to non-existing item
const std::string baaar = GetChild(foo, "baaar"); // <= access to non-existing item
}
catch(const std::exception& e)
{
std::cerr << " > exception 2: " << e.what() << std::endl;
}
If that's NOT what you want, and you just want a default value when the child isn't there, you should just use the .value()
function, which takes the name of the child and a default value to use when the child isn't there.
try
{
const json foo = ex["foo"];
const std::string bar = foo.value("bar", ""); // <= access to existing item
const std::string baar = foo.value("baar", ""); // <= access to non-existing item
const std::string baaar = foo.value("baaar", ""); // <= access to non-existing item
}
catch(const std::exception& e)
{
std::cerr << " > exception 2: " << e.what() << std::endl;
}
The non-const version of operator[] is robust to a missing element, to me having the same for the const version makes sense (and it also removes an undefined behavior in the library).
Removing the non-const version would match std::map
and also remove the undefined behavior. As would removing the const
from your object, though that would mean that suddenly reading the data could modify it, which is likely not what you want.
Also I don't bother the implied small overhead.
Others have different priorities. With the current behavior, you have the ability to add that overhead if you want/need it. Adding the check means that others that don't want the overhead can't avoid it.
Concerning the std::vector example, to me a list a special case where you can iterate over each element. With dictionaries you have to look for a particular element and nothing guarantees that an element will be present which lead into a lot of sanity check code.
That is absolutely not true, you can iterate over the dictionary just as well.
I think this whole discussion is about priorities.
To me having an undefined behavior by just adding a const
to my json object is a real problem. It means that my object behaves differently when I read its content, the object being const or not. And it's not a small difference, in one case my application will throw an exception and in the other it will either stops abruptly (with the assert) or behave in an undefined manner.
You will probably tell me that there is no issue since this is documented, but for a library which is intended to be largely used I believe it is an issue.
In the STL the std::map::operator[]
is only available for non-const objects and std::map::at()
throws an exception if the item doesn't exists. I believe this is a safe design choice, you get no surprise during execution if you add a const
to your object.
Well, the design decision is documented, and this is the behavior of the library since day 1. I can understand your point, but using unchecked access for a non-existing key is a problem in itself. The library offers (and documents) several APIs how to cope with this.
And what do you expect when you access j["foo"]
for a const json
object without key foo
?
- An exception? Use
at
. - A default value? Use
value
. - ???
If all cases are already covered with at()
and value()
, why propose operator[]
for const
objects ?
Probably because one day you thought it would be nicer to use it compared to at()
. If it's the case then I think it should have been done all the way handling non-existing key in a defined manner.
I'm insisting maybe too much on this, but I'm used to MISRA coding rules which require variables to be declared as const
if you don't modify them. In all the projects I've worked on, adding a const
wouldn't change the behavior. Here I see situations where people would add a const
to comply with coding rules having no idea their program would now crash if it receives a json frame with parameters missing or malformed.
Having no way to prevent it except from telling people to read properly the documentation scares me.
No: operator[] is for unchecked access. Don't pay for what you don't use.
Fine, but I still deeply believe this is a bad design choice.