JSON over HTTP
joeconwaystk opened this issue · comments
JSON over HTTP Proposal
Abstract
The purpose of this proposal is to define an API for a productive JSON deserialization in Dart client-side code. The proposed API is not specific to JSON; it could be extended to any form of dynamic data binding. JSON makes a good example. This solution is currently not implementable in Dart 1.x. This solution should include support for all compilation targets.
Consider the result of some HTTP request as a list of JSON objects:
[
{
"id": 1,
"name": "Thing"
},
{
"id": 2,
"name": "Thingamabob"
}
]
A productive way of making this request and getting back this list of JSON objects as Dart objects might look like the following:
var request = new Request<Thing>("/things");
var thingResponse = await request.get();
if (thingResponse.statusCode == 200) {
// type annotation added for clarity, is unnecessary
List<Thing> things = thingResponse.body.asList();
}
Where a Response.body.asList()
is of type List<T>
as in Request<T>
. T
must extend some type that has behavior for deserialization behavior. De-serialization of a single object follows the same pattern, but accessed with asObject()
.
var request = new Request<Thing>("/things/1");
Thing thing = (await request.get()).body.asObject();
Declaring the parameterized type for Request
requires minimal effort. In a simple case, it declares managed
properties and extends Codable
:
class Thing extends Codable {
managed int id;
managed String name;
String notManaged;
}
Other Codable
subclasses can be managed
:
class Foo extends Codable {
managed int id;
managed Thing thing;
}
Such that the following JSON is deserializable:
{
"id": "foo1",
"thing": {
"id": 1,
"name": "Thing"
}
}
As an alternative syntax, all managed
properties can be declared as so:
class Foo extends Codable {
managed {
int id;
Thing thing;
}
}
The name of the key in the payload can be specified with metadata:
class Foo extends Codable {
managed {
int id;
@CodableKey("thingamajig")
Thing thing;
}
}
Motivation
Moving dynamic data into and out of an application is a necessary component of a Dart application. Constraints on meta-programming on most Dart compilation targets require manual or code generated solutions. Manual solutions are tedious and prone to error, while code generation requires deeper knowledge of the platform, extra tooling and therefore deters new users.
A built-in solution to JSON deserialization is an expected feature of any modern application programming language; so much so that Swift modified their compiler to support it. The value-add of a unified mechanism to productively model JSON data as Dart objects is significant. A new user can immediately see results with an existing API (or one they are able to build quickly), which will impact their long-term usage of Dart.
Proposal
The managed
keyword is applied to properties and accessors of a type. These properties and accessors are available to the interface of the type, but storage is deferred to some other mechanism. The following is syntactically valid:
class T {
managed int i;
}
var t = new T()..i = 1;
The base class Object
adds a new property, managedMembers
, which contains runtime-available metadata for all managed
declarations. For AOT targets, this information is created during compilation. This effectively moves code generation into a compiler pre-processing step.
The invocation t.i=
is not a simple property assignment, but is resolved to an invocation of:
t.setValueForKey(#i, 1);
Likewise, the getter t.i
is an invocation of:
t.valueForKey(#i);
Because T
does not currently implement setValueForKey
or valueForKey
, this implementation must be provided by T
or a base class. Codable
implements these two methods as little more than map access:
class Codable {
Map<Symbol, dynamic> _data = {};
void setValueForKey(Symbol key, dynamic value) {
_data[key] = value;
}
dynamic valueForKey(Symbol key) => _data[key];
}
Thus, access to managed
properties is always dynamic. (This part is debatable, but it allows for a tiered approach where this functionality is available earlier and can be optimized later without an API change.)
A Codable
is instantiated with a Coder
. This would require that constructors be inherited; that Dart constructors are not inherited is surprising and requires workarounds for some design patterns. Codable
would define two constructors:
class Codable {
Codable();
Codable.fromCoder(Coder coder) {
managedMembers.forEach((k, metadata) {
var value = coder.decode(k, metadata);
setValueForKey(k, value);
});
}
}
A Coder
is an abstract key-value container object. A simple implementation would wrap a Map
:
class KeyedCoder implements Coder {
KeyedCoder(this._keyValues);
Map<String, dynamic> _keyValues = {};
dynamic decode(String key, ManagedMemberInfo metadata) {
var value = _keyValues[key];
if (!metadata.isAssignableWith(value)) {
throw 'invalid value';
}
if (metadata.transformation != null) {
return metadata.transformation(value);
}
return value;
}
....
}
(Important: this does not limit concrete Coder
s from implementing key-lookup in other ways. Additionally, a Coder
should not copy data.)
The Request<T>
object from earlier deserializes data as such:
class Request<T extends Codable> {
Request(this.path);
Future<Response<T>> get() async {
var response = await http.get(...);
// Decodes according to content-type
var decodedBody = await decode(response);
if (decodedBody is List) {
return new Response<T>()
..statusCode = response.statusCode
.._objects = decodedBody
.map((o) => new T.fromCoder(new KeyedCoder(o)))
.toList();
} else if (decodedBody is Map) {
return new Response<T>()
..statusCode = response.statusCode
.._object = new T.fromCoder(new KeyedCoder(decodedBody));
}
}
}
This requires that parameterized types can be instantiated and that their constructors are known.
Summary of Proposed Changes
- Adds
managed
language keyword.- Allows for dynamic storage/implementation.
- Adds
Object.managedMembers
.- A limited set of metadata to be generated at compile-time for AOT targets from
managed
properties.
- A limited set of metadata to be generated at compile-time for AOT targets from
- Allows constructors to be inherited.
- Allows parameterized types to be instantiated.
- Adds
Codable
andCodableKey
.
@joeconwaystk also see my comment here: matanlurey/dart_serialize_proposal#8 (comment)
I think this issue is covering 2 things:
- Automatic deserializing of data over HTTP calls instead of explicitly grabbing the String or Stream of bytes and deserializing. I think this is probably unlikely to gain much traction.
- Better ability to deserialize JSON to Dart types instead of
Map<String, dynamic>
. This is something that has been discussed in a number of places and I think we still hope to be able to do something better around this, but it's not clear what that will look like. See for instance #31876