Use $select query parameter
matthew-white opened this issue · comments
This issue is for the Frontend aspect of getodk/central#205. @lognaturel writes:
it won’t make a noticeable difference with small forms and fast connections but it will with big forms and slow connections
Now that getodk/central-backend#494 is merged, the next step is to think about Frontend. I'll use this issue for related notes. The $select query parameter will be used in two components:
SubmissionShow (submission details page)
From the submission OData, I think SubmissionShow just uses __id, __system, and meta/instanceName.
However, things aren't as simple as requesting $select=__id, __system, meta/instanceName. I believe that some forms don't have a meta/instanceName field. For such a form, requesting $select=meta/instanceName would result in a 400.23 Problem response from Backend.
Another idea might be to $select all of meta rather than just meta/instanceName. If the resulting meta group has an instanceName field, then we'll use it; otherwise we'll know that there isn't an instanceName field. However, I also think it's possible that some forms won't have a meta group. For such a form, requesting $select=meta would again result in a 400.23 Problem. Backend also doesn't currently support $select=meta: right now, the only ComplexType that can be $selected is __system.
Question: does the OData spec require that $selecting a field that doesn't exist result in an error response? Could we have Frontend request $select=__id, __system, meta/instanceName, and if meta/instanceName doesn't exist, Backend will ignore the $select or select just __id and __system?
When SubmissionShow sends a request for the submission OData, it also sends a request for the form fields. We could serialize those requests such that we request the form fields first, then request the submission OData once there's a response, $selecting meta/instanceName or not depending on whether it is among the form fields. However, that would require two round trips to the server before rendering the page, so I don't think we should take that approach.
Maybe the easiest thing would be to pull the instance name off the latest submission version, which SubmissionShow also requests, rather than off the OData.
Navigation to SubmissionShow will often happen from SubmissionList (the submissions page). By the time it renders a link to SubmissionShow, SubmissionList will know whether the form has a meta/instanceName field. It could add a query parameter to the link if the form has a meta/instanceName field (something like ?instanceName=true). We could then have SubmissionShow send a request with $select=__id, __system, meta/instanceName if instanceName=true, send a request with $select=id, __system if instanceName=false, and send a request without $select if instanceName is not specified (for example, if the user navigates directly to SubmissionShow).
SubmissionList (submissions + draft testing pages)
SubmissionList has a dropdown to select the fields to view. By default, this is the first 10 fields (excluding repeat groups). It'd be great if when we request the submission OData, we $select just those 10 fields (+ __id and __system). However, as with SubmissionShow, right now we request the first chunk of submission OData at the same time as we request the form fields. We could serialize those requests, but that doesn't seem ideal.
One approach might be to not specify $select for the first chunk of OData, but then specify it for later OData requests: for later chunks in the infinite scroll, when the field selection changes, or when a filter changes. Only the first OData request wouldn't specify $select.
Somewhat separately, I think we could also considering saving the column selection in a Frontend query parameter. Right now, when a filter changes, that change is reflected in the Frontend URL: a query parameter changes. That allows users to bookmark a filtered view of the submissions page. If we added the column selection to the Frontend URL as well, then users could bookmark a view of the submissions page with the columns that they wish to see. We could have SubmissionList look to that Frontend query parameter when it sends the request for the first chunk of OData: if the query parameter is specified, then Frontend will pass it along to Backend via $select. (Something to consider: the query parameter could be quite long. Could we encode or compress it somehow?)
@lognaturel also asked:
issue request using most recently available selected fields falling back to no $select; cancel and reissue if fields come in and list doesn’t match?
I think something like that would work well once we start storing user preferences in Backend. (Though possibly wouldn't be needed if we make the other changes above?) We've also talked about adding saveable views of the submissions table, which I think would intersect with this.
One approach might be to not specify $select for the first chunk of OData, but then specify it for later OData requests: for later chunks in the infinite scroll, when the field selection changes, or when a filter changes. Only the first OData request wouldn't specify $select.
That certainly seems like a reasonable, valuable approach to me. Maybe it's just a first step but it should really help a lot for very wide forms on a slow connection.
once we start storing user preferences in Backend.
I was trying to communicate an idea that would be frontend-only to possibly use select on the first OData request but only if it takes a long time without the select. I don't think it's necessary, though. Focusing on subsequent queries seems sufficient.
Somewhat separately, I think we could also considering saving the column selection in a Frontend query parameter.
If we did this, we would need to figure out how to represent a selection of no columns. For the "Submitted by" and "Review State" filters, if you select none, then it falls back to all, because actually selecting none would necessarily result in zero rows. For these filters, the absence of query parameters (the default) means that all options are selected, not none. For the column selection dropdown, I think the absence of a query parameter should likewise fall back to the default behavior, selecting the first 10 fields. However, unlike with the filters, there are cases for this dropdown where it can be desirable to select none. Even if you deselect all the columns in the column selection dropdown, you will still see the fixed columns, including "Instance ID" and "State and actions". If we want that to continue to be the case, and we want to add a query parameter for the column selection, then we will need to think about how to represent a selection of no columns in the query parameter.
Maybe the easiest thing would be to pull the instance name off the latest submission version, which
SubmissionShowalso requests, rather than off the OData.
It looks like SubmissionShow actually requests the first submission version, not the latest. We could also request the latest, but it seems preferable to avoid the additional request.
Another idea might be to
$selectall ofmetarather than justmeta/instanceName. If the resultingmetagroup has aninstanceNamefield, then we'll use it; otherwise we'll know that there isn't aninstanceNamefield. However, I also think it's possible that some forms won't have ametagroup. For such a form, requesting$select=metawould again result in a 400.23 Problem. Backend also doesn't currently support$select=meta: right now, the onlyComplexTypethat can be$selected is__system.
As of getodk/central-backend#717, Backend now supports $select=meta, so this approach is appealing. How likely is it that a form won't have a meta group? If it's quite unlikely, I think we should just specify $select=__id,__system,meta. If it's a possibility that we want to account for, then we could try to send a request with $select, then if that fails with a particular Problem code, resend the request without $select. (But if we can avoid that logic, that would be nice.)
I answered this somewhere else but for completeness: it's safe to assume a form has a meta group. It's also safe to assume it has instanceID but it may or may not have instanceName.