Stdlib organization with re-export idiom

[metagn]

Based off of #497 (comment)

A common idiom in Nimble packages is to have submodules for individual features of the package, then a main module that re-exports them all (i.e. import foo/[a, b, c] == import foo). This means users both can pick only the features they need, and don't have to worry about which features they might need if they just import the main module.

Applying this idiom in the standard library has already been discussed for os, times, json etc. but there might be some other existing applications that are easier to implement and less likely to cause breakages.

1. Sugar

The sugar module currently consists of the following independent definitions:

• => and ->
• dump and dumpToString
• capture
• dup (depends on std/private/underscored_calls which is also used by std/with)
• collect

I propose these are put in their own submodules and then re-exported in sugar, with respectively the following module names:

• std/sugar/arrows or std/sugar/lambdas? Unsure
• std/sugar/dump
• std/sugar/capture
• std/sugar/dup
• std/sugar/collect

We could also drop the sugar/ and just put these directly in std/ as they do not conflict with other modules. However this could be confusing, people could import std/[collect, dup, sugar] and not realize.

I also think the following, existing modules should be re-exported in sugar:

• std/with
• std/enumerate
• Maybe std/decls?
• Maybe std/wrapnils?

The point of "sugar" is that it's lazy/convenient, and the most convenient thing would be to have a unified import for all of these. But also we shouldn't have to import everything just for collect or dump or arrows etc.

2. Strings

strutils contains a lot of important definitions for strings that aren't just "utilities", and so it is extremely popular along with strformat. On top of strutils, there is strmisc which is tiny and honestly isn't that disconnected from the rest of strutils.

While it might be nice to split strutils for specific things people might need, it would be a huge effort and there is the nimrtl stuff that maybe shouldn't be disturbed. However there are some minor things that could be done:

• Sets of characters, i.e. AllChars, Letters, Digits, Whitespace etc. along with procs like allCharsInSet could go in a separate module and get re-exported. Sometimes modules will import strutils just for these (example being wordwrap).
• The outplaced parseutils wrappers could go in parseutils or a separate module and get re-exported/wrapped again. (also fromHex and parseHexInt have the same implementation yet do not mention each other in documentation)
• std/formatfloat is unrelated to strutils.formatFloat. We could rename the current std/formatfloat to std/floatdollar or something, make a new std/formatfloat with the formatFloat procs and re-exported floatdollar, then we can re-export that in strutils.
• formatSize is out of place along with its enum BinaryFormatMode. It could have its own module, and may or may not be re-exported.

To make them easier to discover, we could also re-export cstrutils, strbasics somewhere, and stuff like editdistance, wordwrap in strmisc. There are also modules like encodings and unicode that people aren't aware of sometimes and ask about, but these might be too big to re-export.

[Andreas Rumpf]

This goes into the right direction but the import/export idiom is not without its problems for discoverability, esp for newcomers.

Also, I still don't know why e.g. cmpIgnoreCase should be connected to editDistance via a single import std / strings. editDistance is not even useful when you think it would be. For typo corrections it typically produces poor results.

[metagn]

Well, it shouldn't. The name std/strings is too broad, it's misleading if it doesn't export the infinite string stuff we have.

Classifications like "miscellaneous string algorithms in the standard library", which to me is what strmisc implies, are finite and unlikely to intersect with anything else. Most stuff already in the standard library like cmpIgnoreCase are going to be fundamental enough that they don't belong there, but rarer stuff like editdistance would belong IMO. Brushing them off as "miscellaneous" also implies they aren't perfect implementations either. However currently strmisc consists of direct proc definitions, so maybe it's not ready to have re-exports (expandTabs and partition could respectively go in std/indents and std/strsplit if strutils is split up).

[konsumlamm]

One major problem with reexports imo is currently that the docs for reexported items don't show up, they just get listed in an "Exports" section (see here for an example). I think this should definitely be fixed before we do something like this. Apart from that, I like the idea of using submodules in the standard library, although I'm not sure about having a submodule for each macro in std/sugar...

[metagn]

I thought the sugar part would be the obvious one. IMO it's better than putting all the random macros in the same module. We could maybe group them together, like => and capture both have to do with lambdas, with and dup are pretty similar, etc.

I think there is an RFC to overhaul the doc page structure that involves getting rid of the proc/template/iterator etc sections, maybe along with that we could add an "Exported" section to the bottom that shows the docs of all the exported symbols.