pieroxy / lz-string

Excuse my poor understanding of this topic in advance.

Something that occurred to me is that it should be possible to exceed the compression we have via compressToEncodedURIComponent() using IRIs. The logic being simply that URIs only allowed for ASCII characters, but IRIs allow for the millions of different unicode characters, so by using these extra characters we should be able to encode some input more efficiently. Notably the goal for this kind of thing is to make the shortest URL, which isn't necessarily the same as making the smallest memory representation (because Unicode characters are larger than ASCII ones in memory). Also, this is different to the existing compressToUTF16 function, because not all unicode characters are legal in IRIs, although most are.

If this isn't particularly difficult, I'm happy to contribute this functionality if I'm able. Having looked at the existing implementation, I wonder if this is as simple as providing a new "conversion" function that encodes integers into strings. This could be done similarly to how you have done it with URIs:

lz-string/libs/lz-string.js

Line 95 in 2f749bc

    
           return LZString._compress(input, 6, function(a){return keyStrUriSafe.charAt(a);});

For reference, the IRI standard that defines the available characters is here: https://datatracker.ietf.org/doc/html/rfc3987#section-2.1. The part that is of relevance is the iquery rule, which defines the legal IRI query string:

   ipchar         = iunreserved / pct-encoded / sub-delims / ":"
                  / "@"

   iquery         = *( ipchar / iprivate / "/" / "?" )

   ifragment      = *( ipchar / "/" / "?" )

   iunreserved    = ALPHA / DIGIT / "-" / "." / "_" / "~" / ucschar

   ucschar        = %xA0-D7FF / %xF900-FDCF / %xFDF0-FFEF
                  / %x10000-1FFFD / %x20000-2FFFD / %x30000-3FFFD
                  / %x40000-4FFFD / %x50000-5FFFD / %x60000-6FFFD
                  / %x70000-7FFFD / %x80000-8FFFD / %x90000-9FFFD
                  / %xA0000-AFFFD / %xB0000-BFFFD / %xC0000-CFFFD
                  / %xD0000-DFFFD / %xE1000-EFFFD

   iprivate       = %xE000-F8FF / %xF0000-FFFFD / %x100000-10FFFD

Another use-case for a standardised dictionary for compressToCustom!

Implement IRI Encoding