haskell / text

Since text-2.0.1, lift implementation for Text has changed to, essentially

unpackCStringLen# $(bytes) $(lenInt)

text/src/Data/Text.hs

Lines 419 to 422 in bc6f403

    
           lift txt = do 
        
             let (ptr, len) = unsafePerformIO $ asForeignPtr txt 
        
             let lenInt = P.fromIntegral len 
        
             TH.appE (TH.appE (TH.varE 'unpackCStringLen#) (TH.litE . TH.bytesPrimL $ TH.mkBytes ptr 0 lenInt)) (TH.lift lenInt)

where lenInt is an Int. The problem here is that Int gets lifted into a numeric literal, which gets desugared as fromInteger <lenInt>. When using RebindableSyntax and there's a non-standard fromInteger in scope, this breaks Text's lift.

Here's a small repro: https://github.com/lierdakil/text-2.0.1-lift-repro

The only way I could figure a workaround is using an unlifted Int# and the primitive constructor. Not too bad, but depends on GHC.Exts or somesuch.

Under this suggestion, the relevant code would look something like this (from the top of my head):

    let !lenInt@(I# lenIntPrim) = P.fromIntegral len
    TH.appE (TH.appE (TH.varE 'unpackCStringLen#) (TH.litE . TH.bytesPrimL $ TH.mkBytes ptr 0 (P.fromIntegral lenInt))) (conE 'I# `appE` TH.lift lenIntPrim)

It seems to me that the issue is with instance Lift Int and RebindableSyntax, not with Text. CC @phadej @RyanGlScott

Hmm. There is an argument that Lift Int should actually insert an Int and not an integral literal. But in that case, that's not the only integral type that would need to be changed, pretty much all of them would, and this change is potentially breaking.

Sigh. I really wish that GHC gave us finer-grained control over what language extensions take effect at Template Haskell splices. I struggle to imagine how we'd avoid issues like this one without it (unless we want to avoid ever using IntegerL, which seems counter-productive).

But in that case, that's not the only integral type that would need to be changed, pretty much all of them would, and this change is potentially breaking.

It's very annoying indeed, but I don't see why it would be breaking.

unless we want to avoid ever using IntegerL

FWIW, using IntegerL for Lift seems, on some abstract level, incorrect in the first place. F.ex. you can just lift a negative Int and splice it somewhere where Word is expected, and you'll only learn something went wrong at runtime (well, chances are, you'll also get a warning with -Woverflowed-literals, but that's not on by default). I.e. if I lift an Int I expect it to stay an Int, not arbitrarily and silently mutate into another integral type.

why it would be breaking

Because at this point, boxed integral types are lift-ed into integral literals, so they're in fact polymorphic at the splice site. Even the code here uses this fact (notice mkBytes accepts lenInt as Word, while unpackCStringLen# accepts it as Int instead), so I pretty much guarantee if types suddenly matter, we'll be facing a lot of broken TH downstream.

Because at this point, boxed integral types are lift-ed into integral literals, so they're in fact polymorphic at the splice site. Even the code here uses this fact (notice mkBytes accepts lenInt as Word, while unpackCStringLen# accepts it as Int instead), so I pretty much guarantee if types suddenly matter, we'll be facing a lot of broken TH downstream.

I see, that's nasty. But that's downstream abusing instance Lift Int: the contract of class Lift does not promise anything like this. There is plenty of breakage in the upcoming release of template-haskell anyways.

CC @mpickering on behalf of template-haskell.

In the meantime a PR for text is welcome.

My 5cents. Lift Int could generate 5 :: Int i.e. explicit type annotation. Alternatively it could generate I# 5#.

In meanwhile text could do either.

The untypedness of Lift is known issue. And there aren't cheap way to fix it, as far as I know.

For example, lift [] will produce []. And that is polymorphic. There isn't I# 5# like fix. The only fix is to generate [] @TheElementType like code, and that means lifting types - a mechanism which is not cheap to add.

EDIT: In fact [] @TheElementType will break with RebindableSyntax too (or OverloadedLists). The lists are nasty, as there isn't non-rebindable or non-overloadable syntax for them (except using : and nil :: [a] defined somewhere, but these may produce different code than a list literal).

generate 5 :: Int

Unfortunately, that won't work as GHC will desugar the literal via fromInteger regardless of the type signature/annotation, so you'll get the equivalent of fromInteger 5 :: Int, which doesn't help any.

	lift txt = do
	let (ptr, len) = unsafePerformIO $ asForeignPtr txt
	let lenInt = P.fromIntegral len
	TH.appE (TH.appE (TH.varE 'unpackCStringLen#) (TH.litE . TH.bytesPrimL $ TH.mkBytes ptr 0 lenInt)) (TH.lift lenInt)

The new `lift` implementation in text-2.0.1 breaks with `RebindableSyntax`