Allow Haskell migrations before and after running each .sql files

[Kostas Dermentzis]

applyMigration runs the migration of a .sql file as a cmd command. This means it cannot combine it in a single transaction with the runHaskellMigration. We could use the Cardano.Db.Run utillities for the files instead. We could also add Haskell migrations that run before the .sql files migrations again in the same transaction.

[Vincent Orr]

We could also add Haskell migrations that run before the .sql files migrations again in the same transaction.

@kderme I'm not entirely clear what is meant by this, if I understand this issue correct we are trying to consolidate all the queries in each .sql file and for them to just run once?

So in the code bellow we'd want to try consolidate everything (maybe as Text) and we could do a single rawSql?

cardano-db-sync/cardano-db/src/Cardano/Db/Migration.hs

Lines 110 to 129 in 045330d

 ranAll <- case (mLogfiledir, allScripts) of 
   (_, []) -> 
     error $ "Empty schema dir " ++ show migrationDir 
   (Nothing, schema : scripts) -> do 
     putStrLn "Running:" 
     applyMigration' Nothing stdout schema 
     (scripts', ranAll) <- filterMigrations scripts 
     forM_ scripts' $ applyMigration' Nothing stdout 
     putStrLn "Success!" 
     pure ranAll 
   (Just logfiledir, schema : scripts) -> do 
     logFilename <- genLogFilename logfiledir 
     withFile logFilename AppendMode $ \logHandle -> do 
       unless quiet $ putStrLn "Running:" 
       applyMigration' (Just logFilename) logHandle schema 
       (scripts', ranAll) <- filterMigrations scripts 
       forM_ scripts' $ applyMigration' (Just logFilename) logHandle 
       unless quiet $ putStrLn "Success!" 
       pure ranAll 
 pure (ranAll, map (takeFileName . snd) (filter isUnofficialMigration allScripts)) 

If we parse all migrations files into a single query would I be right we could strip away:

CREATE FUNCTION migrate() RETURNS void AS $$
DECLARE
  next_version int ;
BEGIN
  SELECT stage_two + 1 INTO next_version FROM schema_version ;
  IF next_version = 32 THEN
  --------------------------------------
  -- the queries here we consolidate  --
  --------------------------------------
    -- Hand written SQL statements can be added here.
    UPDATE schema_version SET stage_two = next_version ;
    RAISE NOTICE 'DB has been migrated to stage_two version %', next_version ;
  END IF ;
END ;
$$ LANGUAGE plpgsql ;

SELECT migrate() ;

DROP FUNCTION migrate() ;