Enhanced Checks

[Nathan Huebner]

Before I fill this out, you're welcome 100% to just ignore this feature request, but it's something I was trying to alter in your code myself, and I only managed to get one part working. The reason I am suggesting this is because we have a huge database, it was originally created by hand, then 30% of it from migrations. Now we want to go back and fix the migrations - this tool makes it possible, but every time a migration gets generated I have to manually go through and add conditional statements so that the migrations can safely run on production and not break these already existing operations. I know you would say "why don't you just add the migration list to the migrations table" - that's an easier way to do it, but I have superstitious coworkers, so we're fortifying our migrations to make sure there's only sunshine :)

So again, the idea is for example:

Schema::create would be wrapped in: NOT Schema::hasTable (php condition)
Schema::table would be wrapped in: Schema::hasTable (php condition)
Foreign Key Drop would be wrapped in: HasForeignKey (we have a helper function for this in DatabaseUtils::hasForeignKey
Foreign Key Create would also be wrapped to make sure it doesn't already exist.

And yes I could just re-run the whole database and output the migrations, that would probably be the right way to go - but we're talking about 1500 tables, 5 databases, on SQL Server 2019 - It's scary as hell to think your program would perfectly rewrite every migration in perfect order and be sound. I would be shocked, because I consider myself a senior, and the code you're writing here is senior larevel code, and I still make mistakes. It would just be shocking though if it worked perfectly with such a terrible scenario (ie: SQL Server, tons of tables, Views, Stored Procedures, Indexes, etc). It's kind of madness in our DB.

Now for the form below - Thanks for reading all of this.

Is your feature request related to a problem? Please describe.
Not your problem, but more along the lines of more manual work I have to perform. You're getting us 99.9% there with this package, so it's not a huge burden to manually modify the files.

Describe the solution you'd like
Perhaps add an option --strict-checks, which basically makes sure any time something is created it does not exist, and any time something is dropped it hasn't already been dropped - any time something is modified, make sure it exists before changing it. The idea is to prevent failure of migration - if the migration was designed to add a column, the column shouldn't exist. This is just the nature of a dirty database and migration catch up.

Describe alternatives you've considered
Manually wrapping statements with conditional statements to get the result I need.

Additional context
Yeh I pasted a snippet below to show you what I did:

SchemaBlueprint -- Don't worry, I butchered your nice code real good LOL, sorry. And my indentions need to be fixed grrr.

if ($this->blueprint !== null) {

            // Wrap Schema Start/End added by Nathan.
            $wrap_schema_start = "if (Schema::hasTable('{$tableWithoutPrefix}')) {";
            $wrap_schema_end = '}';
            if (str_contains($schema, 'create')) {
                $wrap_schema_start = "if (!Schema::hasTable('{$tableWithoutPrefix}')) {";
                $wrap_schema_end = '}';
            }

            if ($wrap_schema_start) {
                $lines[] = $wrap_schema_start;
            }
            $lines[] = ($wrap_schema_start ? Space::TAB() : '') . "$schema('$tableWithoutPrefix', static function (Blueprint \$table) {";
            // Add 1 tabulation to indent(prettify) blueprint definition.
            $lines[] = ($wrap_schema_start ? Space::TAB() : '') . Space::TAB() . $this->blueprint->toString();
            $lines[] = ($wrap_schema_start ? Space::TAB() : '') . "});";
            if ($wrap_schema_end) {
                $lines[] = $wrap_schema_end;
            }
            return $lines;
        }

The above code updates generated this up statement. I'm not really interested in down statements at this time, but if you do decide to make updates on my concept, it might be good to include those in the down as well. Think of it as hardening / concretely handling the migrations so they can succeed and still be effective. Order of running them is important as well.

if (!Schema::hasTable('wh_totes')) { // THIS IS IMPORTANT
            Schema::create('wh_totes', static function (Blueprint $table) {
                $table->bigIncrements('id');
                $table->timestamps();
                $table->integer('warehouse_id');
                $table->string('name');
                $table->float('length', 0, 0);
                $table->float('width', 0, 0);
                $table->float('height', 0, 0);
                $table->boolean('can_consolidate')->default(true);
            });
        }

        if (Schema::hasTable('wh_totes')) { // THIS MIGHT BE IMPORTANT
            Schema::table('wh_totes', static function (Blueprint $table) {
                
                if (Schema::hasTable('wh_warehouses')) { // THIS IS IMPORTANT
                    $table->foreign(['warehouse_id'])->references(['id'])->on('wh_warehouses')->onUpdate('NO ACTION')->onDelete('NO ACTION');
                }
            });
        }

[Nathan Huebner]

Either way, I'm thankful you gave me support for MSSQL. I hate SQL Server, I will use anything else - but that's what my job gave us, until we can get to MySQL.

Thank you so much again!!!!

If you decide not to implement this feature, no big deal. If you like this feature and think it's something I could handle, let me know how I should start. Your code is very object oriented, and I don't fully understand completely yet how you generate so much of it without raw strings. It's impressive the way it was built.

[Nathan Huebner]

Also another way to go about this might be a dependency management system.

For example, if I generate a table migration, it may have a dependency to another table (via foreign key). Your code generates the migration, but it has no idea if that dependency table exists.

Perhaps you could add --dependency-protection and then generate a new function called checkDependencies after down(), run that during up() and if for example a dependency table is missing, the migration halts.

In theory, the migration should halt when it encounters a missing dependency, but I don't believe migrations are Transactional. Maybe it's possible to make the migrations transactional? If it errors out, cancel out the transaction? --use-transactions?

Not sure, lots to think of here to keep people from screwing up their database.

Maybe you have some ideas Kitlooong. Thank you again!

[Nathan Huebner]

In the end, migrations are the site owner / developer / DBA's responsibility.

All of this stuff might help someone later down the road, but then again your system is getting us all 99.9% to the finish line. And if we really put our trust in it, maybe 100%.

[Kit Loong]

Hi @NHuebner1983 , thank you for your suggestion.

Finally, I have time to sit down and read thoroughly your post.

First of all, thanks for using the Laravel Migrations Generator, I am happy this tool helps you and your team.

Before your issue, I would never thought to perform the Schema::hasTable check because a migration is meant to create/alter something new against the database, an "exist check" seems redundant to me.

I believe you already know: when you run php artisan migrate, any migration listed in the migrations table will not execute, this is the main reason this package has an option for you to log the migrations in advance.

I would have first generated migration files and the migrations table in a dev environment, make sure things are working as expected then import the migrations table into production. I have been using this feature in multiple production projects and this never fails.

Now I do understand the case is different in yours given you have 1k+ tables in multiple databases, which I would definitely test well in the dev environment before dumping into production.

Perhaps add an option --strict-checks, which basically makes sure any time something is created it does not exist, and any time something is dropped it hasn't already been dropped.

I am not fully convinced yet, but I might be wrong. Currently, I am working on stored procedure creation. For the table, we have a convenient built-in Schema::hasTable check, but Views and Stored Procedures are not.

Perhaps you could add --dependency-protection and then generate a new function called checkDependencies after down()

Injecting a custom function into the migration file is out of an option. Imagine:

1. We install the generator with composer require --dev in the dev environment.
2. Generate migrations.
3. composer install --optimize-autoloader --no-dev in another environment.
4. Run php artisan migrate

Migrations should be executed successfully without the package.

but I don't believe migrations are Transactional

I think DDL is not transactional, at least not for all DBs.

To summarize, I would not fully turn down the idea of having a --strict-checks, but when this solves the table issue, it should solve the view and stored procedure too. Right now, I can't see a straightforward way to implement checking of the later 2 assets.

I really appreciate your feedback, and you have proof your suggestion has changed my mind about having the --strict-checks.

I will keep this in my roadmap and definitely will revisit it when I (or you) have a complete implementation of --strict-checks.

[Nathan Huebner]

Kitloong - Feel free to come back here later. You're right about everything.

I can't wait to hear how Stored Procedures go. We're hoping to get rid of them at some point and have them built in Laravel instead. What you're working on is way more important than my suggestion so, good luck and keep going.

Thank you again <3