Issue with drop partition when it contains null and type is INT/DATE

[thomaskreutzer]

-- In order to work around the issue change the column type to string
ALTER TABLE tablename PARTITION COLUMN (partn STRING);
-- Drop the offending partitions
ALTER TABLE tablename DROP PARTITION (partn='HIVE_DEFAULT_PARTITION');
-- Change the column type back to int
ALTER TABLE tablename PARTITION COLUMN (partn INT);

Possibly hive SRE can be enhanced to detect this and write the above statements when it would not be possible for the empty partitions to be dropped due to mismatch of data type.

[Alex Dedov]

Not sure if thats related, but after ID'ing missing HDFS locations based on Hive partition metadata analysis, SRE generates a bunch of mkdir directives. Some of these commands, if applied, seem to be capable of exacerbating the problem. For example:

mkdir -p "hdfs://namotdev2/data/gfctocon/work/hive/marketquoteevent/quotedate=__HIVE_DEFAULT_PARTITION__/marketquotesource=BLOOMBERG/loaddate=__HIVE_DEFAULT_PARTITION__"
In the above table, partition columns loaddate and quotedate are defined as DATE. I've see similar issue with INTs as well.