Writing Id::Classification values greater 31 to LAS 1.4

[Alexander Nehrbaß]

Describe the bug
We are running into a potential bug when trying to write Id::Classification values greater 31 into a LAS 1.4 file with PDAL starting from version 2.6.0.

I converted how we us PDAL for this into an unit test to illustrate the issue:

TEST(LasWriterTest, las14_classification_greater_31)
{
    using namespace Dimension;

    const std::string FILENAME(Support::temppath("synthetic_test.las"));
    PointTable table;

    table.layout()->registerDims({Id::X, Id::Y, Id::Z, Id::Classification});

    BufferReader bufferReader;

    PointViewPtr view(new PointView(table));
    view->setField(Id::X, 0, 1.0);
    view->setField(Id::Y, 0, 2.0);
    view->setField(Id::Z, 0, 3.0);
    view->setField(Id::Classification, 0, 32);
    bufferReader.addView(view);

    Options writerOps;
    writerOps.add("filename", FILENAME);
    writerOps.add("minor_version", "4");

    LasWriter writer;
    writer.setOptions(writerOps);
    writer.setInput(bufferReader);

    writer.prepare(table);
    writer.execute(table);

    Options readerOps;
    readerOps.add("filename", FILENAME);

    PointTable readTable;

    LasReader reader;
    reader.setOptions(readerOps);

    reader.prepare(readTable);
    PointViewSet viewSet = reader.execute(readTable);
    EXPECT_EQ(viewSet.size(), 1u);
    view = *viewSet.begin();
    EXPECT_EQ(view->size(), 1u);
    EXPECT_EQ(32,
        view->getFieldAs<uint8_t>(Id::Classification, 0));

    FileUtils::deleteFile(FILENAME);
}

This unit test is based on the existing LasWriterTest, las14_classflags_from_las14_classflags-test with the following changes:

• we write 32 into Id::Classification
• we don't set Id::Synthetic
• we don't set "dataformat_id" in the Options

This unit test passes in PDAL 2.5.5 and starts failing starting from PDAL 2.6.0 with the following output:

Expected equality of these values:
 32
 view->getFieldAs<uint8_t>(Id::Classification, 0)
   Which is: '\x1' (1)

I.e. we read the classification back as Unclassified.

I traced this change in behavior down to this PR #4186.

Expected behavior
The expected behavior is hard to describe, as I am a bit lost at this point.
Some thoughts:

• Was the behavior of PDAL 2.5.5 incorrect? Should this test have never passed?
• Are we using PDAL incorrectly?
  I can make the test pass in PDAL 2.6.0 (and later) as well by setting writerOps.add("dataformat_id", 7); explicitly.

System/installation information:
PDAL built from source on Ubuntu 22.04

[Andrew Bell]

LAS classification bits are confused and confusing. PDAL 2.6 split each of the classification bits into separate dimensions to help deal with the ambiguity.

It appears that you are writing dataformat_id 3 (the default). Dataformat ID 0-5 only use five bits for classification, so the value "32" won't fit (it is a classification of "0" with the "synthetic" bit set). If what you're trying to do is say that a point is "synthetic", you will now have to use the dimension made for that purpose when loading the data. PDAL does this automatically when you use the LAS reader (readers.las), but since you're loading data from a buffer, PDAL has no idea what you mean and just sets the internal "classification" value to 32, but the high three bits of classification get zero'd when written to a LAS file with dataformat ID 5 or less, which explains the behavior you're seeing.

In previous versions, PDAL would simply pass whatever classification value that was provided to output. This could cause problems as it's not clear whether the high three bits of the classification field represented flags or were part of the classification value itself.

[Alexander Nehrbaß]

@abellgithub Thanks a lot for this clarifying explanation.