morganstanley / hobbes

Here is the test case, you can see there is no issue for raw storage, but will generate core dump(not always but very offen) file for compressd file :

`
#include
#include <hobbes/hobbes.H>
#include <hobbes/storage.H>
#include <hobbes/db/file.H>
#include <hobbes/db/series.H>
#include <hobbes/lang/tylift.H>

const int option = 100;
struct TestEnum {
enum Enum : uint32_t{
Invalid = 0,
Option = option
};
Enum _value;
using is_enum = void;
};

namespace hobbes {
template
struct liftTestEnum
{
static MonoTypePtr type(typedb&) {
Variant::Members vms;
auto i = 0;
vms.push_back(Variant::Member("Invalid", primty("unit"), 0));
vms.push_back(Variant::Member("Option", primty("unit"), option));
return MonoTypePtr(Variant::make(vms));
}
};
template
struct lift<T*, false, typename T::is_enum> : public liftTestEnum {};
template
struct lift<T, true, typename T::is_enum> : public liftTestEnum {};
}

DEFINE_STRUCT(
FixarrayS,
(TestEnum, value),
(const hobbes::array*, body)
);

void run(const std::string& fname, bool compressed) {
hobbes::cc c;
FixarrayS v1;
v1.body = hobbes::makeString("Hello world!");
v1.value._value = TestEnum::Option;

hobbes::writer writer{fname};
hobbes::series ss1(&c, &writer, "udata", 10000, compressed? hobbes::StoredSeries::Compressed : hobbes::StoredSeries::Raw);
ss1(v1);
std::cout << "------------testing -----------------" << std::endl;
c.define("f", "inputFile :: (LoadFile "" + fname + "" w) => w");
c.compileFn<void()>("print([(x.value, (unsafeCast(x.value)::{t:int}).t) | x<-f.udata])")();
}

int main()
{
std::cout << "=============Create Raw Stream [OK]=============" << std::endl;
run("/var/tmp/test.log", false);
std::cout << "=============Create Compressed Stream [Failed]=============" << std::endl;
run("/var/tmp/testc.log", true);
return 0;
}
`

it seems need change here:

hobbes/lib/hobbes/db/cbindings.C

Line 17 in 90c9865

uint8_t maxSymbol;

right now hobbes only support the enum without customized values

https://github.com/morganstanley/hobbes/blob/main/include/hobbes/cfregion.H#L850

Your enum here is an Int in disguise, not a variant. With the changes made here, you can keep your type as enums while keeping the static information as well.

Is this something you can do?

hi : an interactive shell for hobbes
      type ':h' for help on commands

> db=inputFile :: (Argument "file" p, LoadFile p _) => _
> :t db.udata
(cseq { test:(test |invalid:(Test 0L), option:(Test 100L)|), string:[char] } ({ pmodels:[:{ freqs:[:int|256L:], activeFreqs:[:int|256L:], c:int }|4L:] } * ({ pmodels:[:{ freqs:[:int|256L:], activeFreqs:[:int|256L:], c:int }|8L:] } * { pmodels:[:{ freqs:[:int|256L:], activeFreqs:[:int|256L:], c:int }|1L:] })) 10000L)

seem use fregion is an options. will have a try

Failed to compress slog file with enum type