String > 1KB gets rejected

[Shubanker Chourasia]

While trying to Insert a field with a longer string, the insertion fails with error:

Failed to insert document with _id '65c212450bde51153a4e28fd': Term of column query_text_values exceeds the byte limit for index. Term size 1.182KiB. Max allowed size 1.000KiB.

I could not find a config to increase this limit.

This seems to be coming from limit of index, is there a way to disable indexing for a field/collection ?

Minimalistic code to reproduce this issue:

const Test = mongoose.model('test', new mongoose.Schema({ text: String }));
const doc = new Test({ text: 'Test'.repeat(300) });
await doc.save();

I also tried skipping mongoose, and created collection with connection.collection from mongoose.connection and inserted data, yet got same error.

[Valeri Karpov]

Disable indexing on the offending property. We actually just had to do this in stargate/stargate-mongoose-sample-apps#211.

await Test.createCollection({
  indexing: {
    deny: ['text']
  }
});

Keep in mind this means you won't be able to query by text property.

[Shubanker Chourasia]

Disable indexing on the offending property. We actually just had to do this in stargate/stargate-mongoose-sample-apps#211.

await Test.createCollection({
  indexing: {
    deny: ['text']
  }
});

Keep in mind this means you won't be able to query by text property.

Somehow this isn't working for me even in the sample code above, I even tried with a new non-existing collection still same.

this is what I ran:

const Test = mongoose.model('test1', new mongoose.Schema({ text: String }));
await Test.createCollection({
    indexing: { deny: ['text'] }
});
// await Promise.all(Object.values(mongoose.connection.models).map(Model => Model.init()));
const doc = new Test({ text: 'Test'.repeat(300) });
await doc.save();

[Valeri Karpov]

Do you have mongoose.set('autoCreate', true) enabled? If so, try disabling that. autoCreate: true makes Mongoose automatically create the collection, which may mean the collection is already in the process of being created by the time you call Test.createCollection({ indexing })

[Shubanker Chourasia]

Do you have mongoose.set('autoCreate', true) enabled? If so, try disabling that. autoCreate: true makes Mongoose automatically create the collection, which may mean the collection is already in the process of being created by the time you call Test.createCollection({ indexing })

Yes I have tried it with autoCreate set to false, still the same.

mongoose.set('autoCreate', true);
const Test = mongoose.model('test1', new mongoose.Schema({ text: String }));
await Test.createCollection({
    indexing: { deny: ['text'] }
});
// await Promise.all(Object.values(mongoose.connection.models).map(Model => Model.init()));
const doc = new Test({ text: 'Test'.repeat(300) });
await doc.save();

[Valeri Karpov]

Your code has mongoose.set('autoCreate', true);, try mongoose.set('autoCreate', false); ?

[Shubanker Chourasia]

Your code has mongoose.set('autoCreate', true);, try mongoose.set('autoCreate', false); ?

Oh my bad in copying, I was trying with both combos, even with false it had no difference (I have been using new collection name each time)

[Valeri Karpov]

Did you delete the collection before trying with mongoose.set('autoCreate', false);? Sorry for the repetitive questions, it's just that I've run into similar issues a couple of times and it was always because I hadn't dropped the old collection with old options first.

[Shubanker Chourasia]

Did you delete the collection before trying with mongoose.set('autoCreate', false);? Sorry for the repetitive questions, it's just that I've run into similar issues a couple of times and it was always because I hadn't dropped the old collection with old options first.

I could try dropping existing collection, however I have been changing collection name each time, tried randomising it just to ensure I don't miss changing then name:

mongoose.set('autoCreate', false)
const Test = mongoose.model(`test_${Math.floor(Math.random() * 1000)}`, new mongoose.Schema({ text: String }));
await Test.createCollection({
    indexing: { deny: ['text'] }
});
const doc = new Test({ text: 'Test'.repeat(300) });
await doc.save();

I also debugged it to check the request passed into jsonAPI, seems it is passing the options correctly.

[Valeri Karpov]

I was able to repro this in stargate-mongoose@0.5.0 using the following script:

'use strict';

const mongoose = require('mongoose');

mongoose.set('autoCreate', false);
mongoose.set('autoIndex', false);

const { driver } = require('stargate-mongoose');
mongoose.setDriver(driver); 

run().catch(err => console.log(err));

async function run() {
  await mongoose.connect(process.env.ASTRA_URI, { isAstra: true });

  const Test = mongoose.model(`test_${Math.floor(Math.random() * 1000)}`, new mongoose.Schema({ text: String }));
  await Test.createCollection({
    indexing: { deny: ['text'] }
  });
  const doc = new Test({ text: 'Test'.repeat(5000) });
  await doc.save();
  console.log('Done');
}

However, I then checked with the recently released stargate-mongoose@0.5.1 and that fixed it. Looks like #183 fixed this issue, so please upgrade to stargate-mongoose@0.5.1 for the fix.