Issue with WithMaxMsgSize
SkinnySackboy opened this issue · comments
Hi,
I have some existing JetStreams which work, but now I need to exceed the default maximum message size. I can see that by default the parameter is -1 but that in theory the default size is 1MB. I am trying to do this in code by calling WithMaxMsgSize(5000000) to increase the limit from 1MB to 5MB when I call jetStreamManagementContext.AddStream(streamConfiguration.Build())
This passes without issue but then fails when I attempt to publish my message. I am pretty confident my message is below the limit, but I still get an NATS.Client.NATSMaxPayloadException: 'Maximum payload size has been exceeded'.
Am I doing something wrong?
Thanks in advance.
Edit: If I manually set the config on the server itself (max_payload = 5MB) this works, so I guess the question is how come setting it programmatically via the streamConfiguration does not?
@SkinnySackboy So the default max payload on the server is 1048576 bytes. All streams will adhere to the server config. The WithMaxMsgSize sets the maximum size for that stream which can be less than or equal to the server setting.
Understood thanks. It wasn't clear to me that it could be <= and not > because it doesn't fail when updating the stream definition, it only fails when when I try to publish a message which is too large.
ok I have more information. So after your comment, I went to dig in some. So the client's have a check in them that prevents a large message. When it goes to publish, it knows about the server configuration and checks your payload against what the server told it it allows (ServerInfo) NATSMaxPayloadException is thrown if this limit is crossed.
The java client even has an connection option that says "don't check max payload" (it's on by default) but .NET doesn't
In java if you set clientSideLimitChecks = false, what happens is the server returns an error on publish.
I've asked the server team this: "How come I can create a stream with max_msg_size greater than the server max_payload ?" am waiting for an answer?
Awesome thank you very much!
So what I've learned so far, is that I've been asked to remove the checks from the client and let the server return an error. Think about this. A client could be connected to a leaf node that has once configuration, but the jetstream server has another, the stream has another. Topologies could change. Server and stream configs could change. So it's a good idea to setup your stream for the limits, but the clients should let the server be the authority when publishing violates that.