There’s sometimes an extra “0” inside the stream shifting all values to the next channel.
satoer opened this issue · comments
Hi, first of all, thank you for the library. From all the DMX library’s for the ESP32 this one seems to work the best. Sadly in my case, not perfect.
I’m using a RS485 module with a ESP32 (the DMX cable is terminated). I want to read the incoming DMX stream, and this seems to work almost fine. The only problem I am having, is that it adding a single “0” somewhere inside the dmx stream at a random position, shifting other values to the next channel this is happening at random times. Sometimes multiple times a second, sometimes it stays quiet for a couple of seconds.
Here is a output (with timestamp) of a readout of 20 channels, I’m using a Arya DMX controller which I configured to add 10 to every channel (so you can see the shift more clearly):
15:54:37.596 -> 10, 20, 30, 40, 50, 60, 70, 80, 90, 100, 110, 120, 130, 0, 140, 150, 160, 170, 180, 190,
15:54:37.701 -> 10, 20, 30, 40, 50, 60, 70, 80, 90, 100, 110, 120, 130, 140, 150, 160, 170, 180, 190, 200,
15:54:39.186 -> 10, 20, 30, 40, 50, 60, 70, 80, 90, 100, 110, 0, 120, 130, 140, 150, 160, 170, 180, 190,
15:54:39.256 -> 10, 20, 30, 40, 50, 60, 70, 80, 90, 100, 110, 120, 130, 140, 150, 160, 170, 180, 190, 200,
15:54:41.587 -> 10, 20, 30, 40, 50, 60, 70, 0, 80, 90, 100, 110, 120, 130, 140, 150, 160, 170, 180, 190,
15:54:41.657 -> 10, 20, 30, 40, 50, 60, 70, 80, 90, 100, 110, 120, 130, 140, 150, 160, 170, 180, 190, 200,
15:54:43.299 -> 10, 20, 30, 40, 0, 50, 60, 70, 80, 90, 100, 110, 120, 130, 140, 150, 160, 170, 180, 190,
15:54:43.367 -> 10, 20, 30, 40, 50, 60, 70, 80, 90, 100, 110, 120, 130, 140, 150, 160, 170, 180, 190, 200,
15:54:46.707 -> 10, 20, 30, 40, 50, 60, 70, 80, 90, 100, 110, 120, 130, 140, 150, 160, 170, 180, 190, 0,
15:54:46.809 -> 10, 20, 30, 40, 50, 60, 70, 80, 90, 100, 110, 120, 130, 140, 150, 160, 170, 180, 190, 200,
15:54:49.406 -> 10, 20, 30, 40, 50, 60, 70, 80, 90, 100, 0, 110, 120, 130, 140, 150, 160, 170, 180, 190,
15:54:49.474 -> 10, 20, 30, 40, 50, 60, 70, 80, 90, 100, 110, 120, 130, 140, 150, 160, 170, 180, 190, 200,
15:54:49.715 -> 10, 20, 30, 40, 50, 60, 70, 80, 90, 0, 100, 110, 120, 130, 140, 150, 160, 170, 180, 190,
15:54:49.782 -> 10, 20, 30, 40, 50, 60, 70, 80, 90, 100, 110, 120, 130, 140, 150, 160, 170, 180, 190, 200,
15:54:54.426 -> 10, 20, 30, 40, 50, 60, 70, 80, 90, 100, 110, 120, 0, 130, 140, 150, 160, 170, 180, 190,
15:54:54.495 -> 10, 20, 30, 40, 50, 60, 70, 80, 90, 100, 110, 120, 130, 140, 150, 160, 170, 180, 190, 200,
15:54:55.995 -> 10, 20, 30, 40, 50, 60, 70, 0, 80, 90, 100, 110, 120, 130, 140, 150, 160, 170, 180, 190,
15:54:56.063 -> 10, 20, 30, 40, 50, 60, 70, 80, 90, 100, 110, 120, 130, 140, 150, 160, 170, 180, 190, 200,
15:55:06.447 -> 10, 20, 30, 40, 50, 60, 0, 70, 80, 90, 100, 110, 120, 130, 140, 150, 160, 170, 180, 190,
15:55:06.516 -> 10, 20, 30, 40, 50, 60, 70, 80, 90, 100, 110, 120, 130, 140, 150, 160, 170, 180, 190, 200,
15:55:07.545 -> 10, 20, 30, 40, 50, 60, 70, 80, 90, 100, 110, 120, 130, 140, 150, 0, 160, 170, 180, 190,
15:55:07.613 -> 10, 20, 30, 40, 50, 60, 70, 80, 90, 100, 110, 120, 130, 140, 150, 160, 170, 180, 190, 200,
15:55:08.881 -> 10, 20, 30, 40, 50, 60, 70, 0, 80, 90, 100, 110, 120, 130, 140, 150, 160, 170, 180, 190,
15:55:08.983 -> 10, 20, 30, 40, 50, 60, 70, 80, 90, 100, 110, 120, 130, 140, 150, 160, 170, 180, 190, 200,
15:55:10.076 -> 10, 20, 30, 40, 50, 60, 70, 80, 90, 0, 100, 110, 120, 130, 140, 150, 160, 170, 180, 190,
15:55:10.177 -> 10, 20, 30, 40, 50, 60, 70, 80, 90, 100, 110, 120, 130, 140, 150, 160, 170, 180, 190, 200,
15:55:14.580 -> 10, 20, 30, 40, 50, 60, 0, 70, 80, 90, 100, 110, 120, 130, 140, 150, 160, 170, 180, 190,
15:55:14.648 -> 10, 20, 30, 40, 50, 60, 70, 80, 90, 100, 110, 120, 130, 140, 150, 160, 170, 180, 190, 200,
15:55:16.055 -> 10, 20, 30, 40, 50, 60, 70, 80, 90, 0, 100, 110, 120, 130, 140, 150, 160, 170, 180, 190,
15:55:16.124 -> 10, 20, 30, 40, 50, 60, 70, 80, 90, 100, 110, 120, 130, 140, 150, 160, 170, 180, 190, 200,
15:55:16.737 -> 10, 20, 30, 40, 50, 60, 70, 80, 90, 100, 110, 120, 130, 140, 150, 0, 160, 170, 180, 190,
15:55:16.805 -> 10, 20, 30, 40, 50, 60, 70, 80, 90, 100, 110, 120, 130, 140, 150, 160, 170, 180, 190, 200,
15:55:19.271 -> 10, 20, 30, 40, 50, 60, 0, 70, 80, 90, 100, 110, 120, 130, 140, 150, 160, 170, 180, 190,
15:55:19.337 -> 10, 20, 30, 40, 50, 60, 70, 80, 90, 100, 110, 120, 130, 140, 150, 160, 170, 180, 190, 200,
15:55:23.969 -> 10, 20, 30, 40, 50, 60, 70, 80, 90, 100, 110, 120, 130, 140, 150, 160, 170, 0, 180, 190,
15:55:24.038 -> 10, 20, 30, 40, 50, 60, 70, 80, 90, 100, 110, 120, 130, 0, 140, 150, 160, 170, 180, 190,
15:55:24.108 -> 10, 20, 30, 40, 50, 60, 70, 80, 90, 100, 110, 120, 130, 140, 150, 160, 170, 180, 190, 200,
As you can see there’s an extra 0 which shifts the values to the next channel.
I've put on a logic analyser, on the dmx side and the serial side. Both output the same data. I recorded couple of seconds when the anomaly happens. Analysed it as DMX data, and the data stays exactly the same (no anomaly in the stream). One thing I noticed. The DMX mixer only outputs 192 values / channels, maybe that's the problem?
Here's a piece of dmx data stream which constantly repeats (with another start code):
0.762522000000000,10,20,30,40,50,60,70,80,90,100,110,120,130,140,150,160,170,180,190,200,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,101,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
0.837180000000000,10,20,30,40,50,60,70,80,90,100,110,120,130,140,150,160,170,180,190,200,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,101,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
This is the end and start of the stream:
0.106832750000000,DMX-512,Slot 191: '0'
0.106864500000000,DMX-512,stop bits
0.106872750000000,DMX-512,MARK Time after Slot
0.107244750000000,DMX-512,start bit
0.107248750000000,DMX-512,Slot 192: '0'
0.107280500000000,DMX-512,stop bits
0.107288750000000,DMX-512,MARK Time after Slot
0.112497750000000,DMX-512,BREAK
0.112634000000000,DMX-512,Mark After Break
0.112642500000000,DMX-512,start bit
0.112646500000000,DMX-512,START CODE: '0'
0.112678250000000,DMX-512,stop bits
0.112686500000000,DMX-512,MARK Time after Slot
0.113030250000000,DMX-512,start bit
0.113034250000000,DMX-512,Slot 1: \n
0.113066000000000,DMX-512,stop bits
0.113074250000000,DMX-512,MARK Time after Slot
0.113446250000000,DMX-512,start bit
0.113450250000000,DMX-512,Slot 2: '20'
0.113482000000000,DMX-512,stop bits
0.113490250000000,DMX-512,MARK Time after Slot
I’m thinking of doing a dirty fix by checking if the neighbouring channel has inherited the value and if so, ignoring the complete DMX stream. But I’m hoping there might be a neater solution.
Hi! I have a such problem too and thinked about this fix. I think, it is not a good idea when we have about 18fps,
Hi again! It works properly when installed Stable release via Boards Manager. I think, problem in a new version of esp32 hal uart file.
The latest commit has hal uart files for both the github version and the boards manager version, depending on which you have installed, you can replace the file appropriately.
It is advisable to save the original version of the esp32-hal-uart.c file in a location that is outside of the build path. That way if it fails to compile, you can replace the original.
You can modify the esp32-hal-uart.c file yourself. There are 3 changes that are clearly marked in the files in the modified_hal-uart folder.
--All 3 changes are marked with the //begin mod ***** comments.
--The modifications are designed to be disabled with a preprocessor macro #define USE_SLIP_FOR_BREAK_DETECT 1
The necessity of using the modified esp32-hal-uart.c file is that it puts a special character in the RTOS queue that designates that a break is detected. When less than 512 addresses are contained in a DMX packet, the break marks the start of a new packet. Without the break detection, there will likely be out of sequence data in the queue. It is possible that on edge cases where a device has a short break in the DMX stream, the esp32 will not detect the break and data may become out of sequence.
Because of the async nature of RTOS and the DMX input stream, there are potential timing issues with DMX serial (the length of the break and mark-after-break) that may be legal by the DMX standard but are not in the middle of typical implementations. Due to this, esp32 may not be the best choice of microcontroller for DMX input applications.
It is possible that the while loop in the modified esp32-hal-uart.c file is reading past when the break is received. You might try modifying it to check for the break flag each trip through the while loop:
#if USE_SLIP_FOR_BREAK_DETECT
// if break detect interrupt is enabled, use SLIP encoding
if ( uart->dev->int_ena.brk_det ) {
while(uart->dev->status.rxfifo_cnt) {
c = uart->dev->fifo.rw_byte;
// encode incoming byte using SLIP
// SLIP is simple and uses a special byte SLIP_END to mark the end
// of a packet. If the special byte appears as data, it is escaped
// as follows:
// if byte == SLIP_END replace with SLIP_ESC followed by SLIP_ESC_END
if ( c == SLIP_END ) {
c = SLIP_ESC;
if(uart->queue != NULL && !xQueueIsQueueFullFromISR(uart->queue)) {
xQueueSendFromISR(uart->queue, &c, &xHigherPriorityTaskWoken);
}
c = SLIP_ESC_END;
if(uart->queue != NULL && !xQueueIsQueueFullFromISR(uart->queue)) {
xQueueSendFromISR(uart->queue, &c, &xHigherPriorityTaskWoken);
}
// if byte == SLIP_ESC replace with SLIP_ESC followed by SLIP_ESC_ESC
} else if ( c == SLIP_ESC ) {
c = SLIP_ESC;
if(uart->queue != NULL && !xQueueIsQueueFullFromISR(uart->queue)) {
xQueueSendFromISR(uart->queue, &c, &xHigherPriorityTaskWoken);
}
c = SLIP_ESC_ESC;
if(uart->queue != NULL && !xQueueIsQueueFullFromISR(uart->queue)) {
xQueueSendFromISR(uart->queue, &c, &xHigherPriorityTaskWoken);
}
//otherwise just queue the byte
} else {
if(uart->queue != NULL && !xQueueIsQueueFullFromISR(uart->queue)) {
xQueueSendFromISR(uart->queue, &c, &xHigherPriorityTaskWoken);
}
}
//NEW CHECK FOR BREAK
if (uart->dev->int_st.brk_det) {
c = SLIP_END;
if(uart->queue != NULL && !xQueueIsQueueFullFromISR(uart->queue)) {
xQueueSendFromISR(uart->queue, &c, &xHigherPriorityTaskWoken);
uart->dev->int_clr.brk_det = 1;
}
//uart->dev->int_clr.brk_det = 1;
}
//END NEW CHECK FOR BREAK
} //end while
//current break check follows while loop
// send SLIP END when break is detected to mark end of packet
if (uart->dev->int_st.brk_det) {
c = SLIP_END;
if(uart->queue != NULL && !xQueueIsQueueFullFromISR(uart->queue)) {
xQueueSendFromISR(uart->queue, &c, &xHigherPriorityTaskWoken);
uart->dev->int_clr.brk_det = 1;
}
//uart->dev->int_clr.brk_det = 1;
}
} else {
// else break detect not enabled do it like not modified
#endif
It is possible that this would cause a different problem if unread bytes are left in the receive buffer when the break is detected. In which case, instead of the break being marked in the input queue late, it woud be marked early.
I'm re-opening this issue so if you test this, you can post your results. If it works, the modification can be included with the library.
Hello,
I just wanted to note that we solved this problem by changing the buffer size in HardwareSerial.cpp:
_uart = uartBegin(_uart_nr, baud ? baud : 9600, config, rxPin, txPin, 256, invert);
changed to
_uart = uartBegin(_uart_nr, baud ? baud : 9600, config, rxPin, txPin, 1024, invert);
void HardwareSerial::begin(unsigned long baud, uint32_t config, int8_t rxPin, int8_t txPin, bool invert, unsigned long timeout_ms)
{
if(0 > _uart_nr || _uart_nr > 2) {
log_e("Serial number is invalid, please use 0, 1 or 2");
return;
}
if(_uart) {
end();
}
if(_uart_nr == 0 && rxPin < 0 && txPin < 0) {
rxPin = 3;
txPin = 1;
}
if(_uart_nr == 1 && rxPin < 0 && txPin < 0) {
rxPin = RX1;
txPin = TX1;
}
if(_uart_nr == 2 && rxPin < 0 && txPin < 0) {
rxPin = RX2;
txPin = TX2;
}
_uart = uartBegin(_uart_nr, baud ? baud : 9600, config, rxPin, txPin, 1024, invert);
if(!baud) {
time_t startMillis = millis();
unsigned long detectedBaudRate = 0;
-Alan