Serialplot won't refresh

Question

Serialplot won't refresh

schwukas opened this issue 4 years ago · comments

Hi there, weird issue:

I have serialplot running on my T430 without any problems. Plugging the same board into my Desktop PC, I only can get new data whenever I press "Open". So it appears that serialplot isn't polling for new data and I have to "refresh" it manually. My T430 is running Arch Linux, my Desktop is on Manjaro. Both have qwt 6.1 and the complete qt5 package installed.

Is this a known problem and is there a fix? I'm also happy to provide additional information, bug reports whatever :). I didn't have time to dig into the code, but I'll gladly free up some, if it helps.

Hasan Yavuz ÖZDERYA · Answer 1 · Mon Nov 02 2020 18:10:32 GMT+0800 (China Standard Time)

@schwukas I've never heard of this one before. This sounds like an issue with the Qt library or maybe even deeper issue (driver, kernel, hw etc).

Can you tell me what kind of serialport are you using on your PCs. USB-Uart converter etc.? If you are using motherboard RS232 connector and your desktop maybe you can try a USB converter?

Also which build of SerialPlot are you using. Did you build it yourself? What is your Qt version? Have you tried running AppImage version?

schwukas · Answer 2 · Mon Nov 02 2020 18:26:13 GMT+0800 (China Standard Time)

What a response time 😲

I'm using a cheap generic USB to TTL dongle on both systems.
serialplot -v returns 0.11.0. The AUR package I used to install it seems to build commit 834.866d3c2d410a-1. It's just cloning the latest code from the sourcehut repo.
I did not build it myself but used the "ready-made" version from the Arch User Repository. I can build it myself later, if that helps.
I'm on Qt 5.15.1 (x86_64-little_endian-lp64 shared (dynamic) release build; by GCC 10.2.0) on "xcb" on both systems.
I have tried the AppImage version taken from hackaday.io. Same issue there.
I also tried everything once again with root, but that didn't make a difference.

Hasan Yavuz ÖZDERYA · Answer 3 · Mon Nov 02 2020 18:32:59 GMT+0800 (China Standard Time)

Have you tried another software, a serial port terminal, what do see then?

What data format are you using? Have you tried another format, just select another format from "Data Format" tab, without changing your input. You should potentially see at least gibberish data plotted (if there is a problem in SerialPlots parsers).

schwukas · Answer 4 · Mon Nov 02 2020 18:39:24 GMT+0800 (China Standard Time)

Yes, sorry I haven't mentioned that it's working fine in both the Arduino Serial Plotter and cutecom.

I first have copied all the values from cutecom to serialplot to make sure it's the same. When that didn't work I just played around with every possible combination without a result. When I click on "open" and receive that one little bit of data, everything looks fine. The data makes sense, the plot is drawn correctly. So I'd guess that my settings are ok.

Another thing that might be helpful:
If I open the port in serialplot, I get the one thing of data, as I mentioned. So it seemed like the port was already closed again at this point. However if I then try to open the same serial port from cutecom, it says, that the port cannot be locked. So to me it seems as if the port is still opened in serialplot but nothing is drawn.

Forget to mention as well: I can get the plot to draw something when using the "Simple Binary" but the values are just jumping around randomly.

Hasan Yavuz ÖZDERYA · Answer 5 · Mon Nov 02 2020 19:13:36 GMT+0800 (China Standard Time)

Forget to mention as well: I can get the plot to draw something when using the "Simple Binary" but the values are just jumping around randomly.

This is actually good news. It means there is probably a problem with SerialPlot ascii parser. Which means it is probably something we can solve : )

Do you see any errors on the "Logs" tab?

And can you post a piece of your data that is captured with another tool? Maybe you can also let me know your code that prints the data (if it is simple)?

schwukas · Answer 6 · Mon Nov 02 2020 19:54:06 GMT+0800 (China Standard Time)

No errors in the debug log. Just the opening and closing messages.

I made a simple saw tooth that's hopefully easier to analyse.

void setup() {
  Serial.begin(115200);
}

void loop() {
  for (int i = 0; i < 250; i++) {
    Serial.println(i);
    delay(5);
  }
}

So values from 0 to 250 in increments of 1 are expected.

I'm sure you know what a sawtooth looks like but for completeness' sake:

Hasan Yavuz ÖZDERYA · Answer 7 · Mon Nov 02 2020 21:29:33 GMT+0800 (China Standard Time)

This is a difficult problem for me to reproduce. I need your help.. I've prepared a custom debug image that prints some messages. Can you run this and let me know its command line output while problem is happening? Maybe switch between data formats and see what happens? I suggest you run it with a small number of samples per seconds as it will print a lot of lines.

https://nextcloud.ozderya.net/index.php/s/n9reAk7WsjAtobr

schwukas · Answer 8 · Mon Nov 02 2020 21:54:40 GMT+0800 (China Standard Time)

ASCII - Column Delimiter: comma

[Debug] SerialPlot 0.11.0
[Debug] Revision 866d3c2d410a+
[Debug] Opened port: "ttyACM0"
[Debug] on readData
[Debug] can read line
[Debug] bytes: "14\r\n"
[Debug] firstReadAfterEnable
[Debug] can read line
[Debug] bytes: "41\r\n"
[Debug] numComingChannels:  1
[Debug] samples nc:  1  size: 1
[Debug] numBytesRead: 8
[Debug] on readData
[Debug] numBytesRead: 0
[Debug] on readData
[Debug] numBytesRead: 0
[Debug] on readData
[Debug] numBytesRead: 0
[Debug] on readData
[Debug] numBytesRead: 0
[Debug] on readData
[Debug] numBytesRead: 0
[Debug] on readData
[Debug] numBytesRead: 0
[Debug] on readData
[Debug] numBytesRead: 0
[Debug] on readData
[Debug] numBytesRead: 0
[Debug] on readData
[Debug] numBytesRead: 0
[Debug] on readData
[Debug] numBytesRead: 0
[Debug] Closed port: "ttyACM0"

ASCII - Column delimiter: space

[Debug] SerialPlot 0.11.0
[Debug] Revision 866d3c2d410a+
[Debug] Opened port: "ttyACM0"
[Debug] on readData
[Debug] can read line
[Debug] bytes: "4\r\n"
[Debug] firstReadAfterEnable
[Debug] can read line
[Debug] bytes: "5\r\n"
[Debug] numComingChannels:  1
[Debug] samples nc:  1  size: 1
[Debug] numBytesRead: 6
[Debug] on readData
[Debug] numBytesRead: 0
[Debug] on readData
[Debug] numBytesRead: 0
[Debug] on readData
[Debug] numBytesRead: 0
[Debug] on readData
[Debug] numBytesRead: 0
[Debug] Closed port: "ttyACM0"

I also tried the other ASCII formats but got nothing different.

Using the Binary Format just shows opening and closing the port, nothing else.
If there's anything else I can help you with, just let me know.

Hasan Yavuz ÖZDERYA · Answer 9 · Mon Nov 02 2020 22:55:18 GMT+0800 (China Standard Time)

Well this is going to be tricky. Unfortunately this looks like a limitation/bug of Qt. Every time new data come from device this function is called:

unsigned AsciiReader::readData()
{
    unsigned numBytesRead = 0;

    while(_device->canReadLine()) {
        ... read data here ...
    }

Problem looks like readData is called correctly, but canReadLine (a Qt function) doesn't return true. I suspect this is a problem with buffering. Why it doesn't happen on your other device but happens on this one I have no idea. I suspect driver versions are different.

Can you please try these 2 build as well. 2nd one only adds a couple debug messages. Can you post its output just like before?

2nd: https://nextcloud.ozderya.net/index.php/s/zFcDP7yZzS6Atts

This 3rd build does something different. It clears buffers immediately when port is opened. If it is a buffering problem as I suspect, this might fix the issue. ~~You don't have to post its output if it behaves the same.~~ Post its output as well, it might be helpful.

3rd: https://nextcloud.ozderya.net/index.php/s/WzXLNPqpwPx9pG7

schwukas · Answer 10 · Mon Nov 02 2020 23:50:22 GMT+0800 (China Standard Time)

2nd:

[Debug] SerialPlot 0.11.0
[Debug] Revision 866d3c2d410a+
[Debug] on open: bytes avilable
[Debug] Opened port: "ttyACM0"
[Debug] on readData available: 1
[Debug] readdata read buffer size: 0
[Debug] can't read line:  "\xFE"
[Debug] numBytesRead: 0
[Debug] on readData available: 8
[Debug] readdata read buffer size: 0
[Debug] can't read line:  "\x00\x00\x80\x80\x00\x00\x80\x00"
[Debug] numBytesRead: 0
[Debug] on readData available: 9
[Debug] readdata read buffer size: 0
[Debug] can't read line:  "\x80\x00\x00\x80\x80\x00\x00\x80\x00"
[Debug] numBytesRead: 0
[Debug] on readData available: 9
[Debug] readdata read buffer size: 0
[Debug] can't read line:  "\x00\x00\x00\x80\x80\x00\x00\x80\x00"
[Debug] numBytesRead: 0
[Debug] on readData available: 9
[Debug] readdata read buffer size: 0
[Debug] can't read line:  "\x80\x00\x00\x80\x80\x00\x00\x80\x00"
[Debug] numBytesRead: 0
[Debug] on readData available: 9
[Debug] readdata read buffer size: 0
[Debug] can't read line:  "\x00\x80\x00\x80\x80\x00\x00\x80\x00"
[Debug] numBytesRead: 0
[Debug] on readData available: 10
[Debug] readdata read buffer size: 0
[Debug] can't read line:  "\x80\x80\x80\x00\x80\x80\x00\x00\x80\x00"
[Debug] numBytesRead: 0
[Debug] on readData available: 6
[Debug] readdata read buffer size: 0
[Debug] can't read line:  "\x00\x80\x00\x80\x80\x00"
[Debug] numBytesRead: 0
[Debug] on readData available: 3
[Debug] readdata read buffer size: 0
[Debug] can't read line:  "\x00\x80\x00"
[Debug] numBytesRead: 0
[Debug] on readData available: 9
[Debug] readdata read buffer size: 0
[Debug] can't read line:  "\x80\x80\x00\x80\x80\x00\x00\x80\x00"
[Debug] numBytesRead: 0
[Debug] on readData available: 8
[Debug] readdata read buffer size: 0
[Debug] can't read line:  "\x00\x00\x80\x80\x00\x00\x80\x00"
[Debug] numBytesRead: 0
[Debug] Closed port: "ttyACM0"

3rd:

[Debug] SerialPlot 0.11.0
[Debug] Revision 866d3c2d410a+
[Debug] on open: bytes avilable
[Debug] clearing buffer
[Debug] Opened port: "ttyACM0"
[Debug] on readData available: 8
[Debug] readdata read buffer size: 0
[Debug] can't read line:  "\x00\x00\x80\x80\x00\x00\x80\x00"
[Debug] numBytesRead: 0
[Debug] on readData available: 9
[Debug] readdata read buffer size: 0
[Debug] can't read line:  "\x80\x00\x00\x80\x80\x00\x00\x80\x00"
[Debug] numBytesRead: 0
[Debug] on readData available: 9
[Debug] readdata read buffer size: 0
[Debug] can't read line:  "\x00\x00\x00\x80\x80\x00\x00\x80\x00"
[Debug] numBytesRead: 0
[Debug] on readData available: 9
[Debug] readdata read buffer size: 0
[Debug] can't read line:  "\x80\x00\x00\x80\x80\x00\x00\x80\x00"
[Debug] numBytesRead: 0
[Debug] on readData available: 9
[Debug] readdata read buffer size: 0
[Debug] can't read line:  "\x00\x80\x00\x80\x80\x00\x00\x80\x00"
[Debug] numBytesRead: 0
[Debug] on readData available: 10
[Debug] readdata read buffer size: 0
[Debug] can't read line:  "\x80\x80\x80\x00\x80\x80\x00\x00\x80\x00"
[Debug] numBytesRead: 0
[Debug] Closed port: "ttyACM0"

If it's not obvious from the logs, none of the builds fixed the problem.

Hasan Yavuz ÖZDERYA · Answer 11 · Tue Nov 03 2020 00:48:44 GMT+0800 (China Standard Time)

@schwukas well this is weird. There is something else in the input data.

0x80, 0x00 these are not valid ASCII characters. They are usually used for control stuff. I also noticed 0xFE which is usually used as "Frame Start Byte" in packet data formats. Are you sure your device isn't sending something else to serial port? Or maybe you have a problem in your connection and we are seeing junk?

Can you please check again with a serialport terminal in raw (hex) mode and see if you can spot similar data as above?

schwukas · Answer 12 · Fri Nov 06 2020 22:17:47 GMT+0800 (China Standard Time)

It took a while for me to get around to it but I checked the output with cutecom:

00000000 30 0d 0a 30 0d 0a 31 0d   0a 32 0d 0a 33 0d 0a 34 	0␍␊0␍␊1␍  ␊2␍␊3␍␊4
00000016 0d 0a 35 0d 0a 36 0d 0a   37 0d 0a 38 0d 0a 39 0d 	␍␊5␍␊6␍␊  7␍␊8␍␊9␍
00000032 0a 31 30 0d 0a 31 31 0d   0a 31 32 0d 0a 31 33 0d 	␊10␍␊11␍  ␊12␍␊13␍
00000048 0a 31 34 0d 0a 31 35 0d   0a 31 36 0d 0a 31 37 0d 	␊14␍␊15␍  ␊16␍␊17␍
00000064 0a 31 38 0d 0a 31 39 0d   0a 32 30 0d 0a 32 31 0d 	␊18␍␊19␍  ␊20␍␊21␍

This looks like valid ASCII to me. I then turned off cutecom and immediately recorded in serialplot but received the same junk from above. Weird..

I tried again with a lower baud rate. I adjusted it in both serialplot and the Arduino code:

[Debug] SerialPlot 0.11.0
[Debug] Revision 866d3c2d410a+
[Debug] on open: bytes avilable
[Debug] Opened port: "ttyACM0"
[Debug] on readData available: 4
[Debug] readdata read buffer size: 0
[Debug] can read line
[Debug] bytes: "35\n"
[Debug] firstReadAfterEnable
[Debug] can read line
[Debug] bytes: "\n"
[Debug] line empty
[Debug] can't read line:  ""
[Debug] numBytesRead: 4
[Debug] on readData available: 1
[Debug] readdata read buffer size: 0
[Debug] can't read line:  "0"
[Debug] numBytesRead: 0
[Debug] on readData available: 2
[Debug] readdata read buffer size: 0
[Debug] can read line
[Debug] bytes: "\r\n"
[Debug] line empty
[Debug] can't read line:  ""
[Debug] numBytesRead: 2
[Debug] on readData available: 3
[Debug] readdata read buffer size: 0
[Debug] can read line
[Debug] bytes: "1\r\n"
[Debug] numComingChannels:  1
[Debug] samples nc:  1  size: 1
[Debug] can't read line:  ""
[Debug] numBytesRead: 3
[Debug] on readData available: 3
[Debug] readdata read buffer size: 0
[Debug] can read line
[Debug] bytes: "2\r\n"
[Debug] numComingChannels:  1
[Debug] samples nc:  1  size: 1
[Debug] can't read line:  ""
[Debug] numBytesRead: 3
[Debug] on readData available: 2
[Debug] readdata read buffer size: 0
[Debug] can't read line:  "3\r"
[Debug] numBytesRead: 0
[Debug] on readData available: 1
[Debug] readdata read buffer size: 0
[Debug] can read line
[Debug] bytes: "\n"
[Debug] line empty
[Debug] can't read line:  ""
[Debug] numBytesRead: 1
[Debug] on readData available: 1
[Debug] readdata read buffer size: 0
[Debug] can't read line:  "4"
[Debug] numBytesRead: 0
[Debug] on readData available: 2
[Debug] readdata read buffer size: 0
[Debug] can read line
[Debug] bytes: "\r\n"
[Debug] line empty
[Debug] can't read line:  ""
[Debug] numBytesRead: 2
[Debug] on readData available: 3
[Debug] readdata read buffer size: 0
[Debug] can read line
[Debug] bytes: "5\r\n"
[Debug] numComingChannels:  1
[Debug] samples nc:  1  size: 1
[Debug] can't read line:  ""
[Debug] numBytesRead: 3
[Debug] on readData available: 3
[Debug] readdata read buffer size: 0
[Debug] can read line
[Debug] bytes: "6\r\n"
[Debug] numComingChannels:  1
[Debug] samples nc:  1  size: 1
[Debug] can't read line:  ""
[Debug] numBytesRead: 3
[Debug] on readData available: 2
[Debug] readdata read buffer size: 0
[Debug] can't read line:  "7\r"
[Debug] numBytesRead: 0
[Debug] on readData available: 1
[Debug] readdata read buffer size: 0
[Debug] can read line
[Debug] bytes: "\n"
[Debug] line empty
[Debug] can't read line:  ""
[Debug] numBytesRead: 1
[Debug] on readData available: 3
[Debug] readdata read buffer size: 0
[Debug] can read line
[Debug] bytes: "8\r\n"
[Debug] numComingChannels:  1
[Debug] samples nc:  1  size: 1
[Debug] can't read line:  ""
[Debug] numBytesRead: 3
[Debug] on readData available: 3
[Debug] readdata read buffer size: 0
[Debug] can read line
[Debug] bytes: "9\r\n"
[Debug] numComingChannels:  1
[Debug] samples nc:  1  size: 1
[Debug] can't read line:  ""
[Debug] numBytesRead: 3
[Debug] on readData available: 2
[Debug] readdata read buffer size: 0
[Debug] can't read line:  "10"
[Debug] numBytesRead: 0
[Debug] on readData available: 2
[Debug] readdata read buffer size: 0
[Debug] can read line
[Debug] bytes: "\r\n"
[Debug] line empty
[Debug] can't read line:  ""
[Debug] numBytesRead: 2
[Debug] on readData available: 1
[Debug] readdata read buffer size: 0
[Debug] can't read line:  "1"
[Debug] numBytesRead: 0
[Debug] on readData available: 3
[Debug] readdata read buffer size: 0
[Debug] can read line
[Debug] bytes: "1\r\n"
[Debug] numComingChannels:  1
[Debug] samples nc:  1  size: 1
[Debug] can't read line:  ""
[Debug] numBytesRead: 3
[Debug] on readData available: 4
[Debug] readdata read buffer size: 0
[Debug] can read line
[Debug] bytes: "12\r\n"
[Debug] numComingChannels:  1
[Debug] samples nc:  1  size: 1
[Debug] can't read line:  ""
[Debug] numBytesRead: 4
[Debug] on readData available: 3
[Debug] readdata read buffer size: 0
[Debug] can't read line:  "13\r"
[Debug] numBytesRead: 0
[Debug] on readData available: 1
[Debug] readdata read buffer size: 0
[Debug] can read line
[Debug] bytes: "\n"
[Debug] line empty
[Debug] can't read line:  ""
[Debug] numBytesRead: 1
[Debug] on readData available: 1
[Debug] readdata read buffer size: 0
[Debug] can't read line:  "1"
[Debug] numBytesRead: 0
[Debug] on readData available: 3
[Debug] readdata read buffer size: 0
[Debug] can read line
[Debug] bytes: "4\r\n"
[Debug] numComingChannels:  1
[Debug] samples nc:  1  size: 1
[Debug] can't read line:  ""
[Debug] numBytesRead: 3
[Debug] on readData available: 4
[Debug] readdata read buffer size: 0
[Debug] can read line
[Debug] bytes: "15\r\n"
[Debug] numComingChannels:  1
[Debug] samples nc:  1  size: 1
[Debug] can't read line:  ""
[Debug] numBytesRead: 4
[Debug] Closed port: "ttyACM0"

Not the output I was expecting but at least it's not invalid ASCII characters. Maybe that's helpful to you.

Hasan Yavuz ÖZDERYA · Answer 13 · Mon Nov 09 2020 16:09:26 GMT+0800 (China Standard Time)

@schwukas This is my bad. Debug executable that I provided to you has a piece of code that read the data from the port that shouldn't be read. Because I assumed it was junk data. Normally we don't do that. That is why parser isn't working.

But input data seems to be correct in this case. Are you still having problem with release executable?

schwukas · Answer 14 · Mon Nov 09 2020 16:31:34 GMT+0800 (China Standard Time)

No worries ☺️

Are you still having problem with release executable?

Unfortunately, none of the three AppImages I have tried, fixed my issue. I'm okay to accept it's a hardware thing. I chose a crappy, cheap motherboard when I built this PC and regretted it on several occasions. Maybe that's just another one of those occasions. Although I would love to use Serialplot, as it offers way more flexibility than Arduino and Cutecom. Oh well.. 🙂