jonhoo / msql-srv

I am trying to run oltpbench on a SQL server written with msql-srv. But it does not connect. It throws error at this code block
https://github.com/jonhoo/msql-srv/blob/master/src/lib.rs#L255-L259

oltpbench uses JDBC drivers. Do you know what is going on?

mysql terminal session or rust crate works just fine.

Could you tell me what error you get? And which of those unwraps you hit?

Rust code throws this error.

thread '<unnamed>' panicked at 'called `Option::unwrap()` on a `None` value', /home/ranadeep/.cargo/registry/src/github.com-1ecc6299db9ec823/msql-srv-0.8.7/src/lib.rs:256:36

Java client code with JDBC is

import java.sql.*;
import java.util.*;

class MysqlCon{
public static void main(String args[]){
    try{
        Class.forName("com.mysql.jdbc.Driver");
        Properties props = new Properties();
        // props.setProperty("ssl","false");
        // props.setProperty("user","root");
        // props.setProperty("password","root");
        Connection con = DriverManager.getConnection("jdbc:mysql://localhost:3306/mydb", props);
        Statement stmt=con.createStatement();
        ResultSet rs=stmt.executeQuery("select * from mytable");
        while(rs.next())
        System.out.println(rs.getInt(1)+"  "+rs.getString(2)+"  "+rs.getString(3));
        con.close();
    }catch(Exception e){
        System.out.println(e);}
    }
}

Rust error With oltpbench

thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: Error(([0, 0, 116, 112, 99, 99, 0], Eof))', /home/ranadeep/.cargo/registry/src/github.com-1ecc6299db9ec823/msql-srv-0.8.7/src/lib.rs:257:30

oltpbench command

./oltpbenchmark -b tpcc -c config/tpcc_config_mysql.xml --create=true --load=true

Huh, that is very strange indeed. Your first example fails at

msql-srv/src/lib.rs

Line 256 in 9daaa16

let (seq, handshake) = self.reader.next()?.unwrap();

Which means that the first packet it sends isn't parsed correctly, and the connection is eventually terminated here

msql-srv/src/packet.rs

Line 126 in 9daaa16

return Ok(None);

This, in turn, implies that the msql-srv MySQL packet parsing code over at

msql-srv/src/packet.rs

Line 196 in 9daaa16

fn packet(i: &[u8]) -> nom::IResult<&[u8], (u8, Packet<'_>)> {

fails. It would be very helpful to have the sequence of bytes that fails to parse. Could you try printing out bytes around here

msql-srv/src/packet.rs

Line 97 in 9daaa16

match packet(bytes) {

To give some more insight into what exact packet it's choking on?

As for the oltpbench panic, that seems different. Specifically, it looks like the client handshake packet isn't being parsed correctly in

msql-srv/src/commands.rs

Line 12 in 9daaa16

pub fn client_handshake(i: &[u8]) -> nom::IResult<&[u8], ClientHandshake<'_>> {

Could you try printing i at the top of that function, as well as figure out which line in the function the error is returned from? That would help a lot in trying to narrow this down.

I wrapped in is with dbg!().

For the first JDBC code,

[/home/ranadeep/phd/projects/msql-srv/src/packet.rs:97] bytes = []
thread '<unnamed>' panicked at 'called `Option::unwrap()` on a `None` value', /home/ranadeep/phd/projects/msql-srv/src/lib.rs:256:36
...

For oltpbench code,

[/home/ranadeep/phd/projects/msql-srv/src/packet.rs:97] bytes = []
[/home/ranadeep/phd/projects/msql-srv/src/packet.rs:97] bytes = [
    16,
    0,
    0,
    1,
    139,
    34,
    255,
    255,
    255,
    114,
    111,
    111,
    116,
    0,
    0,
    116,
    112,
    99,
    99,
    0,
]
[/home/ranadeep/phd/projects/msql-srv/src/commands.rs:13] i = [
    139,
    34,
    255,
    255,
    255,
    114,
    111,
    111,
    116,
    0,
    0,
    116,
    112,
    99,
    99,
    0,
]
thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: Error(([0, 0, 116, 112, 99, 99, 0], Eof))', /home/ranadeep/phd/projects/msql-srv/src/lib.rs:257:30
...

Hmm... Very strange indeed. I pushed a new version (two, actually, one minor + one major 0.9.0). These should help give slightly better errors, though they won't actually fix the issues above. But at least the panics will turn into returned errors instead.

For the first JDBC code

This is bizarre. The client is responding to the server handshake by disconnecting. This makes me think that it's actually JDBC that doesn't like the server handshake that we are sending, and that it's actually the JDBC side that's choking. Perhaps understandably, as it has probably only been tested against "real" MySQL servers. Could you try enabling error logging on the Java side and see if that says why it decides to drop the connection?

For oltpbench code [...] Error(([0, 0, 116, 112, 99, 99, 0], Eof))

This too smells to me like the JDBC code doing something funky. From my read of the handshake packet spec, that particular sequence of bytes simply isn't valid for the client to send when it does. Or ever I don't think. Here too I think we need some insight from the Java side as to why JDBC decides to send that packet in seeming violation of the spec.

I wonder if there's a way to ping a JDBC dev here — would be useful to have some eyes on this from someone with familiarity with how JDBC works internally.

I wonder if this could also be related to the fact that msql-srv does not currently support the MySQL SSL handshake. Adding it shouldn't be too difficult, though I don't personally have the bandwidth to do so at the moment. If you want to take a stab at writing a PR, I'd be happy to take a look! See also mit-pdos/noria-mysql#15.

I think I have found the culprit at least for the oltpbench. It's because JDBC from that package is still using old HandshakeResponse320. Basically one should check CapabilityFlags for CLIENT_PROTOCOL_41 here and proceed accordingly.

msql-srv/src/commands.rs

Line 13 in 9daaa16

let (i, cap) = nom::number::complete::le_u32(i)?;

Update: JDBC sets CLIENT_PROTOCOL_41 flag though. It is really weird.

Ah, yes, msql-srv does not currently support the old handshake. It shouldn't be too hard to make it though, if you fancy writing up a PR? But if it also sets CLIENT_PROTOCOL_41, then that seems very suspicious indeed!

Sure :) I will create a pull request later when it's ready.

can not connect with JDBC driver