Giters

twingly / ecco

:dolphin: MySQL replication binlog parser in JRuby

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Avoid triggering on_save_position if given faulty position

jage opened this issue · comments

commented

Scenario:

  • The MySQL binlog position is stored in a database, it's saved when on_save_position is called.
  • Ecco is configured with a faulty position on start with set_binlog_position.

Since Ecco triggers on_save_position on start it will cause the faulty position to be persisted in the database.

It would be a very kind gesture by Ecco if it could avoid triggering on_save_position if given a faulty filename or position. It should only store positions retrieved from the MySQL server.

commented

Since Ecco triggers on_save_position on start ...

Found that out here: #5 (comment)

commented
  • Ecco is configured with a faulty position on start with set_binlog_position.

Since Ecco triggers on_save_position on start it will cause the faulty position to be persisted in the database.

Hmm... so the reasoning here is that ecco_client.set_binlog_position(position) (or ecco_client.set_binlog_filename(filename)) will trigger on_save_position, but I don't think so, as it just looks like plain setters in the java library: https://github.com/shyiko/mysql-binlog-connector-java/blob/0.2.4/src/main/java/com/github/shyiko/mysql/binlog/BinaryLogClient.java#L175-L213

commented

I think it has to be up to the user of Ecco to discover / act on "faulty position". #23 could help with that.

commented

Hmm... so the reasoning here is that ecco_client.set_binlog_position(position) (or ecco_client.set_binlog_filename(filename)) will trigger on_save_position

No, It is triggered on start, not when it's set?

Since Ecco triggers on_save_position on start it will cause the faulty position to be persisted in the database.

commented

Yes, on start (connect), we get the ROTATE event, from MySQL. Is the suggestion to not trigger on_save_position then?

commented

Yes, on start (connect), we get the ROTATE event, from MySQL. Is the suggestion to not trigger on_save_position then?

Yes, that's what it says in the first comment. :-)

commented

But I'm not sure it is working the way you think it works.

You are saying (if I understand correctly): App gives Ecco faulty position -> Ecco connects to MySQL with faulty position -> We get ROTATE event with faulty position -> on_save_position triggers with faulty position -> we save faulty position

First of all, in our case, the faulty position was already saved in our DB to start with, but we can set that aside.

I think the information in the ROTATE event has nothing to do with what we configured Ecco client with? But I might be wrong.

commented

Yes, on start (connect), we get the ROTATE event, from MySQL. Is the suggestion to not trigger on_save_position then?

Yes

I don't see what we gain, if we would accomplish to not trigger on_save_position for the first ROTATE event we see after connect. The faulty position we are talking about, is already stored in our DB, before we connect. So it doesn't make any difference.

We would also loose these features:

  • filename: nil

    turns on automatic resolution (resulting in the last known binlog and position)

  • filename: ""

    instructs server to stream events starting from the oldest known binlog

I think it's up to the consumer of ecco to do the right thing.

I think we should close this issue without action.

commented

I must have missed this during parental leave. :-)

I think I agree with @dentarg here, it feels like a case of garbage in, garbage out?

commented

I think we should close this issue without action.

Sure.