Zone import rewrite entries if not in the same order

[Cyril Peponnet]

Issue type

• Bug report

cli53 version (cli53 --version)

0.8.7

OS / Platform

Steps to reproduce

Let's say I have the following A records on a hosted zone.

test A 10.0.0.1
test A 10.0.0.2
test A 10.0.0.3

a cli53 export will put them in oder.

If I change the order im the zone file and try to import it with cli53 import, it will delete the 3 records to recreate them.

Dry-run, changes that would be made:
+ test        600     IN      A       10.0.0.2
+ test        600     IN      A       10.0.0.3
+ test        600     IN      A       10.0.0.1
- test        600     IN      A       10.0.0.1
- test        600     IN      A       10.0.0.2
- test        600     IN      A       10.0.0.3

Expected behaviour

Do not update the records as they are fine.

Actual behaviour

Delete / recreate records

Have you checked if the documentation has the information you require?

Yes

Could you contribute a fix or help testing with this issue?

help testing for sure... no time for a PR :/

Note: this is certainly due to the fact AWS is storing multiples IP addresses in a an array.

[Cyril Peponnet]

The main issue is that your are doing a list of existing rr based in rrset.String() ex:

{
  Name: "test.domain.tld",
  ResourceRecords: [{
      Value: "10.0.0.1"
    },{
      Value: "10.0.0.2"
    }],
  TTL: 30,
  Type: "A"
}

But when parsing the bind file for import, if the entries are not in the same order. the index key mismatch and it will try to delete/create the record again.

{
  Name: "test.domain.tld",
  ResourceRecords: [{
      Value: "10.0.0.2"
    },{
      Value: "10.0.0.1"
    }],
  TTL: 30,
  Type: "A"
}

Note that this is also an issue when I want to add a third IP to a DNS RR, it will delete / create again all the entries.

IMO import should also manage UPSERT to modify records.

[Barnaby Gray]

I don't think it's my desire to change this behaviour - otherwise exports of zones that have records differing in order (which Amazon would preserve as entered from the UI) would either compare as different, or we apply an ordering on the record that doesn't reflect reality.

[Barnaby Gray]

Concerning "IMO import should also manage UPSERT to modify records." - this is dangerous behaviour. The reason we always do a DELETE/CREATE is because this is safer if any concurrent changes happen at the same time - the whole transaction would abort in that case.

UPSERT runs the risk of trampling over any concurrent changes made in the period between the list operation to find differences and upserts actually running.

Also originally UPSERT didn't exist.

[Cyril Peponnet]

While I understand your statement, in my use case where I'm using cli53 to regularly sync a zone file with aws, I cannot afford a delete / create if for instance only the ttl changed. (that leads to dns propagation time).

Maybe my fix can be improved using an optional flag...

[Barnaby Gray]

Not sure I follow 'cannot afford'? As far as I know AWS don't bill changes.
An UPSERT would propagate just the same as a DELETE/CREATE.

[Cyril Peponnet]

For instance, I have an app which is RR served with 2 A records with same name.

If I want to add / or change one of the record, through zone import (automated process). It leads to a delete/create, and basically intermediate dns are caching the delete for a certain time... means that my application is not reachable anymore during that time.

[Barnaby Gray]

You misunderstand how ChangeResourceRecordSets works:
http://docs.aws.amazon.com/Route53/latest/APIReference/API_ChangeResourceRecordSets.html
"Change batches are considered transactional changes. When using the Amazon Route 53 API to change resource record sets, Amazon Route 53 either makes all or none of the changes in a change batch request. This ensures that Amazon Route 53 never partially implements the intended changes to the resource record sets in a hosted zone."

[Cyril Peponnet]

You are right... I shouldn't see this behavior inside a same ChangeRRS.

And upsert should ressemble to a delete/create when done in a same batch from a client perspective.

[Johannes Emerich]

I observed some strange behavior that seems related to this order dependence.

We update two zones from a shared zonefile. After the latest update (with cli53 import --replace) AWS changed the order of entries for one of the zones (also visible when doing an export). Now a dry run import reports that several entries need to be recreated for this zone, because the order differs. Performing the import does not actually change the order on AWS however, so I see no way of converging on a stable situation again.

If the import can not actually influence the order, why does the dry-run report that there is work to do?

Am I doing something wrong? (Note that the zone with an altered order has an additional entry not present in the shared zonefile. This entry however is not part of the "order conflict" cli53 reports in the dry-run.)