Unable to find transaction selection form

[Fabian Köster]

Since yesterday, I get this error message:

Start time: 25.08.2017  
PIN: 
Starting login as user xxx...
Navigating to 'Umsätze'...
Selecting transactions in time frame 25.08.2017 - 01.09.2017...
Traceback (most recent call last):
  File "/usr/bin/dkb.py", line 416, in <module>
    fetcher.select_transactions(args.cardid, from_date, args.to_date)
  File "/usr/bin/dkb.py", line 173, in select_transactions
    br.form = form = self._get_transaction_selection_form()
  File "/usr/bin/dkb.py", line 99, in _get_transaction_selection_form
    raise RuntimeError("Unable to find transaction selection form")
RuntimeError: Unable to find transaction selection form

Did not find the time to look into this in detail, but maybe DKB changed the HTML structure.

[MM]

Actually I get it since Tuesday and confirmed with the hotline, that they indeed changed their HTML. I enabled debugging but the element "slAllAccounts" is still there. so I have not really an idea why it fails.

[Fabian Köster]

Yesterday it suddenly worked for me again, but today I get the same error again. And I also see the form control with name=slAllAccounts.

[Christian Hoffmann]

Thanks for your report and your investigation. I can reproduce what you said -- sometimes it works, sometimes it doesn't. Seems like DKB is performing some kind of A/B testing?
Anyway. I think I've found the issue and have committed a fix which works reliably for me.

DKB's change seems to consist of adding an additional <input type="hidden" name="slAllAccounts" ...> field (maybe in preparation of more JavaScript logic?). This confuses the current code, as it does not expect to find multiple form fields with the same name. The fix consists of looking also for the form fields type ('select' int his case).

Please let me know if you still have any issues.

[MM]

Thanks for looking at it. Yes I still have issues. sometimes it works, sometimes it doesn't. I enabled debugging again and captured a working and a non-working case. The difference seems to be the following:

@@ -10171,9 +10171,10 @@
                                 <label for="id-1615473160_bankAccount" id="id-1615473160_bankAccount-label" class="label" tid="id-1615473160_bankAccount-label" >
                 Konto / Karte              </label>
 
-                            <select class="field text" name="slAllAccounts" onchange="searchbuttonclick();" id="id-1615473160_slAllAccounts" tid="slAllAccounts"   > 
+                            <span class="mandatory"><select class="field text" name="slAllAccounts" onchange="searchbuttonclick();" id="id-1615473160_slAllAccounts" tid="slAllAccounts"   >  
 <option id="id-1615473160_slAllAccounts-0" value="0" selected="selected"  tid="0" >DExx 1203 0000 xxxx xxxx xx / Girokonto</option>DExx 1203 0000 xxxx xxxx xx / Girokonto<option id="id-1615473160_slAllAccounts-1" value="1"  tid="1" >xxxx********xxxx / Kreditkarte</option>xxxx********xxxx / Kreditkarte
 </select>
+</span>        <input type="hidden" name="slAllAccounts" value="-2147482982"  tid="slAllAccounts" tid="slAllAccounts" tid="-2147482982"/>
 
                         
                                        </span>

The lines prefixed with + are in the non-working version of the HTML.

[Christian Hoffmann]

Did you still get the same error message as mentioned in the initial report? I was unable to trigger it again and the fix I've committed should cover that case.

However, after trying more often I was able to trigger another bug in the credit card selection form. Basically, we have the same issue there. The fix consists of another change to make the value assignment work even with the non-unique name.

Can you try again with the latest version (dc2ec16)?

I am able to run it ten times in a row without issues now.

[MM]

No, sorry my mistake, the last error was different. I have the following error in the log from the last failed try:

...
*****************************************************
*****************************************************
Selecting transactions in time frame 15.08.2017 - 02.09.2017...
Selecting transactions in time frame 15.08.2017 - 02.09.2017...
Traceback (most recent call last):
  File "./dkb.py", line 416, in <module>
    fetcher.select_transactions(args.cardid, from_date, args.to_date)
  File "./dkb.py", line 174, in select_transactions
    self._select_credit_card(form, cardid)
  File "./dkb.py", line 146, in _select_credit_card
    raise RuntimeError("Unable to find credit card selection form")
RuntimeError: Unable to find credit card selection form

I made four tries with the lastest version and it worked flawless on every try.
Thanks a lot for your script and support.

[Fabian Köster]

Thanks, works for me!