[BUG] TDVT add_ds command corrupts tds files

[Michael Habiger]

Describe the bug

For context, Step 4 of the instructions Test with TDVT Suite states:

Run tdvt action --add_ds mydb. Choose to generate the password file and choose the logical query config. This creates a mydb.ini file under /config and will modify your two TDS files to rename the connection and link them to the tds/mydb.password password file.

This command is corrupting the information for the connection attribute in the *.tds files that were generated from steps 1-3 of the same document. The parsing algorithm incorrectly removes the class sub-attribute/value pair from the *.tds files and leaves one single quote (').

Console Capture

C:\tableau\wksp> python -m tdvt.tdvt action --add_ds mahds
Make sure you have already saved the appropriate TDS files in the tds directory.
Adding a new datasource [mahds] ...
Would you like to setup a password file? (y/n) This is suitable for a single connection per tds (standard). y
Enter the datasource password: xxxxxx
Created file: tds/mahds.password
Would you like to run TDVT against a schema other than TestV1? (y/n) y
Enter the schema name: mhabiger
Do you have the Staples & Calcs tables loaded using different column names (e.g. num0_col instead of num0)? (y/n) n
Would you like to run TDVT against a custom table? (y/n) n
Enter the logical config to use or type 'list' to see the options or 's' to skip selecting one now: s
Created ini file: config/mahds.ini
Please set the LogicalQueryFormat value to the expected format.
Modifying C:\tableau\wksp\tds\cast_calcs.mahds.tds
Modifying C:\tableau\wksp\tds\Staples.mahds.tds
Generating test files from: C:\tableau\.venv\lib\site-packages\tdvt\logicaltests\generate\input\calcs
Writing test files to: C:\tableau\.venv\lib\site-packages\tdvt\logicaltests\setup\calcs
Generating test files from: C:\tableau\.venv\lib\site-packages\tdvt\logicaltests\generate\input\lod
Writing test files to: C:\tableau\.venv\lib\site-packages\tdvt\logicaltests\setup\lod
Generating test files from: C:\tableau\.venv\lib\site-packages\tdvt\logicaltests\generate\input\pretest
Writing test files to: C:\tableau\.venv\lib\site-packages\tdvt\logicaltests\setup\pretest
Generating test files from: C:\tableau\.venv\lib\site-packages\tdvt\logicaltests\generate\input\staples
Writing test files to: C:\tableau\.venv\lib\site-packages\tdvt\logicaltests\setup\staples
Generating test files from: C:\tableau\.venv\lib\site-packages\tdvt\logicaltests\generate\input\union
Writing test files to: C:\tableau\.venv\lib\site-packages\tdvt\logicaltests\setup\union
Generating test files from: C:\tableau\.venv\lib\site-packages\tdvt\logicaltests\generate\input\pretest\connection_test
Writing test files to: C:\tableau\.venv\lib\site-packages\tdvt\logicaltests\setup\connection_test

C:\tableau\wksp\tds> diff Staples.mahds.tds.ORIGINAL Staples.mahds.tds.AFTER_ADD_DS
12,13c12,13
<       <named-connection caption='localhost' name='actian_odbc.0fnv9im1yvs1ny19w0xdy1jb27vo'>
<         <connection class='actian_odbc' dbname='testv1' one-time-sql='' port='21064' schema='mhabiger' server='localhost' username='mahtest' />
---
>       <named-connection caption='localhost' name='leaf'>
>         <connection  tdvtconnection='mahds_connection' ' dbname='testv1' one-time-sql='' port='21064' schema='mhabiger' server='localhost' username='mahtest' />
.....

Desktop (please complete the following information):

• OS: MS Windows 10 version 20H2
• Tableau Version: Tableau Desktop 2023.2.0 (20232.23.0611.2007)
• Python version 3.10.7
• Python packages

C:\tableau\wksp\tds> python -m pip list
Package    Version
---------- -------
defusedxml 0.7.1
pip        23.2.1
setuptools 63.2.0
tdvt       2.7.0

About you:
Name: Michael Habiger
Company: Actian

[Luke Petschauer]

Hi, Michael I'll take a look, thanks

[Luke Petschauer]

Internal tracking: W-13900361
Looks like this was introduced in #1144. I'll push a branch with a fix.

[Luke Petschauer]

@hab6 could you check out to the fix-tds-regex branch and see if that solves your issue? Looks to fix things in my dev environment.

[Michael Habiger]

@lukewrites The fix works for me. Thank you for the quick turnaround.

(.venv) λ diff cast_calcs.mahds.tds.ORIGINAL cast_calcs.mahds.tds.AFTER_ADD_DS
12,13c12,13
<       <named-connection caption='mhabiger-p15' name='actian_odbc.15t4y8e0768f8q1eyipt40rq7tva'>
<         <connection class='actian_odbc' dbname='testv1' one-time-sql='' port='21064' schema='mhabiger' server='mhabiger-p15' username='mahtest' />
---
>       <named-connection caption='mhabiger-p15' name='leaf'>
>         <connection class='actian_odbc' dbname='testv1' one-time-sql='' port='21064' schema='mhabiger' server='mhabiger-p15' username='mahtest' tdvtconnection='mahds_connection'  />

(.venv) λ diff Staples.mahds.tds.ORIGINAL Staples.mahds.tds.AFTER_ADD_DS
12,13c12,13
<       <named-connection caption='mhabiger-p15' name='actian_odbc.1xoafkv1h2cqyt1fntz6y0ihyobz'>
<         <connection class='actian_odbc' dbname='testv1' one-time-sql='' port='21064' schema='mhabiger' server='mhabiger-p15' username='mahtest' />
---
>       <named-connection caption='mhabiger-p15' name='leaf'>
>         <connection class='actian_odbc' dbname='testv1' one-time-sql='' port='21064' schema='mhabiger' server='mhabiger-p15' username='mahtest' tdvtconnection='mahds_connection'  />

[Luke Petschauer]

Thanks for validating @hab6 change is now in master.