csv-importer

csv-importer.py is a simple script to assist in importing objects via CSV files to StoredSafe.

Imports to StoredSafe can either be done thru the reference web UI implementation when creating new objects by pasting JSON structures according to a specified format or by utilizing StoredSafe's REST API which would require a connection and a valid API key to the affected StoredSafe server.

csv-importer.py can assist in both modes. By specifying the --no-rest option, csv-importer.py will operate in in off-line mode and translating input to the required JSON structures and store it in a file (--json option), or output to stdout (which is the default option).

csv-importer.py can also utilize StoredSafe's REST API to directly import objects. If the --no-rest option is not specified, csv-importer.py will operate in REST API mode and will require either that pre-authentication has been performed by the StoredSafe token handler CLI module (storedsafe-tokenhandler.py) and stored in an init file which location can be specified with the --rc option.

Other authentication options includes specifying a valid token (--token) or perform an on-line one-shot authentication (--user and --apikey)

The script is written in Python v3 and has been tested on macOS Catalina (10.15.5) and on Linux (any fairly recent version of Ubuntu or Red Hat should work fine).

Support for previous versions of StoredSafe (pre v2.1.0) can be found in the legacy branch.

Installation instructions

This script requires Python v3 and some libraries. It has been developed and tested using Python v3.7.7, on macOS Catalina 10.15.5.

Most of the required libraries are installed by default, but requests might require manual installation.

requests:

sudo pip install -r requirements.txt

sudo -H pip install requests

Syntax

$ csv-importer.py --help
Usage: csv-importer.py [-vdsuat]
 --verbose                      (Boolean) Enable verbose output.
 --debug                        (Boolean) Enable debug output.
 --no-rest                      Operate in off-line mode, do not attempt to use the REST API.
 --rc <rc file>                 Use this file to obtain a valid token and a server address.
 --storedsafe (or -s) <Server>  Use this StoredSafe server.
 --user (or -u) <user>          Authenticate as this user to the StoredSafe server.
 --apikey (or -a) <API Key>     Use this unique API key when communicating with the StoredSafe server.
 --token (or -t) <Auth Token>   Use pre-authenticated token instead of --user and --apikey.
 --basic-auth-user <user:pw>    Specify the user name and password to use for HTTP Basic Authentication.
 --csv <file>                   File in CSV format to import.
 --separator <char>             Use this character as CSV delimiter. (defaults to ,)
 --escapechar <char>            Use this character to escape special characters. (defaults to None)
 --json <file>                  Store output (JSON) in this file.
 --fieldnames <fields>          Specify the mapping between columns and field names. Has to match exactly. Defaults to the Server template.
 --objectname <field>           Use this field as objectname when storing objects. Defaults to the host field from the Server template
 --template <template>          Use this template name for import. Name has to match exactly. Defaults to the Server template.
 --templateid <template-ID>     Use this template-ID when importing. (*)
 --vault <Vaultname>            Store imported objects in this vault. Name has to match exactly. (*)
 --vaultid <Vault-ID>           Store imported objects in this Vault-ID. (*)
 --create-vault                 (Boolean) Create missing vaults. (*)
 --policy <policy-id>           Use this password policy for newly created vaults. (Default to policy #7) (*)
 --description <text>           Use this as description for any newly created vault. (Default to "Created by csv-importer.") (*)
 --allow-duplicates             (Boolean) Allow duplicates when importing. (*)
 --skip-first-line              Skip first line of input. A CSV file usually has headers, use this to skip them.
 --remove-extra-columns         Remove any extra columns the if CSV file has more columns than the template.
 --stuff-extra <field>          Add data from extranous columns to this field.
 --not-empty <fielda,fieldb>    These fields must not be blank in imports. (Separate fields with ",")
 --fill-with <string>           For fields specified with --not-empty, use this string as filler. (Defaults to "n/a")
 --list-vaults                  List all vaults accessible to the authenticated user. (*)
 --list-templates               List all available templates.
 --list-fieldnames              List all fieldnames in the specified template. (--template or --templateid)

(*) REST mode only.

Using REST API and interactive login:
$ csv-importer.py --storedsafe safe.domain.cc --user bob --apikey myapikey --csv file.csv --vault "Public Web Servers" --verbose

Using REST API and pre-authenticated login:
$ csv-importer.py --rc ~/.storedsafe-client.rc --vault "Public Web Servers" --csv file.csv

Off-line mode:
$ csv-importer.py --no-rest --csv file.csv --json file.json --template Login --fieldnames host,username,password

--verbose

Add verbose output.

--debug

Add debug output.

--no-rest

Do not use the REST API, operate completely in off-line mode. Result displayed to screen (default) or can be saved in a file. (--json)

--rc <RC file>

Obtain credentials (token) and server information from this file. (Enabled by default to ~/.storedsafe-client.rc)

--storedsafe|-s <server>

Upload certificates to this StoredSafe server.

--user|-u <user>

Authenticate as this StoredSafe user.

--apikey|-a <apikey>

Use this unique API key when communicating with StoredSafe. (Unique per application and installation)

--token <token>

Use pre-authenticated token instead of --user and --apikey, also removes requirement to login with passphrase and OTP.

--basic-auth-user <user:pw>

Specify the user name and password to use for HTTP Basic Authentication.

--csv <CSV file>

Specify one or more IPv4 or IPv6 networks. Overlapping will be resolved.

--separator

Use this character as CSV separator. (Single character)

--escapechar

Use this character as CSV escape char. (Single character)

--json

Output resulting JSON to this file.

--fieldnames

Use this comma separated list as input field names. See --list-fieldnames for valid field names per template.

--objectname

Use this field as the primary name for the object. Defaults to the "host" field from the Server template.

--template

Use this StoredSafe template for import. See --list-templates for a complete list of supported templates on the server. (Case sensitive and name has to match exactly)

--templateid

Instead of using the template name (--template), you can specify the Template-ID which is unique per template.

--vault|-v <Vaultname>

Store any found certificates in this vault. Name has to match exactly. See --list-vaults for a complete list of accessible vaults on the StoredSafe server.

--vaultid <Vault-ID>

Store any found certificates in this Vault-ID.

--create-vault

Create missing vaults.

--policy <policy-id>

Use this password policy for newly created vaults. (Default to 7)

--description <text>

Use this as description for any newly created vault. (Default to "Created by csv-importer.")

--allow-duplicates

Allow importing the same certificate to the same vault multiple times.

--skip-first-line

Normally the first line in a CSV file has headers, use this option to skip these.

--remove-extra-columns

If the input CSV file has more columns than matching fields in StoredSafe, remove them.

--stuff-extra <field>

Add data from extranous columns to this field.

--not-empty <fielda,fieldb>

Certain fields in templates can not be blank in imports. If import complains on empty fields, this option can be used to fill those fields with data. (Separate fields with ",")

--fill-with <string>

For fields specified with --not-empty, use this string as filler. (Defaults to 'n/a')

--list-vaults

List all vaults accessible to the authenticated user on the StoredSafe server.

--list-templates

List all templates accessible to the authenticated user on the StoredSafe server.

--list-fieldnames

List all fields in the specified template. Obtained by querying the template on the StoredSafe server.

Usage

Prepare a CSV file for importing objects to StoredSafe, this can be done manually or by exporting from an other password manager, or an excel spreadsheet.

$ cat file.csv
host,username,password,info,cryptedinfo
fw.safe.domain.cc,admin,j5nJ2QQnRhp7xYwG8fExygDvD,Firewall for Stockholm office.,iLO password is k3PUibwrWMCYCtxlYgsrOiKRZZnIxA
resolver.safe.domain.cc,root,Xf6PVBdlad40sS3C7u1H6mVsD,"unbound, recursive resolver for Stockholm office",
safe.domain.cc,rolf,BoDJCrHLF4VNOyeBOuKzZocqc,Rolf Rolfsson,cryptedinfo
safe.domain.cc,sven,lKuBdy1k6HMxbVh6vRB5yW7q6,Sven Hell,cryptedinfo
kdc.safe.domain.cc,root,uFwWyzrAjyU4RKVdnnClXMuJ5,KDC located in D4K3,Backup GPG passphrase is 3Bxq8Df2LrPpf99qT0mml8SI5Di9hY7QwDMioHvARxj1fmXRFy

Import objects into the StoredSafe appliance using the file "file.csv" as input (content as below), use a pre-authenticated session (--rc) and store imported objects in the "Stockholm Office" Vault on the StoredSafe server "safe.domain.cc".

$ csv-importer.py --rc ~/.storedsafe-client.rc --csv file.csv --vault "Stockholm Office" --skip-first-line --verbose
Importing "fw.safe.domain.cc" into the Vault "Stockholm Office" (Vault-ID 182).
Importing "resolver.safe.domain.cc" into the Vault "Stockholm Office" (Vault-ID 182).
Importing "safe.domain.cc" into the Vault "Stockholm Office" (Vault-ID 182).
Importing "safe.domain.cc" into the Vault "Stockholm Office" (Vault-ID 182).
Importing "kdc.safe.domain.cc" into the Vault "Stockholm Office" (Vault-ID 182).
Imported 5 object/s

Re-running the import, csv-importer.py will detect that the objects are already present in the vault "Stockholm Office" and will avoid storing duplicates by default. (can be changed with --allow-duplicates)

$ csv-importer.py --rc ~/.storedsafe-client.rc --csv file.csv --vault "Stockholm Office" --skip-first-line --verbose
WARNING: Object "fw.safe.domain.cc" (Object-ID 696) (4 field/s matched "fw.safe.domain.cc") - duplicate.
WARNING: Found 1 possible duplicate object/s in "Stockholm Office" when trying to import "fw.safe.domain.cc". (Use "--allow-duplicates" to force import)
WARNING: Object "resolver.safe.domain.cc" (Object-ID 697) (4 field/s matched "resolver.safe.domain.cc") - duplicate.
WARNING: Found 1 possible duplicate object/s in "Stockholm Office" when trying to import "resolver.safe.domain.cc". (Use "--allow-duplicates" to force import)
WARNING: Object "safe.domain.cc" (Object-ID 698) (4 field/s matched "safe.domain.cc") - duplicate.
WARNING: Found 1 possible duplicate object/s in "Stockholm Office" when trying to import "safe.domain.cc". (Use "--allow-duplicates" to force import)
WARNING: Object "safe.domain.cc" (Object-ID 699) (4 field/s matched "safe.domain.cc") - duplicate.
WARNING: Found 1 possible duplicate object/s in "Stockholm Office" when trying to import "safe.domain.cc". (Use "--allow-duplicates" to force import)
WARNING: Object "kdc.safe.domain.cc" (Object-ID 700) (4 field/s matched "kdc.safe.domain.cc") - duplicate.
WARNING: Found 1 possible duplicate object/s in "Stockholm Office" when trying to import "kdc.safe.domain.cc". (Use "--allow-duplicates" to force import)
WARNING: Skipped 5 duplicate object/s.

List all available templates on the server.

$ csv-importer.py --rc ~/.storedsafe-client.rc --list-templates
Found Template "Credit Card" as Template-ID "11"
Found Template "Short login" as Template-ID "10"
Found Template "Server" as Template-ID "1"
Found Template "Folder" as Template-ID "2"
Found Template "Quicknote" as Template-ID "5"
Found Template "Login" as Template-ID "4"
Found Template "PIN code" as Template-ID "9"
Found Template "Note" as Template-ID "8"
Found Template "Server/IP" as Template-ID "1001"

List all field names in a selected template.

$ csv-importer.py --rc ~/.storedsafe-client.rc --list-fieldnames --templateid 1001
username,info,ip,host,password,cryptedinfo

List all vaults available to the current authenticated users.

$ csv-importer.py --rc ~/.storedsafe-client.rc --list-vaults
Vault "StoredSafe" (Vault-ID "1") with "Admin" permissions.
Vault "Testing grounds" (Vault-ID "19") with "Admin" permissions.
Vault "Stockholm Office" (Vault-ID "182") with "Admin" permissions.
Vault "Public Web Servers" (Vault-ID "181") with "Admin" permissions.
Vault "Firewalls in ZA" (Vault-ID "179") with "Admin" permissions.

Using the off-line mode, format the input CSV file according to the StoredSafe JSON specifications, which latter can be validated and imported thru the standard StoredSafe web UI.

$ csv-importer.py --no-rest --csv file.csv --skip-first-line
{
    "Server": [
        {
            "username": "admin",
            "info": "Firewall for Stockholm office.",
            "host": "fw.safe.domain.cc",
            "password": "j5nJ2QQnRhp7xYwG8fExygDvD",
            "cryptedinfo": "iLO password is k3PUibwrWMCYCtxlYgsrOiKRZZnIxA"
        },
        {
            "username": "root",
            "info": "unbound, recursive resolver for Stockholm office",
            "host": "resolver.safe.domain.cc",
            "password": "Xf6PVBdlad40sS3C7u1H6mVsD",
            "cryptedinfo": ""
        },
        {
            "username": "rolf",
            "info": "Rolf Rolfsson",
            "host": "safe.domain.cc",
            "password": "BoDJCrHLF4VNOyeBOuKzZocqc",
            "cryptedinfo": "cryptedinfo"
        },
        {
            "username": "sven",
            "info": "Sven Hell",
            "host": "safe.domain.cc",
            "password": "lKuBdy1k6HMxbVh6vRB5yW7q6",
            "cryptedinfo": "cryptedinfo"
        },
        {
            "username": "root",
            "info": "KDC located in D4K3",
            "host": "kdc.safe.domain.cc",
            "password": "uFwWyzrAjyU4RKVdnnClXMuJ5",
            "cryptedinfo": "Backup GPG passphrase is 3Bxq8Df2LrPpf99qT0mml8SI5Di9hY7QwDMioHvARxj1fmXRFy"
        }
    ]
}

When importing CSV files with extra fields, you will receive a warning.

WARNING: Extra, unmatched columns detected. Import will most likely fail due to this.
WARNING: Consider using "--remove-extra-columns" or "--stuff-extra".

To remediate, you can either delete the extra fields:

$ csv-importer.py --no-rest --template 'Credit Card' --csv file.csv --json file.json --remove-extra-columns

Or concatenate them into an existing field:

$ csv-importer.py --no-rest --template Note --csv file.csv --json file.json --stuff-extra note

List all built in templates in off-line mode:

$ python3 csv-importer.py --no-rest --list-templates
Builtin templates: Server, Login, Short Login, PIN Code, Note, Credit Card

List fieldnames from built in templates, in off-line mode:

$ csv-importer.py --no-rest --list-fieldnames --template "Credit Card"
service,cardno,expires,cvc,owner,pincode,note1,note2

Since certain fields in different templates requires a value, it is possible to specify fields that can not be empty, and also supply an appropriate string as a filler.

$ cat file.csv
exocet,,5b9VcuPpwQG1R0MBCk8TEMtT7w7hd0j1i3iRERqm,https://exocet.domain.tld/login,Serialno: u54898945
,root,UiA7NrjVcOMWcd1aUZaW1lFUuDzXFJkGzZ7aSjmU,,same password for the admin user in the webui
$ csv-importer.py --no-rest --csv file.csv --not-empty host,username --fill-with "blank"
{
    "Server": [
        {
            "host": "exocet",
            "username": "blank",
            "password": "5b9VcuPpwQG1R0MBCk8TEMtT7w7hd0j1i3iRERqm",
            "info": "https://exocet.domain.tld/login",
            "cryptedinfo": "Serialno: u54898945"
        },
        {
            "host": "blank",
            "username": "root",
            "password": "UiA7NrjVcOMWcd1aUZaW1lFUuDzXFJkGzZ7aSjmU",
            "info": "",
            "cryptedinfo": "same password for the admin user in the webui"
        }
    ]
}

Limitations / Known issues

No known limitation.

License

GPL

storedsafe / csv-importer