Quantipy3

Python for people data

Quantipy is an open-source data processing, analysis and reporting software project that builds on the excellent pandas and numpy libraries. Aimed at people data, Quantipy offers support for native handling of special data types like multiple choice variables, statistical analysis using case or observation weights, DataFrame metadata and pretty data exports.

Quantipy for Python 3

This repository is a port of Quantipy from Python 2.x to Python 3.

Key features

Reads plain .csv, converts from Dimensions, SPSS, Decipher, or Ascribe
Open metadata format to describe and manage datasets
Powerful, metadata-driven cleaning, editing, recoding and transformation of datasets
Computation and assessment of data weights
Easy-to-use analysis interface

Features not yet supported in Python 3 version

Structured analysis and reporting via Chain and Cluster containers
Exports to SPSS, Dimensions ddf/mdd, MS Excel and Powerpoint with flexible layouts and various options
Python 3.8 is not yet fully supported, but 3.5, 3.6, and 3.7 are.

Origins

Quantipy was concieved of and instigated by Gary Nelson: http://www.datasmoothie.com

Contributors

Alexander Buchhammer, Alasdair Eaglestone, James Griffiths, Kerstin Müller : https://yougov.co.uk
Datasmoothie’s Birgir Hrafn Sigurðsson and Geir Freysson: http://www.datasmoothie.com

Installation

pip install quantipy3

python3 -m pip install quantipy3

Note that the package is called quantipy3 on pip.

Create a virtual envirionment

If you want to create a virtual environment when using Quantipy:

conda

conda create -n envqp python=3

with venv

python -m venv [your_env_name]

5-minutes to Quantipy

Get started

If you are working with SPSS, import your sav file.

import quantipy as qp
dataset = qp.DataSet("My dataset, wave 1")
dataset.read_spss('my_file.sav')

You can start straight away by exploring what variables are in your file.

dataset.variables()

['gender',
 'agecat',
 'price_satisfaction',
 'numitems_satisfaction',
 'org_satisfaction',
 'service_satisfaction',
 'quality_satisfaction',
 'overall_satisfaction',
 'weight']

If you want more details on a variable, explore it's meta data.

dataset.meta('agecat')

single	codes	texts	missing
agecat: Age category
1	1	18-24	None
2	2	25-34	None
3	3	35-49	None
4	4	50-64	None
5	5	64+	None

Quantipy knows out-of-the-box what SPSS's meta data means and uses it correctly. All codes and labels are the same as in the sav file.

Calculate some results, counts or percentages

dataset.crosstab('price_satisfaction', 'gender')

	Question	agecat. Age category
	Values	All	18-24	25-34	35-49	50-64	64+
Question	Values
price_satisfaction. Price satisfaction	All	582.0	46.0	127.0	230.0	147.0	32.0
	Strongly Negative	72.0	8.0	20.0	22.0	17.0	5.0
	Somewhat Negative	135.0	10.0	30.0	52.0	38.0	5.0
	Neutral	140.0	9.0	32.0	59.0	36.0	4.0
	Somewhat Positive	145.0	12.0	25.0	63.0	33.0	12.0
	Strongly Positive	90.0	7.0	20.0	34.0	23.0	6.0

You can also filter

dataset.crosstab('price_satisfaction', 'agecat', f={'gender':1})

and use a weight column

dataset.crosstab('price_satisfaction', 'agecat', f={'gender':1}, w="weight")

Variables can be created, recoded or edited with DataSet methods, e.g. derive():

mapper = [(1,  '18-35 year old', {'agecat': [1,2]}),
          (2, '36 and older', {'agecat': [3,4,5]})]

dataset.derive('two_age_groups', 'single', dataset.text("Older or younger than 35"), mapper)
dataset.meta('two_age_groups')

single                                              codes     texts              missing
two_age_groups: "Older or youngar than 35"
1                                                       1     18-35 years old    None
2                                                       2     36 and older       None

The DataSet case data component can be inspected with the []-indexer, as known from a pd.DataFrame:

dataset[['gender', 'age']].head(5)

        gender  age
0       1.0    1.0
1       2.0    1.0
2       2.0    2.0
3       1.0    NaN
4       NaN    1.0

Weighting

If your data hasn't been weighted yet, you can use Quantipy's RIM weighting algorithm.

Assuming we have the same variables as before, gender and agecat we can weight the dataset with these two variables:

from quantipy.core.weights.rim import Rim

age_targets = {'agecat':{1:5.0, 2:30.0, 3:26.0, 4:19.0, 5:20.0}}
gender_targets = {'gender':{0:49, 1:51}}
scheme = Rim('gender_and_age')
scheme.set_targets(targets=[age_targets, gender_targets])
dataset.weight(scheme,unique_key='respondentId',
               weight_name="my_weight",
               inplace=True)

Quantipy will show you a weighting report:

Weight variable       weights_gender_and_age
Weight group                  _default_name_
Weight filter                           None
Total: unweighted                 582.000000
Total: weighted                   582.000000
Weighting efficiency               60.009826
Iterations required                14.000000
Mean weight factor                  1.000000
Minimum weight factor               0.465818
Maximum weight factor               6.187700
Weight factor ratio                13.283522

And you can test whether the weighting has worked by running crosstabs:

dataset.crosstab('agecat', ci=['c%'], w='my_new_weight')

	Question	agecat. Age category
Question	Values
agecat. Age category	All	100.0
	18-24	5.0
	25-34	30.0
	35-49	26.0
	50-64	19.0
	64+	20.0

dataset.crosstab('gender', ci=['c%'], w='my_new_weight')

	Question	gender. Gender
Question	Values
gender. Gender	All	100.0
	Male	49.0
	Female	51.0

Contributing

The test suite for Quantipy can be run with the command

python3 -m pytest tests

But when developing a specific aspect of Quantipy, it might be quicker to run (e.g. for the DataSet)

python3 -m unittest tests.test_dataset

Tests for unsupported features are skipped, see here for what tests are supported.

We welcome volunteers and supporters. Please include a test case with any pull request, especially those that run calculations.

Mustyy / quantipy3