Access quantized weights

[amnamasood]

I have been trying to access the quantized weights, but the quant layer has no attribute 'weight'. Looking at the code, I think that the quant layer only quantizes the input (i.e. the result of previous layers) in the forward pass, and not the weights of layers which I want quantized. Is there any workaround for this?

[Aaron Chen]

You can refer to quantize.py.

if args.param_bits < 32:
    state_dict = model_raw.state_dict()
    state_dict_quant = OrderedDict()
    sf_dict = OrderedDict()
    for k, v in state_dict.items():
        if 'running' in k:
            if args.bn_bits >=32:
                print("Ignoring {}".format(k))
                state_dict_quant[k] = v
                continue
            else:
                bits = args.bn_bits
        else:
            bits = args.param_bits

        if args.quant_method == 'linear':
            sf = bits - 1. - quant.compute_integral_part(v, overflow_rate=args.overflow_rate)
            v_quant  = quant.linear_quantize(v, sf, bits=bits)
        elif args.quant_method == 'log':
            v_quant = quant.log_minmax_quantize(v, bits=bits)
        elif args.quant_method == 'minmax':
            v_quant = quant.min_max_quantize(v, bits=bits)
        else:
            v_quant = quant.tanh_quantize(v, bits=bits)
        state_dict_quant[k] = v_quant
        print(k, bits)
    model_raw.load_state_dict(state_dict_quant)

[amnamasood]

Thank you. It worked, but I have some confusion regarding 'sf'. My param_bits is 8 but the sf is often greater than 8. Can you please explain what 'sf' represents?

[amnamasood]

Hello again. I am checking my quantized network after specifying sf=6 in LinearQuant. the problem is that when I am quantizing forward iteration and evaluating the accuracy for vgg16 network, it comes out to be 68%. But when I am done quantizing forward iteration and evaluate the model again, the accuracy drops to 2%. Your help would be appreciated.

[Aaron Chen]

sf decide the least precision, bits decide the range. So if sf=2 and bits=8, it means the least precision is 0.25, and range is [-127, 128] * 0.25