I have trouble with multi class training
dotorl127 opened this issue · comments
Hello, I wanna training multi class(car, pedestrian, cyclist).
so, I've modified proto file in config as below.
model: {
second: {
voxel_generator {
point_cloud_range : [0, -40, -3, 70, 40, 1]
voxel_size : [0.16, 0.16, 4]
max_number_of_points_per_voxel : 100
}
num_class: 3
voxel_feature_extractor: {
module_class_name: "PillarFeatureNet"
num_filters: [64]
with_distance: false
}
middle_feature_extractor: {
module_class_name: "PointPillarsScatter"
}
rpn: {
module_class_name: "RPN"
layer_nums: [3, 5, 5]
layer_strides: [2, 2, 2]
num_filters: [64, 128, 256]
upsample_strides: [1, 2, 4]
num_upsample_filters: [128, 128, 128]
use_groupnorm: false
num_groups: 32
}
loss: {
classification_loss: {
weighted_sigmoid_focal: {
alpha: 0.25
gamma: 2.0
anchorwise_output: true
}
}
localization_loss: {
weighted_smooth_l1: {
sigma: 3.0
code_weight: [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0]
}
}
classification_weight: 1.0
localization_weight: 2.0
}
# Outputs
use_sigmoid_score: true
encode_background_as_zeros: true
encode_rad_error_by_sin: true
use_direction_classifier: true
direction_loss_weight: 0.2
use_aux_classifier: false
# Loss
pos_class_weight: 1.0
neg_class_weight: 1.0
loss_norm_type: NormByNumPositives
# Postprocess
post_center_limit_range: [0, -40, -5, 70, 40, 5]
use_rotate_nms: false
use_multi_class_nms: false
nms_pre_max_size: 1000
nms_post_max_size: 300
nms_score_threshold: 0.05
nms_iou_threshold: 0.5
use_bev: false
num_point_features: 4
without_reflectivity: false
box_coder: {
ground_box3d_coder: {
linear_dim: false
encode_angle_vector: false
}
}
target_assigner: {
anchor_generators: {
anchor_generator_stride: {
sizes: [1.6, 3.9, 1.56] # wlh
strides: [0.16, 0.16, 0.0] # if generate only 1 z_center, z_stride will be ignored
offsets: [0.8, -39.2, -1.78] # origin_offset + strides / 2
rotations: [0, 1.57] # 0, pi/2
matched_threshold : 0.6
unmatched_threshold : 0.45
}
}
anchor_generators: {
anchor_generator_stride: {
sizes: [0.6, 1.76, 1.73] # wlh
strides: [0.16, 0.16, 0.0] # if generate only 1 z_center, z_stride will be ignored
offsets: [0.08, -39.2, -1.78] # origin_offset + strides / 2
rotations: [0, 1.57] # 0, pi/2
matched_threshold : 0.5
unmatched_threshold : 0.35
}
}
anchor_generators: {
anchor_generator_stride: {
sizes: [0.6, 0.8, 1.73] # wlh
strides: [0.16, 0.16, 0.0] # if generate only 1 z_center, z_stride will be ignored
offsets: [0.08, -39.2, -1.78] # origin_offset + strides / 2
rotations: [0, 1.57] # 0, pi/2
matched_threshold : 0.5
unmatched_threshold : 0.35
}
}
sample_positive_fraction : -1
sample_size : 512
region_similarity_calculator: {
nearest_iou_similarity: {
}
}
}
}
}
train_input_reader: {
#record_file_path: "/data/sets/kitti_second/kitti_train.tfrecord"
class_names: ["Car", "Cyclist", "Pedestrian"]
max_num_epochs : 160
batch_size: 2
prefetch_size : 25
max_number_of_voxels: 12000
shuffle_points: true
num_workers: 2
groundtruth_localization_noise_std: [0.25, 0.25, 0.25]
groundtruth_rotation_uniform_noise: [-0.15707963267, 0.15707963267]
global_rotation_uniform_noise: [-0.78539816, 0.78539816]
global_scaling_uniform_noise: [0.95, 1.05]
global_random_rotation_range_per_object: [0, 0]
anchor_area_threshold: 1
remove_points_after_sample: false
groundtruth_points_drop_percentage: 0.0
groundtruth_drop_max_keep_points: 15
database_sampler {
database_info_path: "/home/moon/nutonomy_pointpillars/second/data/sets/kitti_second/kitti_dbinfos_train.pkl"
sample_groups {
name_to_max_num {
key: "Car"
value: 15
}
}
sample_groups {
name_to_max_num {
key: "Pedestrian"
value: 8
}
}
sample_groups {
name_to_max_num {
key: "Cyclist"
value: 8
}
}
database_prep_steps {
filter_by_min_num_points {
min_num_point_pairs {
key: "Car"
value: 5
}
min_num_point_pairs {
key: "Pedestrian"
value: 5
}
min_num_point_pairs {
key: "Cyclist"
value: 5
}
}
}
database_prep_steps {
filter_by_difficulty {
removed_difficulties: [-1]
}
}
global_random_rotation_range_per_object: [0, 0]
rate: 1.0
}
remove_unknown_examples: false
remove_environment: false
kitti_info_path: "/home/moon/nutonomy_pointpillars/second/data/sets/kitti_second/kitti_infos_train.pkl"
kitti_root_path: "/home/moon/nutonomy_pointpillars/second/data/sets/kitti_second"
}
train_config: {
optimizer: {
adam_optimizer: {
learning_rate: {
exponential_decay_learning_rate: {
initial_learning_rate: 0.0002
decay_steps: 27840 # 1856 steps per epoch * 15 epochs
decay_factor: 0.8
staircase: true
}
}
weight_decay: 0.0001
}
use_moving_average: false
}
inter_op_parallelism_threads: 4
intra_op_parallelism_threads: 4
steps: 296960 # 1856 steps per epoch * 160 epochs
steps_per_eval: 9280 # 1856 steps per epoch * 5 epochs
save_checkpoints_secs : 1800 # half hour
save_summary_steps : 10
enable_mixed_precision: false
loss_scale_factor : 512.0
clear_metrics_every_epoch: false
}
eval_input_reader: {
#record_file_path: "/data/sets/kitti_second/kitti_val.tfrecord"
class_names: ["Car", "Cyclist", "Pedestrian"]
batch_size: 2
max_num_epochs : 160
prefetch_size : 25
max_number_of_voxels: 12000
shuffle_points: false
num_workers: 3
anchor_area_threshold: 1
remove_environment: false
kitti_info_path: "/home/moon/nutonomy_pointpillars/second/data/sets/kitti_second/kitti_infos_val.pkl"
kitti_root_path: "/home/moon/nutonomy_pointpillars/second/data/sets/kitti_second"
}
and When I've tried training, error occurred as below.
middle_class_name PointPillarsScatter
num_trainable parameters: 74
{'Cyclist': 5, 'Pedestrian': 5, 'Car': 5}
[-1]
load 14357 Car database infos
load 2207 Pedestrian database infos
load 734 Cyclist database infos
load 1297 Van database infos
load 56 Person_sitting database infos
load 488 Truck database infos
load 224 Tram database infos
load 337 Misc database infos
After filter database:
load 10520 Car database infos
load 2066 Pedestrian database infos
load 580 Cyclist database infos
load 826 Van database infos
load 53 Person_sitting database infos
load 321 Truck database infos
load 199 Tram database infos
load 259 Misc database infos
remain number of infos: 3712
remain number of infos: 3769
WORKER 0 seed: 1592207469
WORKER 1 seed: 1592207470
/home/moon/nutonomy_pointpillars/second/core/geometry.py:97: NumbaWarning:
Compilation is falling back to object mode WITH looplifting enabled because Function "points_in_convex_polygon_3d_jit" failed type inference due to: Invalid use of type(CPUDispatcher(<function surface_equ_3d_jit at 0x7f6203c33158>)) with parameters (array(float64, 4d, A))
- parameterized
[1] During: resolving callee type: type(CPUDispatcher(<function surface_equ_3d_jit at 0x7f6203c33158>))
[2] During: typing of call at /home/moon/nutonomy_pointpillars/second/core/geometry.py (118)
File "core/geometry.py", line 118:
def points_in_convex_polygon_3d_jit(points,
num_surfaces = np.full((num_polygons,), 9999999, dtype=np.int64)
normal_vec, d = surface_equ_3d_jit(polygon_surfaces[:, :, :3, :])
^
@numba.jit(nopython=False)
/home/moon/nutonomy_pointpillars/second/core/geometry.py:97: NumbaWarning:
Compilation is falling back to object mode WITH looplifting enabled because Function "points_in_convex_polygon_3d_jit" failed type inference due to: Invalid use of type(CPUDispatcher(<function surface_equ_3d_jit at 0x7f6203c33158>)) with parameters (array(float64, 4d, A))
- parameterized
[1] During: resolving callee type: type(CPUDispatcher(<function surface_equ_3d_jit at 0x7f6203c33158>))
[2] During: typing of call at /home/moon/nutonomy_pointpillars/second/core/geometry.py (118)
File "core/geometry.py", line 118:
def points_in_convex_polygon_3d_jit(points,
num_surfaces = np.full((num_polygons,), 9999999, dtype=np.int64)
normal_vec, d = surface_equ_3d_jit(polygon_surfaces[:, :, :3, :])
^
@numba.jit(nopython=False)
/home/moon/nutonomy_pointpillars/second/core/geometry.py:97: NumbaWarning:
Compilation is falling back to object mode WITHOUT looplifting enabled because Function "points_in_convex_polygon_3d_jit" failed type inference due to: cannot determine Numba type of <class 'numba.dispatcher.LiftedLoop'>
File "core/geometry.py", line 123:
def points_in_convex_polygon_3d_jit(points,
sign = 0.0
for i in range(num_points):
^
@numba.jit(nopython=False)
/home/moon/nutonomy_pointpillars/second/core/geometry.py:97: NumbaWarning:
Compilation is falling back to object mode WITHOUT looplifting enabled because Function "points_in_convex_polygon_3d_jit" failed type inference due to: cannot determine Numba type of <class 'numba.dispatcher.LiftedLoop'>
File "core/geometry.py", line 123:
def points_in_convex_polygon_3d_jit(points,
sign = 0.0
for i in range(num_points):
^
@numba.jit(nopython=False)
/home/moon/anaconda3/envs/pointpillars/lib/python3.6/site-packages/numba/object_mode_passes.py:178: NumbaWarning: Function "points_in_convex_polygon_3d_jit" was compiled in object mode without forceobj=True, but has lifted loops.
File "core/geometry.py", line 113:
def points_in_convex_polygon_3d_jit(points,
"""
max_num_surfaces, max_num_points_of_surface = polygon_surfaces.shape[1:3]
^
state.func_ir.loc))
/home/moon/anaconda3/envs/pointpillars/lib/python3.6/site-packages/numba/object_mode_passes.py:188: NumbaDeprecationWarning:
Fall-back from the nopython compilation path to the object mode compilation path has been detected, this is deprecated behaviour.
For more information visit http://numba.pydata.org/numba-doc/latest/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit
File "core/geometry.py", line 113:
def points_in_convex_polygon_3d_jit(points,
"""
max_num_surfaces, max_num_points_of_surface = polygon_surfaces.shape[1:3]
^
state.func_ir.loc))
/home/moon/anaconda3/envs/pointpillars/lib/python3.6/site-packages/numba/object_mode_passes.py:178: NumbaWarning: Function "points_in_convex_polygon_3d_jit" was compiled in object mode without forceobj=True, but has lifted loops.
File "core/geometry.py", line 113:
def points_in_convex_polygon_3d_jit(points,
"""
max_num_surfaces, max_num_points_of_surface = polygon_surfaces.shape[1:3]
^
state.func_ir.loc))
/home/moon/anaconda3/envs/pointpillars/lib/python3.6/site-packages/numba/object_mode_passes.py:188: NumbaDeprecationWarning:
Fall-back from the nopython compilation path to the object mode compilation path has been detected, this is deprecated behaviour.
For more information visit http://numba.pydata.org/numba-doc/latest/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit
File "core/geometry.py", line 113:
def points_in_convex_polygon_3d_jit(points,
"""
max_num_surfaces, max_num_points_of_surface = polygon_surfaces.shape[1:3]
^
state.func_ir.loc))
/home/moon/nutonomy_pointpillars/second/core/preprocess.py:472: NumbaPerformanceWarning: '@' is faster on contiguous arrays, called on (array(float32, 2d, A), array(float32, 2d, C))
points[i:i + 1, :3] = points[i:i + 1, :3] @ rot_mat_T[j]
/home/moon/nutonomy_pointpillars/second/core/preprocess.py:472: NumbaPerformanceWarning: '@' is faster on contiguous arrays, called on (array(float32, 2d, A), array(float32, 2d, C))
points[i:i + 1, :3] = points[i:i + 1, :3] @ rot_mat_T[j]
/home/moon/anaconda3/envs/pointpillars/lib/python3.6/site-packages/numba/typing/npydecl.py:958: NumbaPerformanceWarning: '@' is faster on contiguous arrays, called on (array(float32, 2d, A), array(float32, 2d, C))
warnings.warn(NumbaPerformanceWarning(msg))
/home/moon/anaconda3/envs/pointpillars/lib/python3.6/site-packages/numba/typing/npydecl.py:958: NumbaPerformanceWarning: '@' is faster on contiguous arrays, called on (array(float32, 2d, A), array(float32, 2d, C))
warnings.warn(NumbaPerformanceWarning(msg))
/home/moon/nutonomy_pointpillars/second/core/geometry.py:137: NumbaWarning:
Compilation is falling back to object mode WITH looplifting enabled because Function "points_in_convex_polygon_jit" failed type inference due to: Invalid use of Function() with argument(s) of type(s): (array(float32, 3d, C), Tuple(slice<a:b>, list(int64), slice<a:b>))
- parameterized
In definition 0:
All templates rejected with literals.
In definition 1:
All templates rejected without literals.
In definition 2:
All templates rejected with literals.
In definition 3:
All templates rejected without literals.
In definition 4:
All templates rejected with literals.
In definition 5:
All templates rejected without literals.
In definition 6:
All templates rejected with literals.
In definition 7:
All templates rejected without literals.
In definition 8:
All templates rejected with literals.
In definition 9:
All templates rejected without literals.
In definition 10:
All templates rejected with literals.
In definition 11:
All templates rejected without literals.
In definition 12:
TypeError: unsupported array index type list(int64) in Tuple(slice<a:b>, list(int64), slice<a:b>)
raised from /home/moon/anaconda3/envs/pointpillars/lib/python3.6/site-packages/numba/typing/arraydecl.py:71
In definition 13:
TypeError: unsupported array index type list(int64) in Tuple(slice<a:b>, list(int64), slice<a:b>)
raised from /home/moon/anaconda3/envs/pointpillars/lib/python3.6/site-packages/numba/typing/arraydecl.py:71
This error is usually caused by passing an argument of a type that is unsupported by the named function.
[1] During: typing of intrinsic-call at /home/moon/nutonomy_pointpillars/second/core/geometry.py (153)
File "core/geometry.py", line 153:
def points_in_convex_polygon_jit(points, polygon, clockwise=True):
vec1 = polygon - polygon[:, [num_points_of_polygon - 1] +
list(range(num_points_of_polygon - 1)), :]
^
@numba.jit
/home/moon/nutonomy_pointpillars/second/core/geometry.py:137: NumbaWarning:
Compilation is falling back to object mode WITH looplifting enabled because Function "points_in_convex_polygon_jit" failed type inference due to: Invalid use of Function() with argument(s) of type(s): (array(float32, 3d, C), Tuple(slice<a:b>, list(int64), slice<a:b>))
- parameterized
In definition 0:
All templates rejected with literals.
In definition 1:
All templates rejected without literals.
In definition 2:
All templates rejected with literals.
In definition 3:
All templates rejected without literals.
In definition 4:
All templates rejected with literals.
In definition 5:
All templates rejected without literals.
In definition 6:
All templates rejected with literals.
In definition 7:
All templates rejected without literals.
In definition 8:
All templates rejected with literals.
In definition 9:
All templates rejected without literals.
In definition 10:
All templates rejected with literals.
In definition 11:
All templates rejected without literals.
In definition 12:
TypeError: unsupported array index type list(int64) in Tuple(slice<a:b>, list(int64), slice<a:b>)
raised from /home/moon/anaconda3/envs/pointpillars/lib/python3.6/site-packages/numba/typing/arraydecl.py:71
In definition 13:
TypeError: unsupported array index type list(int64) in Tuple(slice<a:b>, list(int64), slice<a:b>)
raised from /home/moon/anaconda3/envs/pointpillars/lib/python3.6/site-packages/numba/typing/arraydecl.py:71
This error is usually caused by passing an argument of a type that is unsupported by the named function.
[1] During: typing of intrinsic-call at /home/moon/nutonomy_pointpillars/second/core/geometry.py (153)
File "core/geometry.py", line 153:
def points_in_convex_polygon_jit(points, polygon, clockwise=True):
vec1 = polygon - polygon[:, [num_points_of_polygon - 1] +
list(range(num_points_of_polygon - 1)), :]
^
@numba.jit
/home/moon/nutonomy_pointpillars/second/core/geometry.py:137: NumbaWarning:
Compilation is falling back to object mode WITHOUT looplifting enabled because Function "points_in_convex_polygon_jit" failed type inference due to: cannot determine Numba type of <class 'numba.dispatcher.LiftedLoop'>
File "core/geometry.py", line 161:
def points_in_convex_polygon_jit(points, polygon, clockwise=True):
cross = 0.0
for i in range(num_points):
^
@numba.jit
/home/moon/nutonomy_pointpillars/second/core/geometry.py:137: NumbaWarning:
Compilation is falling back to object mode WITHOUT looplifting enabled because Function "points_in_convex_polygon_jit" failed type inference due to: cannot determine Numba type of <class 'numba.dispatcher.LiftedLoop'>
File "core/geometry.py", line 161:
def points_in_convex_polygon_jit(points, polygon, clockwise=True):
cross = 0.0
for i in range(num_points):
^
@numba.jit
/home/moon/anaconda3/envs/pointpillars/lib/python3.6/site-packages/numba/object_mode_passes.py:178: NumbaWarning: Function "points_in_convex_polygon_jit" was compiled in object mode without forceobj=True, but has lifted loops.
File "core/geometry.py", line 148:
def points_in_convex_polygon_jit(points, polygon, clockwise=True):
# first convert polygon to directed lines
num_points_of_polygon = polygon.shape[1]
^
state.func_ir.loc))
/home/moon/anaconda3/envs/pointpillars/lib/python3.6/site-packages/numba/object_mode_passes.py:188: NumbaDeprecationWarning:
Fall-back from the nopython compilation path to the object mode compilation path has been detected, this is deprecated behaviour.
For more information visit http://numba.pydata.org/numba-doc/latest/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit
File "core/geometry.py", line 148:
def points_in_convex_polygon_jit(points, polygon, clockwise=True):
# first convert polygon to directed lines
num_points_of_polygon = polygon.shape[1]
^
state.func_ir.loc))
/home/moon/anaconda3/envs/pointpillars/lib/python3.6/site-packages/numba/object_mode_passes.py:178: NumbaWarning: Function "points_in_convex_polygon_jit" was compiled in object mode without forceobj=True, but has lifted loops.
File "core/geometry.py", line 148:
def points_in_convex_polygon_jit(points, polygon, clockwise=True):
# first convert polygon to directed lines
num_points_of_polygon = polygon.shape[1]
^
state.func_ir.loc))
/home/moon/anaconda3/envs/pointpillars/lib/python3.6/site-packages/numba/object_mode_passes.py:188: NumbaDeprecationWarning:
Fall-back from the nopython compilation path to the object mode compilation path has been detected, this is deprecated behaviour.
For more information visit http://numba.pydata.org/numba-doc/latest/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit
File "core/geometry.py", line 148:
def points_in_convex_polygon_jit(points, polygon, clockwise=True):
# first convert polygon to directed lines
num_points_of_polygon = polygon.shape[1]
^
state.func_ir.loc))
Traceback (most recent call last):
File "pytorch/train.py", line 894, in
fire.Fire()
File "/home/moon/anaconda3/envs/pointpillars/lib/python3.6/site-packages/fire/core.py", line 138, in Fire
component_trace = _Fire(component, args, parsed_flag_args, context, name)
File "/home/moon/anaconda3/envs/pointpillars/lib/python3.6/site-packages/fire/core.py", line 468, in _Fire
target=component.name)
File "/home/moon/anaconda3/envs/pointpillars/lib/python3.6/site-packages/fire/core.py", line 672, in _CallAndUpdateTrace
component = fn(*varargs, **kwargs)
File "pytorch/train.py", line 423, in train
raise e
File "pytorch/train.py", line 323, in train
ret_dict = net(input)
File "/home/moon/anaconda3/envs/pointpillars/lib/python3.6/site-packages/torch/nn/modules/module.py", line 532, in call
result = self.forward(*input, **kwargs)
File "/home/moon/nutonomy_pointpillars/second/pytorch/models/voxelnet.py", line 701, in forward
preds_dict = self.rpn(spatial_features)
File "/home/moon/anaconda3/envs/pointpillars/lib/python3.6/site-packages/torch/nn/modules/module.py", line 532, in call
result = self.forward(*input, **kwargs)
File "/home/moon/nutonomy_pointpillars/second/pytorch/models/voxelnet.py", line 472, in forward
x = torch.cat([up1, up2, up3], dim=1)
RuntimeError: invalid argument 0: Sizes of tensors must match except in dimension 1. Got 252 and 250 in dimension 2 at /opt/conda/conda-bld/pytorch_1579027003190/work/aten/src/THC/generic/THCTensorMath.cu:71
How should I fix proto file to multi class training?
Is anyone success training with multi class?
Best regards.
@dotorl127 I haven't had this error, maybe you can change the old numba version.
@dotorl127 I haven't had this error, maybe you can change the old numba version.
HEY, I trained a car-pedestrian-cyclist model under this repo but the accuracy of pedestrian and cyclist is very low(under 1%).
I check the original second repo and the author improved multiclass feature on version 1.5. And this repo is based on version 1.0. I was wondering did you come up with any solutions for it?
