aosokin / os2d

Hi Anton

I am stuck with this error message:
assert class_images is not None, "If class_conv_layer is None than class_images cannot be None"

I want to convert your pytorch model into tensorflow model using onnx. I try to create dummy_input and get this error message.
Can you please explain it? Do I always need to specify class_images?

Here is an example of my conversion code:

model = Os2dModel()
dummy_input = Variable(torch.randn(64, 3, 7, 7, device='cuda'))
torch.onnx.export(Os2dModel(logger, is_cuda = True), dummy_input, "os2d_v2-train.onnx")

Hi Jenny,
Please note that the forward method of the Os2dModel class requires more than one argument:

os2d/os2d/modeling/model.py

Lines 235 to 258 in 96c488b

    
               def forward(self, images=None, class_images=None, 
        
                                 feature_maps=None, class_head=None, 
        
                                 train_mode=False, fine_tune_features=True): 
        
                   """ Forward pass of the OS2D model. Cant function in several different regimes: 
        
                       [training mode] Extract features from input and class images, and applies the model to get  
        
                           clasificaton/localization scores of all classes on all images 
        
                           Args: 
        
                               images (tensor) - batch of input images 
        
                               class_images (list of tensors) - list of class images (possibly of different sizes) 
        
                               train_mode (bool) - should be True 
        
                               fine_tune_features (bool) - flag showing whether to enable gradients over features 
        
                       [evaluation mode] 
        
                               feature_maps (tensor) - pre-extracted feature maps, sized batch_size x feature_dim x height x width 
        
                               class_head (Os2dHead) - head created to detect some classes, 
        
                                   inside has class_feature_maps, sized class_batch_size x feature_dim x class_height x class_width 
        
                               train_mode (bool) - should be False 
        
                   Outputs: 
        
                       loc_scores (tensor) - localization prediction, sized batch_size x num_classes x 4 x num_anchors (bbox parameterization) 
        
                       class_scores (tensor) - classification prediction, sized batch_size x num_classes x num_anchors 
        
                       class_scores_transform_detached (tensor) - same, but with transofrms detached from the computational graph 
        
                           used not to tune transofrmation on the negative examples 
        
                       fm_sizes (FeatureMapSize) - size of the output score map, num_anchors == fm_sizes.w * fm_sizes.h 
        
                       transform_corners (tensor) - points defining parallelograms showing transformations, sized batch_size x num_classes x 8 x num_anchors 
        
                   """

My guess is that you need to provide class_images as the error message says. It should be a list of tensors containing images of classes-to-detect.

Best,
Anton

Hi Anton

For training mode, I understand, I need to specify the classes but what about inference mode? As I am interested in inference mode, not training.

I think you can still pass images, class_images and train_mode=False into this function at it should work. This way the features from the class images will be extracted at the same call. Alternatively you can precompute these class features and pass the class_head as done here:

os2d/os2d/modeling/model.py

Lines 268 to 269 in 96c488b

    
           class_feature_maps = self.net_label_features(class_images) 
        
           class_head = self.os2d_head_creator.create_os2d_head(class_feature_maps)

	def forward(self, images=None, class_images=None,
	feature_maps=None, class_head=None,
	train_mode=False, fine_tune_features=True):
	""" Forward pass of the OS2D model. Cant function in several different regimes:
	[training mode] Extract features from input and class images, and applies the model to get
	clasificaton/localization scores of all classes on all images
	Args:
	images (tensor) - batch of input images
	class_images (list of tensors) - list of class images (possibly of different sizes)
	train_mode (bool) - should be True
	fine_tune_features (bool) - flag showing whether to enable gradients over features
	[evaluation mode]
	feature_maps (tensor) - pre-extracted feature maps, sized batch_size x feature_dim x height x width
	class_head (Os2dHead) - head created to detect some classes,
	inside has class_feature_maps, sized class_batch_size x feature_dim x class_height x class_width
	train_mode (bool) - should be False
	Outputs:
	loc_scores (tensor) - localization prediction, sized batch_size x num_classes x 4 x num_anchors (bbox parameterization)
	class_scores (tensor) - classification prediction, sized batch_size x num_classes x num_anchors
	class_scores_transform_detached (tensor) - same, but with transofrms detached from the computational graph
	used not to tune transofrmation on the negative examples
	fm_sizes (FeatureMapSize) - size of the output score map, num_anchors == fm_sizes.w * fm_sizes.h
	transform_corners (tensor) - points defining parallelograms showing transformations, sized batch_size x num_classes x 8 x num_anchors
	"""

	class_feature_maps = self.net_label_features(class_images)
	class_head = self.os2d_head_creator.create_os2d_head(class_feature_maps)

Error Message