Giters

RobotLocomotion / pytorch-dense-correspondence

Code for "Dense Object Nets: Learning Dense Visual Object Descriptors By and For Robotic Manipulation"

Home Page:https://arxiv.org/pdf/1806.08756.pdf

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Trouble applying cross-object loss

darren-harton opened this issue · comments

commented

Hello and thank you for making this repo available!

I'm having trouble replicating Figure 4(b) from the paper.
image

Using notebooks/dense_correspondence/experiments/multi_object/training_multi_object.ipynb as an example, I've trained a model setting the number of dimensions to 2 and the data type probabilities for SINGLE_OBJECT_WITHIN_SCENE and DIFFERENT_OBJECT both to 0.5, as described in the figure summary.

However, when evaluating the model based on the examples provided in notebooks/dense_correspondence/evaluation/evaluation_clusters_2d.ipynb, the feature space for each object appears to overlap even more than without cross-object loss enabled. Do you have any suggestions for how to remedy this?

Here is my training code and the resulting 2d cluster plot:

isolated_dataset_config_filename = os.path.join(utils.getDenseCorrespondenceSourceDir(), 'config', 'dense_correspondence', 
                               'dataset', 'composite', "caterpillar_baymax_starbot_all_front_single_only.yaml")

cluttered_dataset_config_filename = os.path.join(utils.getDenseCorrespondenceSourceDir(), 'config', 'dense_correspondence', 
                               'dataset', 'composite', "caterpillar_baymax_starbot_all_front.yaml")

train_config_file = os.path.join(utils.getDenseCorrespondenceSourceDir(), 'config', 'dense_correspondence', 
                               'training', 'training.yaml')



logging_dir = "trained_models/cluttered_scene"
num_iterations = 5000
num_image_pairs = 100
debug = False

TRAIN = True
EVALUATE = True

d = 2
M_background = 2.0



# load dataset and training config
dataset_config = utils.getDictFromYamlFilename(isolated_dataset_config_filename)
dataset = SpartanDataset(config=dataset_config)
train_config = utils.getDictFromYamlFilename(train_config_file)

name = "multi_object_isolated_M_background_%.1f_%d" %(M_background, d)
print "training %s" %(name)
train = DenseCorrespondenceTraining(dataset=dataset, config=train_config)
train._config["training"]["logging_dir"] = logging_dir
train._config["training"]["logging_dir_name"] = name
train._config["training"]["num_iterations"] = num_iterations
train._config["dense_correspondence_network"]["descriptor_dimension"] = d

train._config["loss_function"]["M_background"] = M_background

train._config["training"]["data_type_probabilities"]["SINGLE_OBJECT_WITHIN_SCENE"] = 0.5
train._config["training"]["data_type_probabilities"]["DIFFERENT_OBJECT"] = 0.5


if TRAIN:
    train.run()
print "finished training descriptor of dimension %d" %(d)

image

commented

Never mind, I've discovered that the problem was due to a change I made.

commented

Ok glad you got it sorted.