I am running the latest master branch (offline), and when the code gets to the training point it crashes when trying to forward() the model and evaluate the loss function:

neptune: Executing in Offline Mode.
2018-08-15 18-34-40 google-ai-odt >>> training
2018-08-15 18-35-03 google-ai-odt >>> Training on a reduced class subset: ['Person', 'Car', 'Dress', 'Footwear']
2018-08-15 18:35:05 steppy >>> initializing Step label_encoder...
2018-08-15 18:35:05 steppy >>> initializing Step label_encoder...
2018-08-15 18:35:05 steppy >>> initializing experiment directories under experiments
2018-08-15 18:35:05 steppy >>> initializing experiment directories under experiments
2018-08-15 18:35:05 steppy >>> done: initializing experiment directories
2018-08-15 18:35:05 steppy >>> done: initializing experiment directories
2018-08-15 18:35:05 steppy >>> Step label_encoder initialized
2018-08-15 18:35:05 steppy >>> Step label_encoder initialized
2018-08-15 18:35:05 steppy >>> initializing Step loader...
2018-08-15 18:35:05 steppy >>> initializing Step loader...
2018-08-15 18:35:05 steppy >>> initializing experiment directories under experiments
2018-08-15 18:35:05 steppy >>> initializing experiment directories under experiments
2018-08-15 18:35:05 steppy >>> done: initializing experiment directories
2018-08-15 18:35:05 steppy >>> done: initializing experiment directories
2018-08-15 18:35:05 steppy >>> Step loader initialized
2018-08-15 18:35:05 steppy >>> Step loader initialized
neptune: Executing in Offline Mode.
2018-08-15 18:35:07 steppy >>> initializing Step retinanet...
2018-08-15 18:35:07 steppy >>> initializing Step retinanet...
2018-08-15 18:35:07 steppy >>> initializing experiment directories under experiments
2018-08-15 18:35:07 steppy >>> initializing experiment directories under experiments
2018-08-15 18:35:07 steppy >>> done: initializing experiment directories
2018-08-15 18:35:07 steppy >>> done: initializing experiment directories
2018-08-15 18:35:07 steppy >>> Step retinanet initialized
2018-08-15 18:35:07 steppy >>> Step retinanet initialized
2018-08-15 18:35:07 steppy >>> cleaning cache...
2018-08-15 18:35:07 steppy >>> cleaning cache...
2018-08-15 18:35:07 steppy >>> cleaning cache done
2018-08-15 18:35:07 steppy >>> cleaning cache done
2018-08-15 18:35:07 steppy >>> Step label_encoder, adapting inputs...
2018-08-15 18:35:07 steppy >>> Step label_encoder, adapting inputs...
2018-08-15 18:35:07 steppy >>> Step label_encoder, fitting and transforming...
2018-08-15 18:35:07 steppy >>> Step label_encoder, fitting and transforming...
2018-08-15 18:35:10 steppy >>> Step label_encoder, persisting transformer to the experiments/transformers/label_encoder
2018-08-15 18:35:10 steppy >>> Step label_encoder, persisting transformer to the experiments/transformers/label_encoder
2018-08-15 18:35:10 steppy >>> Step loader, adapting inputs...
2018-08-15 18:35:10 steppy >>> Step loader, adapting inputs...
2018-08-15 18:35:10 steppy >>> Step loader, transforming...
2018-08-15 18:35:10 steppy >>> Step loader, transforming...
2018-08-15 18:35:10 steppy >>> Step retinanet, unpacking inputs...
2018-08-15 18:35:10 steppy >>> Step retinanet, unpacking inputs...
2018-08-15 18:35:10 steppy >>> Step retinanet, fitting and transforming...
2018-08-15 18:35:10 steppy >>> Step retinanet, fitting and transforming...
2018-08-15 18:35:13 steppy >>> starting training...
2018-08-15 18:35:13 steppy >>> starting training...
2018-08-15 18:35:13 steppy >>> initial lr: 1e-05
2018-08-15 18:35:13 steppy >>> initial lr: 1e-05
2018-08-15 18:35:13 steppy >>> epoch 0 ...
2018-08-15 18:35:13 steppy >>> epoch 0 ...
2018-08-15 18:35:13 steppy >>> epoch 0 batch 0 ...
2018-08-15 18:35:13 steppy >>> epoch 0 batch 0 ...
Traceback (most recent call last):
  File "main.py", line 78, in <module>
    main()
  File "/home/m09170/anaconda3/lib/python3.6/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/home/m09170/anaconda3/lib/python3.6/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/home/m09170/anaconda3/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/m09170/anaconda3/lib/python3.6/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/m09170/anaconda3/lib/python3.6/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "main.py", line 16, in train
    pipeline_manager.train(pipeline_name, dev_mode)
  File "/media/nvme1/kaggle-openimages/src/open-solution-googleai-object-detection/src/pipeline_manager.py", line 21, in train
    train(pipeline_name, dev_mode)
  File "/media/nvme1/kaggle-openimages/src/open-solution-googleai-object-detection/src/pipeline_manager.py", line 85, in train
    pipeline.fit_transform(data)
  File "/media/nvme1/kaggle-openimages/src/open-solution-googleai-object-detection/src/steppy_dev/base.py", line 280, in fit_transform
    step_output_data = self._cached_fit_transform(step_inputs)
  File "/media/nvme1/kaggle-openimages/src/open-solution-googleai-object-detection/src/steppy_dev/base.py", line 390, in _cached_fit_transform
    step_output_data = self.transformer.fit_transform(**step_inputs)
  File "/home/m09170/anaconda3/lib/python3.6/site-packages/steppy/base.py", line 605, in fit_transform
    self.fit(*args, **kwargs)
  File "/media/nvme1/kaggle-openimages/src/open-solution-googleai-object-detection/src/models.py", line 32, in fit
    metrics = self._fit_loop(data)
  File "/media/nvme1/kaggle-openimages/src/open-solution-googleai-object-detection/src/models.py", line 63, in _fit_loop
    batch_loss = loss_function(outputs_batch, target) * weight
  File "/home/m09170/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 357, in __call__
    result = self.forward(*input, **kwargs)
  File "/media/nvme1/kaggle-openimages/src/open-solution-googleai-object-detection/src/parallel.py", line 137, in forward
    outputs = _criterion_parallel_apply(replicas, inputs, targets, kwargs)
  File "/media/nvme1/kaggle-openimages/src/open-solution-googleai-object-detection/src/parallel.py", line 192, in _criterion_parallel_apply
    raise output
  File "/media/nvme1/kaggle-openimages/src/open-solution-googleai-object-detection/src/parallel.py", line 167, in _worker
    output = module(*(input + target), **kwargs)
TypeError: can only concatenate tuple (not "dict") to tuple

It looks like the "target" variable used for the loss function is supposed to be a tuple, but instead it is a dictionary. I have to admit I'm not sure what exactly's causing this, but I wanted to see if you have any immediate ideas before I spend time going through the code line by line. Execution command is just: python main.py -- train --pipeline_name retinanet, and the whole config has been filled out with (supposedly) the correct files.

Thanks!

The above script (src/object_detection/dataset_tools/oid_hierarchical_labels_expansion.py) doesn't expand hierarchically image-level labels and boxes as it is supposed to be doing. It just replicates the input file and outputs a duplicated file.

I believe the bug is in the below part of the code :

    with open(parsed_args.input_annotations, 'r') as source:
        with open(parsed_args.output_annotations, 'w') as target:
            header = None
            for line in source:
                if not header:
                    target.writelines(line)
                    continue
                if labels_file:
                    expanded_lines = expansion_generator.expand_labels_from_csv(line)
                else:
                    expanded_lines = expansion_generator.expand_boxes_from_csv(line)
                target.writelines(expanded_lines)

The header is always None and doesn't get set to any other value, hence script just replicates the input file. Possible solution is to just add header=line just before the continue statement.

As our project is prepared for the community, I think we should take care of some community-focused issues, like possibility to visualise the results. In my opinion a proper jupyter notebook or separate cli command for writing to .png would be sufficient, like neptune run main.py visualise_results.

Adding vis for one (or more classes)

neptune run --config "./configs/neptune_config_local.yaml" main.py -- visualize --pipeline_name retinanet --classes_to_visualize='Picture frame,Cat'

If we don't run predict on whole test set submission fails because every image from test set has to have corresponding row in submission file. Adding rows with image id and empty prediction string should be enough.

ids = score > CLS_THRESH
ids = ids.nonzero().squeeze()             # [#obj,]
keep = box_nms(boxes[ids], score[ids], threshold=NMS_THRESH)
return boxes[ids][keep], labels[ids][keep]

When there are no positive ids (where score > CSL_TRESH) code tries to index with empty tensor and that raises a PyTorch error

In COCO dataset there are 80 different classes, some architectures that worked previously on COCO might not work on Open Images in scale 1:1. Having an option to train on just a subset of classes (e.g. 80) and sampler from #1 that will generate images with instances from this subset could help evaluating base architectures.

So this little hack seems quite useful and might give a (small) boost. I've only seen it beeing implemented here: https://github.com/fizyr/keras-retinanet/blob/ebb4512a0ab0efc08bdb8e800e77ab89b6ca2cc1/keras_retinanet/preprocessing/generator.py#L210

The idea is to avoid destructive resizing by precomputing aspect ratios (image dimensions) for all images, and then yielding images with similar AR's to use as little padding as possible.

Bumps protobuf from 3.6.0 to 3.15.0.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Bumps pillow from 5.2.0 to 6.2.0.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

I got your training script to run, but I was wondering what the meaning of "sum" is. I.e. "2018-08-18 09:23:40 steps >>> epoch 1 batch 12 sum: 10.1"

Is that the sum of the losses in the given batch or how can I interpret it? I looked into the code and couldn't find the information. I'm assuming it's part of the steppy toolkit?

I belive this https://github.com/neptune-ml/open-solution-googleai-object-detection/blob/master/src/pipeline_manager.py#L36 line is VERY dangerous, it can easly delete someones home-dir if he does not understands what we ask for.

• We could create new dir for each run (in a basedir provided)
• We should store the config file in the experiment_dir too (just make a copy?)