Hello, was excited to see you reimplemented LinkNet in PyTorch.

Can you verify whether the model has been tested? I ran into a few issues, including that linknet is not included in the get_model() function in models/init.py and also this line in model/utils.py is broken. I seem to have fixed these two but am running into Cuda bad params world. I wanted to ask first whether or not it's been tested (and if you could put me to a script that works with it) before diving in deeper. Thanks!

Modifications:

1. In cityscapes_loader.py, add args for mean version and img_norm, and input type for scipy.misc.imresize needs to be uint8 with RGB mode (since original pretrained model uses pascal mean image, and image doesn't normalize to [0,1])
2. Fix wrong number n_blocks for bottoleNeckIdentityPSP in residualBlockPSP function (n_blocks -> n_blocks-1)
3. Add auxiliary training layers for training PSPNet
4. Modify tile_predict with flip arg and support for batch with tensor type
5. Add PSPNet support for training and testing (extra args: img_norm, eval_flip, measure_time)

Validation results on cityscapes validation set (mIoU/pAcc): Without flip: 78.65/96.31 With flip: 78.80/96.34 I feed images into network input size: 1025x2049 and single scale (since model input is odd numbers (713x713)) Run on single GTX 1080TI, time is about 1.2~1.3 fps python validate.py --model_path checkpoints/pspnet_101_cityscapes.pth --dataset cityscapes --img_rows 1025 --img_cols 2049 --no-img_norm --eval_flip --measure_time --batch_size 2 --split val

Pretrained models in pytorch: pspnet_50_ade20k.pth pspnet_101_cityscapes.pth pspnet_101_pascalvoc.pth

model： fcn8s default parameter, n_epoch=50

when epoch=2，mean IoU=0.19 epoch=5，meanIoU=0.425 epoch=6,7,8....49,mean IoU acc and other metrics unchanged

Has someone trained successfully?

I've installed all requirement package, but as I run train.py --arch fcn8s ( I set ade20k as default), there are some warning and all result are nan like this:

/home/zqk/work/pytorch_seg/pytorch-semseg/ptsemseg/metrics.py:31:RuntimeWarning: invalid value encountered in double_scalars acc = np.diag(hist).sum() / hist.sum() /home/zqk/work/pytorch_seg/pytorch-semseg/ptsemseg/metrics.py:32: RuntimeWarning: invalid value encountered in divide acc_cls = np.diag(hist) / hist.sum(axis=1) /home/zqk/work/pytorch_seg/pytorch-semseg/ptsemseg/metrics.py:33: RuntimeWarning: Mean of empty slice acc_cls = np.nanmean(acc_cls) /home/zqk/work/pytorch_seg/pytorch-semseg/ptsemseg/metrics.py:34: RuntimeWarning: invalid value encountered in divide iu = np.diag(hist) / (hist.sum(axis=1) + hist.sum(axis=0) - np.diag(hist)) /home/zqk/work/pytorch_seg/pytorch-semseg/ptsemseg/metrics.py:35: RuntimeWarning: Mean of empty slice mean_iu = np.nanmean(iu) /home/zqk/work/pytorch_seg/pytorch-semseg/ptsemseg/metrics.py:36: RuntimeWarning: invalid value encountered in divide freq = hist.sum(axis=1) / hist.sum() /home/zqk/work/pytorch_seg/pytorch-semseg/ptsemseg/metrics.py:37: RuntimeWarning: invalid value encountered in greater fwavacc = (freq[freq > 0] * iu[freq > 0]).sum() ('FreqW Acc : \t', 0.0) ('Overall Acc: \t', nan) ('Mean Acc : \t', nan) ('Mean IoU : \t', nan) 0it [00:00, ?it/s] ('FreqW Acc : \t', 0.0) ('Overall Acc: \t', nan) ('Mean Acc : \t', nan) ('Mean IoU : \t', nan) 0it [00:00, ?it/s] ('FreqW Acc : \t', 0.0) ('Overall Acc: \t', nan) ('Mean Acc : \t', nan) ('Mean IoU : \t', nan) 0it [00:00, ?it/s] ('FreqW Acc : \t', 0.0) ('Overall Acc: \t', nan) ('Mean Acc : \t', nan) ('Mean IoU : \t', nan)

what's wrong with this? and how to fix it,thanks for your help

@meetshah1995 @josephreisinger The first epoch is normal： Epoch [1/100] Loss: 3.4047 Epoch [1/100] Loss: 2.2006 Epoch [1/100] Loss: 1.7521 Epoch [1/100] Loss: 1.9977 Epoch [1/100] Loss: 1.9035 Epoch [1/100] Loss: 1.6435 Epoch [1/100] Loss: 1.4620 Epoch [1/100] Loss: 1.9718 Epoch [1/100] Loss: 0.9839 Epoch [1/100] Loss: 1.4327 Epoch [1/100] Loss: 1.5200 Epoch [1/100] Loss: 0.9417 Epoch [1/100] Loss: 1.3892 Epoch [1/100] Loss: 1.5654 Epoch [1/100] Loss: 1.4325 Epoch [1/100] Loss: 1.0973 Epoch [1/100] Loss: 1.2371 Epoch [1/100] Loss: 1.5081

But second epoch is wrong：

RuntimeError: Traceback (most recent call last): File "/home/hpl/anaconda2/envs/hpl36/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 55, in _worker_loop samples = collate_fn([dataset[i] for i in batch_indices]) File "/home/hpl/anaconda2/envs/hpl36/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 55, in samples = collate_fn([dataset[i] for i in batch_indices]) File "/home/hpl/code/2018/Face_parsing/pytorch-semseg-master/ptsemseg/loader/camvid_loader.py", line 47, in getitem img, lbl = self.transform(img, lbl) File "/home/hpl/code/2018/Face_parsing/pytorch-semseg-master/ptsemseg/loader/camvid_loader.py", line 64, in transform lbl = torch.from_numpy(lbl).long() RuntimeError: can't convert a given np.ndarray to a tensor - it has an invalid type. The only supported types are: double, float, int64, int32, and uint8.

Many thanks for great opensource implementation of the semantic segmentation in pytorch ever!

I'm trying to proceed through training 'segnet' model on 'pascal' dataset. What I've done:

1. installed pytorch: 0.2.0_4 and python: 2.7.13
2. downloaded VOCtrainval_11-May-2012.tar from http://host.robots.ox.ac.uk/pascal/VOC/voc2012/#Development%20Kit
3. downloaded "Semantic Boundaries Dataset and Benchmark" from http://home.bharathh.info/pubs/codes/SBD/download.html
4. as stated in Readme, extracted and pointed to them in config.json file
5. started training process

• started visdom server

python -m visdom.server

started training as:

python train.py --arch segnet --dataset pascal

training successfully started and going looks well:

after completion, generated 100 segnet_pascal_1_%2d.pkl files

6. So, after that I'm trying to test newly trained model on the simple pictures:

python test.py --model_path segnet_pascal_1_99.pkl --dataset pascal --img_path 2007_000033.jpg --out_path result_33.jpg

But result is quite wrong:

for some reasons, output resolution differ and segmentation was not produced correctly.

Could you please give me some advises what I'm doing wrong?

Many thanks, Ivan

Hello,

When I run python train.py --arch icnet --dataset pascal --n_epoch 500 I get the following output (checked on Windows and Linux, with PyTorch 0.3.1 and 0.4.1) The dataset is the one here.

Using custom loss
Traceback (most recent call last):
  File "train.py", line 160, in <module>
    train(args)
  File "train.py", line 86, in train
    outputs = model(images)
  File "C:\ProgramData\Anaconda3\lib\site-packages\torch\nn\modules\module.py", line 357, in __call__
    result = self.forward(*input, **kwargs)
  File "C:\ProgramData\Anaconda3\lib\site-packages\torch\nn\parallel\data_parallel.py", line 68, in forward
    return self.module(*inputs, **kwargs)
  File "C:\ProgramData\Anaconda3\lib\site-packages\torch\nn\modules\module.py", line 357, in __call__
    result = self.forward(*input, **kwargs)
  File "D:\nn\github\pytorch-semseg\ptsemseg\models\icnet.py", line 120, in forward
    x_sub24, sub4_cls = self.cff_sub24(x_sub4, x_sub2)
  File "C:\ProgramData\Anaconda3\lib\site-packages\torch\nn\modules\module.py", line 357, in __call__
    result = self.forward(*input, **kwargs)
  File "D:\nn\github\pytorch-semseg\ptsemseg\models\utils.py", line 500, in forward
    high_fused_fm = F.relu(low_fm+high_fm, inplace=True)
RuntimeError: The size of tensor a (15) must match the size of tensor b (16) at non-singleton dimension 3

Am I doing something wrong? Please let me know if you need more info.

i run the pspnet in pytorch0.2.0, but I met the error RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation. thanks

When I run the code, it raise ValueError("Too many dimensions: %d > %d." % (ndim, ndmax)). It show that too many dimensions: 3 > 2. who can help me? Thank you!

Hello, when I run train.py with pascal dataset, filenofounderror 'no such file or directory config.json' is shown. The error is located in get_data_path function in pascal_voc_loader.py. So how to fix it? Thx!

I have a dataset of images containing varying number of segments. I also have pixel wise labels for each of them. How can I use this dataset with this library?

The poly learning rate doesn't work as intended. The current implementation is as follows:

def get_lr(self):
        if self.last_epoch % self.decay_iter or self.last_epoch % self.max_iter:
            return [base_lr for base_lr in self.base_lrs]
        else:
            factor = (1 - self.last_epoch / float(self.max_iter)) ** self.gamma
            return [base_lr * factor for base_lr in self.base_lrs]

Notice that the else condition will never get hit since self.last_epoch % self.max_iter will almost always return a non-zero number.

When I run cdo = cdo(), it appears TypeError . How can you solve it?

# In
from cdo import Cdo
import os
import numpy as np
import re

cdo = Cdo()

# Out
TypeError                                 Traceback (most recent call last)
Input In [19], in <cell line: 6>()
      3 import numpy as np
      4 import re
----> 6 cdo = Cdo()

File ~\anaconda3\envs\CMIP6\lib\site-packages\cdo.py:187, in Cdo.__init__(self, cdo, returnNoneOnError, forceOutput, env, debug, tempdir, logging, logFile, cmd, options)
    184 self._cmd = cmd
    185 self._options = options
--> 187 self.operators         = self.__getOperators()
    188 self.noOutputOperators = [op for op in self.operators.keys() if 0 == self.operators[op]]
    189 self.returnNoneOnError = returnNoneOnError

File ~\anaconda3\envs\CMIP6\lib\site-packages\cdo.py:278, in Cdo.__getOperators(self)
    275 def __getOperators(self):  # {{{
    276   operators = {}
--> 278   version = parse_version(getCdoVersion(self.CDO))
    279   if (version < parse_version('1.7.2')):
    280     proc = subprocess.Popen([self.CDO, '-h'], stderr=subprocess.PIPE, stdout=subprocess.PIPE)

File ~\anaconda3\envs\CMIP6\lib\site-packages\cdo.py:78, in getCdoVersion(path2cdo, verbose)
     77 def getCdoVersion(path2cdo, verbose=False):
---> 78   proc = subprocess.Popen([path2cdo, '-V'], stderr=subprocess.PIPE, stdout=subprocess.PIPE)
     79   ret = proc.communicate()
     81   cdo_help_stdout = ret[0].decode("utf-8")

File ~\anaconda3\envs\CMIP6\lib\subprocess.py:951, in Popen.__init__(self, args, bufsize, executable, stdin, stdout, stderr, preexec_fn, close_fds, shell, cwd, env, universal_newlines, startupinfo, creationflags, restore_signals, start_new_session, pass_fds, user, group, extra_groups, encoding, errors, text, umask)
    947         if self.text_mode:
    948             self.stderr = io.TextIOWrapper(self.stderr,
    949                     encoding=encoding, errors=errors)
--> 951     self._execute_child(args, executable, preexec_fn, close_fds,
    952                         pass_fds, cwd, env,
    953                         startupinfo, creationflags, shell,
    954                         p2cread, p2cwrite,
    955                         c2pread, c2pwrite,
    956                         errread, errwrite,
    957                         restore_signals,
    958                         gid, gids, uid, umask,
    959                         start_new_session)
    960 except:
    961     # Cleanup if the child failed starting.
    962     for f in filter(None, (self.stdin, self.stdout, self.stderr)):

File ~\anaconda3\envs\CMIP6\lib\subprocess.py:1360, in Popen._execute_child(self, args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, unused_restore_signals, unused_gid, unused_gids, unused_uid, unused_umask, unused_start_new_session)
   1358     args = list2cmdline([args])
   1359 else:
-> 1360     args = list2cmdline(args)
   1362 if executable is not None:
   1363     executable = os.fsdecode(executable)

File ~\anaconda3\envs\CMIP6\lib\subprocess.py:565, in list2cmdline(seq)
    563 result = []
    564 needquote = False
--> 565 for arg in map(os.fsdecode, seq):
    566     bs_buf = []
    568     # Add a space to separate this argument from the others

File ~\anaconda3\envs\CMIP6\lib\os.py:822, in _fscodec.<locals>.fsdecode(filename)
    816 def fsdecode(filename):
    817     """Decode filename (an os.PathLike, bytes, or str) from the filesystem
    818     encoding with 'surrogateescape' error handler, return str unchanged. On
    819     Windows, use 'strict' error handler if the file system encoding is
    820     'mbcs' (which is the default encoding).
    821     """
--> 822     filename = fspath(filename)  # Does type-checking of `filename`.
    823     if isinstance(filename, bytes):
    824         return filename.decode(encoding, errors)

TypeError: expected str, bytes or os.PathLike object, not NoneType

For the validation, we need to pass the path of saved model after training, but i am unable to find the path. Though in config file, i have added resume: saved_model.pt but i am unable to find the location of the saved model.

Traceback (most recent call last): File "E:/Semantic Segmentation/pytorch-semseg-master/train.py", line 229, in train(cfg, writer, logger) File "E:/Semantic Segmentation/pytorch-semseg-master/train.py", line 80, in train optimizer_cls = get_optimizer(cfg) File "E:\Semantic Segmentation\pytorch-semseg-master\ptsemseg\optimizers_init_.py", line 24, in get_optimizer opt_name = cfg["training"]["optimizer"]["name"] KeyError: 'name'

I'm currently trying to train HardNet on CamVid but I always get the below error :

Traceback (most recent call last): File "train.py", line 267, in train(cfg, writer, logger) File "train.py", line 138, in train for (images, labels, _) in trainloader: ValueError: not enough values to unpack (expected 3, got 2) Did anyone encounter this issue or succeed in finding the problem ?

Any help is appreciated.