Hi - I appreciate there's already an open issue related to speed, but mine is slightly different.

When I run python tools/time_net.py --cfg configs/dds_baselines/regnetx/RegNetX-1.6GF_dds_8gpu.yaml having changed GPUS: from 8 to 1, I get the following dump. I am running this on a batch of size 64, with input resolution 224x224, on a V100, as stated in the paper.

This implies a forward pass of ~62ms, not the 33ms stated in MODEL_ZOO. Have I done something wrong? Not sure why the times are so different. The other numbers (acts, params, flops) all seem fine. The latency differences are seen for other models as well - here is 800MF (39ms vs model zoo's 21ms):

I am using commit a492b56f580d43fb4e003eabda4373b25b4bedec, not the latest version of the repo, but MODEL_ZOO has not been changed since before this commit. This is because it is useful being able to time the models on dummy data, rather than having to construct a dataset. Would it be possible to have an option to do this? I can open a separate issue as a feature request for consideration if necessary.

Hi, After going through the code, I noticed this line: https://github.com/facebookresearch/pycls/blob/cd1cfb185ab5ebef328e2c3a38f68112bbd43712/pycls/datasets/imagenet.py#L55

It seems that the imagenet val dataset does not have images stored in different subdirectories as does with train set. Why is the dataset implement like this? Would you please tell us how to prepare the imagenet dataset so that we could reproduce the result in the model zoo?

Is it possible to use pycls models without Internet access? I'm using pretrained=False parameter and load weights manually, but I'm still getting URLError: <urlopen error [Errno -3] Temporary failure in name resolution>.

Adding test_net.py to evaluate a trained model. Example command:

python tools/test_net.py \
    --cfg configs/baselines/imagenet/R-50-1x64d_bs32_1gpu.yaml \
    TRAIN.START_CHECKPOINT save/resnet50/checkpoints/model_epoch_0096.pyth \
    TEST.BATCH_SIZE 256

I wonder if we want to import duplicated functions from train.py or not. I isolate them for now.

Hi, thanks for your wonderful repo. In your code of update_model_ema https://github.com/facebookresearch/pycls/blob/ee770af5b55cd1959e71af73bf9d5b7d7ac10dc3/pycls/core/net.py#L101-L114

I notice that you are using a magic code adjust = cfg.TRAIN.BATCH_SIZE / cfg.OPTIM.MAX_EPOCH * update_period to modify alpha value. Is there any insight of doing this? If there are some paper of this, could you please help telling me?

Thanks : )

This is a major update and introduces powerful new functionality to pycls.

The pycls codebase now provides powerful support for studying design spaces and more generally population statistics of models as introduced in On Network Design Spaces for Visual Recognition and Designing Network Design Spaces. This idea is that instead of planning a single pycls job (e.g., testing a specific model configuration), one can study the behavior of an entire population of models. This allows for quite powerful and succinct experimental design, and elevates the study of individual model behavior to the study of the behavior of model populations. Please see SWEEP_INFO for details.

This is commit 1 of 2 for the sweep code. It is focused on the sweep config, setting up the sweep, and launching it.

Co-authored-by: Raj Prateek Kosaraju [email protected] Co-authored-by: Piotr Dollar [email protected]

This is a major update and introduces powerful new functionality to pycls.

The pycls codebase now provides powerful support for studying design spaces and more generally population statistics of models as introduced in On Network Design Spaces for Visual Recognition and Designing Network Design Spaces. This idea is that instead of planning a single pycls job (e.g., testing a specific model configuration), one can study the behavior of an entire population of models. This allows for quite powerful and succinct experimental design, and elevates the study of individual model behavior to the study of the behavior of model populations. Please see SWEEP_INFO for details.

EMA as used in "Fast and Accurate Model Scaling" to improve accuracy. Note that EMA of model weights is nearly free computationally (if not computed every iter), hence EMA weights area always computed/stored. Saving/loading checkpoints has been updated, but the code is backward compatible with checkpoints that do not store the ema weights.

Details: -config.py: added EMA options -meters.py: generalized to allow for ema meter -net.py: added update_model_ema() to compute model ema -trainer.py: added updating/testing/logging of ema model -checkpoint.py: save/load_checkpoint() also save/load ema weights

See GETTING_STARTED.md for example usage.

Summary: -paper reference: https://arxiv.org/abs/2103.06877 -regnet.py: added regnet_cfg_to_anynet_cfg() -scaler.py: implements model scaler -scale_net.py: entry point for model scaler -GETTING_STARTED.md: added example usage for scaler

Hi @rajprateek , @ir413 , Thanks for your team's great work, it provides many insights to the community. I am sure that the model zoos and the current codebase could inspire future research a lot.

I am also a little bit curious about the future plans of your codebase. So I want to ask that do you have any plans to support the design space comparison in this repo? For example, to allow users to sample & train models from different design spaces and compare these design spaces as described in the Sec. 3.1, as shown in Fig. 5, 7, and 9 in the paper. I think this feature could help the community to reproduce the comparison process and further improve this codebase's impact.

Hi, I noticed that 100 models are sampled to get the results as shown in Figure 11. (sec 4).

However, as the flops in the figure span a wide range(0.2B~12.8B), I don't know whether

the total number of models in all the flops regime is 100, or

for each of the flops regime, you sampled 100 models?

RuntimeError: Cannot re-initialize CUDA in forked subprocess. To use CUDA with multiprocessing, you must use the 'spawn' start method

how to solve this problem?

an answer says: torch.multiprocessing.set_start_method('spawn') but where to add this line?

Does this framework support the integration of other data types? i.e if I want to input data other than images, can I easily extend the framework to do this?

device env: win10 py3.8 found in my env, when downloading yaml and pyth, raise "urllib.error.HTTPError: HTTP Error 404: Not Found",

it turned out that when concat url,

config_url = os.path.join(_URL_CONFIGS, _MODEL_ZOO_CONFIGS[name])

the sep added in the url in windows is '', which cause this error,

to solve in my env: config_url = _URL_CONFIGS + '/ + _MODEL_ZOO_CONFIGS[name] or as implemented in https://stackoverflow.com/questions/8223939/how-to-join-absolute-and-relative-urls

Hi, in regnet, the paper[1] mentioned it picks the best model of 25 random models (Section 5) as the final result. However, I cannot find the relative setting to train these 25 random models. Is that used the same setting to get EDF, showed in Figure.9 ?

[1] https://arxiv.org/pdf/2003.13678.pdf