Avoid PyCharm Overloading
=========================
This speeds up PyCharm's package index processes and avoids CPU & memory
overloading.
On my machine (4c/32GB) with 5 conda environments and 10 projects, this reduces
PyCharm's package scanning from 15 minutes to 45 seconds.
Why?
----
When PyCharm has multiple conda environments, it will sometimes launch concurrent
package scanning across those environments. To do so, it uses the `conda` cli,
and a tool called conda_packaging_tool.py, available from the JetBrains
intellij-community github repository (see below).
Unfortunately, these tools do not behave well when launched in parallel, as
they tend to overload CPU & memory in this case.
How?
----
To avoid stalling these processes, we modify their sources as follows:
1. modify the conda cli
$ conda activate base
$ pip install runfast
$ nano `which conda`
Modify the main section:
if __name__ == '__main__':
from conda.cli import main
from runfast import cached
cached(main)
2. modify the conda_packaging_tool.py
if __name__ == '__main__':
from runfast import cached
cached(main)
How does this work?
-------------------
Two simple steps:
1. When called in parallel, only one process is allowed to proceed.
2. `runfast.cached` caches the output of these tools (stdout, stderr) for 1 minute,
given the same command line parameters
That is, when PyCharm launches 5 package scanning (`conda list`) commands, only
one of them will run immediately. If some of the scans are for the same environment,
only one of them will actually run, while the others simply return the same output.
How to clear the cache or avoid caching at all?
$ export RUNFAST_NOCACHE=1
Testing
-------
a) conda packaging tool
# first time
$ time python /opt/pycharm-community-2021.2.1/plugins/python-ce/helpers/conda_packaging_tool.py
real 0m13.271s
user 0m9.631s
sys 0m1.936s
# second time
$ time python /opt/pycharm-community-2021.2.1/plugins/python-ce/helpers/conda_packaging_tool.py
real 0m3.159s
user 0m0.139s
sys 0m1.298s
b) conda cli
# first time
$ time conda list -p /path/to/env
real 0m7.985s
user 0m7.883s
sys 0m0.087s
# second time
real 0m0.152s
user 0m0.116s
sys 0m0.032s
References
----------
* https://github.com/JetBrains/intellij-community/blob/master/python/helpers/conda_packaging_tool.py
* https://github.com/conda/conda/blob/33a142c16530fcdada6c377486f1c1a385738a96/conda/core/index.py#L53
This speeds up PyCharm's package index processes and avoids CPU & memory overloading
Overview
Python decorator for `TODO`s
Python decorator for `TODO`s. Don't let your TODOs rot in your python projects anymore !
The next generation Canto RSS daemon
Canto Daemon This is the RSS backend for Canto clients. Canto-curses is the default client at: http://github.com/themoken/canto-curses Requirements De
Todos os exercícios do Curso de Python, do canal Curso em Vídeo, resolvidos em Python, Javascript, Java, C++, C# e mais...
Exercícios - CeV Oferecido por Linguagens utilizadas atualmente O que vai encontrar aqui? 👀 Esse repositório é dedicado a armazenar todos os enunciad
A simple and usefull python calculator.
simplepy-calculator Your simple and fresh calculator. Getting Started Install python3 from the oficial python website or via terminal. Clone this repo
Performance data for WASM SIMD instructions.
WASM SIMD Data This repository contains code and data which can be used to generate a JSON file containing information about the WASM SIMD proposal. F
A collection of convenient parsers for Advent of Code problems.
Advent of Code Parsers A collection of convenient Python parsers for Advent of Code problems. Installation pip install aocp Quickstart You can import
Lookup for interesting stuff in SMB shares
SMBSR - what is that? Well, SMBSR is a python script which given a CIDR/IP/IP_file/HOSTNAME(s) enumerates all the SMB services listening (445) among t
Radiosonde Telemetry Decoders
Radiosonde Telemetry Frame Decoders This repository is an attempt to collate the various sources of information on how to decode radiosonde telemetry
🏃 Python3 Solutions of All Problems in GKS 2022 (In Progress)
GoogleKickStart 2022 Python3 solutions of Google Kick Start 2022. Solution begins with * means it will get TLE in the largest data set. Total computat
Auto Join Zoom Meeting
Auto-Join-Zoom-Meeting Join a zoom meeting with out filling in meeting id's or passcodes, one button for it all! Setup See attached excel document. MA
Test reproducibility of leiden/umap on different systems
Demonstrate that UMAP and Leiden analysis is not reproducible between different cpu architectures.
Materials and information for my PyCascades 2021 Presentation
Materials and information for PyCascades 2021 Presentation: Sparking Creativity in LED Art with CircuitPython
Your one and only Discord Bot that helps you concentrate!
Your one and only Discord Bot thats helps you concentrate! Consider leaving a ⭐ if you found the project helpful. concy-bot A bot which constructively
Repo to store back end infrastructure for Message in a Bottle
Message in a Bottle Backend API RESTful API for Message in a Bottle frontend application consumption. About The Project • Tools Used • Local Set Up •
This app converts an pdf file into the audio file.
PDF-to-Audio This app takes an pdf as an input and convert it into audio, and the library text-to-speech starts speaking the preffered page given in t
Easily Generate Revolut Business Cards
RevBusinessCardGen Easily Generate Revolut Business Cards Prerequisites Before you begin, ensure you have met the following requirements: You have ins
PressurePlate is a multi-agent environment that requires agents to cooperate during the traversal of a gridworld.
PressurePlate is a multi-agent environment that requires agents to cooperate during the traversal of a gridworld. The grid is partitioned into several rooms, and each room contains a plate and a clos
Script to change official Kali repository to mirrors
Script to change official Kali repository to mirrors. This helps increase packages update and downloading for some user.
Deis v1, the CoreOS and Docker PaaS: Your PaaS. Your Rules.
This repository (deis/deis) is no longer developed or maintained. The Deis v1 PaaS based on CoreOS Container Linux and Fleet has been replaced by Deis
Library for managing git hooks
Autohooks Library for managing and writing git hooks in Python. Looking for automatic formatting or linting, e.g., with black and pylint, while creati