Right now if you want to upgrade a package which has mypackage[extras] you need to update it as mypackage. I think we can provide better support for the extras to align better as there could be cases where the items in extras are not being bumped properly.

@ak-gupta thoughts?

Closes #47

This feature adds in TOML support to co-exist along side setup.cfg support:

• created mirrored functions between the 2 to keep the logic separate.
  • _convert_toml_array_to_string, parse_toml, and upgrade_pyproject_toml
• new dependency is added with tomlkit.
• added in some new unit tests

Description

Adding in a new hook specifically for the update/upgrade portion. this will enable plugins to inject custom code

Fixes #39

Type of change

• [ ] Bug fix (non-breaking change which fixes an issue)
• [x] New feature (non-breaking change which adds functionality)
• [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
• [x] This change requires a documentation update

Checklist:

• [x] I have performed a self-review of my own code
• [x] I have commented my code, particularly in hard-to-understand areas
• [x] I have made corresponding changes to the documentation
• [x] My changes generate no new warnings
• [x] I have added tests that prove my fix is effective or that my feature works
• [x] New and existing unit tests pass locally with my changes
• [x] New and existing integration tests pass locally with my changes
• [ ] Any dependent changes have been merged and published in downstream modules

Based on PEP 518 here: https://peps.python.org/pep-0518/#overview-of-file-formats-considered It seems like pyproject.toml is preferred over setup.cfg. We should include some support here for various types. So far:

• setup.cfg (existing)
• pyproject.toml (new)

@ak-gupta thoughts?

edgetest opened this PR to update the dask[dataframe] version yesterday

https://github.com/capitalone/rubicon-ml/pull/221

I had to manually apply those version updates to the environment files in my repo, which caused the whitesource scan to fail. so, I left the PR open overnight to come back to it today and finish. overnight, edgetest again noticed that dask[dataframe] needed to be updated so it created a branch with the same name as the one I had committed my manual changes to yesterday and forced pushed, overwriting the branch

edgetest should be able to recognize that it already opened the PR to update dask[dataframe] and not force push the same update

I also imagine that if a second library needed to be updated while the PR for a previous one is still open we'd see the same behavior since the branch always seems to be named edgetest-patch

So currently the packages are always upgraded via pip here.

Proposal is to make this a hook, something like:

@hookspec(firstresult=True)
def run_update(python_path: str, upgrade: List):
    """Update packages from upgrade list.

    Parameters
    ----------
    python_path : str
        The path to the python executable.
    upgrade : list
        The list of packages to upgrade
   
    Raises
    ------
    RuntimeError
        Error raised if the packages cannot be updated.
    """

This directly links back to a request in the conda plugin: https://github.com/capitalone/edgetest-conda/issues/30 Enabling this functionality will enable the plugin to provide a spec to upgrade via conda.

Release 2022.6.0

Feature:

• github report format output option (#36)

House keeping:

• edgetest bumps (#24, #29, #33)
• pre-commit bumps (#34)
• WhiteSource Configuration Migration (#31)

This feature is needed more of a result for providing the report output in the github format and tied to a request in edgetest-hub: https://github.com/capitalone/edgetest-hub/issues/28

it'd be nice if I only had to specify my library's dependencies in [edgetest.envs.core] instead of needing to remember to add every new test dependency. there's already a considerable amount of env declaration duplication required throughout the various package managing tools, so it'd be nice if there wasn't another layer of duplication here

I'd like to only need to specify the libraries I want to try updating in [edgetest.envs.core]. I'd like to inherit the rest of the dependencies from an environment file. for example:

[edgetest.envs.core]
python_version = 3.9
conda_install = 
	./environment.yml
extras = 
	all
upgrade = 
	click
	dask[dataframe]
	fsspec
	intake[dataframe]
	pandas
	pyarrow
	PyYAML
	s3fs
	prefect
	dash
	dash-bootstrap-components

I'm imagining a scenario where packages using edgetest with a lot of dependencies may need to release their package quite frequently to account for updated dependencies. for example, if I have pyarrow<=7.0.0,>=0.18.0 defined in my setup with the upper bound as required by edgetest and all of a sudden a major bug is found in pyarrow 7.0.0 that needs an immediate upgrade to 7.0.1, I now require an immediate update to and release of my package before users can use it with the fixed pyarrow

I think it could be useful to be able to specify pyarrow<=7.0,>=0.18.0 (explicitly leaving off the patch version) and having that mean that edgetest will only look for, test, and PR upgrades to the major/minor versions and my setup can automatically allow any update to the patch version to be useable. (I'm assuming that if I were to say pyarrow<=7.0,>=0.18.0 edgetest would currently look for pyarrow 7.0.1 next time it ran, not 7.1 - if that's incorrect then let me know)

I'm coming from a mindset of avoiding upper pins in general to avoid issues like this, but I absolutely see the value in having them (and especially in having them tested) so I think it'd be nice to be able to configure some kind of "compromise" between the two ideas

I'm also aware that different versioning schemes could cause headaches here if we were to try something super automatic, so that's something to consider

thoughts @fdosani @ak-gupta ?