Trading environnement for RL agents, backtesting and training.

Related tags

Deep Learningalgorithmreinforcement-learningbitcointradingtrading-botreinforcement-learning-algorithmstrading-algorithmsbacktestingbitcoin-botreinforcement-learning-agenttrading-env
Overview

TradzQAI

Trading environnement for RL agents, backtesting and training.

Live session with coinbasepro-python is finaly arrived !

  • Available sessions:

    • Local
      • 1M bar datasets any size.
        • header should be ['Time', 'Open', 'High', 'Low', 'Close', 'Volume'] with ';' as separator
      • Tick datasets any size.
        • header should be ['Time', 'BID', 'ASK', 'VOL'] or ['Time', 'Price', 'Volume'] with ',' as separator
    • Live
      • Gdax API

  • Available agents:

    • DDPG
    • DQFD
    • DQN
    • DQNN
    • NAF
    • PPO
    • TRPO
    • VPG

  • Available contract type:

    • CFD
    • Classic

TradzQAI has been inspired by q-trader.

Indicators lib come from pyti

More datasets available here

Status

Alpha in development
    GUI rework on track

Getting Started

  • Dependencies :

    Installation :

    pip install -r requirements.txt

  • Running the project

    Usage:
  python run.py -h (Show usage)
  python run.py -b agent_name (to manually build config file, it build config files from agent, default PPO)
  python run.py -s live (for live session) 
  python run.py -m eval (for eval mode)
  python run.py -c config_dir/ # load config from directory, make sure you have agent, env and network json files in it
  python run.py (Run as default)

    When you run it for the first time, a config directory is created, you can change it to changes environnement settings and some agents settings. It save settings (env, agent, network) in a save directory, and create a new directory if make any changes.

    • Do you own decision function for maker side. For more info look at this function

      from core import Local_session as Session
from mymodule import myfunc
session = Session(mode=args.mode, config=args.config)
session.initApi(key=key, b64=b64, passphrase=passphrase, url=url,
        product_id=product_id)
session.getApi().setBestPriceFunc(myfunc)

    • Do your own runner.

    from core import Local_session as Session
session = Session() # Run with default values
session.loadSession() # loading environnement, worker and agent
session.start() # Start the session thread
    • Do your own worker.
    from core import Local_env
env = Local_env() # run with default values
for e in episode:
  state = env.reset()
  for s in step:
    action = agent.act(state)
    next_state, terminal, reward = env.execute(action)
    agent.observe(reward, terminal)
    if terminal or env.stop:
      break
  if env.stop or e == episode - 1:
    env.logger._running = False #Close the logger thread
    break
    • How to use networks.

      • You have to define your input to fit with columns name of your datasets, it will automaticaly it grab input from network and compare it with your dataset columns in getState function, it allow you to do complex network like this :

        [
    [
        {"names": ["Price"], "type": "input"},
        {"activation": "relu", "size": 8, "type": "dense"},
        {"activation": "relu", "size": 8, "type": "dense"},
        {"name": "pricenet", "type": "output"}
    ],
    [
        {"names": ["Volume"], "type": "input"},
        {"activation": "relu", "size": 8, "type": "dense"},
        {"activation": "relu", "size": 8, "type": "dense"},
        {"name": "volnet", "type": "output"}
    ],
    [
        {"names": ["pricenet", "volnet"], "type": "input"},
        {"activation": "relu", "size": 64, "type": "dense"},
        {"activation": "relu", "size": 32, "type": "dense"},
        {"activation": "relu", "size": 8, "type": "dense"},
        {"name": "prediction", "type": "output"}
    ]
]
        • Simple network are handled as well without defining any input:
          [
      {"activation": "relu", "size": 64, "type": "dense"},
      {"activation": "relu", "size": 64, "type": "dense"}
  ]
    • Also TradzQAI support pre trained keras model:
      • You can build settings for your model by using py run.py -b DEEP. Your model have to be placed in the same directory as the one you use to launch it and have to be called deep_model.h5.

Relevant project

This project isn't perfect so keep this in mind.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Comments
  • We cant afford any contract. Please check wallet settings.

    We cant afford any contract. Please check wallet settings.

    Hi, i encountered another issue using the new code. I tried to increase the capital from 200 to 20000 (for example) and it still happened the same way. Any idea? Thanks.

    /TradzQAI-master$ python3.6 run.py -m train DAT_ASCII_GRXEUR_M1_201711.csv : 0%| | 0/1 [00:00<?, ?it/sException in thread Thread-4: | 0/1 [00:00<?, ?it/s] Traceback (most recent call last): | 0/18354 [00:00<?, ?it/s] File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner self.run() File "/usr/lib/python3.6/threading.py", line 864, in run self._target(*self._args, **self._kwargs) File "/home/lamhk/TradzQAI-master/core/worker/local_worker.py", line 45, in run self.step() File "/home/lamhk/TradzQAI-master/core/worker/local_worker.py", line 71, in step state, terminal, reward = self.env.execute(action) File "/home/lamhk/TradzQAI-master/core/environnement/local_env.py", line 184, in execute self.wallet.manage_exposure(self.contract_settings) File "/home/lamhk/TradzQAI-master/core/environnement/base/wallet.py", line 197, in manage_exposure raise ValueError('current_max_pos : {:.3f} We cant afford any contract. Please check wallet settings.'.format(self.risk_managment['current_max_pos'])) ValueError: current_max_pos : 0.000 We cant afford any contract. Please check wallet settings.

    DAT_ASCII_GRXEUR_M1_201711.csv : 100%|████████████| 1/1 [00:00<00:00, 5.25it/s]

    Regards, lamhk

    question 
    opened by lamhk 9
  • How to change the agent type?

    How to change the agent type?

    Hi, I tried to build "-b BUILD" using DQN instead of PPO inside run.py line-10 / agent = "PPO". However, when I try to run the run.py, it still create directory for PPO_n. Where I should change it to switch the agent effectively. Thanks.

    question 
    opened by lamhk 6
  • command 'wc -1 data/BTC_EUR_2018_07/BTC_EUR_2018_07_01.csv' returned a non-zero exit status 1

    command 'wc -1 data/BTC_EUR_2018_07/BTC_EUR_2018_07_01.csv' returned a non-zero exit status 1

    Hi, i tried to run the code without any argument and i had this error : Command 'wc -1 data/BTC_EUR_2018_07/BTC_EUR_2018_07_01.csv' returned a non-zero exit status 1

    Do you know how to fix it?

    question 
    opened by greg2paris 5
  • We cant afford any contract.

    We cant afford any contract.

    I get this error even though i set the wallet to a high amount. Can anybody help?

    File "C:\Users\Miner_cunt\projects\TradzQAI\TradzQAI\core\environnement\base\wallet.py", line 199, in manage_exposure raise ValueError('current_max_pos : {:.3f} We cant afford any contract. Please check wallet settings.'.format(self.risk_managment['current_max_pos'])) ValueError: current_max_pos : 0.000 We cant afford any contract. Please check wallet settings.

    Thanks

    duplicate question 
    opened by dropcunt 3
  • Exception when running live version

    Exception when running live version

    I created a config for the PPO agent and trained it on the provided data without problems.

    When I run 'python run.py -s live' I get the exception

      File "run.py", line 110, in <module>
    session.loadSession()
  File "/Users/Oscar/Documents/Cryptotrading/Tradz/TradzQAI/core/session/live.py", line 61, in loadSession
    self.initAgent()
  File "/Users/Oscar/Documents/Cryptotrading/Tradz/TradzQAI/core/session/live.py", line 74, in initAgent
    self.agent = self.agent(env=self.env, device=self.device)._get()
  File "/Users/Oscar/Documents/Cryptotrading/Tradz/TradzQAI/agents/PPO.py", line 6, in __init__
    Agent.__init__(self, env=env, device=device)
  File "/Users/Oscar/Documents/Cryptotrading/Tradz/TradzQAI/agents/agent.py", line 24, in __init__
    states=self.env.states,
AttributeError: 'Live_env' object has no attribute 'states'

    I made sure my api-key, pass and secret are correct. I am connecting to https://api.pro.coinbase.com as endpoint.

    Anything I missed?

    bug 
    opened by oscarberonius 3
  • Training on GPU

    Training on GPU

    when I set the device to my GPU in run.py i get the following error:

    InvalidArgumentError (see above for traceback): Cannot assign a device for operation 'ppo/initialize/action-action/Initializer/zeros': Could not satisfy explicit device specification '' because the node was colocated with a group of nodes that required incompatible device '/device:GPU:0' Colocation Debug Info: Colocation group had the following types and devices: Assign: CPU Const: CPU Fill: GPU CPU Switch: GPU CPU VariableV2: CPU Identity: GPU CPU RefSwitch: GPU CPU StridedSliceAssign: GPU CPU IsVariableInitialized: CPU

    Does anybody has the same issue? Do you know were you can set something like allow_soft_placement=True? Thanks for your help!

    question 
    opened by dropcunt 2
  • Choosing different agent than PPO produces faulty behavior.

    Choosing different agent than PPO produces faulty behavior.

    I can run the default agent without issues, but had some trouble when running TRPO and DQN.

    I tried removing both the save and config directories and rebuilding with trpo using 'python run.py -b TRPO'. The config directory seemed to be correctly created. The containing agent.json file had the correct property "type": "trpo_agent".

    When I tried running the agent (python run.py) a save directory containing a PPO_0 directory was created. I tried changing the default agent in run.py to be TRPO as well but had the same result. The model could not evaluate and produced the following error.

    Traceback (most recent call last):
  File "run.py", line 110, in <module>
    session.loadSession()
  File "/Users/Oscar/Documents/Cryptotrading/Tradz/TradzQAI/core/session/local.py", line 46, in loadSession
    self.initAgent()
  File "/Users/Oscar/Documents/Cryptotrading/Tradz/TradzQAI/core/session/local.py", line 57, in initAgent
    self.agent = self.agent(env=self.env, device=self.device)._get()
  File "/Users/Oscar/Documents/Cryptotrading/Tradz/TradzQAI/agents/PPO.py", line 6, in __init__
    Agent.__init__(self, env=env, device=device)
  File "/Users/Oscar/Documents/Cryptotrading/Tradz/TradzQAI/agents/agent.py", line 27, in __init__
    device=device
  File "/Users/Oscar/Documents/Cryptotrading/Tradz/lib/python3.6/site-packages/tensorforce/agents/agent.py", line 283, in from_spec
    kwargs=kwargs
  File "/Users/Oscar/Documents/Cryptotrading/Tradz/lib/python3.6/site-packages/tensorforce/util.py", line 192, in get_object
    return obj(*args, **kwargs)
  File "/Users/Oscar/Documents/Cryptotrading/Tradz/lib/python3.6/site-packages/tensorforce/agents/trpo_agent.py", line 161, in __init__
    entropy_regularization=entropy_regularization
  File "/Users/Oscar/Documents/Cryptotrading/Tradz/lib/python3.6/site-packages/tensorforce/agents/learning_agent.py", line 141, in __init__
    batching_capacity=batching_capacity
  File "/Users/Oscar/Documents/Cryptotrading/Tradz/lib/python3.6/site-packages/tensorforce/agents/agent.py", line 80, in __init__
    self.model = self.initialize_model()
  File "/Users/Oscar/Documents/Cryptotrading/Tradz/lib/python3.6/site-packages/tensorforce/agents/trpo_agent.py", line 189, in initialize_model
    likelihood_ratio_clipping=self.likelihood_ratio_clipping
  File "/Users/Oscar/Documents/Cryptotrading/Tradz/lib/python3.6/site-packages/tensorforce/models/pg_prob_ratio_model.py", line 88, in __init__
    gae_lambda=gae_lambda
  File "/Users/Oscar/Documents/Cryptotrading/Tradz/lib/python3.6/site-packages/tensorforce/models/pg_model.py", line 98, in __init__
    requires_deterministic=False
  File "/Users/Oscar/Documents/Cryptotrading/Tradz/lib/python3.6/site-packages/tensorforce/models/distribution_model.py", line 90, in __init__
    discount=discount
  File "/Users/Oscar/Documents/Cryptotrading/Tradz/lib/python3.6/site-packages/tensorforce/models/memory_model.py", line 114, in __init__
    reward_preprocessing=reward_preprocessing
  File "/Users/Oscar/Documents/Cryptotrading/Tradz/lib/python3.6/site-packages/tensorforce/models/model.py", line 217, in __init__
    self.setup()
  File "/Users/Oscar/Documents/Cryptotrading/Tradz/lib/python3.6/site-packages/tensorforce/models/model.py", line 290, in setup
    independent=independent
  File "/Users/Oscar/Documents/Cryptotrading/Tradz/lib/python3.6/site-packages/tensorforce/models/memory_model.py", line 605, in create_operations
    independent=independent
  File "/Users/Oscar/Documents/Cryptotrading/Tradz/lib/python3.6/site-packages/tensorforce/models/model.py", line 1195, in create_operations
    self.create_observe_operations(reward=reward, terminal=terminal)
  File "/Users/Oscar/Documents/Cryptotrading/Tradz/lib/python3.6/site-packages/tensorforce/models/model.py", line 1129, in create_observe_operations
    reward=reward
  File "/Users/Oscar/Documents/Cryptotrading/Tradz/lib/python3.6/site-packages/tensorflow/python/ops/template.py", line 368, in __call__
    return self._call_func(args, kwargs)
  File "/Users/Oscar/Documents/Cryptotrading/Tradz/lib/python3.6/site-packages/tensorflow/python/ops/template.py", line 311, in _call_func
    result = self._func(*args, **kwargs)
  File "/Users/Oscar/Documents/Cryptotrading/Tradz/lib/python3.6/site-packages/tensorforce/models/memory_model.py", line 566, in tf_observe_timestep
    return tf.cond(pred=optimize, true_fn=true_fn, false_fn=tf.no_op)
  File "/Users/Oscar/Documents/Cryptotrading/Tradz/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py", line 488, in new_func
    return func(*args, **kwargs)
  File "/Users/Oscar/Documents/Cryptotrading/Tradz/lib/python3.6/site-packages/tensorflow/python/ops/control_flow_ops.py", line 2076, in cond
    orig_res_t, res_t = context_t.BuildCondBranch(true_fn)
  File "/Users/Oscar/Documents/Cryptotrading/Tradz/lib/python3.6/site-packages/tensorflow/python/ops/control_flow_ops.py", line 1920, in BuildCondBranch
    original_result = fn()
  File "/Users/Oscar/Documents/Cryptotrading/Tradz/lib/python3.6/site-packages/tensorforce/models/memory_model.py", line 562, in true_fn
    optimize = self.fn_optimization(**batch)
  File "/Users/Oscar/Documents/Cryptotrading/Tradz/lib/python3.6/site-packages/tensorflow/python/ops/template.py", line 368, in __call__
    return self._call_func(args, kwargs)
  File "/Users/Oscar/Documents/Cryptotrading/Tradz/lib/python3.6/site-packages/tensorflow/python/ops/template.py", line 311, in _call_func
    result = self._func(*args, **kwargs)
  File "/Users/Oscar/Documents/Cryptotrading/Tradz/lib/python3.6/site-packages/tensorforce/models/pg_model.py", line 305, in tf_optimization
    next_internals=next_internals
  File "/Users/Oscar/Documents/Cryptotrading/Tradz/lib/python3.6/site-packages/tensorforce/models/memory_model.py", line 491, in tf_optimization
    return self.optimizer.minimize(**arguments)
  File "/Users/Oscar/Documents/Cryptotrading/Tradz/lib/python3.6/site-packages/tensorforce/core/optimizers/optimizer.py", line 139, in minimize
    deltas = self.step(time=time, variables=variables, **kwargs)
  File "/Users/Oscar/Documents/Cryptotrading/Tradz/lib/python3.6/site-packages/tensorflow/python/ops/template.py", line 368, in __call__
    return self._call_func(args, kwargs)
  File "/Users/Oscar/Documents/Cryptotrading/Tradz/lib/python3.6/site-packages/tensorflow/python/ops/template.py", line 311, in _call_func
    result = self._func(*args, **kwargs)
  File "/Users/Oscar/Documents/Cryptotrading/Tradz/lib/python3.6/site-packages/tensorforce/core/optimizers/optimized_step.py", line 102, in tf_step
    **kwargs
  File "/Users/Oscar/Documents/Cryptotrading/Tradz/lib/python3.6/site-packages/tensorflow/python/ops/template.py", line 368, in __call__
    return self._call_func(args, kwargs)
  File "/Users/Oscar/Documents/Cryptotrading/Tradz/lib/python3.6/site-packages/tensorflow/python/ops/template.py", line 311, in _call_func
    result = self._func(*args, **kwargs)
  File "/Users/Oscar/Documents/Cryptotrading/Tradz/lib/python3.6/site-packages/tensorforce/core/optimizers/natural_gradient.py", line 123, in tf_step
    deltas = self.solver.solve(fn_x=fisher_matrix_product, x_init=None, b=[-grad for grad in loss_gradients])
  File "/Users/Oscar/Documents/Cryptotrading/Tradz/lib/python3.6/site-packages/tensorflow/python/ops/template.py", line 368, in __call__
    return self._call_func(args, kwargs)
  File "/Users/Oscar/Documents/Cryptotrading/Tradz/lib/python3.6/site-packages/tensorflow/python/ops/template.py", line 311, in _call_func
    result = self._func(*args, **kwargs)
  File "/Users/Oscar/Documents/Cryptotrading/Tradz/lib/python3.6/site-packages/tensorforce/core/optimizers/solvers/conjugate_gradient.py", line 80, in tf_solve
    return super(ConjugateGradient, self).tf_solve(fn_x, x_init, b)
  File "/Users/Oscar/Documents/Cryptotrading/Tradz/lib/python3.6/site-packages/tensorforce/core/optimizers/solvers/iterative.py", line 78, in tf_solve
    args = tf.while_loop(cond=self.next_step, body=self.step, loop_vars=args)
  File "/Users/Oscar/Documents/Cryptotrading/Tradz/lib/python3.6/site-packages/tensorflow/python/ops/control_flow_ops.py", line 3274, in while_loop
    return_same_structure)
  File "/Users/Oscar/Documents/Cryptotrading/Tradz/lib/python3.6/site-packages/tensorflow/python/ops/control_flow_ops.py", line 2994, in BuildLoop
    pred, body, original_loop_vars, loop_vars, shape_invariants)
  File "/Users/Oscar/Documents/Cryptotrading/Tradz/lib/python3.6/site-packages/tensorflow/python/ops/control_flow_ops.py", line 2929, in _BuildLoop
    body_result = body(*packed_vars_for_body)
  File "/Users/Oscar/Documents/Cryptotrading/Tradz/lib/python3.6/site-packages/tensorflow/python/ops/template.py", line 368, in __call__
    return self._call_func(args, kwargs)
  File "/Users/Oscar/Documents/Cryptotrading/Tradz/lib/python3.6/site-packages/tensorflow/python/ops/template.py", line 311, in _call_func
    result = self._func(*args, **kwargs)
  File "/Users/Oscar/Documents/Cryptotrading/Tradz/lib/python3.6/site-packages/tensorforce/core/optimizers/solvers/conjugate_gradient.py", line 128, in tf_step
    A_conjugate = self.fn_x(conjugate)
  File "/Users/Oscar/Documents/Cryptotrading/Tradz/lib/python3.6/site-packages/tensorforce/core/optimizers/natural_gradient.py", line 112, in fisher_matrix_product
    return tf.gradients(ys=delta_kldiv_gradients, xs=variables)
  File "/Users/Oscar/Documents/Cryptotrading/Tradz/lib/python3.6/site-packages/tensorflow/python/ops/gradients_impl.py", line 596, in gradients
    gate_gradients, aggregation_method, stop_gradients)
  File "/Users/Oscar/Documents/Cryptotrading/Tradz/lib/python3.6/site-packages/tensorflow/python/ops/gradients_impl.py", line 749, in _GradientsHelper
    _RaiseNoGradWrtInitialLoopValError(op, from_ops, xs)
  File "/Users/Oscar/Documents/Cryptotrading/Tradz/lib/python3.6/site-packages/tensorflow/python/ops/gradients_impl.py", line 441, in _RaiseNoGradWrtInitialLoopValError
    % target_op.name)
ValueError: Cannot compute gradient inside while loop with respect to op 'trpo/actions-and-internals/layered-network/apply/dense0/apply/linear/apply/W'. We do not support taking the gradient wrt or through the initial value of a loop variable. Gradients can be computed through loop invariants or wrt the input parameters to the loop body.

    Switching to dqn produced the same problems with the file structure, although the model could evaluate.

    bug 
    opened by oscarberonius 2
  • Using other exchanges

    Using other exchanges

    Is it possible to change the exchanges for the life exchanges? Where to start? If you give me a starting point i could try and do it myself and make a pull request afterwards

    opened by AlconDivino 1
  • complex network

    complex network "names" issue

    When I try to change the network to a complex network I get this error:

    File "/Users/dropper/projects/crypto_bot/tforce_btc_trader/tensorforce/tensorforce/util.py", line 159, in get_object return obj(*args, **kwargs) TypeError: __init__() got an unexpected keyword argument 'names'

    Do I have to change something in the dataloader.py?

    Thank you

    opened by LucaKern 1
  • Performing training automatically evaluates the model.

    Performing training automatically evaluates the model.

    So I have a little trouble understanding how the training/evaluation works.

    If I run 'python run.py -m train', a log file "train_date.txt" is created with the resulting award in /save/agent_#. Likewise a log file "eval_date.txt" is created when I run the evaluation command.

    I can run the eval command and get results even without running the training command first. The results of the evaluation seem to be the same regardless of how many times the evaluation is run, but the results of the training differ every time.

    Can you explain what the difference is and what data is being used to evaluate the model? To me it seems like both commands train a model on all the provided data and then evaluate on parts of that same data, but I can't figure out what really is happening.

    question 
    opened by oscarberonius 1
  • Cannot modify training data

    Cannot modify training data

    Hi.

    I can successfully run the PPO agent using the provided file structure of the data directory. For this I am using "data_directory": "data/" as argument in environnement.json.

    However, I tried pulling out the directory BTC_EUR_2018_07 and placed it in a new directory eval_data at the same level as the data dir. Now when I do 'python run.py -m train' I get the following error.

      File "run.py", line 110, in <module>
    session.loadSession()
  File "/Users/Oscar/Documents/Cryptotrading/Tradz/TradzQAI/core/session/local.py", line 46, in loadSession
    self.initAgent()
  File "/Users/Oscar/Documents/Cryptotrading/Tradz/TradzQAI/core/session/local.py", line 57, in initAgent
    self.agent = self.agent(env=self.env, device=self.device)._get()
  File "/Users/Oscar/Documents/Cryptotrading/Tradz/TradzQAI/agents/PPO.py", line 6, in __init__
    Agent.__init__(self, env=env, device=device)
  File "/Users/Oscar/Documents/Cryptotrading/Tradz/TradzQAI/agents/agent.py", line 27, in __init__
    device=device
  File "/Users/Oscar/Documents/Cryptotrading/Tradz/lib/python3.6/site-packages/tensorforce/agents/agent.py", line 283, in from_spec
    kwargs=kwargs
  File "/Users/Oscar/Documents/Cryptotrading/Tradz/lib/python3.6/site-packages/tensorforce/util.py", line 192, in get_object
    return obj(*args, **kwargs)
  File "/Users/Oscar/Documents/Cryptotrading/Tradz/lib/python3.6/site-packages/tensorforce/agents/ppo_agent.py", line 155, in __init__
    entropy_regularization=entropy_regularization
  File "/Users/Oscar/Documents/Cryptotrading/Tradz/lib/python3.6/site-packages/tensorforce/agents/learning_agent.py", line 141, in __init__
    batching_capacity=batching_capacity
  File "/Users/Oscar/Documents/Cryptotrading/Tradz/lib/python3.6/site-packages/tensorforce/agents/agent.py", line 80, in __init__
    self.model = self.initialize_model()
  File "/Users/Oscar/Documents/Cryptotrading/Tradz/lib/python3.6/site-packages/tensorforce/agents/ppo_agent.py", line 183, in initialize_model
    likelihood_ratio_clipping=self.likelihood_ratio_clipping
  File "/Users/Oscar/Documents/Cryptotrading/Tradz/lib/python3.6/site-packages/tensorforce/models/pg_prob_ratio_model.py", line 88, in __init__
    gae_lambda=gae_lambda
  File "/Users/Oscar/Documents/Cryptotrading/Tradz/lib/python3.6/site-packages/tensorforce/models/pg_model.py", line 98, in __init__
    requires_deterministic=False
  File "/Users/Oscar/Documents/Cryptotrading/Tradz/lib/python3.6/site-packages/tensorforce/models/distribution_model.py", line 90, in __init__
    discount=discount
  File "/Users/Oscar/Documents/Cryptotrading/Tradz/lib/python3.6/site-packages/tensorforce/models/memory_model.py", line 114, in __init__
    reward_preprocessing=reward_preprocessing
  File "/Users/Oscar/Documents/Cryptotrading/Tradz/lib/python3.6/site-packages/tensorforce/models/model.py", line 217, in __init__
    self.setup()
  File "/Users/Oscar/Documents/Cryptotrading/Tradz/lib/python3.6/site-packages/tensorforce/models/model.py", line 242, in setup
    self.setup_placeholders()
  File "/Users/Oscar/Documents/Cryptotrading/Tradz/lib/python3.6/site-packages/tensorforce/models/model.py", line 448, in setup_placeholders
    kwargs=dict(shape=self.states_spec[name]['shape']))
UnboundLocalError: local variable 'name' referenced before assignment

    Any idea?

    question 
    opened by oscarberonius 1
  • Bump numpy from 1.14.5 to 1.22.0

    Bump numpy from 1.14.5 to 1.22.0

    Bumps numpy from 1.14.5 to 1.22.0.

    Release notes

    Sourced from numpy's releases.

    v1.22.0

    NumPy 1.22.0 Release Notes

    NumPy 1.22.0 is a big release featuring the work of 153 contributors spread over 609 pull requests. There have been many improvements, highlights are:

    • Annotations of the main namespace are essentially complete. Upstream is a moving target, so there will likely be further improvements, but the major work is done. This is probably the most user visible enhancement in this release.
    • A preliminary version of the proposed Array-API is provided. This is a step in creating a standard collection of functions that can be used across application such as CuPy and JAX.
    • NumPy now has a DLPack backend. DLPack provides a common interchange format for array (tensor) data.
    • New methods for quantile, percentile, and related functions. The new methods provide a complete set of the methods commonly found in the literature.
    • A new configurable allocator for use by downstream projects.

    These are in addition to the ongoing work to provide SIMD support for commonly used functions, improvements to F2PY, and better documentation.

    The Python versions supported in this release are 3.8-3.10, Python 3.7 has been dropped. Note that 32 bit wheels are only provided for Python 3.8 and 3.9 on Windows, all other wheels are 64 bits on account of Ubuntu, Fedora, and other Linux distributions dropping 32 bit support. All 64 bit wheels are also linked with 64 bit integer OpenBLAS, which should fix the occasional problems encountered by folks using truly huge arrays.

    Expired deprecations

    Deprecated numeric style dtype strings have been removed

    Using the strings "Bytes0", "Datetime64", "Str0", "Uint32", and "Uint64" as a dtype will now raise a TypeError.

    (gh-19539)

    Expired deprecations for loads, ndfromtxt, and mafromtxt in npyio

    numpy.loads was deprecated in v1.15, with the recommendation that users use pickle.loads instead. ndfromtxt and mafromtxt were both deprecated in v1.17 - users should use numpy.genfromtxt instead with the appropriate value for the usemask parameter.

    (gh-19615)

    ... (truncated)

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 0
  • base_env.py check_time_before_closing() idx not defined

    base_env.py check_time_before_closing() idx not defined

    I loaded a Dataset with LTCUSDT data from binance (Format: Timestamp;Open;Low,High;Close;Volume) I now got the following error:

    agent not loaded
binanceData_LTCUSDT_1m.csv :   0%|                        | 0/1 [00:00<?, ?it/sException in thread Thread-3:                              | 0/1 [00:00<?, ?it/s]
Traceback (most recent call last):█████▉| 200994/200998 [54:19<00:00, 27.90it/s]
  File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.6/threading.py", line 864, in run
    self._target(*self._args, **self._kwargs)
  File "/home/alcon/Desktop/TradzQAI/TradzQAI/core/worker/local_worker.py", line 23, in run
    self.step()
  File "/home/alcon/Desktop/TradzQAI/TradzQAI/core/worker/local_worker.py", line 46, in step
    state, terminal, reward = self.env.execute(action)
  File "/home/alcon/Desktop/TradzQAI/TradzQAI/core/environnement/local_env.py", line 178, in execute
    self.check_time_before_closing()
  File "/home/alcon/Desktop/TradzQAI/TradzQAI/core/environnement/base/base_env.py", line 148, in check_time_before_closing
    self.step_left = idx - self.current_step['step'] + 1
UnboundLocalError: local variable 'idx' referenced before assignment

binanceData_LTCUSDT_1m.csv : 100%|██████████████| 1/1 [54:19<00:00, 3259.48s/it]
^[[A Processing : 100%|████████████████▉| 200996/200998 [54:19<00:00, 61.67it/s]

    Looking at the function i see that the runnervar idx is really not defined in the last line. I then defined the var at the begining as idx=0

        def check_time_before_closing(self):
        if self.current_step['step'] == self.len_data - 1:
            return
        for idx in range(self.current_step['step'] + 1 , self.len_data - 1):
            if self._date[idx - 1][7] != self._date[idx][7]:
                break
        self.step_left = idx - self.current_step['step'] + 1

    The process ran smothly with my own data and the dax data in the repo but when running python3 run.py -m eval and checking the save logs everything is 0 like this:

    2019:04:09 17:47:40 000000 Starting episode : 1
2019:04:09 17:47:56 000001 ######################################################
2019:04:09 17:47:56 000002 Total reward : 0
2019:04:09 17:47:56 000003 Average reward : 0.000
2019:04:09 17:47:56 000004 Avg reward 500 : 0.0
2019:04:09 17:47:56 000005 Avg reward 100 : 0.0
2019:04:09 17:47:56 000006 Avg reward 50 : 0.0
2019:04:09 17:47:56 000007 Avg reward 10 : 0.0
2019:04:09 17:47:56 000008 Total profit : 0
2019:04:09 17:47:56 000009 Total trade : 0
2019:04:09 17:47:56 000010 Sharp ratio : 0.000
2019:04:09 17:47:56 000011 Mean return : 0.000
2019:04:09 17:47:56 000012 Max Drawdown : 0.000
2019:04:09 17:47:56 000013 Max return : 0.000
2019:04:09 17:47:56 000014 Percent return : 0.000
2019:04:09 17:47:56 000015 Trade W/L : 0.000
2019:04:09 17:47:56 000016 Step : 18353
2019:04:09 17:47:56 000017 ######################################################

    Any idea what i can do?

    help wanted 
    opened by AlconDivino 2
  • Error

    Error

    python run.py

    Traceback (most recent call last): File "run.py", line 30, in from TradzQAI import Local_session as Session File "/home/desktop/Downloads/TradzQAI-master/TradzQAI/init.py", line 1, in from .core import Local_session File "/home/desktop/Downloads/TradzQAI-master/TradzQAI/core/init.py", line 1, in from .environnement import Local_env File "/home/desktop/Downloads/TradzQAI-master/TradzQAI/core/environnement/init.py", line 1, in from .local_env import Local_env File "/home/desktop/Downloads/TradzQAI-master/TradzQAI/core/environnement/local_env.py", line 1, in from TradzQAI.tools import act_processing File "/home/desktop/Downloads/TradzQAI-master/TradzQAI/tools/init.py", line 5, in from .utils import * File "/home/desktop/Downloads/TradzQAI-master/TradzQAI/tools/utils.py", line 22 SyntaxError: Non-ASCII character '\xe2' in file /home/desktop/Downloads/TradzQAI-master/TradzQAI/tools/utils.py on line 22, but no encoding declared; see http://python.org/dev/peps/pep-0263/ for details

    question 
    opened by ghost 3
  • live session error

    live session error

    I have this problem when run with the live session:

      File "run.py", line 42, in <module>
    product_id=product_id)
  File "C:\Users\dannox\Google Drive\stockOptionsTrading\TradzQAI\TradzQAI\core\session\live.py", line 119, in initApi
    self.initEnv()
  File "C:\Users\dannox\Google Drive\stockOptionsTrading\TradzQAI\TradzQAI\core\session\live.py", line 124, in initEnv
    logger=self.logger, saver=self.saver, dataloader=self.dl, api=self.api)
  File "C:\Users\dannox\Google Drive\stockOptionsTrading\TradzQAI\TradzQAI\core\environnement\live_env.py", line 99, in __init__
    self.dl.loadHistorical()
  File "C:\Users\dannox\Google Drive\stockOptionsTrading\TradzQAI\TradzQAI\core\environnement\base\dataloader.py", line 245, in loadHistorical
    self.ticks = tick.join(self.indics.build_indicators(tick['Price']))
  File "C:\Users\dannox\Google Drive\stockOptionsTrading\TradzQAI\TradzQAI\tools\indicators\build_indicators.py", line 76, in build_indicators
    indicators[idx[1]] = idx[0](data, idx[2][0], idx[2][1])
  File "C:\Users\dannox\Google Drive\stockOptionsTrading\TradzQAI\TradzQAI\tools\indicators\moving_average_convergence_divergence.py", line 12, in moving_average_convergence_divergence
    check_for_period_error(data, long_period)
  File "C:\Users\dannox\Google Drive\stockOptionsTrading\TradzQAI\TradzQAI\tools\indicators\catch_errors.py", line 14, in check_for_period_error
    raise Exception("Error: data_len < period")
Exception: Error: data_len < period
Error: messages missing (4948236479 - 4948236523). Re-initializing  book at sequence.
    bug 
    opened by dgiunchi 1
  • Possible bug in dataloader.py

    Possible bug in dataloader.py

    Hi,

    I ran into a problem when I tried running the new core. I successfully built the PPO agent, but when I executed an eval session I received the following exception.

    #PRINT# lines.split(sep) = ['20180701000108', '5467.87', '0.062', '56.86932808248668', '1.828228906843833', '0.12368473729161092', '5468.798899294792', '5457.467818250765', '5444.815160592744']
Traceback (most recent call last):
  File "run.py", line 42, in <module>
    session.loadSession()
  File "/Users/Oscar/Documents/Cryptotrading/Tradz/TradzQAI/TradzQAI/core/session/local.py", line 71, in loadSession
    self.initEnv()
  File "/Users/Oscar/Documents/Cryptotrading/Tradz/TradzQAI/TradzQAI/core/session/local.py", line 102, in initEnv
    logger=self.logger, saver=self.saver, dataloader=self.dl)
  File "/Users/Oscar/Documents/Cryptotrading/Tradz/TradzQAI/TradzQAI/core/environnement/local_env.py", line 84, in __init__
    self.dl.loadFile()
  File "/Users/Oscar/Documents/Cryptotrading/Tradz/TradzQAI/TradzQAI/core/environnement/base/dataloader.py", line 177, in loadFile
    tmp_data, tmp_raw, tmp_time = self.getStockDataVec(self.files[self.files_index])
  File "/Users/Oscar/Documents/Cryptotrading/Tradz/TradzQAI/TradzQAI/core/environnement/base/dataloader.py", line 150, in getStockDataVec
    df.columns = names
UnboundLocalError: local variable 'names' referenced before assignment

    I'm not sure if the code is supposed to be able to handle a row length of 10 or not, but there shouldn't be any problem with the data as I tested it before and made sure the path is correct.

    enhancement 
    opened by oscarberonius 5
Releases(v0.2)

  • v0.2(Dec 19, 2017)

    • Moved GUI to PyQT5

    • 1 month data saved

      • DAX30 2017 10

    • multiple agents implemented

      • DQN
      • DDQN
      • DRQN
      • DDRQN

    • Added some managment fonction

      • Wallet managment
      • Risk managment
      • Close position on better profit
    Source code(tar.gz)
    Source code(zip)

  • v0.1(Dec 19, 2017)

    • Tkinter GUI

      • Start window with some settings
      • Overview window with some view on env, and model behaviour

    • 1 month data without saved model

      • DAX30

    • Just one agent (DQN)

    • Takes long and short positions

    Source code(tar.gz)
    Source code(zip)
Owner
Tony Denion
Machine learning engineer.
Tony Denion
GitHub Repository
mmfewshot is an open source few shot learning toolbox based on PyTorch

OpenMMLab FewShot Learning Toolbox and Benchmark

OpenMMLab 514 Dec 28, 2022
Semi-supervised Representation Learning for Remote Sensing Image Classification Based on Generative Adversarial Networks

SSRL-for-image-classification Semi-supervised Representation Learning for Remote Sensing Image Classification Based on Generative Adversarial Networks

Feng 2 Nov 19, 2021
Top #1 Submission code for the first https://alphamev.ai MEV competition with best AUC (0.9893) and MSE (0.0982).

alphamev-winning-submission Top #1 Submission code for the first alphamev MEV competition with best AUC (0.9893) and MSE (0.0982). The code won't run

70 Oct 29, 2022
A collection of models for image<->text generation in ACM MM 2021.

Bi-directional Image and Text Generation UMT-BITG (image & text generator) Unifying Multimodal Transformer for Bi-directional Image and Text Generatio

Multimedia Research 63 Oct 30, 2022
Flower - A Friendly Federated Learning Framework

Flower - A Friendly Federated Learning Framework Flower (flwr) is a framework for building federated learning systems. The design of Flower is based o

Adap 1.8k Jan 01, 2023
YOLOv5🚀 reproduction by Guo Quanhao using PaddlePaddle

YOLOv5-Paddle YOLOv5 🚀 reproduction by Guo Quanhao using PaddlePaddle 支持AutoBatch 支持AutoAnchor 支持GPU Memory 快速开始 使用AIStudio高性能环境快速构建YOLOv5训练(PaddlePa

QuanHao Guo 20 Nov 14, 2022
Generating Videos with Scene Dynamics

Generating Videos with Scene Dynamics This repository contains an implementation of Generating Videos with Scene Dynamics by Carl Vondrick, Hamed Pirs

Carl Vondrick 706 Jan 04, 2023
Pytorch implementation of "Attention-Based Recurrent Neural Network Models for Joint Intent Detection and Slot Filling"

RNN-for-Joint-NLU Pytorch implementation of "Attention-Based Recurrent Neural Network Models for Joint Intent Detection and Slot Filling"

Kim SungDong 194 Dec 28, 2022
Multi-Objective Loss Balancing for Physics-Informed Deep Learning

Multi-Objective Loss Balancing for Physics-Informed Deep Learning Code for ReLoBRaLo. Abstract Physics Informed Neural Networks (PINN) are algorithms

Rafael Bischof 16 Dec 12, 2022
CrossMLP - The repository offers the official implementation of our BMVC 2021 paper (oral) in PyTorch.

CrossMLP Cascaded Cross MLP-Mixer GANs for Cross-View Image Translation Bin Ren1, Hao Tang2, Nicu Sebe1. 1University of Trento, Italy, 2ETH, Switzerla

Bingoren 16 Jul 27, 2022
This project aims at building a real-time wide band channel sounder using USRPs

RFNoC-HLS-WINLAB Bhargav Gokalgandhi [email protected] Pr

Xilinx 15 May 13, 2022
Tools for manipulating UVs in the Blender viewport.

UV Tool Suite for Blender A set of tools to make editing UVs easier in Blender. These tools can be accessed wither through the Kitfox - UV panel on th

35 Oct 29, 2022
OpenCV, MediaPipe Pose Estimation, Affine Transform for Icon Overlay

Yoga Pose Identification and Icon Matching Project Goal Detect yoga poses performed by a user and overlay a corresponding icon image. Running the main

Anna Garverick 1 Dec 03, 2021
Physics-Informed Neural Networks (PINN) and Deep BSDE Solvers of Differential Equations for Scientific Machine Learning (SciML) accelerated simulation

NeuralPDE NeuralPDE.jl is a solver package which consists of neural network solvers for partial differential equations using scientific machine learni

SciML Open Source Scientific Machine Learning 680 Jan 02, 2023
UnivNet: A Neural Vocoder with Multi-Resolution Spectrogram Discriminators for High-Fidelity Waveform Generation

UnivNet UnivNet: A Neural Vocoder with Multi-Resolution Spectrogram Discriminators for High-Fidelity Waveform Generation. Training python train.py --c

Rishikesh (ऋषिकेश) 55 Dec 26, 2022
The PyTorch re-implement of a 3D CNN Tracker to extract coronary artery centerlines with state-of-the-art (SOTA) performance. (paper: 'Coronary artery centerline extraction in cardiac CT angiography using a CNN-based orientation classifier')

The PyTorch re-implement of a 3D CNN Tracker to extract coronary artery centerlines with state-of-the-art (SOTA) performance. (paper: 'Coronary artery centerline extraction in cardiac CT angiography

James 135 Dec 23, 2022
TensorFlow implementation for Bayesian Modeling and Uncertainty Quantification for Learning to Optimize: What, Why, and How

Bayesian Modeling and Uncertainty Quantification for Learning to Optimize: What, Why, and How TensorFlow implementation for Bayesian Modeling and Unce

Shen Lab at Texas A&M University 8 Sep 02, 2022
LaneAF: Robust Multi-Lane Detection with Affinity Fields

LaneAF: Robust Multi-Lane Detection with Affinity Fields This repository contains Pytorch code for training and testing LaneAF lane detection models i

155 Dec 17, 2022
A FAIR dataset of TCV experimental results for validating edge/divertor turbulence models.

TCV-X21 validation for divertor turbulence simulations Quick links Intro Welcome to TCV-X21. We're glad you've found us! This repository is designed t

0 Dec 18, 2021
Code for our ACL 2021 paper "One2Set: Generating Diverse Keyphrases as a Set"

One2Set This repository contains the code for our ACL 2021 paper “One2Set: Generating Diverse Keyphrases as a Set”. Our implementation is built on the

Jiacheng Ye 63 Jan 05, 2023