Async boto3 with Autogenerated Data Classes

Overview

awspydk

Async boto3 with Autogenerated JIT Data Classes


Motivation

This library is forked from an internal project that works with a lot of backend AWS APIs, and I got tired of having to constantly parse the returned responses before being able to work with them for (most of the time), a few seconds. Any API-driven application that uses boto3 tends to suffer from being syncronous only. So this library solves a few major problems:

  • Enables both sync and async from the same client.

  • Client can be called implicitly without needing initialization, i.e. you can directly use AwsClient without needing to initialize.

  • Dynamically generates class functions based on the boto3.client child functions, and allows you to call them. This is useful in repl or ipython environments where type-hints are always helpful, especially when the names are so long.

  • Automatically initializes the aws client if its not initialized from the defaults, simply by calling it.

  • Translates all results into Automatically Generated Dataclasses through lazycls.

    • Can be disabled by setting aws.config.AutoCls = False

Quickstart

pip install --upgrade awspydk
from aws import AwsClient

# Sync Method
buckets = AwsClient.v1.s3_list_buckets(as_cls=True)

# Async Method
buckets = await AwsClient.v1.async_s3_list_buckets(as_cls=True)

"""
Both yield the same results.
The underlying classes are auto-generated from Pydantic BaseModels, so anything you can do with Pydantic Models, you can do with these.

{
    'Buckets': [
        AwsS3Bucket(CreationDate=datetime.datetime(2021, 8, 25, 16, 42, 46, tzinfo=tzutc()), Name='...'),
        AwsS3Bucket(CreationDate=datetime.datetime(2021, 9, 2, 17, 54, 56, tzinfo=tzutc()), Name='...',
        AwsS3Bucket(CreationDate=datetime.datetime(2021, 9, 3, 4, 20, 10, tzinfo=tzutc()), Name='...'),
        AwsS3Bucket(CreationDate=datetime.datetime(2021, 9, 1, 20, 50, 33, tzinfo=tzutc()), Name='...'),
        AwsS3Bucket(CreationDate=datetime.datetime(2021, 9, 2, 4, 2, 28, tzinfo=tzutc()), Name='...')
    ],
    'Owner': AwsS3Owner(DisplayName='...', ID='...')
}
"""

## Change Regions
AwsClient.reset(region='us-west-1')

## Change the defaut clients created
from aws.config import DefaultClients

## Modify to only create ec2 client
DefaultClients = {
    'ec2': 'ec2'
}

## Reset implicitly
AwsClient.reset()


BotoKwargs = {
    'AWS_PROFILE': ...,
}

## Reset Explicitly
AwsClient.reset(clients=DefaultClients, boto_kwargs=BotoKwargs)

Client Defaults

These are found in aws.config

AwsRegion = envToStr('AWS_REGION', 'us-east-1')
AutoCls = envToBool('AWSSDK_AUTOCLS', 'true')

## These are the default clients that will be autogenerated.
## Key is the shorthand, value is the actual AWS API Name in boto3
DefaultClients = {
    'ec2': 'ec2', 
    'ecr': 'ecr', 
    'r53' :'route53', 
    'acm': 'acm',
    'elb': 'elb',
    'elbv2': 'elbv2',
    'asg': 'autoscaling',
    's3': 's3'
}

## These are the default resources that will be autogenerated.
## Key is the shorthand, value is the actual AWS Resource API Name in boto3
DefaultResources = {
    'Ec2': 'ec2',
    'S3': 's3',
    'Iam': 'iam'
}

# These are the default filter args for querying
DefaultFilterArgs = {
    'string_only': True,
    'remove_null': True
}
Owner
Chief Architect @ Growth Engine
ML-Test-Client

ML-Test-Client Introduction What is this? This Test Client App is to be used to crowd-test machine learning models with the goal of finding the best c

11 Jul 15, 2022
❤️A next gen powerful telegram group manager bot for manage your groups and have fun with other cool modules

Natsuki Based on Python Telegram Bot Contributors Video Tutorial: Complete guide on deploying @TheNatsukiBot's clone on Heroku. ☆ Video by Sadew Jayas

Pawan Theekshana 8 Oct 06, 2022
A small python script which runs a speedtest using speedtest.net and inserts it into a Google Docs Spreadsheet.

speedtest-google-sheets This is a small python script which runs a speedtest using speedtest.net and inserts it into a Google Docs Spreadsheet. Setup

marie 2 Feb 10, 2022
This Server Cloner can clone the server you want with all the perms of roles in every particular channel.

Server-Cloner-with-perms 🚀 This Server Cloner can clone the server you want with all the perms of roles in every particular channel. Features Clone C

Gripz 0 Feb 17, 2022
Solcast rooftop api for HA

Solcast Solar Home Assistant(https://www.home-assistant.io/) Component This custom component integrates the Solcast API into Home Assistant. Modified

Greg 1 Oct 11, 2021
A Pythonic client for the official https://data.gov.gr API.

pydatagovgr An unofficial Pythonic client for the official data.gov.gr API. Aims to be an easy, intuitive and out-of-the-box way to: find data publish

Ilias Antonopoulos 40 Nov 10, 2022
Parse 11.000 free proxies!

Proxy Machine Description I did this project in order to boost views with the teleboost ✈️ in my Telegram channel. You can use it not only for boostin

VLDSLV 77 Jan 08, 2023
A Phyton script working for stream twits from twitter by tweepy to mongoDB

twitter-to-mongo A python script that uses the Tweepy library to pull Tweets with specific keywords from Twitter's Streaming API, and then stores the

3 Feb 03, 2022
Python JIRA Library is the easiest way to automate JIRA. Support for py27 was dropped on 2019-10-14, do not raise bugs related to it.

Jira Python Library This library eases the use of the Jira REST API from Python and it has been used in production for years. As this is an open-sourc

PyContribs 1.7k Jan 06, 2023
Código que Utiliza Programação Dinâmica para resolver o problema da Moeda

Programação Dinâmica: Modelo baseado em recursão Utiliza a técnica de Memorização Não pode ser aplicada quando existe dependência entre as respostas G

Hemili Beatriz 1 Jan 08, 2022
The Social-Engineer Toolkit (SET) is specifically designed to perform advanced attacks against the human element.

The Social-Engineer Toolkit (SET) The Social-Engineer Toolkit (SET) is specifically designed to perform advanced attacks against the human element. SE

Professor 6 Nov 28, 2022
Bitbucket Server API Wrapper

A simple wrapper for the Atlassian's Bitbucket Server / Bitbucket Datacenter (formerly Stash) REST API, written in Python.

Schweitzer Engineering Laboratories 4 Jan 06, 2023
Box SDK for Python

Box Python SDK Installing Getting Started Authorization Server-to-Server Auth with JWT Traditional 3-legged OAuth2 Other Auth Options Usage Documentat

Box 371 Dec 29, 2022
This discord bot will help you to control your target through PickleC2

PickleC2-Bot This discord bot will help you to control your target through PickleC2 WHAT's a PickleC2? PickleC2 is a simple C2 framework written in py

4 Jun 25, 2022
Criando Lambda Functions para Ingerir Dados de APIs com AWS CDK

LIVE001 - AWS Lambda para Ingerir Dados de APIs Fazer o deploy de uma função lambda com infraestrutura como código Lambda vai numa API externa e extra

Andre Sionek 12 Nov 20, 2022
This is a discord token generator(requests) which works and makes 200 tokens per minute

Discord Email verified token generator Creates email verified discord accounts (unlocked) Report Bug · Discord server Features Profile pictures and na

131 Dec 10, 2022
Discord bot ( discord.py ), uses pandas library from python for data-management.

Discord_bot A Best and the most easy-to-use Discord bot !! Some simple basic auto moderations, Chat functions. It includes a game similar to Casino, g

Jaitej 4 Aug 30, 2022
A lightweight Python wrapper for the IG Markets API

trading_ig A lightweight Python wrapper for the IG Markets API. Simplifies access to the IG REST and Streaming APIs with a live or demo account. What

IG Python 247 Dec 08, 2022
PyFIR - Python implementations of Finite Impulse Response (FIR) filters

pyFIR Python implementations of Finite Impulse Response (FIR) filters. The algorithms are mainly (but not strictly) the ones described in WEFERS, Fran

Davi Carvalho 4 Feb 12, 2022
Neofetch/pfetch, but for weather

Wfetch Neofetch/pfetch, but for weather Features Information about the weather outside: Weather condition Temperature Humidity Pressure Wind Sunrise-s

G_cat 72 Nov 18, 2022