A Python dictionary implementation designed to act as an in-memory cache for FaaS environments

Related tags

Data Structuresfaas-cache-dict
Overview

faas-cache-dict

A Python dictionary implementation designed to act as an in-memory cache for FaaS environments.

Formally you would describe this a memory constrained LRU TTL cache dict.

This is implementation only uses core Python stdlib with no external dependencies.

Background

This was originally designed to be used as an in-memory cache for AWS Lambda.

In most FaaS environments, successive quick calls of the function persists variables in the global scope. We can leverage this to cache data for future calls.

FaaS runtimes have limited RAM capacities so this library allows you to set a max mebibyte size for the dict. It also allows setting an optional max items length.

Items are kept in order with the LRU at the HEAD of the list.

Items are deleted if they expire, or from the head (LRU) if the cache is out of space.

Usage

Limitations

  • Performance degrades with size, you will need to test this for your use case. Though in most circumstances this will be much faster than performing a network call to a cache.
  • The library should be thread-safe, but limited testing has gone into this.

Support

CPython 3.7 or greater. No extra dependencies are required.

Contributions

This code is distributed under an open license. Feel free to fork it or preferably open a PR.

Inspirations

Thanks to mobilityhouse/ttldict for their implementation which served as a starting point.

You might also like...
Leetcode solutions - All algorithms implemented in Python 3 (for education)

Leetcode solutions - All algorithms implemented in Python 3 (for education)

Vineet Dhaimodker 3 Oct 21, 2022
This Repository consists of my solutions in Python 3 to various problems in Data Structures and Algorithms
This Repository consists of my solutions in Python 3 to various problems in Data Structures and Algorithms

Problems and it's solutions. Problem solving, a great Speed comes with a good Accuracy. The more Accurate you can write code, the more Speed you will

SAMIR PAUL 1.3k Jan 1, 2023
Python library for doing things with Grid-like structures

gridthings Python library for doing things with Grid-like structures Development This project uses poetry for dependency management, pre-commit for li

Matt Kafonek 2 Dec 21, 2021
A Python library for electronic structure pre/post-processing
A Python library for electronic structure pre/post-processing

PyProcar PyProcar is a robust, open-source Python library used for pre- and post-processing of the electronic structure data coming from DFT calculati

Romero Group 124 Dec 7, 2022
Programming of a spanning tree algorithm with Python : In depth first with a root node.

ST Algorithm Programming of a spanning tree algorithm with Python : In depth first with a root node. Description This programm reads informations abou

Mathieu Lamon 1 Dec 16, 2021
Data Structure With Python

Data-Structure-With-Python- Python programs also include in this repo Stack A stack is a linear data structure that stores items in a Last-In/First-Ou

Sumit Nautiyal 2 Jan 9, 2022
🔬 Fixed struct serialization system, using Python 3.9 annotated type hints

py-struct Fixed-size struct serialization, using Python 3.9 annotated type hints This was originally uploaded as a Gist because it's not intended as a

Alba Mendez 4 Jan 14, 2022
Final Project for Practical Python Programming and Algorithms for Data Analysis
Final Project for Practical Python Programming and Algorithms for Data Analysis

Final Project for Practical Python Programming and Algorithms for Data Analysis (PHW2781L, Summer 2020) Redlining, Race-Exclusive Deed Restriction Lan

Aislyn Schalck 1 Jan 27, 2022
Basic sort and search algorithms written in python.

Basic sort and search algorithms written in python. These were all developed as part of my Computer Science course to demonstrate understanding so they aren't 100% efficent

Ben Jones 0 Dec 14, 2022
Releases(0.2.3)

  • 0.2.3(Apr 27, 2022)

    Included

    • Fix min version installable on Py 3.8 + (due to pickle protocol versioning)

    Todo:

    • Discover better documents solution

    This has not yet been tested in a prod environment, and as such should be considered beta software for now.

    Source code(tar.gz)
    Source code(zip)

  • 0.2.2(Apr 25, 2022)

    Included

    • Add purge() to empty all contents of cache dict

    Todo:

    • Discover better documents solution

    This has not yet been tested in a prod environment, and as such should be considered beta software for now.

    Source code(tar.gz)
    Source code(zip)

  • 0.2.1(Apr 22, 2022)

    Included

    • Fix set_ttl() should be able to make a value immortal by passing None
    • Fix is_expired() no longer errors if the key has been purged, now returns True if missing
    • Add ability to change the max (LRU) length of the cache once created, purging the LRU as required.
    • Add extensive new documentation

    Todo:

    • Discover better documents solution

    This has not yet been tested in a prod environment, and as such should be considered beta software for now.

    Source code(tar.gz)
    Source code(zip)

  • 0.2.0(Apr 22, 2022)

    Included

    • Add FileBackedFaaSCache
    • Add support for pickling faas cache objects
    • Bump minimum supported version to Python 3.8
    • Fix minor docs

    Todo:

    • Add full documentation

    This has not yet been tested in a prod environment, and as such should be considered beta software for now.

    Source code(tar.gz)
    Source code(zip)

  • 0.1.3(Feb 15, 2022)

    Included

    • Cleanup deprecated leftovers from objsize migration

    Todo:

    • Add full documentation

    This has not yet been tested in a prod environment, and as such should be considered beta software for now.

    Source code(tar.gz)
    Source code(zip)

  • 0.1.2(Feb 15, 2022)

    Included

    • Replaced primitive deep size algorithm with objsize

    Todo:

    • Add full documentation

    This has not yet been tested in a prod environment, and as such should be considered beta software for now.

    Source code(tar.gz)
    Source code(zip)

  • 0.0.6(Feb 15, 2022)

    Included

    • Fixed "falsey" key causing badly reported item expiry. Thanks @lokhman
    • Fixed get_deep_size incorrectly passing params on deeply nested dicts.
    • Fix handling of attempted nested deletion of already deleted items.
    • Refactor constants to own file and general code cleanup.

    Todo:

    • Add full documentation

    This has not yet been tested in a prod environment, and as such should be considered beta software for now.

    Source code(tar.gz)
    Source code(zip)

  • 0.0.5(Dec 20, 2021)

    Included

    • Fix usage in README

    Todo:

    • Add full documentation

    This has not yet been tested in a prod environment, and as such should be considered beta software for now.

    Source code(tar.gz)
    Source code(zip)

  • 0.0.4(Dec 20, 2021)

    Included

    • Add module level python version lockout
    • Improve object __repr__
    • Add usage docstring

    Todo:

    • Add full documentation

    This has not yet been tested in a prod environment, and as such should be considered beta software for now.

    Source code(tar.gz)
    Source code(zip)

  • 0.0.3(Dec 20, 2021)

    Included

    • Added new support for memory constraint shortcuts, 1M for 1 Mebibyte, 1K, 1G and 1T also supported respectively.
    • Added support for float TTL and float memory constraints.
    • Fixed nasty bug which could cause livelock if memory constraint made too small after data had been added.
    • Added max items to object __repr___.
    • Standardised user input linting.
    • Unit tests for the new functionality.
    • Excluded scripts folder from the distributable.
    • Refactoring of project to split out logic into smaller files.

    Todo:

    • Add usage documentation.

    This has not yet been tested in a prod environment, and as such should be considered beta software for now.

    Source code(tar.gz)
    Source code(zip)

  • 0.0.2(Dec 17, 2021)

    Initial release.

    Included

    • Support implemented for TTL, LRU and max mebibyte size.
    • Unit tests.

    Todo:

    • Add usage documentation

    This has not yet been tested in a prod environment, and as such should be considered beta software for now.

    Source code(tar.gz)
    Source code(zip)
Owner
Juan
Security Engineer, Python charmer, JSON wizard.
Juan
GitHub Repository
Webtesting for course Data Structures & Algorithms

Selenium job to automate queries to check last posts of Module Data Structures & Algorithms Web-testing for course Data Structures & Algorithms Struct

1 Dec 15, 2021
Integrating C Buffer Data Into the instruction of `.text` segment instead of on `.data`, `.rodata` to avoid copy.

gcc-bufdata-integrating2text Integrating C Buffer Data Into the instruction of .text segment instead of on .data, .rodata to avoid copy. Usage In your

Jack Ren 1 Jan 31, 2022
A DSA repository but everything is in python.

DSA Status Contents A: Mathematics B: Bit Magic C: Recursion D: Arrays E: Searching F: Sorting G: Matrix H: Hashing I: String J: Linked List K: Stack

Shubhashish Dixit 63 Dec 23, 2022
Supporting information (calculation outputs, structures)

Supporting information (calculation outputs, structures)

Eric Berquist 2 Feb 02, 2022
Simple spill-to-disk dictionary

Chest A dictionary that spills to disk. Chest acts likes a dictionary but it can write its contents to disk. This is useful in the following two occas

Blaze 59 Dec 19, 2022
A Python implementation of red-black trees

Python red-black trees A Python implementation of red-black trees. This code was originally copied from programiz.com, but I have made a few tweaks to

Emily Dolson 7 Oct 20, 2022
A high-performance immutable mapping type for Python.

immutables An immutable mapping type for Python. The underlying datastructure is a Hash Array Mapped Trie (HAMT) used in Clojure, Scala, Haskell, and

magicstack 996 Jan 02, 2023
One-Stop Destination for codes of all Data Structures & Algorithms

CodingSimplified_GK This repository is aimed at creating a One stop Destination of codes of all Data structures and Algorithms along with basic explai

Geetika Kaushik 21 Sep 26, 2022
RLStructures is a library to facilitate the implementation of new reinforcement learning algorithms.

RLStructures is a lightweight Python library that provides simple APIs as well as data structures that make as few assumptions as possibl

Facebook Research 262 Nov 18, 2022
This repository contains code for CTF platform.

CTF-platform Repository for backend of CTF hosting website For starting the project first time : Clone the repo in which you have to work in your syst

Yash Jain 3 Feb 18, 2022
An command-line utility that schedules your exams preparation routines

studyplan A tiny utility that schedules your exams preparation routines. You only need to specify the tasks and the deadline. App will output a iCal f

Ilya Breitburg 3 May 18, 2022
Python library for doing things with Grid-like structures

gridthings Python library for doing things with Grid-like structures Development This project uses poetry for dependency management, pre-commit for li

Matt Kafonek 2 Dec 21, 2021
nocasedict - A case-insensitive ordered dictionary for Python

nocasedict - A case-insensitive ordered dictionary for Python Overview Class NocaseDict is a case-insensitive ordered dictionary that preserves the or

PyWBEM Projects 2 Dec 12, 2021
Array is a functional mutable sequence inheriting from Python's built-in list.

funct.Array Array is a functional mutable sequence inheriting from Python's built-in list. Array provides 100+ higher-order methods and more functiona

182 Nov 21, 2022
An esoteric data type built entirely of NaNs.

NaNsAreNumbers An esoteric data type built entirely of NaNs. Installation pip install nans_are_numbers Explanation A floating point number is just co

Travis Hoppe 72 Jan 01, 2023
This repository is for adding codes of data structures and algorithms, leetCode, hackerrank etc solutions in different languages

DSA-Code-Snippet This repository is for adding codes of data structures and algorithms, leetCode, hackerrank etc solutions in different languages Cont

DSCSRMNCR 3 Oct 22, 2021
Decided to include my solutions for leetcode problems.

LeetCode_Solutions Decided to include my solutions for leetcode problems. LeetCode # 1 TwoSum First leetcode problem and it was kind of a struggle. Th

DandaIT04 0 Jan 01, 2022
dict subclass with keylist/keypath support, normalized I/O operations (base64, csv, ini, json, pickle, plist, query-string, toml, xml, yaml) and many utilities.

python-benedict python-benedict is a dict subclass with keylist/keypath support, I/O shortcuts (base64, csv, ini, json, pickle, plist, query-string, t

Fabio Caccamo 799 Jan 09, 2023
This repo represents all we learned and are learning in Data Structure course.

DataStructure Journey This repo represents all we learned and are learning in Data Structure course which is based on CLRS book and is being taught by

Aprime Afr (Alireza Afroozi) 3 Jan 22, 2022
Leetcode solutions - All algorithms implemented in Python 3 (for education)

Leetcode solutions - All algorithms implemented in Python 3 (for education)

Vineet Dhaimodker 3 Oct 21, 2022