updates

Author: nocomplexity

constructs/multiprocessing.md

@@ -0,0 +1,87 @@
+---
+title: Multiprocessing Module `Connection.recv()` 
+short_title: Multiprocessing Module
+---
+
+
+The `multiprocessing` module provides powerful tools for inter-process communication (IPC) in Python. However, its use of the `pickle` module for serialization introduces significant security risks if not handled with care.
+
+## Security Concerns
+
+The `Connection.recv()` method (and its counterparts like `recv_bytes()`) automatically unpickles the received data. Deserializing untrusted data with `pickle` can lead to arbitrary code execution, making it a high-severity vulnerability when the source of the data cannot be fully trusted.
+
+This risk exists because `pickle` is **not** a safe serialization format for untrusted input. Maliciously crafted data can trigger the execution of arbitrary Python code during unpickling.
+
+Even when using `send()` / `recv()` pairs within the same application, design flaws (such as insufficient input validation or exposure of listening endpoints) can allow attackers to inject harmful payloads.
+
+:::{important}
+The `multiprocessing` module **always** requires careful design, thorough code review, and robust security controls.
+:::
+
+
+## Preventive Measures
+
+* **Use `Pipe()` where possible**: Connections created with `multiprocessing.Pipe()` are considered safer within a single process tree because both ends are typically controlled by the same application. Even then, treat data as untrusted.
+
++++
+
+* **Implement strong authentication**: Before calling `recv()` on any `Connection` object (especially those from network listeners, queues, or shared resources), perform proper authentication of the remote process. This may include cryptographic verification, mutual TLS, or other access control mechanisms.
+
++++
+
+* **Avoid `recv()` on untrusted connections**: Prefer safer alternatives such as:
+  * Structured data formats like JSON, MessagePack (with schema validation), or Protocol Buffers.
+  * Higher-level libraries that handle authentication and secure serialization.
+
++++
+
+* **Sandbox or restrict privileges**: Run processes using the `multiprocessing` module with the principle of least privilege.
+
++++
+
+* **Input validation**: Always validate and sanitise data *after* receiving but *before* any further processing, even if authentication is in place.
+
++++
+
+* **Prefer `send_bytes()` / `recv_bytes()` with manual deserialization**: This gives you more control, though it does not eliminate the need for careful handling.
+
+## Example
+
+```python
+from multiprocessing import Pipe, Process
+
+def worker(conn):
+    # Good practice: send trusted data only
+    conn.send({"command": "hello", "value": 42})  # Simple dict, safe
+
+def main():
+    parent_conn, child_conn = Pipe()
+    p = Process(target=worker, args=(child_conn,))
+    p.start()
+
+    # Safe because we control both ends and use Pipe()
+    data = parent_conn.recv()  # Still verify structure in production!
+    print(data)
+
+    p.join()
+```
+
+**Warning example (to avoid)**
+Dangerous example: receiving from an untrusted or network-exposed connection
+
+```python
+conn = ...  # from Listener or external source
+data = conn.recv()  # Arbitrary code execution possible!
+```
+
+## Discussion
+
+The core issue stems from `pickle`'s design: it can execute code via `__reduce__` methods and other mechanisms. The Python documentation explicitly warns about this risk.
+
+While `Pipe()` reduces the attack surface in simple cases, any exposure to external input, shared memory, or less-controlled process relationships reintroduces the danger. Modern Python security practices favour explicit, auditable serialization over `pickle` for cross-process or cross-machine communication.
+
+Code audit tools relying on Python's `ast` module (without full type inference or data-flow analysis) can only detect common patterns. They may miss sophisticated uses of `multiprocessing.Connection` objects stored in variables or passed through helper functions.
+
+## More Information
+
+* [Python Documentation: multiprocessing](https://docs.python.org/3/library/multiprocessing.html#multiprocessing-recv-pickle-security)

constructs/pickle.md

@@ -0,0 +1,56 @@
+# Pickle Function
+
+:::{danger} 
+Unpickling will import any class or function that it finds in the pickle data. This is a severe security concern as it permits the unpickler to import and invoke arbitrary code. 
+
+**Never use `pickle.load()`, `pickle.loads()` or `pickle.Unpickler` on data received from an untrusted or unauthenticated source.**
+:::
+
+## Security Concerns
+
+The security concern with the `pickle` module in Python revolves around **deserialisation of untrusted data**. When you use `pickle.load()` to deserialise a byte stream, the `pickle` module reconstructs Python objects from that stream by importing any class or function it encounters. A malicious attacker can craft a pickled payload that, when deserialised, can:
+
+1. **Execute arbitrary code:** The `pickle` protocol can be manipulated to cause the deserialiser to import arbitrary modules and call arbitrary functions with arbitrary arguments. This permits an attacker to execute system commands, delete files, or perform any action the Python process running `pickle.load()` has permissions to carry out. This is commonly referred to as a "deserialisation vulnerability" or "arbitrary code execution."
+
+2. **Cause Denial of Service (DoS):** An attacker could create a pickled object that, when deserialised, consumes excessive memory or CPU resources, leading to your application crashing or becoming unresponsive.
+
+## Preventive Measures
+
+The most effective preventive measure is to **avoid using `pickle` for deserialising untrusted data entirely**. Where this is not possible, consider the following approaches:
+
+- **Use alternative serialisation formats:** Consider using `json`, `msgpack`, or `protobuf` for data exchange with untrusted sources. These formats are limited to data structures and do not support arbitrary code execution.
+  
+- **Implement cryptographic signing:** If you must use `pickle`, sign your pickled data with a cryptographic signature (e.g., using `hmac`) before transmission and verify the signature before deserialisation. This ensures the data has not been tampered with.
+
+- **Run deserialisation in a sandbox:** Isolate the unpickling process in a restricted environment, such as a container or a separate process with limited system privileges.
+
+- **Use `pickle.restricted` (Python 3.x):** The `pickle` module provides a `restricted` attribute that can be used to limit the classes and functions that can be imported during unpickling. However, this is not a complete security solution and should be used with caution.
+
+## Example
+
+Consider the following malicious pickled payload:
+
+```python
+import pickle
+import os
+
+class Malicious:
+    def __reduce__(self):
+        return (os.system, ('rm -rf /',))
+
+malicious_data = pickle.dumps(Malicious())
+pickle.loads(malicious_data)  # Executes 'rm -rf /'
+```
+
+In this example, the `__reduce__` method returns a tuple instructing the unpickler to call `os.system` with the argument `'rm -rf /'`. When deserialised, this would execute a destructive system command.
+
+## Discussion
+
+The root cause of the security issue lies in Python's dynamic nature: `pickle` was designed for flexibility, allowing serialisation of complex Python objects, including functions and classes. This flexibility comes at the cost of security when handling untrusted data.
+
+The security weakness is not in `pickle` itself but in how it is used. The Python documentation explicitly warns against using `pickle` with untrusted data. 
+
+
+## More Information
+
+- [Python `pickle` Documentation - Security Considerations](https://docs.python.org/3/library/pickle.html#pickle-restrict)

constructs/random.md

@@ -0,0 +1,98 @@
+# Random Statement
+
+:::{warning}
+The `random` module is **not** suitable for security or cryptographic purposes. Its pseudo-random number generator is deterministic and predictable if an attacker observes enough output. Never use it for generating session tokens, encryption keys, passwords, or any other security-sensitive values.
+:::
+
+## Security Concerns
+
+The `random` module in Python is unsafe for security or cryptographic purposes, such as generating session tokens, encryption keys, or passwords.
+
+This is because the `random` module uses a pseudo-random number generator (PRNG) called the Mersenne Twister. This algorithm is **deterministic**—if an attacker can observe a sufficient amount of its output, they can completely determine its internal state (the seed) and accurately predict all future and even past values. This predictability can lead to severe vulnerabilities, including session hijacking, credential compromise, and unauthorised access.
+
+The following `random` functions can introduce weaknesses when used in security contexts:
+- `random.seed`
+- `random.Random`
+- `random.randbytes`
+- `random.randint`
+- `random.random`
+- `random.randrange`
+- `random.triangular`
+- `random.uniform`
+
+The `random` module is specifically designed for non-security-sensitive applications like simulations, statistical modelling, and simple games, prioritising speed and good statistical distribution over true unpredictability.
+
+For all security-sensitive tasks, you **must** use the `secrets` module, which relies on a Cryptographically Secure Pseudo-Random Number Generator (CSPRNG) provided by the operating system.
+
+## Preventive Measures
+
+To avoid introducing predictable randomness vulnerabilities in your applications, follow these guidelines:
+
+- **Use `secrets` for security purposes:** For any security-sensitive randomness—such as passwords, tokens, or cryptographic keys—always use the `secrets` module. This module provides functions like `secrets.token_bytes()`, `secrets.token_hex()`, and `secrets.choice()` that are designed for cryptographic security.
+
++++
+
+
+- **Use `random.SystemRandom` with caution:** The `SystemRandom` class uses `os.urandom()` and can be used as a drop-in replacement for the `random` module's functions. However, it may not be available on all systems, and using `secrets` directly is the preferred, more explicit approach.
+
++++
+
+- **Review third-party dependencies:** Ensure that any third-party libraries you use are not relying on the `random` module for security-critical operations. Vulnerabilities can be introduced through transitive dependencies.
+
+
+## Example
+
+Consider the following problematic code that uses `random` to generate a browser cookie:
+
+```python
+"""Problematic code using random module"""
+import random
+
+browser_cookie = random.randint(min_value, max_value)
+```
+
+
+This code is vulnerable because an attacker could predict the cookie value if they observe enough outputs from the same PRNG instance.
+
+To improve this code, use the `secrets` module:
+
+
+Use the  `SystemRandom` class. This class uses the system function `os.urandom()` to generate random numbers from sources provided by the operating system.
+
+```python
+from random import SystemRandom
+safe_random = SystemRandom()
+
+browser_cookie = safe_random.randint(min_value, max_value)
+```
+
+Alternatively, if you need a drop-in replacement for `random` functions, use `SystemRandom`:
+
+```python
+from random import SystemRandom
+safe_random = SystemRandom()
+
+browser_cookie = safe_random.randint(min_value, max_value)
+```
+
+However, note that `secrets` is the recommended approach for all new code requiring cryptographic randomness.
+
+
+## Discussion
+
+The root cause of the security issue lies in the design goals of the Mersenne Twister algorithm. It was developed for speed and statistical quality in simulations, not for unpredictability. Its state can be fully recovered from just 624 consecutive outputs, making it trivial for an attacker to predict future values once they have observed enough random numbers.
+
+The Python standard library provides the `secrets` module specifically to address this gap. Introduced in Python 3.6 via [PEP 506](https://peps.python.org/pep-0506/), `secrets` was designed to be the "go-to" module for all cryptographic randomness needs. It abstracts away platform-specific CSPRNG implementations and provides a simple, safe API for developers.
+
+A common misconception is that using `random.seed()` with a high-entropy seed makes the `random` module secure. This is false—while it may make prediction more difficult initially, the Mersenne Twister's state can still be recovered from its outputs, regardless of the seed's quality.
+
+## More Information
+
+* https://docs.python.org/3/library/random.html
+* https://docs.python.org/3/library/secrets.html#module-secrets
+* [ CWE-330: Use of Insufficiently Random Values](https://cwe.mitre.org/data/definitions/330.html)
+* [CWE-338: Use of Cryptographically Weak Pseudo-Random Number Generator (PRNG)](https://cwe.mitre.org/data/definitions/338.html)
+* [CVE-2022-23472](https://nvd.nist.gov/vuln/detail/CVE-2022-23472)
+* [PEP 506 – Adding A Secrets Module To The Standard Library](https://peps.python.org/pep-0506/)
+* https://www.codiga.io/blog/python-avoid-random/
+

constructs/shelve.md

@@ -0,0 +1,66 @@
+# Shelve Module
+
+The `shelve` module offers a simple, persistent, dictionary-like interface for storing and retrieving Python objects to and from disk. While convenient for basic local persistence, it is built directly on top of the `pickle` module and inherits all of its serious security risks.
+
+## Security Concerns
+
+The `shelve` module uses `pickle` for both serialization (`shelve.sync()`, object storage) and deserialization (`shelve.open()` and item retrieval). This means that loading data from a shelf file can trigger arbitrary code execution if the file originates from an untrusted source.
+
+Key risks include:
+
+- **Arbitrary code execution**: Malicious shelf files can exploit `pickle`'s powerful reconstruction capabilities (via `__reduce__`, custom classes, etc.) to run arbitrary Python code during unpickling.
+- **No built-in authentication or validation**: `shelve` performs no integrity checks or sandboxing by default.
+- **File-based attack surface**: Any process with write access to the shelf file (or the ability to supply a malicious `.db` file) can compromise the application that opens it.
+- **Silent failures and hidden dangers**: The convenience of the dict-like API can lead developers to treat shelf data as trusted without proper validation.
+
+
+## Preventive Measures
+
+* **Never load untrusted shelves**: Avoid using `shelve.open()` on files received from users, networks, downloads, or any untrusted location.
+
+* **Prefer safer alternatives**:
+  * Use `json`, `toml`, `yaml` (with safe loaders), or structured formats like Protocol Buffers / MessagePack with schema validation.
+  * For more complex object persistence, consider ORMs with proper input sanitisation (e.g., SQLAlchemy) or secure serialization libraries.
+
+* **Validate and sanitise data**: Even when using `shelve` for trusted data, implement strict schema validation after loading.
+
+* **Use with `pickle` restrictions**: If you must use `shelve`, restrict the `pickle` protocol to the safest level and consider custom `Unpickler` classes that limit global imports (though this is complex and not foolproof).
+* **File permissions and isolation**: Store shelf files with strict permissions and run the application with the principle of least privilege.
+
+* **Consider `dbm` directly**: For simple key-value storage of bytes/strings, use the lower-level `dbm` modules instead of `shelve`.
+
+## Example
+
+**Safe usage (trusted local data only)**:
+
+```python
+import shelve
+
+with shelve.open('local_config.db') as db:
+    db['user_settings'] = {'theme': 'dark', 'timeout': 30}
+    settings = db['user_settings']  # Safe because file is trusted
+```
+
+**Dangerous usage (to avoid)**:
+
+```python
+import shelve
+
+# Never do this with files from untrusted sources
+with shelve.open('untrusted_data.db') as db:  # Arbitrary code execution possible!
+    malicious_data = db['payload']
+```
+
+## Discussion
+
+`Shelve` is only appropriate for **fully trusted** environments — typically single-user applications where the shelf file is never exposed to external input or third parties. In practice this is near to impossible for most use cases.
+
+The fundamental problem with `shelve` is that it provides an attractive, high-level interface that hides the dangerous `pickle` implementation underneath. Developers often underestimate the risk because the API feels like a simple dictionary.
+
+Unlike `pickle` used directly, `shelve` adds the complication of persistent files that may be tampered with over time or replaced by attackers. This makes it especially dangerous in desktop applications, plugins, or any scenario where files can be swapped.
+
+Modern Python security guidance strongly recommends avoiding `pickle`-based solutions for anything beyond fully controlled, internal use cases. The convenience of `shelve` rarely outweighs the long-term maintenance and security burden it introduces.
+
+## More Information
+
+* [Python Documentation: shelve — Security Considerations](https://docs.python.org/3/library/shelve.html#shelve-security)