Python Data Types Under the Hood: Memory, Mutability, and Gotchas

When I wrote my first data parsing script on an 8GB RAM laptop, I tried loading 15 million user IDs into a Python list of integers.

In C or Rust, an integer takes 4 bytes. In theory, 15 million integers should take around 60MB of RAM. But five seconds into running the script, Python ate 1.2GB of memory, my laptop began thrashing swap memory, and the process died with MemoryError.

That was the day I realized Python does not store plain numbers. In Python, everything is a rich heap-allocated object. To write fast, memory-conscious Python backend services, you have to understand how data types actually operate under the CPython runtime.

The PyObject Memory Overhead

Open a Python 3.12 shell and inspect the memory footprint of an integer:

>>> import sys
>>> sys.getsizeof(0)
24
>>> sys.getsizeof(42)
28
>>> sys.getsizeof(2**100)
40

Why does storing the number 42 require 28 bytes? Because CPython wraps every value inside a PyObject structure containing:

• ob_refcnt (8 bytes): Reference count for garbage collection.
• ob_type (8 bytes): Pointer to the type object (<class 'int'>).
• ob_size (8 bytes): Number of 30-bit digits representing the arbitrary-precision integer.
• ob_digit (4 bytes): The actual numerical payload.

This dynamic abstraction gives Python arbitrary precision arithmetic out of the box. You will never encounter an integer overflow bug in Python, but you pay a 7x memory tax per number.

The Float Precision Trap in Financial Code

If you build e-commerce payment checkouts or UPI billing scripts, never store rupee amounts in standard Python float types:

>>> 0.1 + 0.2
0.30000000000000004
>>> 0.1 + 0.2 == 0.3
False

Python floats follow the IEEE 754 standard for binary 64-bit floating-point numbers. Most decimal fractions (like 0.1) cannot be represented cleanly in base-2 binary, leading to tiny rounding discrepancies. Over 100,000 billing transactions, those pennies produce audit failures.

Always use Python's built-in decimal.Decimal for financial logic:

from decimal import Decimal, ROUND_HALF_UP

# Always pass strings to Decimal to prevent float conversion error
item_price = Decimal('199.99')
tax_rate = Decimal('0.18') # 18% GST

total_tax = (item_price * tax_rate).quantize(Decimal('0.01'), rounding=ROUND_HALF_UP)
final_payable = item_price + total_tax

print(f"Payable: ₹{final_payable}") # Payable: ₹235.99

The Classic Mutability Bug: Default Mutable Arguments

Here is one of the most common interview questions asked at product companies in Pune and Hyderabad:

# DANGEROUS: Default list evaluated ONCE at function definition
def add_user_log(user_id: int, tags: list = []):
    tags.append(user_id)
    return tags

print(add_user_log(101)) # [101]
print(add_user_log(102)) # [101, 102] <-- BUG: Shared mutable list across calls!

In Python, default arguments are not created anew every time a function executes. They are evaluated exactly once when the Python interpreter parses the function definition. If you modify that mutable list, every subsequent call shares that identical memory reference.

The standard idiomatic fix uses None as a sentinel value:

def add_user_log(user_id: int, tags: list | None = None):
    if tags is None:
        tags = []
    tags.append(user_id)
    return tags

Collection Comparison: Lists vs Tuples vs Sets vs Dictionaries

Choosing the right container for your data directly impacts algorithmic time complexity and RAM consumption:

If you have 100,000 records and need to check whether a given ID exists in that collection, checking if id in my_list: takes linear time O(N). If you convert that collection to a set() once, lookups drop to near-instant O(1) time.

Slashing Memory by 60% with __slots__

By default, every custom class instance in Python maintains a dynamic dictionary (__dict__) to allow arbitrary runtime attribute assignment. If you instantiate millions of data objects, this dictionary wastes gigabytes of RAM.

Use __slots__ or modern dataclasses with slots=True to eliminate the dynamic dictionary:

from dataclasses import dataclass
import sys

@dataclass(slots=True)
class OptimizedRecord:
    user_id: int
    email: str
    score: float

# Compare memory footprint
rec = OptimizedRecord(user_id=1, email="ankur@dropoutdeveloper.in", score=98.5)
print(f"Optimized object size: {sys.getsizeof(rec)} bytes")

Using slots=True tells CPython to allocate a fixed-size C array for attributes instead of an open-ended dictionary, cutting per-object memory usage by more than half.

Identity vs Equality: is vs ==

Never confuse value equality with object identity:

• == checks if the values of two objects are equal.
• is checks if two variables point to the exact same address in memory (id(a) == id(b)).

>>> a = 256
>>> b = 256
>>> a is b
True

>>> x = 257
>>> y = 257
>>> x is y
False

Why does 256 is 256 return True while 257 is 257 returns False in standard REPL sessions? Because CPython maintains an internal global cache for small integers between -5 and 256. When you assign an integer in that range, Python reuses the existing singleton object. Outside that range, Python allocates a brand new memory block. Always use == for numbers and reserve is for singletons like None.

Practical Rules for Clean Python Code

1. Use Decimal whenever currency, prices, or taxes are involved.
2. Default function arguments must always be immutable primitives or None.
3. Prefer tuple over list for read-only static collections to conserve memory.
4. Use set when running frequent membership checks (in operator).
5. Add slots=True to data-heavy classes when processing large datasets.

Next Steps

• Master decision branching in Mastering Conditional Logic and Boolean Short-Circuiting.
• Format complex JSON data using our JSON Formatter.
• Measure LLM prompt limits using our Token Counter.