Introduction: The Elegant Complexity of Python's Meta-Design

Python stands as the predominant language in the artificial intelligence domain, yet for developers coming from strongly-typed languages like C#, it presents numerous points of friction and philosophical disagreement. However, there exists one aspect of Python's design that represents an absolute triumph of language architecture: its metaclass-based metamodel. This sophisticated system, while often misunderstood or underutilized, reveals profound insights into how Python thinks about objects, classes, and the very nature of type creation.

This comprehensive exploration delves into the intricate workings of Python's metamodel, unpacking the relationships between instances, classes, metaclasses, and the ultimate metaclass—type itself. Through detailed analysis and practical examples, we will illuminate how this system achieves a self-consistent circularity that mirrors philosophical concepts of creation and existence.

The Metaverse Analogy: Understanding Meta Through Cosmology

To grasp Python's metamodel, consider an analogy drawn from the concept of the metaverse—a notion that, despite its recent decline in popular discourse, offers valuable insights into hierarchical creation.

From a "creation" perspective, the metaverse defines the laws governing our universe (reality). The universe itself is an instance constructed from metadata. The terms "meta" and "instance" are not absolute categories; the metaverse is itself a universe, and its laws are defined by a higher-order "meta-metaverse." The metaverse becomes an instance of this meta-metaverse. This abstraction can continue indefinitely until we reach an ultimate origin point where we cannot identify the source of laws. At this terminus, some attribute creation to a "creator," while Laozi termed it "Dao" (the Way). We might call this the "Genesis Universe."

Since we cannot find a meta for the Genesis Universe, we allow it to serve as its own meta, creating a self-consistent closed loop. If we view the universe as an instance of its metaverse, then the Genesis Universe's instances include itself.

The meta can be simpler than the instance—the principle of "great truths are simple." As the Tao Te Ching states: "The Dao begets One; One begets Two; Two begets Three; Three begets all things." Alternatively, the meta can be more complex than the instance, containing intricate laws from which we select subsets to construct instances. Readers familiar with the novel "Perfect World" will recall that the "Lower Realm's Eight Domains," serving as a place of exile for prisoners, constitutes a "dimensionally-reduced universe" constructed with the "Upper Realm" as its metaverse. Due to incomplete laws, cultivation in the Lower Realm can only reach the "Venerable Realm"—godhood remains unattainable.

Python's "metaverse" operates on similar principles. We use meta to define rules for instances and serve as a factory for creating instances. Thus, a class is the meta of its instances. We can use metaclasses to create classes, meaning conventional classes are instances of metaclasses, and metaclasses are the meta of conventional classes. In this sense, "metaclass" might be more accurately termed "class-meta," but the term "metaclass" expresses both "the meta of classes" and the concept that metaclasses themselves are classes. Since metaclasses are classes, they naturally can have their own metaclasses. Therefore, class is meta, and meta is also class—just as "the metaverse is also a universe, and a universe can serve as a metaverse."

Python's Genesis Universe is type. Our classes are constructed by it, making type the metaclass of classes. Since type is a metaclass, it possesses the attributes of a class. Due to its transcendent position in this Genesis Universe, type can be viewed as its own instance.

The Definition and Mechanics of Metaclasses

By default, when we use a class to create an instance, the underlying process unfolds as follows:

  1. The class's __new__ method is called with the class and parameters (including any added keyword arguments) to create a base object
  2. This base object and parameters are passed to the class's __init__ method for initialization
  3. The initialized object is returned

It's crucial to emphasize that although __new__ appears syntactically similar to a class method, it is本质上 a static method. It lacks the standard @classmethod decorator, and when called, the first parameter must be explicitly specified.

Consider the following Foobar class definition. Two methods of creating instances (foobar1 and foobar2) are equivalent:

from typing_extensions import Self
from typing import Any

class Foobar:
    def __new__(cls, *args: Any, **kwargs: Any) -> Self:
        return super().__new__(cls)
    
    def __init__(self, foo: int, bar: int) -> None:
        self.foo = foo
        self.bar = bar
    
    def __eq__(self, value: object) -> bool:
        if not isinstance(value, Foobar):
            return NotImplemented
        return self.foo == value.foo and self.bar == value.bar

foobar1 = Foobar(foo=111, bar=222)
foobar2 = Foobar.__new__(Foobar, foo=111, bar=222)
Foobar.__init__(foobar2, foo=111, bar=222)
assert foobar1 == foobar2

Similarly, a metaclass, being a class itself, can define __new__ and __init__ methods. While a class's these methods initialize its instances, a metaclass's these methods serve the same purpose—except the metaclass's instances are the classes that have it as their metaclass. When the Python interpreter encounters a class definition with a specified metaclass, it employs a similar process to create that class:

  1. Call the metaclass's __new__ method with fixed parameters to construct a class object. Parameters依次为:

    • The current metaclass
    • The class name
    • A dictionary representing class members
    • Keyword arguments specified during class definition

  2. Call the metaclass's __init__ method with fixed parameters to initialize the base class object created above. Parameters include:

    • self as the class object
    • The class name
    • A dictionary representing class members
    • Keyword arguments specified during class definition

Consider the following metaclass Meta as an illustrative example. We define __new__ and __init__ methods that print all output parameters. In __new__, we add keyword arguments to the namespaces dictionary serving as class members, then call the base class (type)'s __new__ method to create the base class object. In __init__, we define a function repr that outputs as a string and assign it as the class's __repr__ method:

from typing import Any

class Meta(type):
    def __new__(cls, name: str, bases: tuple[type, ...], namespaces: dict[str, Any], /, **kwds: Any):
        print(f"""
__new__
 cls: {cls}
 name: {name}
 bases: {bases}
 namespaces: {namespaces}
 kwds: {kwds}
""")
        for key, value in kwds.items():
            namespaces[key] = value
        return super().__new__(cls, name, bases, namespaces)
    
    def __init__(self, name: str, bases: tuple[type, ...], dict: dict[str, Any], /, **kwds: Any) -> None:
        print(f"""
__init__
 cls: {self}
 name: {name}
 bases: {bases}
 dict: {dict}
 kwds: {kwds}
""")
        def repr(self) -> str:
            express = ", ".join(f"{key}={value!r}" for key, value in kwds.items())
            return f"({express.strip()})"
        setattr(self, "__repr__", repr)

class Foo:
    ...

class Bar(Foo, metaclass=Meta, x=-1, y=-1):
    ...

print(Bar())

Here, Meta serves as the metaclass for class Bar, while Foo is Bar's base class. When defining Bar, we also specify two keyword arguments. According to Meta's __new__ method definition, they become type field members. We create a Bar object and pass it as a parameter to the print method. Program execution produces the following output:

__new__
 cls: <class '__main__.Meta'>
 name: Bar
 bases: (<class '__main__.Foo'>,)
 namespaces: {'__module__': '__main__', '__qualname__': 'Bar', '__firstlineno__': 33, '__static_attributes__': ()}
 kwds: {'x': -1, 'y': -1}

__init__
 cls: <class '__main__.Bar'>
 name: Bar
 bases: (<class '__main__.Foo'>,)
 dict: {'__module__': '__main__', '__qualname__': 'Bar', '__firstlineno__': 33, '__static_attributes__': (), 'x': -1, 'y': -1}
 kwds: {'x': -1, 'y': -1}

(x=-1, y=-1)

Instances Are Created by Metaclasses: The Fundamental Truth

The extensive discussion above aims to illustrate the instance creation process: first call __new__ to construct a base object, then deliver this object to __init__ for initialization. However, this is merely the surface phenomenon, not the essence. The fundamental truth is: instances of a class are created by its metaclass.

When we speak of "using a metaclass object to construct," we essentially treat the metaclass as a factory function for instance construction. In Python's world, everything is an object. Functions are no exception—a function is an instance of the function class. The function (or callable object) class is a class possessing a __call__ method (either self-defined or inherited from a base class). Calling a function is本质上 calling the function object's __call__ method. When invoking a metaclass to construct instances in function form, it means instances are created through the __call__ method defined in the metaclass.

The following demonstration fully illustrates this point:

from typing import Any

class Foo:
    ...

class Meta(type):
    def __call__(self, *args: Any, **kwds: Any) -> Any:
        assert self is Bar
        assert type(self) is Meta
        assert args == ("111", "222")
        assert kwds == {"c": "333", "d": "444"}
        return Foo()

class Bar(metaclass=Meta):
    def __init__(self, a: str, b: str, **kwargs) -> None:
        self.x = a
        self.y = b
        self.kwargs = kwargs

assert isinstance(Bar("111", "222", c="333", d="444"), Foo)

As a method, its first parameter is always the subject object of the method call (for class methods, the subject is the class; for instance methods, the subject is the class's instance). From the assertions in the __call__ method, we can see that the subject object calling this method is the Bar class object itself. The type(self) returned is naturally the metaclass Meta that defines this class. The positional and keyword arguments injected into Bar("111", "222", c="333", d="444") are assigned to args and kwds in tuple and dictionary form respectively.

So why does calling Foo() return a Foo object for ordinary classes like Foo? The aforementioned rules still apply: although Foo doesn't specify a concrete metaclass, type serves as the fallback metaclass. The instance returned by Foo() is actually created by the __call__ method defined in type. This method constructs objects as follows:

class type:
    def __call__(self, *args: Any, **kwds: Any) -> Any:
        instance = self.__new__(self, *args, **kwds)  # type: ignore
        self.__init__(instance, *args, **kwds)
        return instance

This method creates objects by:

  1. First calling the class's __new__ to construct a base object
  2. Then passing that object as a parameter to the class's __init__ method for initialization, finally returning this object

We need not concern ourselves with whether a class defines __new__ and __init__, because the ultimate base class object provides fallback behavior:

  • A defined parameterless __new__ allocates memory as the created empty object
  • A defined parameterless __init__ does nothing

The Ultimate Metaclass: type in Depth

As the ultimate metaclass, type is also a class. As mentioned above: when we treat a class as an executable function, the metaclass's __call__ method is called behind the scenes. Since type's metaclass is itself, calling the type function is本质上 calling type's __call__ method.

type.__new__ Method

Let's first examine the logic of the __new__ method defined in the type class. This method is used to construct a class object. Our custom metaclass Meta's __new__ method ultimately calls this method (return super().__new__(cls)). The method signature is as follows:

class type:
    def __new__(
        cls: type[Self],
        name: str,
        bases: tuple[type, ...],
        namespace: dict[str, Any], /,
        **kwds: Any
    ) -> Self: ...

The five parameters are respectively:

  • cls: The metaclass constructing the class
  • name: Used to name the ultimately constructed class
  • bases: Base classes serving as the foundation for the constructed type
  • namespaces: Type members
  • kwds: Additional keyword arguments

If the cls parameter is the type class object itself, the constructed result is a conventional type. It is named by the name parameter, uses bases as base classes, and possesses type members defined by namespaces. The keyword arguments specified by the kwds parameter are meaningless. The following demonstration code illustrates this:

class Foo:
    x = -1

cls = type.__new__(type, "Bar", (Foo,), {"y": -1})
assert cls.__name__ == "Bar"
assert cls.__bases__ == (Foo,)
assert type(cls) is type

bar = cls()
assert bar.x == -1
assert bar.y == -1

Things become more interesting if the cls parameter specifies a custom metaclass. Since both metaclasses and type's __new__ method can create types, conflicts arise. Obviously, the latter has higher priority (after all, we explicitly called the __new__ method), so the custom metaclass's __new__ and __init__ will not take effect. The following demonstration program proves this:

class Baz:
    ...

log = []

class Meta(type):
    def __new__(cls, name: str, bases, namespaces, /, **kwds):
        log.append(f"Meta.__new__ is called")
        return Baz
    
    def __init__(self, *args, **kwargs):
        log.append(f"Meta.__init__ is called")
        self.z = -1

class Foo:
    x = -1

cls = type.__new__(Meta, "Bar", (Foo,), {"y": -1})

assert len(log) == 0
assert cls.__name__ == "Bar"
assert cls.__bases__ == (Foo,)
assert type(cls) is Meta

bar = cls()
assert bar.x == -1
assert bar.y == -1
assert not hasattr(bar, "z")

Although the metaclass-defined __new__ and __init__ do not take effect, the specified metaclass is indeed set as the generating type's metaclass (assert type(cls) is Meta). If we override the __call__ method in the metaclass, according to the rules introduced above: when we use the generated class for instantiation, this method is called behind the scenes. This is reflected in the following demonstration program:

class Baz:
    ...

class Meta(type):
    def __call__(self, *args, **kwargs):
        return Baz()

class Foo:
    x = -1

cls = type.__new__(Meta, "Bar", (Foo,), {"y": -1})
assert isinstance(cls(), Baz)

The type Function

As the ultimate metaclass, type is also a class. As mentioned above: when we treat a class as an executable function, the metaclass's __call__ method is called. Since type's metaclass is itself, calling the type function is本质上 calling type's __call__ method.

The type function's logic operates as follows:

  1. If a single object is passed, it returns the object's class. Specifically: if a conventional object is specified, its class is returned; if a class is specified, its metaclass is returned; if no metaclass is explicitly specified, the fallback metaclass type is naturally returned. This rule is well demonstrated in the following code:
class Meta(type):
    pass

class Foobar(metaclass=Meta):
    pass

assert type(Foobar()) is Foobar
assert type(Foobar) is Meta
assert type(type) is type
  1. If the single parameter requirement is not met, the type function ultimately creates a class, requiring input parameters依次为: class name, base classes, class members, keywords (adding type in front, exactly matching the __new__ method parameters):
class Base():
    foo = -1

cls = type("Foobar", (Base,), {"bar": -1})
assert cls.__name__ == "Foobar"
assert cls.__bases__ == (Base,)
assert cls.bar == -1  # type: ignore

instance = cls()
assert instance.foo == -1
assert instance.bar == -1  # type: ignore

Therefore, type's __call__ determines whether to return the class of the specified object or create a class based on the parameter format. The following definition approximates the real implementation:

class type:
    def __call__(self, *args, **kwargs):
        # Single parameter: return the parameter's class
        if len(args) == 1 and not kwargs:
            obj = args[0]
            return getattr(obj, "__class__", type(obj))
        
        # Multiple parameters: create a new class
        instance = self.__new__(self, *args, **kwargs)
        self.__init__(instance, *args, **kwargs)
        return instance

Reorganizing Class Generation and Instantiation: A Comprehensive Summary

Let me now summarize class generation and class-based instantiation:

For a class code snippet written using the class keyword, the Python interpreter constructs this class as follows:

  1. Extract the metaclass (if not explicitly specified, the type class serves as the metaclass), and call the metaclass's __new__ method with it and the class definition information (class name, base classes, all members defined for the class, and keyword arguments) as parameters:

    • For a custom metaclass, if it (or its custom base class) overrides the __new__ method, it can原则上 return any class object
    • If no metaclass is explicitly specified, or the specified custom metaclass (including its custom base class) does not override the __new__ method, then type class's __new__ method is called. According to the logic above, it strictly generates and returns the corresponding class object based on our definition

  2. Pass the class object returned by the __new__ method, along with the class definition information, as parameters to call the metaclass's __init__ method:

    • If the specified custom metaclass (or its custom base class) overrides the __init__ method, it can原则上 perform arbitrary processing on the passed class object if not using slots mode; once the class object is created based on slots mode, since the class's memory layout is fixed, class members cannot be added or deleted
    • If no metaclass is explicitly specified, or the specified custom metaclass (including its custom base class) does not override the __init__ method, then type class's __init__ method is called. This method is an empty method with no operations

When we use a class object as a factory function for instantiation, the metaclass's __call__ method is called, where the first parameter is the class object serving as the factory function:

  • If the specified custom metaclass (or its custom base class) overrides the __call__ method, it can原則上 return any object

  • If no metaclass is explicitly specified, or the specified custom metaclass (including its custom base class) does not override the __call__ method, then type class's __call__ method is called, reflecting the default instantiation process:

    • Call the class's __new__ method to construct a base object:

      • If the class (or its custom base class) overrides the __new__ method, it can原則上 return any object
      • Otherwise, the __new__ method inherited from the object class is used to create this base object

    • Pass the base object constructed by the __new__ method, along with specified parameters, into the class's __init__ method:

      • If the class (or its custom base class) overrides the __init__ method, it can原則上 perform any processing on the provided base object
      • Otherwise, the __init__ method inherited from object is called, but it does nothing

Metaclass Instance Methods: Becoming Class Methods

Since a metaclass's instances are the classes that have it as their metaclass, for classes created by it, instance methods defined in the metaclass become its class methods. Consider the following PointMeta metaclass as an example. parse is its instance method, but for Point which takes it as a metaclass, parse becomes its class method:

class PointMeta(type):
    def __new__(cls, name, bases, namespace, **kwds):
        namespace["x"] = 0
        namespace["y"] = 0
        return super().__new__(cls, name, bases, namespace)
    
    def parse(cls, s):
        x_str, y_str = s.split(",")
        point = cls()
        point.x = int(x_str)
        point.y = int(y_str)
        return point

class Point(metaclass=PointMeta):
    pass

p = Point.parse("1,2")
assert p.x == 1
assert p.y == 2

Determining Instance Type: The Memory Layout Perspective

Have you ever wondered how isinstance or the type function determines which class a specified object belongs to? Some might say: doesn't every object have a __class__ field? Correct, but where does this field originate?

Regardless of the programming language, an object always corresponds to one block (continuous) or multiple blocks (discontinuous) of memory space. All information provided by the object originates from this, including the class to which the object belongs. Therefore, understanding an object's memory layout reveals all information about the object. For a conventional object (not a class object—class object layout is much more complex), its memory layout depends on whether it uses slots mode.

Non-Slots Mode: Dynamic Dictionary Layout (Default)

This is Python's most commonly used mode, with flexibility as its core. Memory is laid out using the following structure:

  • PyObject Header: Contains reference count and a pointer to the type object
  • dict pointer: Points to a real Python dictionary object
  • weakref pointer: Used to support weak references

Dynamic capability is this layout's greatest advantage. Since attributes are not stored in the instance itself but in the dictionary pointed to by the __dict__ pointer, we can add arbitrary members at any time. However, the resulting problem is high memory overhead. Since dictionaries reserve space to reduce conflicts, each instance must additionally maintain a hash table object. Each data member positioning operation involves hashing, affecting access speed.

Slots Mode

This mode adopts a compact array layout. When we define __slots__ = ('a', 'b'), Python uses the following structure to lay out object memory:

  • PyObject Header: Reference count and type pointer
  • Fixed offset attribute slots: Directly reserve positions for a and b in memory (storing pointers to specific objects)

There is no __dict__ (unless you explicitly add 'dict' in slots). This is the memory layout mode adopted by statically compiled languages, resulting in instances only being able to possess members defined in slots. Attempting to add new attributes throws an AttributeError. Although dynamic capability is lost, the extremely compact memory layout removes the overhead of the entire dictionary object. When owning millions of small objects, memory usage typically decreases by 40%-70%. Attribute access becomes direct memory addressing with base address plus fixed offset, eliminating hash computation requirements and improving performance.

Returning to the title question: How to determine an instance's type? Simply put, regardless of which memory layout mode is adopted, it has a PyObject Header containing a pointer to the class object. This is the basis for judging object type. So in the aforementioned instantiation process, who writes this pointer?

Python's instantiation for a certain class always calls object's __new__ method. It calculates the memory size required for the object based on the specified type, allocates a block of memory of matching size, and then writes the class object's address into the type pointer in the PyObject Header. Some might ask: didn't we just say that non-slots mode object memory is dynamically allocated? While this is true, the dynamism here refers to the dictionary pointed to by __dict__. The memory here only includes the PyObject Header and two additional pointers.

class object:
    def __new__(cls) -> Self

Conclusion: The Beauty of Self-Consistent Design

Python's metamodel represents a triumph of language design—a self-referential, circular system where classes create instances, metaclasses create classes, and type creates everything while being its own meta. This elegant architecture enables powerful metaprogramming capabilities while maintaining conceptual consistency.

Understanding this system unlocks advanced Python techniques: custom metaclasses for framework development, automatic registration systems, ORM implementations, and sophisticated API design patterns. More importantly, it reveals the philosophical depth underlying Python's object model—a system where the creator and creation merge into a unified, self-sustaining whole.

For developers transitioning from other languages, grasping Python's metamodel may require shifting mental models. But once understood, it offers unparalleled flexibility and expressiveness, embodying Python's zen: "Simple is better than complex, but complex is better than complicated." The metamodel is complex, yes—but never complicated. Every piece serves a purpose, every relationship has meaning, and the whole forms a coherent, beautiful system.