Python has emerged as the predominant language in artificial intelligence and machine learning domains. However, from the perspective of developers experienced with statically-typed languages like C#, Python presents numerous design choices worthy of critical examination. Despite these observations, Python's metaclass-based metamodel represents an absolute highlight—a design decision that stands as perhaps the language's most elegant feature. This comprehensive exploration examines the intricacies of Python's metamodel architecture.

1. The Metaverse Analogy

To understand metaclasses, consider the concept of the metaverse that gained significant attention in recent years. From a "creation" perspective, the metaverse defines the laws governing our universe (reality). The universe itself represents an instance constructed from metadata. The terms "meta" and "instance" do not represent absolute concepts—the metaverse is itself a universe, with its laws defined by a higher-level "meta-metaverse," making the metaverse an instance of this meta-metaverse.

This abstraction continues indefinitely until reaching an ultimate origin where we cannot identify the source of laws. Some designate this as the "Creator," while Laozi referred to it as "Dao" (the Way). We might call this the "Genesis Universe." Unable to find a meta for the Genesis Universe, we treat it as its own meta, creating a self-consistent closed loop. Since we view the universe as an instance of its metaverse, the Genesis Universe's instances include itself.

The meta can be simpler than the instance—the principle of "great truths are simple" embodies this concept. The Dao generates One, One generates Two, Two generates Three, Three generates all things. Conversely, the meta can be more complex than the instance, containing intricate laws from which we select portions to construct instances.

Readers familiar with certain cultivation novels understand the concept of "Lower Realm Eight Domains" as a "dimension-reduced universe" constructed using the "Upper Realm" as its metaverse. Due to incomplete laws, cultivation in the Lower Realm can only reach the "Venerable Realm," preventing ascension to godhood.

Python's "metaverse" operates similarly. We use meta to define rules for instances, serving as a factory for creating instances. Thus, classes serve as the meta for instances. We can create classes using metaclasses, making regular classes instances of metaclasses, while metaclasses serve as the meta for regular classes. In this sense, "metaclass" might be more accurately termed "class-meta," but the term "metaclass" additionally expresses that metaclasses themselves are classes. Since metaclasses are classes, they naturally possess their own metaclasses. Therefore, class equals meta, and meta equals class—just as "the metaverse is also a universe, and a universe can serve as a metaverse."

Python's Genesis Universe is type. Our classes are constructed by it, making type the metaclass of classes. Since type is a metaclass, it possesses class attributes. Occupying this transcendent position in the Genesis Universe, type can be viewed as its own instance.

2. Metaclass Definition

By default, when using a class to construct instances, the underlying process follows these steps:

  1. Pass the class and parameters (including any added keyword arguments) to the class's __new__ method to create a base object
  2. Pass this base object and parameters to the class's __init__ method for initialization
  3. Return the initialized object

Consider the following Foobar class definition. Two methods of creating instances (foobar1 and foobar2) prove equivalent:

from typing_extensions import Self
from typing import Any

class Foobar:
    def __new__(cls, *args: Any, **kwargs: Any) -> Self:
        return super().__new__(cls)

    def __init__(self, foo: int, bar: int) -> None:
        self.foo = foo
        self.bar = bar
    
    def __eq__(self, value: object) -> bool:
        if not isinstance(value, Foobar):
            return NotImplemented
        return self.foo == value.foo and self.bar == value.bar

foobar1 = Foobar(foo=111, bar=222)
foobar2 = Foobar.__new__(Foobar, foo=111, bar=222)
Foobar.__init__(foobar2, foo=111, bar=222)
assert foobar1 == foobar2

Note that while __new__ appears similar to a class method, it is本质上 a static method. It lacks the standard @classmethod decorator, and the first parameter must be explicitly specified during invocation.

Similarly, as a class, a metaclass can define __new__ and __init__ methods. These methods in regular classes initialize their instances. For metaclasses, these methods serve the same purpose, except the metaclass's instances are classes that use it as their metaclass.

When the Python interpreter encounters a class definition with a specified metaclass, it employs a similar approach to create the class:

  1. Call the metaclass's __new__ method with fixed parameters to construct a class object:

    • Current metaclass
    • Class name
    • Class members represented as a dictionary
    • Keyword arguments specified during class definition

  2. Call the metaclass's __init__ method to initialize the base class object created above:

    • self is the class object
    • Remaining parameters include class name, class members dictionary, and keyword arguments

Consider the following metaclass Meta example. The defined __new__ and __init__ methods print all output parameters. In __new__, keyword arguments are added to the namespaces dictionary serving as class members. Finally, the base class's __new__ method creates the base class object. In __init__, a function repr is defined as a string output and set as the class's __repr__ method:

from typing import Any

class Meta(type):
    def __new__(cls, name: str, bases: tuple[type, ...], namespaces: dict[str, Any], /, **kwds: Any):
        print(f"""
__new__
 cls: {cls}
 name: {name}
 bases: {bases}
 namespaces: {namespaces}
 kwds: {kwds}
""")
        for key, value in kwds.items():
            namespaces[key] = value
        return super().__new__(cls, name, bases, namespaces)
    
    def __init__(self, name: str, bases: tuple[type, ...], dict: dict[str, Any], /, **kwds: Any) -> None:
        print(f"""
__init__
 cls: {self}
 name: {name}
 bases: {bases}
 dict: {dict}
 kwds: {kwds}
""")
        def repr(self) -> str:
            express = ", ".join(f"{key}={value!r}" for key, value in kwds.items())
            return f"({express.strip()})"
        setattr(self, "__repr__", repr)

class Foo:
    ...

class Bar(Foo, metaclass=Meta, x=-1, y=-1):
    ...

print(Bar())

Here, Meta serves as the metaclass for class Bar, while Foo is Bar's base class. During Bar's definition, two keyword arguments are specified. According to Meta's __new__ method definition, they become type field members. After creating the Bar object and passing it to the print method, program execution produces:

__new__
 cls: <class '__main__.Meta'>
 name: Bar
 bases: (<class '__main__.Foo'>,)
 namespaces: {'__module__': '__main__', '__qualname__': 'Bar', '__firstlineno__': 33, '__static_attributes__': ()}
 kwds: {'x': -1, 'y': -1}

__init__
 cls: <class '__main__.Bar'>
 name: Bar
 bases: (<class '__main__.Foo'>,)
 dict: {'__module__': '__main__', '__qualname__': 'Bar', '__firstlineno__': 33, '__static_attributes__': (), 'x': -1, 'y': -1}
 kwds: {'x': -1, 'y': -1}

(x=-1, y=-1)

3. Instances Are Created by Metaclasses

The extensive discussion above aimed to illustrate the class instance creation process: first calling __new__ to construct a base object, then passing this object to __init__ for initialization. However, this represents appearance rather than essence. The fundamental truth is: instances of a class are created by the metaclass.

When discussing "using metaclass objects to construct," we treat the metaclass as an instance construction factory function. In Python's world, everything is an object. Functions are no exception—a function is an instance of the function class. The function (or callable object) class possesses a __call__ method (either self-defined or inherited from base class). Calling a function本质上 means invoking the function object's __call__ method.

When invoking a metaclass to construct instances in function form, instances are created through the __call__ method defined in the metaclass. The following demonstration fully illustrates this point:

from typing import Any

class Foo:
    ...

class Meta(type):
    def __call__(self, *args: Any, **kwds: Any) -> Any:
        assert self is Bar
        assert type(self) is Meta
        assert args == ("111", "222")
        assert kwds == {"c": "333", "d": "444"}
        return Foo()

class Bar(metaclass=Meta):
    def __init__(self, a: str, b: str, **kwargs) -> None:
        self.x = a
        self.y = b
        self.kwargs = kwargs

assert isinstance(Bar("111", "222", c="333", d="444"), Foo)

As a method, its first parameter is always the calling subject object (class method's subject is the class, instance method's subject is the class instance). From the assertions in the __call__ method, the calling subject is the Bar class object. The type(self) returned naturally represents the metaclass Meta that defined this class. Positional and keyword arguments injected into Bar("111", "222", c="333", d="444") are assigned to args and kwds as tuple and dictionary respectively.

Why does calling Foo() return a Foo object for regular classes like Foo? The above rules still apply: although Foo doesn't specify a concrete metaclass, type serves as the fallback metaclass. The instance returned by Foo() is actually created by the __call__ method defined in type, which operates as follows:

class type:
    def __call__(self, *args: Any, **kwds: Any) -> Any:
        instance = self.__new__(self, *args, **kwds)
        self.__init__(instance, *args, **kwds)
        return instance

This method creates objects through:

  1. First calling the class's __new__ to construct a base object
  2. Then passing this object as a parameter to the class's __init__ method for initialization, finally returning this object

We needn't consider whether the class defines __new__ and __init__, as the ultimate base class object provides fallback:

  • A defined parameterless __new__ allocates memory as the created empty object
  • A defined parameterless __init__ does nothing

4. The Ultimate Metaclass: type

The __call__ method logic discussed above, along with the type class itself, extends far beyond this simple explanation.

4.1 The type.__new__ Method

First, let's examine the logic of the __new__ method defined in the type class. This method constructs a class object. The __new__ method of our custom metaclass Meta ultimately calls this method (return super().__new__(cls)). The method signature appears as follows:

class type:
    def __new__(
        cls: type[Self],
        name: str,
        bases: tuple[type, ...],
        namespace: dict[str, Any],
        /,
        **kwds: Any
    ) -> Self:
        ...

The five parameters are:

  • cls: The metaclass constructing the class
  • name: Names the ultimately constructed class
  • bases: Base classes for constructing the type
  • namespaces: Type members
  • kwds: Additional keyword arguments

If the cls parameter is the type class object itself, the constructed result is a regular type. It's named by the name parameter, uses bases as base classes, and possesses type members defined by namespaces. Keyword arguments specified by the kwds parameter hold no significance, as demonstrated:

class Foo:
    x = -1

cls = type.__new__(type, "Bar", (Foo,), {"y": -1})
assert cls.__name__ == "Bar"
assert cls.__bases__ == (Foo,)
assert type(cls) is type

bar = cls()
assert bar.x == -1
assert bar.y == -1

Specifying a custom metaclass for the cls parameter becomes more interesting. Since both metaclasses and type's __new__ method can create types, conflict arises. Clearly, the latter holds higher priority (since we explicitly called __new__), so the custom metaclass's __new__ and __init__ won't take effect. The following demonstration proves this:

class Baz:
    ...

log = []

class Meta(type):
    def __new__(cls, name: str, bases, namespaces, /, **kwds):
        log.append(f"Meta.__new__ is called")
        return Baz
    
    def __init__(self, *args, **kwargs):
        log.append(f"Meta.__init__ is called")
        self.z = -1

class Foo:
    x = -1

cls = type.__new__(Meta, "Bar", (Foo,), {"y": -1})

assert len(log) == 0
assert cls.__name__ == "Bar"
assert cls.__bases__ == (Foo,)
assert type(cls) is Meta

bar = cls()
assert bar.x == -1
assert bar.y == -1
assert not hasattr(bar, "z")

Although the metaclass-defined __new__ and __init__ won't take effect, the specified metaclass is indeed set as the generated type's metaclass (assert type(cls) is Meta). If we override the __call__ method in the metaclass, according to the rules introduced above: when instantiating using the generated class, this method is called behind the scenes, as reflected in the following demonstration:

class Baz:
    ...

class Meta(type):
    def __call__(self, *args, **kwargs):
        return Baz()

class Foo:
    x = -1

cls = type.__new__(Meta, "Bar", (Foo,), {"y": -1})
assert isinstance(cls(), Baz)

4.2 The type Function

As the ultimate metaclass, type is also a class. As mentioned above: when treating a class as an executable function, the metaclass's __call__ method is called behind the scenes. Since type's metaclass is itself, calling the type function本质上 means invoking type's __call__ method.

The type function's logic operates as follows:

  • Single Object Input: Returns the object's class. Specifically, specifying a regular object returns that object's class; specifying a class returns the metaclass; if no metaclass is explicitly specified, the fallback metaclass type is naturally returned. This rule is reflected in the following demonstration:
class Meta(type):
    pass

class Foobar(metaclass=Meta):
    pass

assert type(Foobar()) is Foobar
assert type(Foobar) is Meta
assert type(type) is type
  • Multiple Parameters: If the single parameter requirement isn't met, the type function ultimately creates a class, requiring input parameters in order: class name, base classes, class members, keywords (adding type in front matches the __new__ method parameters exactly):
class Base():
    foo = -1

cls = type("Foobar", (Base,), {"bar": -1})
assert cls.__name__ == "Foobar"
assert cls.__bases__ == (Base,)
assert cls.bar == -1

instance = cls()
assert instance.foo == -1
assert instance.bar == -1

Therefore, type's __call__ determines whether to return the specified object's class or create a new class based on parameter format. The following definition更接近 the actual implementation:

class type:
    def __call__(self, *args, **kwargs):
        # Single parameter, return parameter's type
        if len(args) == 1 and not kwargs:
            obj = args[0]
            return getattr(obj, "__class__", type(obj))
        
        # Multiple parameters, create a new class
        instance = self.__new__(self, *args, **kwargs)
        self.__init__(instance, *args, **kwargs)
        return instance

5. Reorganizing Class Generation and Instantiation

A summary of class generation and class-based instantiation follows:

For a class code snippet written using the class keyword, the Python interpreter constructs this class as follows:

  1. Extract the metaclass (if not explicitly specified, the type class serves as metaclass), and call the metaclass's __new__ method with it and the class definition information (class name, base classes, all members defined for the class, and keyword arguments) as parameters:

    • For custom metaclasses, if it (or its custom base class) overrides the __new__ method, it can原则上 return any class object
    • If no metaclass is explicitly specified, or the specified custom metaclass (including its custom base class) doesn't override __new__, then type class's __new__ method is called. According to the logic above, it strictly generates and returns the corresponding class object based on our definition

  2. Call the metaclass's __init__ method with the class object returned by __new__ and class definition information as parameters:

    • If the specified custom metaclass (or its custom base class) overrides the __init__ method, it can原则上 perform arbitrary processing on the passed class object without using slots mode
    • Once the class object is created based on slots mode, since the class's memory layout is fixed, class members cannot be added or deleted
    • If no metaclass is explicitly specified, or the specified custom metaclass (including its custom base class) doesn't override __init__, then type class's __init__ method is called. This method is empty with no operations

When instantiating by treating the class object as a factory function, the metaclass's __call__ method is invoked, with the first parameter being the class object serving as the factory function:

  • If the specified custom metaclass (or its custom base class) overrides the __call__ method, it can原則上 return any object

  • If no metaclass is explicitly specified, or the specified custom metaclass (including its custom base class) doesn't override __call__, then type class's __call__ method is called, reflecting the default instantiation process:

    • Call the class's __new__ method to construct a base object:

      • If the class (or its custom base class) overrides __new__, it can原則上 return any object
      • Otherwise, the __new__ method inherited from the object class creates this base object

    • Pass the base object constructed by __new__, along with specified parameters, to the class's __init__ method:

      • If the class (or its custom base class) overrides __init__, it can原則上 perform any processing on the provided base object
      • Otherwise, the __init__ inherited from object is called, but it does nothing

6. Metaclass Instance Methods

Since the metaclass's instances are classes that use it as their metaclass, for classes created by it, instance methods defined in the metaclass become its class methods. Consider the following PointMeta metaclass example, where parse is its instance method. However, for Point which uses it as a metaclass, parse becomes its class method:

class PointMeta(type):
    def __new__(cls, name, bases, namespace, **kwds):
        namespace["x"] = 0
        namespace["y"] = 0
        return super().__new__(cls, name, bases, namespace)

    def parse(cls, s):
        x_str, y_str = s.split(",")
        point = cls()
        point.x = int(x_str)
        point.y = int(y_str)
        return point

class Point(metaclass=PointMeta):
    pass

p = Point.parse("1,2")
assert p.x == 1
assert p.y == 2

7. Determining Instance Types

Have you ever considered how isinstance or type functions determine an object's class when called? Some might say doesn't every object have a __class__ attribute? Correct, but where does this attribute originate?

Regardless of programming language, an object always corresponds to one (contiguous) or multiple (non-contiguous) memory spaces. All information provided by the object originates from here, including the class to which the object belongs. Therefore, understanding an object's memory layout reveals all its information. For regular objects (not class objects, as class object layouts are far more complex), memory layout depends on whether slots mode is employed.

7.1 Non-Slots Mode: Dynamic Dictionary Layout (Default)

This represents Python's most commonly used mode, with flexibility as its core. Memory adopts the following structure:

  • PyObject Header: Contains reference count and pointer to type object
  • dict Pointer: Points to a real Python dictionary object
  • weakref Pointer: Supports weak references

Dynamic capability represents this layout's greatest advantage. Since attributes aren't stored in the instance itself but in the dictionary pointed to by __dict__, we can add arbitrary members at any time. However, the tradeoff involves significant memory overhead. Since dictionaries reserve space to reduce conflicts, each instance must additionally maintain a hash table object. Hash operations performed during each data member lookup also impact access speed.

7.2 Slots Mode

This mode employs compact array layout. When defining __slots__ = ('a', 'b'), Python adopts the following structure for object memory:

  • PyObject Header: Reference count and type pointer
  • Fixed Offset Attribute Slots: Directly reserve positions for a and b in memory (storing pointers to specific objects)

No __dict__ exists (unless you explicitly add 'dict' to slots). This represents the memory layout adopted by statically compiled languages, resulting in instances only possessing members defined in slots. Attempting to add new attributes raises AttributeError.

While sacrificing dynamic capability, the extremely compact memory layout eliminates the entire dictionary object's overhead. When owning millions of small objects, memory usage typically decreases by 40%-70%. Attribute access becomes base address plus fixed offset direct memory addressing, eliminating hash calculation requirements and improving performance.

Returning to the title question: How to determine an instance's type? Simply put, regardless of which memory layout is adopted, all possess a PyObject Header containing a pointer to the class object. This serves as the basis for judging object types. In the instantiation process described above, who writes this pointer?

Python's instantiation for a certain class always calls object's __new__ method. It calculates the memory size required by the object based on the specified type, allocates a matching memory segment, then writes the class object's address into the type pointer in PyObject Header.

Some might ask: wasn't it just stated that non-slots mode object memory is dynamically allocated? While correct, the dynamism here refers to the dictionary pointed to by __dict__. The memory here only includes PyObject Header and two additional pointers.

class object:
    def __new__(cls) -> Self:
        # Allocates memory and writes type pointer to PyObject Header
        ...

Conclusion

Python's metaclass system represents one of the language's most sophisticated design decisions. By treating classes as instances of metaclasses, Python achieves remarkable flexibility in type creation and manipulation. Understanding this system unlocks powerful metaprogramming capabilities while providing deeper insight into Python's object model internals.

The metaclass architecture enables developers to customize class creation behavior, implement design patterns at the type level, and create sophisticated frameworks that would be impossible with conventional class definitions alone. While metaclasses introduce complexity, they reward careful study with unparalleled control over Python's type system.