Skip to content

gh-142418 Enhance markcoroutinefunction to support functools.partial #142505

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
chaope wants to merge 1 commit into
base: main
Choose a base branch
from
+14 −0
Open

gh-142418 Enhance markcoroutinefunction to support functools.partial #142505

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Lib/inspect.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -317,6 +317,7 @@ def markcoroutinefunction(func):
"""
if hasattr(func, '__func__'):
func = func.__func__
func = functools._unwrap_partial(func)
Copy link

@x42005e1f x42005e1f Dec 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please read the example in the original issue carefully. The original function may not always return a coroutine object depending on the parameters passed. Marking it will break the expected behavior because the return type of the partial function and the original function are different.

Copy link
Author

@chaope chaope Dec 10, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please read the example in the original issue carefully. The original function may not always return a coroutine object depending on the parameters passed. Marking it will break the expected behavior because the return type of the partial function and the original function are different.

My understanding of the design of markcoroutinefunction and iscoroutinefunction is that it gives you a way to mark a function as coroutine, but it doesn't do any check to prevent you from falsely mark a random object as coroutine. We can have another issue discussing the design of it and if we do want to change the design of it, maybe we need a PEP.

But the issue you reported indeed reveals a bug that after a function is marked as coroutine, it is not correctly recognized by iscoroutinefunction (which is against what markcoroutinefunction and iscoroutinefunction are designed for).

To demonstrate that markcoroutinefunction can be applied to random object in python, you can try the following code, the object doesn't even need to be a callable.

>>> from inspect import iscoroutinefunction, markcoroutinefunction
>>> 
>>> class Object(object):
...     pass
... 
>>> o = Object()
>>> o = markcoroutinefunction(o)
>>> iscoroutinefunction(o)
True

Copy link

@x42005e1f x42005e1f Dec 10, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My understanding of the design of markcoroutinefunction and iscoroutinefunction is that it gives you a way to mark a function as coroutine, but it doesn't do any check to prevent you from falsely mark a random object as coroutine. We can have another issue discussing the design of it and if we do want to change the design of it, maybe we need a PEP.

How does this relate to my comment? If you unwrap an object in markcoroutinefunction(), you do not mark the passed object (instead, you mark the one it references). And this violates both the expected and current behavior.

Copy link
Author

@chaope chaope Dec 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As markcoroutinefunction already has

    if hasattr(func, '__func__'):
        func = func.__func__

I don't think marking the passed object is the expected behavior.

When we do markcoroutinefunction for object A, the expected behavior is that iscoroutinefunction returns True for object A.

current iscoroutinefunction code does the unwrap, which means that all of the partial function sharing the same base function should return same result for iscoroutinefunction, I think this is the right behavior, and I want to preserve it, so that my change is done on markcoroutinefunction.

Copy link

@x42005e1f x42005e1f Dec 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Current behavior:

>>> def f(): ...
>>> g = markcoroutinefunction(partial(f))
>>> iscoroutinefunction(g)
False
>>> iscoroutinefunction(f)
False

Expected behavior:

>>> def f(): ...
>>> g = markcoroutinefunction(partial(f))
>>> iscoroutinefunction(g)
True
>>> iscoroutinefunction(f)
False

Your behavior:

>>> def f(): ...
>>> g = markcoroutinefunction(partial(f))
>>> iscoroutinefunction(g)
True
>>> iscoroutinefunction(f)
True

Copy link

@x42005e1f x42005e1f Dec 10, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please note that the main difference between method objects and partial objects is that the former cannot change the return value (they expect the same parameters, except for self), while the latter can narrow it down (since they can directly affect the parameters).

(However, in special cases, method objects can also narrow the return type, but these are very exotic cases, which cannot be said about partial objects).

Copy link

@x42005e1f x42005e1f Dec 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

current iscoroutinefunction code does the unwrap, which means that all of the partial function sharing the same base function should return same result for iscoroutinefunction, I think this is the right behavior, and I want to preserve it, so that my change is done on markcoroutinefunction.

Let me clarify this point. The reason why iscoroutinefunction() returns True for any wrapping partial object may be that the marked function already has CoroutineType as its return type (you can see this even in the typeshed annotations). No matter how much we narrow this type, it will still remain a coroutine. And this rule must not be broken, otherwise we may get type errors that cannot be detected statically.

Copy link
Author

@chaope chaope Dec 10, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand your concern and the expected behavior you provided. Using the variable name in your issue, ideally, iscoroutinefunction(manufacturer_of_jokes) should not be True.

But then it's a question of if we want to unwrap partial function in iscoroutinefunction.

  • If we don't unwrap, then the specific issue can be resolved by only looking at the object itself. But that will almost defeat the purpose of iscoroutinefunction.
  • If we unwrap, then your joke can again be wrapped in partial function, what should the wrapper function return then?
from functools import partial
from inspect import iscoroutinefunction, markcoroutinefunction

async def wedonotlikesnakecase():
    return "the_funniest_joke_in_the_world"

def sync_func():
    return "sync_func"

def manufacturer_of_jokes(somefunc, another_func=None):
    global manufacturer_of_jokes
    del manufacturer_of_jokes
    if another_func is not None:
        return another_func()
    return somefunc()

joke = partial(manufacturer_of_jokes, wedonotlikesnakecase)
joke = markcoroutinefunction(joke)
joke1 = partial(joke, wedonotlikesnakecase) # This will be coroutine.
joke2 = partial(joke, sync_func)            # This will not be coroutine.
print(iscoroutinefunction(joke1), iscoroutinefunction(joke2))

Copy link

@x42005e1f x42005e1f Dec 10, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we don't unwrap, then the specific issue can be resolved by only looking at the object itself. But that will almost defeat the purpose of iscoroutinefunction.

A solution is to check the marker for each unwrapped object (including the passed object itself, if it is not a method object). You can see this in the code attached to the original issue and in the parallel PR. Once the marker is found, any subsequent narrowed type is guaranteed to be CoroutineType (as long as the function is marked correctly).

If we unwrap, then your joke can again be wrapped in partial function, what should the wrapper function return then?

See the answer above. All explained via the type narrowing principle. Your joke() has an arbitrary return type (Any) because it can accept an arbitrary function whose value it returns when passed (applying markcoroutinefunction() to it is a big mistake). joke1() is a coroutine function and should be marked accordingly. joke2() is a string function and should not be considered a coroutine function. Your behavior violates the latter.

Copy link

@x42005e1f x42005e1f Dec 10, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If iscoroutinefunction() returns True, this should be read as "the return type is always CoroutineType (or one of its subclasses)" rather than "the return type may be CoroutineType (or one of its subclasses)". Imagine you are doing the following:

if iscoroutinefunction(func):
    await func(*args, **kwargs)

This is error-prone if the return value is not always a coroutine (and in general, what is the point of different behavior, why would iscoroutinefunction() be needed at all if it does not provide such a guarantee as long as functions are marked correctly?).

func._is_coroutine_marker = _is_coroutine_mark
return func

Expand Down
11 changes: 11 additions & 0 deletions Lib/test/test_inspect/test_inspect.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -334,6 +334,17 @@ def do_something_static():
self.assertTrue(inspect.iscoroutinefunction(Cl3.do_something_classy))
self.assertTrue(inspect.iscoroutinefunction(Cl3.do_something_static))

# Test markcoroutinefunction with functools.partial
async def _fn4():
pass

def fn4():
return _fn4()

partial_fn4 = functools.partial(fn4)
marked_partial_fn4 = inspect.markcoroutinefunction(partial_fn4)
self.assertTrue(inspect.iscoroutinefunction(marked_partial_fn4))

self.assertFalse(
inspect.iscoroutinefunction(unittest.mock.Mock()))
self.assertTrue(
Expand Down
2 changes: 2 additions & 0 deletions Misc/NEWS.d/next/Library/2025-12-10-02-31-08.gh-issue-142418.XLGNh3.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Fix markcoroutinefunction by letting it set the coroutine function marker on
underlying function correctly.
Loading