X Tutup
Skip to content

bpo-44676: Add ability to serialise types.Union #27244

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ambv merged 3 commits into from
Jul 23, 2021
Merged

bpo-44676: Add ability to serialise types.Union #27244

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
03e7227
bpo-44676: Add ability to serialise types.Union
uriyyo Jul 19, 2021
432ef67
Make from_args protected
uriyyo Jul 20, 2021
3184435
Merge remote-tracking branch 'upstream/main' into fix-issue-44676
uriyyo Jul 23, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 35 additions & 0 deletions Lib/test/test_types.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,9 @@
import weakref
import typing


T = typing.TypeVar("T")

class Example:
pass

Expand Down Expand Up @@ -802,6 +805,38 @@ def eq(actual, expected):
eq(x[NT], int | NT | bytes)
eq(x[S], int | S | bytes)

def test_union_pickle(self):
alias = list[T] | int
s = pickle.dumps(alias)
loaded = pickle.loads(s)
self.assertEqual(alias, loaded)
self.assertEqual(alias.__args__, loaded.__args__)
self.assertEqual(alias.__parameters__, loaded.__parameters__)
Copy link
Copy Markdown
Contributor

@ambv ambv Jul 20, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Those are good checks 👍🏻
I'd also put a plain self.assertEqual(alias, loaded). At the moment it's redundant with comparing __args__ but this might not be true later, so this will keep future-proof.

Copy link
Copy Markdown
Member Author

@uriyyo uriyyo Jul 20, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I added self.assertEqual(alias, loaded) line to test.


def test_union_from_args(self):
with self.assertRaisesRegex(
TypeError,
r"^Each union argument must be a type, got 1$",
):
types.Union._from_args((1,))

with self.assertRaisesRegex(
TypeError,
r"Union._from_args\(\) argument 'args' must be tuple, not int$",
):
types.Union._from_args(1)

with self.assertRaisesRegex(ValueError, r"args must be not empty"):
types.Union._from_args(())

alias = types.Union._from_args((int, str, T))

self.assertEqual(alias.__args__, (int, str, T))
self.assertEqual(alias.__parameters__, (T,))

result = types.Union._from_args((int,))
self.assertIs(int, result)

def test_union_parameter_substitution_errors(self):
T = typing.TypeVar("T")
x = int | T
Expand Down
4 changes: 2 additions & 2 deletions Lib/typing.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -329,7 +329,7 @@ def _eval_type(t, globalns, localns, recursive_guard=frozenset()):
if isinstance(t, GenericAlias):
return GenericAlias(t.__origin__, ev_args)
if isinstance(t, types.Union):
return functools.reduce(operator.or_, ev_args)
return types.Union._from_args(ev_args)
else:
return t.copy_with(ev_args)
return t
Expand Down Expand Up @@ -1808,7 +1808,7 @@ def _strip_annotations(t):
stripped_args = tuple(_strip_annotations(a) for a in t.__args__)
if stripped_args == t.__args__:
return t
return functools.reduce(operator.or_, stripped_args)
return types.Union._from_args(stripped_args)

return t

Expand Down
2 changes: 2 additions & 0 deletions Misc/NEWS.d/next/Core and Builtins/2021-07-19-19-53-46.bpo-44676.WgIMvh.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Add ability to serialise ``types.Union`` objects. Patch provided by Yurii
Karabas.
51 changes: 51 additions & 0 deletions Objects/unionobject.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -299,6 +299,24 @@ is_unionable(PyObject *obj)
return 0;
}

static int
is_args_unionable(PyObject *args)
Copy link
Copy Markdown
Contributor

@ambv ambv Jul 20, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note to self: this is extracted from L450 below.

{
Py_ssize_t nargs = PyTuple_GET_SIZE(args);
for (Py_ssize_t iarg = 0; iarg < nargs; iarg++) {
PyObject *arg = PyTuple_GET_ITEM(args, iarg);
int is_arg_unionable = is_unionable(arg);
if (is_arg_unionable <= 0) {
if (is_arg_unionable == 0) {
PyErr_Format(PyExc_TypeError,
"Each union argument must be a type, got %.100R", arg);
}
return 0;
}
}
return 1;
}

PyObject *
_Py_union_type_or(PyObject* self, PyObject* other)
{
Expand Down Expand Up @@ -418,14 +436,47 @@ union_repr(PyObject *self)
return NULL;
}

static PyObject *
union_reduce(PyObject *self, PyObject *Py_UNUSED(ignored))
{
unionobject *alias = (unionobject *)self;
PyObject* from_args = PyObject_GetAttrString(self, "_from_args");
if (from_args == NULL) {
return NULL;
}

return Py_BuildValue("O(O)", from_args, alias->args);
Copy link
Copy Markdown
Member

@Fidget-Spinner Fidget-Spinner Jul 24, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pablogsal and @uriyyo the reference leak is here, we need to decref from_args. I am submitting a PR now.

Copy link
Copy Markdown
Member

@pablogsal pablogsal Jul 24, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I just catched it as well: #27332

Copy link
Copy Markdown
Member

@pablogsal pablogsal Jul 24, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @Fidget-Spinner for looking at it!

Copy link
Copy Markdown
Member Author

@uriyyo uriyyo Jul 24, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Fidget-Spinner thanks for solving this issue

}

static PyMemberDef union_members[] = {
{"__args__", T_OBJECT, offsetof(unionobject, args), READONLY},
{0}
};

static PyObject *
union_from_args(PyObject *cls, PyObject *args)
{
if (!PyTuple_CheckExact(args)) {
_PyArg_BadArgument("Union._from_args", "argument 'args'", "tuple", args);
return NULL;
}
if (!PyTuple_GET_SIZE(args)) {
PyErr_SetString(PyExc_ValueError, "args must be not empty");
return NULL;
}

if (is_args_unionable(args) <= 0) {
return NULL;
}

return make_union(args);
}

static PyMethodDef union_methods[] = {
{"_from_args", union_from_args, METH_O | METH_CLASS},
{"__instancecheck__", union_instancecheck, METH_O},
{"__subclasscheck__", union_subclasscheck, METH_O},
{"__reduce__", union_reduce, METH_NOARGS},
{0}};


Expand Down
X Tutup