Remove ttconv and implement Type-42 embedding using fontTools by jkseppan · Pull Request #20866 · matplotlib/matplotlib

jkseppan · 2021-08-20T12:39:43Z

PR Summary

ttconv is now only being used for outputting fonts in Type-42 format in PostScript files, and with fontTools this turns out to be quite doable in pure Python.

PR Checklist

Has pytest style unit tests (and pytest passes).
Is Flake 8 compliant (run flake8 on changed files to check).
[N/A] New features are documented, with examples if plot related.
Documentation is sphinx and numpydoc compliant (the docs should build without error).
Conforms to Matplotlib style conventions (install flake8-docstrings and run flake8 --docstring-convention=all).
[N/A] New features have an entry in doc/users/next_whats_new/ (follow instructions in README.rst there).
API changes documented in doc/api/next_api_changes/ (follow instructions in README.rst there).

jkseppan · 2021-08-21T09:59:44Z

Now this works with almost all TrueType fonts on my Mac, with the exception of CFF fonts (#20870) and Apple Color Emoji.ttc and NISC18030.ttf both of which cause a crash in FT2Font (#7305).

lib/matplotlib/backends/_backend_pdf_ps.py

QuLogic · 2021-12-24T00:31:36Z

lib/matplotlib/backends/_backend_pdf_ps.py

+    if fontfile.endswith('ttc'):
+        # fix this once we support loading more fonts from a collection
+        # https://github.com/matplotlib/matplotlib/issues/3135#issuecomment-571085541
+        options.font_number = 0


I think we test a ttc in CI already, so it could be used to test this line?

QuLogic · 2021-12-24T00:43:42Z

lib/matplotlib/backends/backend_ps.py

+        The Type-42 formatted font
+    """
+    version, breakpoints = _version_and_breakpoints(font.get('loca'), fontdata)
+    post, name = font['post'], font['name']


Are these tables guaranteed to exist?

Both post and name are required: https://docs.fileformat.com/font/ttf/

lib/matplotlib/backends/backend_ps.py

QuLogic · 2021-12-24T00:47:06Z

lib/matplotlib/backends/backend_ps.py

+    tuple
+        ((v1, v2), breakpoints) where v1 is the major version number,
+        v2 is the minor version number and breakpoints is a sorted list
+        of offsets into fontdata; if loca is not available, just the table
+        boundaries


For multiple return, you should list the entries separately, like Parameters.

lib/matplotlib/backends/backend_ps.py

lib/matplotlib/tests/test_backend_ps.py

tacaswell · 2021-12-24T01:16:21Z

Wow, I missed that this came in. 💯 on dropping an extension module!

jklymak · 2022-02-01T11:53:20Z

Dropping to draft until the comments and rebase are addressed. Anyone should feel free to move back to active...

QuLogic · 2022-03-15T02:04:09Z

Ping @jkseppan?

oscargus · 2022-03-15T09:04:44Z

lib/matplotlib/backends/_backend_pdf_ps.py

+
+    Returns
+    -------
+    fontTools.ttLib.ttFont.TTFont


Long term, it could make sense to link to the documentation? (Although fontTools is not in the intersphinx_mapping yet.)

tacaswell · 2022-12-09T20:31:51Z

We want to try and do mpl 3.7 feature freeze around Jan 1 so I think it is unlikely we will have the bandwidth to get this rebased and reviewed by then, pushing to 3.8.

jklymak · 2023-02-21T18:08:45Z

Do we have someone who can pick this up?

QuLogic · 2024-10-09T21:02:21Z

FYI, I think I'm okay with this as it stands, but as I did some followup cleanup work, I'm not sure if I should approve it myself.

oscargus

I am still open to the possibility to add fontTools to the intersphinx-stuff and provide a proper link, but I am also open to do that myself in another PR.

oscargus · 2024-10-10T07:13:28Z

Btw, do we (i.e. someone with a Mac(?)) know if #20866 (comment) still holds?

QuLogic · 2024-10-10T07:15:57Z

Btw, do we (i.e. someone with a Mac(?)) know if #20866 (comment) still holds?

Maybe @greglucas or @ksunden can have a look at it?

greglucas · 2024-10-10T15:09:48Z

Using the example from the linked issue:

# this needs the patch above

import matplotlib
from pathlib import Path
matplotlib.use('ps')
from matplotlib import pyplot as plt

# the following font is in CFF format on my Mac
font = Path('/System/Library/Fonts/AppleSDGothicNeo.ttc')

fig = plt.figure()
plt.axis((0,10,0,10))
plt.text(3,3,'Glib jocks quiz nymph to vex dwarf.', font=font)

matplotlib.rc('ps', fonttype=42)
plt.savefig('fmt42.ps')

matplotlib.rc('ps', fonttype=3)
plt.savefig('fmt3.ps')

The type42 is completely blank for me (no axes either). The type3 shows up as expected. I don't get an error though 🤷

tacaswell · 2024-10-11T19:32:58Z

That example fails for me on main. Trying to open the fmt42.ps with ghostscript gives me:

@ ghostscript fmt42.ps
GPL Ghostscript 10.04.0 (2024-09-18)
Copyright (C) 2024 Artifex Software, Inc.  All rights reserved.
This software is supplied under the GNU AGPLv3 and comes with NO WARRANTY:
see the file COPYING for details.
Error: /invalidfont in definefont
Operand stack:
   unknown   --dict:11/15(L)--   Font
Execution stack:
   %interp_exit   .runexec2   --nostringval--   definefont   --nostringval--   2   %stopped_push   --nostringval--   definefont   definefont   false   1   %stopped_push   1949   1   3   %oparray_pop   1948   1   3   %oparray_pop   1933   1   3   %oparray_pop   1803   1   3   %oparray_pop   --nostringval--   %errorexec_pop   .runexec2   --nostringval--   definefont   --nostringval--   2   %stopped_push   --nostringval--   1897   2   4   %oparray_pop
Dictionary stack:
   --dict:745/1123(ro)(G)--   --dict:0/20(G)--   --dict:86/200(L)--   --dict:8/10(L)--
Current allocation mode is local
Current file position is 11715
GPL Ghostscript 10.04.0: Unrecoverable error, exit code 1

and with this branch I get

@ ghostscript fmt42.ps
GPL Ghostscript 10.04.0 (2024-09-18)
Copyright (C) 2024 Artifex Software, Inc.  All rights reserved.
This software is supplied under the GNU AGPLv3 and comes with NO WARRANTY:
see the file COPYING for details.
Error: /invalidfont in definefont
Operand stack:
   AppleSDGothicNeo-Regular   --dict:12/16(L)--   Font
Execution stack:
   %interp_exit   .runexec2   --nostringval--   definefont   --nostringval--   2   %stopped_push   --nostringval--   definefont   definefont   false   1   %stopped_push   1949   1   3   %oparray_pop   1948   1   3   %oparray_pop   1933   1   3   %oparray_pop   1803   1   3   %oparray_pop   --nostringval--   %errorexec_pop   .runexec2   --nostringval--   definefont   --nostringval--   2   %stopped_push   --nostringval--   1897   2   4   %oparray_pop
Dictionary stack:
   --dict:745/1123(ro)(G)--   --dict:0/20(G)--   --dict:86/200(L)--   --dict:8/10(L)--
Current allocation mode is local
Current file position is 18773
GPL Ghostscript 10.04.0: Unrecoverable error, exit code 1

ksunden · 2024-10-11T19:38:08Z

Okay, looks like the answer is that this has not fixed #20866 (comment), however that is broken on main anyway, so I'd lean towards not blocking for merging this if we are otherwise satisfied.

tacaswell · 2024-10-11T20:01:17Z

The issue also appears to be font specific, dejavu being embedded as type42 works OK.

Split _backend_pdf_ps.get_glyphs_subset into two functions: get_glyphs_subset returns the font object (which needs to be closed after using) and font_as_file serializes this object into a BytesIO file-like object.

Move the test in lib/matplotlib/tests/test_ttconv.py to lib/matplotlib/tests/test_backend_pdf.py. Actually it has not been a test of ttconv for a while, since type-3 conversion has not been using ttconv.

Fonttools cannot subset these and drops them; avoid the warning.

Also don't end up in an infinite loop if there is a larger gap between breakpoints than we would like.

FT2Font throws RuntimeError for some fonts

Not the subsetted one, since in some cases fontTools.subset does not produce a good name table.

De-wrap a few lines now that we allow longer lines.

This was missed in matplotlib#20866

jkseppan added backend: ps topic: text/fonts labels Aug 20, 2021

jkseppan force-pushed the remove-ttconv branch from 447fd1b to a826065 Compare August 20, 2021 12:42

QuLogic added this to the v3.6.0 milestone Aug 20, 2021

jkseppan force-pushed the remove-ttconv branch from 8c44828 to 2535508 Compare August 21, 2021 09:55

jkseppan force-pushed the remove-ttconv branch from de16cf0 to daee4e5 Compare August 21, 2021 15:57

jkseppan marked this pull request as ready for review August 21, 2021 16:21

jkseppan force-pushed the remove-ttconv branch 2 times, most recently from 016713a to 7cea2d4 Compare August 22, 2021 14:20

QuLogic added the status: needs rebase label Dec 24, 2021

QuLogic reviewed Dec 24, 2021

View reviewed changes

jklymak added the status: needs revision label Feb 1, 2022

jklymak marked this pull request as draft February 1, 2022 11:52

QuLogic mentioned this pull request Mar 4, 2022

[MNT]: Replace str(n)cpy etc with safe versions (C++) #22603

Closed

oscargus reviewed Mar 15, 2022

View reviewed changes

QuLogic modified the milestones: v3.6.0, v3.7.0 Jul 5, 2022

tacaswell modified the milestones: v3.7.0, v3.8.0 Dec 9, 2022

tacaswell force-pushed the remove-ttconv branch from 7cea2d4 to a1b41fe Compare December 9, 2022 20:44

github-actions bot removed the status: needs rebase label Dec 9, 2022

github-actions bot added the status: needs rebase label Jan 21, 2023

anntzer mentioned this pull request Feb 20, 2023

Use pybind11 in ttconv module #25253

Merged

QuLogic mentioned this pull request Jun 7, 2023

replace ttconv for ps/pdf #12418

Closed

QuLogic modified the milestones: future releases, v3.10.0 Sep 21, 2024

QuLogic removed the status: needs revision label Sep 21, 2024

QuLogic force-pushed the remove-ttconv branch from 7f2b2a2 to 1393702 Compare September 26, 2024 09:25

oscargus approved these changes Oct 10, 2024

View reviewed changes

jkseppan and others added 10 commits October 16, 2024 04:04

Type-42 fonts using fontTools

e3bda66

Split _backend_pdf_ps.get_glyphs_subset into two functions: get_glyphs_subset returns the font object (which needs to be closed after using) and font_as_file serializes this object into a BytesIO file-like object.

Remove ttconv

786f202

Move the test in lib/matplotlib/tests/test_ttconv.py to lib/matplotlib/tests/test_backend_pdf.py. Actually it has not been a test of ttconv for a while, since type-3 conversion has not been using ttconv.

Add several more droppable tables

566c2a6

Fonttools cannot subset these and drops them; avoid the warning.

Don't crash if the loca table is missing

485e1b7

Also don't end up in an infinite loop if there is a larger gap between breakpoints than we would like.

Don't crash if glyph bounds cannot be found

f4a04c3

Log the name of unsupported fonts

94698fa

FT2Font throws RuntimeError for some fonts

Get names from the original font

c6ab62b

Not the subsetted one, since in some cases fontTools.subset does not produce a good name table.

Document ttconv removal

19e4679

Apply fixes from code review

9fa0b2b

Clean up some ttconv replacements

45a1654

De-wrap a few lines now that we allow longer lines.

QuLogic force-pushed the remove-ttconv branch from 1393702 to 45a1654 Compare October 16, 2024 08:04

ksunden approved these changes Oct 16, 2024

View reviewed changes

ksunden merged commit 5e5ce02 into matplotlib:main Oct 16, 2024

tacaswell added a commit to tacaswell/matplotlib that referenced this pull request Nov 7, 2024

MNT: remove _tconv.pyi

231f595

This was missed in matplotlib#20866

tacaswell mentioned this pull request Nov 7, 2024

MNT: remove _ttconv.pyi #29099

Merged

tacaswell added a commit to tacaswell/matplotlib that referenced this pull request Nov 7, 2024

MNT: remove _ttconv.pyi

f0f1b5c

This was missed in matplotlib#20866

jkseppan mentioned this pull request Jun 12, 2025

Update font-related documentation #30150

Merged

QuLogic mentioned this pull request Jul 1, 2025

Request to support PostScript-flavoured (CFF) OpenType fonts in PDF backend #2869

Open

Uh oh!

Conversation

jkseppan commented Aug 20, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Summary

PR Checklist

Uh oh!

jkseppan commented Aug 21, 2021

Uh oh!

Uh oh!

QuLogic Dec 24, 2021

Choose a reason for hiding this comment

Uh oh!

QuLogic Dec 24, 2021

Choose a reason for hiding this comment

Uh oh!

oscargus Sep 6, 2024

Choose a reason for hiding this comment

Uh oh!

Uh oh!

QuLogic Dec 24, 2021

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

tacaswell commented Dec 24, 2021

Uh oh!

jklymak commented Feb 1, 2022

Uh oh!

QuLogic commented Mar 15, 2022

Uh oh!

oscargus Mar 15, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tacaswell commented Dec 9, 2022

Uh oh!

jklymak commented Feb 21, 2023

Uh oh!

QuLogic commented Oct 9, 2024

Uh oh!

oscargus left a comment

Choose a reason for hiding this comment

Uh oh!

oscargus commented Oct 10, 2024

Uh oh!

QuLogic commented Oct 10, 2024

Uh oh!

greglucas commented Oct 10, 2024

Uh oh!

tacaswell commented Oct 11, 2024

Uh oh!

ksunden commented Oct 11, 2024

Uh oh!

tacaswell commented Oct 11, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

jkseppan commented Aug 20, 2021 •

edited

Loading

oscargus Mar 15, 2022 •

edited

Loading