[tex-live] (x)dvipdfmx and fully embeded fonts in pdftex.map

[Ulrike Fischer]

In texlive xdvipdfmx and dvipdfmx use pdftex.map as fontmap.

But pdftex.map contains map entries which uses << to embed a font
fully and neither xdvipdfmx nor dvipdfmx seems to like this as
discovered by this question:

http://tex.stackexchange.com/questions/183000/how-to-get-adobe-garamond-pro-to-work-for-mathdesign-on-xelatex

A dvipdfmx example is this

\documentclass{article}
\begin{document}
\font\test=augie8r
\test This is a test.
\end{document}

A compilation with latex + dvipdfmx fails with

[1kpathsea:make_tex: Invalid fontname `<augie___.pfb', contains '<'

Changing the entry to

augie8r Augie " TeXBase1Encoding ReEncodeFont " <8r.enc
<augie___.pfb

corrects the problem.

But in the case of the adobe garamondm utopia, GFSBodoni and other
fonts this it not an option: Here otf-fonts are loaded and pdftex
can handle them only if they are fully embeded.

Imho one must overload this map entries in dvipdfmx.cfg to allow
xelatex to use them.

(miktex still uses a dedicated dvipdfm.map and so is not affected).

[Ulrike Fischer]

Am Wed, 4 Jun 2014 11:15:06 +0200 schrieb Ulrike Fischer:

Addition:

Even if I correct the map-file xelatex can't use the fonts. It
always start mktexpk:

fontmap: mdwgdr8t -> AGaramondPro-Regular.otf(a_csqf63.enc)
(Encoding:a_csqf63.enc[AutoEnc_csqf633h5kovjwhnfrdbqmo3bg])
kpathsea: Running mktexpk --mfmode / --bdpi 600 --mag 1+0/600 --dpi
600 AGaramon
dPro-Regular.otf

The command name is D:\texlive\2014\bin\win32\mktexpk

kpathsea: Running mktexmf AGaramondPro-Regular.mf


But latex + dvipdfmx works find and embeds the fonts. What's wrong
with xdvipdfmx?

[Ulrike Fischer]

Post by Ulrike Fischer
Even if I correct the map-file xelatex can't use the fonts. It
fontmap: mdwgdr8t -> AGaramondPro-Regular.otf(a_csqf63.enc)
(Encoding:a_csqf63.enc[AutoEnc_csqf633h5kovjwhnfrdbqmo3bg])
kpathsea: Running mktexpk --mfmode / --bdpi 600 --mag 1+0/600 --dpi
600 AGaramon
dPro-Regular.otf
The command name is D:\texlive\2014\bin\win32\mktexpk
kpathsea: Running mktexmf AGaramondPro-Regular.mf
But latex + dvipdfmx works find and embeds the fonts. What's wrong
with xdvipdfmx?

I build a test example to analyse the problem:

1. in pdftex.map I changed the Skaknew-Figure entry to:

SkakNew-Figurine SkakNew-Figurine " TeXnANSIEncoding ReEncodeFont "
<texnansi.enc <SkakNew-Figurine.otf

2. Then I compiled this example

\documentclass{article}
\begin{document}
\font\test=SkakNew-Figurine
\test K Q
\end{document}

xetex + xdvipdfmx fails.
latex + dvipdfmx works fine.

When I set --kpathsea-debug -1 in dvipdfmx and xdvipdfmx one can see
the reason:

dvipdfmx:
kdebug:kpse_find_file: searching for SkakNew-Figurine.otf of type
opentype fonts (from texmf.cnf)

xdvipdfmx:
kdebug:kpse_find_file: searching for SkakNew-Figurine.otf of type
tfm (from texmf.cnf)

So in xdvipdfmx otf has the wrong type. And bingo: If I move the otf
to fonts/tfm it is found.

So who can correct the type for otf (and ttf probably too) in
xdvipdfmx?

[Ulrike Fischer]

Post by Ulrike Fischer
kdebug:kpse_find_file: searching for SkakNew-Figurine.otf of type
opentype fonts (from texmf.cnf)
kdebug:kpse_find_file: searching for SkakNew-Figurine.otf of type
tfm (from texmf.cnf)
So in xdvipdfmx otf has the wrong type. And bingo: If I move the otf
to fonts/tfm it is found.

I think I misinterpreted: that it seemed worked by moving the file
was only a side effect. The main reason is that it doesn't work
xdvipdfmx seems to be that it doesn't look for open type fonts.

[Akira Kakuto]

Dear Ulrike,

Related:

psfonts.map contains lines such as
md-grbb7m GFSBodoni-Bold " AutoEnc_2rwgawua6ejo6lcmev7rxg6fvb ReEncodeFont " <[a_2rwgaw.enc <<GFSBodoniBold.otf

AFAIK dvips cannot download open type fonts.
Thus the lines like above may not be intended
to use by dvips.
There is LCDF type tools to help in such cases.

IMHO latex+dvipdfmx supports native open type fonts
in map files. This fact has been used in
Japanese type setting, e.g.,
rml H KozMinPr6N-Regular.otf

Can other cases use open type fonts in map files directly?

Best,
Akira

[Ulrike Fischer]

Post by Akira Kakuto
psfonts.map contains lines such as
md-grbb7m GFSBodoni-Bold " AutoEnc_2rwgawua6ejo6lcmev7rxg6fvb ReEncodeFont " <[a_2rwgaw.enc <<GFSBodoniBold.otf
AFAIK dvips cannot download open type fonts.
Thus the lines like above may not be intended
to use by dvips.

No. The fonts will not work with dvips. But as long as one doesn't
try to use the fonts such map entries don't do much harm. (The same
is naturally true for the AGaramond entries: As long as xdvipdfmx
doesn't try to use them ...)

Post by Akira Kakuto
There is LCDF type tools to help in such cases.

Yes, but it is naturally more work. Also it is not quite clear to me
if it is always allowed to convert the fonts to other font types.
And imho LCDF only works with postscript flavour open type and not
with true type.

Post by Akira Kakuto
IMHO latex+dvipdfmx supports native open type fonts
in map files.

Yes, when the map-entries are correct (no <<) then they work fine
with latex+dvipdfmx. But xdvipdfmx simply doesn't look for them.
(That's a pity as you can reencode fonts in such map-entries).

Post by Akira Kakuto
Can other cases use open type fonts in map files directly?

pdflatex + lualatex can do it. (But at least pdflatex only if the
fonts are embedded completly with <<).

[Karl Berry]

Hi Ulrike,

if it is always allowed to convert the fonts to other font types.

I realize you were talking about a font not in TL (AGaramond), but just
to be clear: for all fonts distributed with TeX Live, it is most
certainly allowed (barring licensing bugs), although sometimes names
have to be changed.

I hope/trust Khaled, Akira, Peter, me, or someone, will fix the
xdvipdfmx lookup problem. Not for TL14 though.

Thanks,
Karl

[Karl Berry]

But pdftex.map contains map entries which uses << to embed a font
fully and neither xdvipdfmx nor dvipdfmx seems to like this as

Thanks for the report. Seems it is another deficiency that needs to be
fixed in the code.

Thanks,
k

[Karl Berry]

kdebug:kpse_find_file: searching for SkakNew-Figurine.otf of type
tfm (from texmf.cnf)

So in xdvipdfmx otf has the wrong type. And bingo: If I move the otf
to fonts/tfm it is found.

Puzzling. I see a different problem. Although my xdvipdfmx does try to
look up the TFM, I thought it really was looking for (and finding) a tfm
there. Also, in my output, the extension has been removed:
kdebug:kpse_find_file: searching for SkakNew-Figurine of type tfm (from
texmf.cnf)

Maybe your result, and the different diagnostics, are due to differences
between w32tex (= where the xdvipdfmx Windows binary in TL came from)
and the Unix TL binaries? Akira?

Anyway, I see plenty of not-critical issues in the filename lookups, but
ultimately, on i386-linux, it says it is finding the right file:
kdebug:fopen(/home/texlive/karl/Master/texmf-dist/fonts/opentype/public/skaknew/SkakNew-Figurine.otf, rb) => 0x9957190

But then it immediately aborts:
xdvipdfmx:fatal: Not a CFF/OpenType font?

dvipdfmx, on the other hand, finds the same file and does not abort.
Khaled?

Looking at the source, I see lots of xdvipdfmx vs. dvipdfmx differences
in this area. And unfortunately the same error message is repeated in
several places, so it's not immediately evident where it's coming from
(error msgs should be uniqified...). I'm not up for running it under
the debugger tonight.

I simplified the input file to use plain tex, called it tryot.tex:
\nopagenumbers \font\test = SkakNew-Figurine \test K \end

Then create a ./pdftex.map containing the line:
SkakNew-Figurine SkakNew-Figurine <SkakNew-Figurine.otf
(the reencode is irrelevant here so far as I can see)

Then run:
xetex --no-pdf tryot.tex
env KPATHSEA_DEBUG=-1 xdvipdfmx tryot.xdv

(My [x]dvipdfmx doesn't recognize --kpathsea-debug, don't know
when or why that happened; another separate thing to deal with.)

And I get the fatal error above.

karl

[Karl Berry]

[1kpathsea:make_tex: Invalid fontname `<augie___.pfb', contains '<'

It is easy enough to change (x)dvipdfmx to accept and ignore the extra <
in what is supposed to mean full embedding in a map file line
(fontmap_parse_mapdef_dps in fontmap.c, looks like).

The bigger question is whether (x)dvipdfmx actually needs to force full
embedding in that case. That does not look so simple to me, but your
message also implies it is not needed, that you are just trying to work
around syntactic incompatibilities for the sake, ultimately, of pdftex.

But in the case of the adobe garamondm utopia, GFSBodoni and other
fonts this it not an option: Here otf-fonts are loaded

gfsbodoni has pfb files, and gfsbodoni.map loads them.
So I don't get that one.

and pdftex can handle them only if they are fully embeded.

On another front, it seems that it would be desirable for pdftex to
fully download otf files entirely, regardless of the map file spec,
instead of just reporting an error and giving up. I admit it's not
clear to me when one would want to play with otf files in pdftex in the
first place, but fine.

thanks,
k

[Ulrike Fischer]

Post by Ulrike Fischer
[1kpathsea:make_tex: Invalid fontname `<augie___.pfb', contains '<'
It is easy enough to change (x)dvipdfmx to accept and ignore the extra <
in what is supposed to mean full embedding in a map file line
(fontmap_parse_mapdef_dps in fontmap.c, looks like).
The bigger question is whether (x)dvipdfmx actually needs to force full
embedding in that case. That does not look so simple to me, but your
message also implies it is not needed, that you are just trying to work
around syntactic incompatibilities for the sake, ultimately, of pdftex.

Well, if I remember everything correctly then there are two
problems:

1. that (x)dvipdfmx gives errors when the <<-syntax is used.

2. that (x)dvipdfmx can't load otf-fonts through map-entries even if
the "<"-syntax is used. That is not really a problem for normal text
fonts - they can be easily loaded through fontspec -- but it is if
the fonts needs to be reencoded e.g. for math packages.

So the bigger question is if one could enable embedding of otf-fonts
through map-entries at all, subsetting can come later ;-)

Post by Ulrike Fischer
But in the case of the adobe garamondm utopia, GFSBodoni and other
fonts this it not an option: Here otf-fonts are loaded
gfsbodoni has pfb files, and gfsbodoni.map loads them.
So I don't get that one.

There are also map-entries which uses otf:

md-grbr7m GFSBodoni-Regular " AutoEnc_py5znvjh3poy4qksuktslsc7kc
ReEncodeFont " <[a_py5znv.enc <<GFSBodoni.otf

Post by Ulrike Fischer
and pdftex can handle them only if they are fully embeded.
On another front, it seems that it would be desirable for pdftex to
fully download otf files entirely, regardless of the map file spec,
instead of just reporting an error and giving up. I admit it's not
clear to me when one would want to play with otf files in pdftex in the
first place, but fine.

Why not? It avoids to have to convert fonts to pfb (which is perhaps
not allowed) and to handle two variants of a font (e.g. how to
prevent xelatex to find the pfb instead of the otf). And at least in
miktex pdftex can access such fonts directly from the system folder.

[Karl Berry]

Hi Ulrike,

There are also map-entries which uses otf:
md-grbr7m GFSBodoni-Regular " AutoEnc_py5znvjh3poy4qksuktslsc7kc
ReEncodeFont " <[a_py5znv.enc <<GFSBodoni.otf

I see. That's part of mathdesign, not gfsbodoni, hence I didn't see it.
(And I highly doubt that using the .otf is required here, but I've
raised enough issues with the mathdesign author already that I'm not
going to add to them unless there is some evidence of movement.)
Anyway.

1. that (x)dvipdfmx gives errors when the <<-syntax is used.

Ack.

2. that (x)dvipdfmx can't load otf-fonts through map-entries even if
the "<"-syntax is used.

I don't see that problem (which would be tantamount to *fmx not
supporting otf fonts at all). Given input tryot.tex:

\nopagenumbers
% let's set up a test for partial download of otf, since none in map file.
\special{pdf:mapline md-grbbi7m GFSBodoni-BoldItalic " AutoEnc_rl4tn2dr7yl7pvjznhtbna64dc ReEncodeFont " <[a_rl4tn2.enc <GFSBodoniBoldIt.otf}
\font\test = md-grbbi7m % fully downloads .otf from map file

\test K
\end

I run
tex tryot
dvipdfmx tryot.dvi
and the result is a tryot.pdf (with the font partially embedded).

If you're thinking of your previous mail about SkakNew-Figurine
(http://tug.org/pipermail/tex-live/2014-June/035488.html), that's a
different case altogether.

Thanks,
k

[Ulrike Fischer]

Post by Ulrike Fischer
2. that (x)dvipdfmx can't load otf-fonts through map-entries even if
the "<"-syntax is used.
I don't see that problem (which would be tantamount to *fmx not
\nopagenumbers
% let's set up a test for partial download of otf, since none in map file.
\special{pdf:mapline md-grbbi7m GFSBodoni-BoldItalic " AutoEnc_rl4tn2dr7yl7pvjznhtbna64dc ReEncodeFont " <[a_rl4tn2.enc <GFSBodoniBoldIt.otf}
\font\test = md-grbbi7m % fully downloads .otf from map file
\test K
\end
I run
tex tryot
dvipdfmx tryot.dvi
and the result is a tryot.pdf (with the font partially embedded).

Yes you are right, I had forgotten that with tex + dvipdfmx it
worked. But xetex fails:

G:\Z-Test>xetex test-utf8
This is XeTeX, Version 3.14159265-2.6-0.99991 (TeX Live 2014/W32TeX)
(preloaded
format=xetex)
restricted \write18 enabled.
entering extended mode
(./test-utf8.tex [1] )
kpathsea: Running mktexpk --mfmode / --bdpi 600 --mag 1+0/600 --dpi
600 GFSBodoniBoldIt.otf

The command name is D:\texlive\2014\bin\win32\mktexpk

kpathsea: Running mktexmf GFSBodoniBoldIt.mf

The command name is D:\texlive\2014\bin\win32\mktexmf
name = GFSBodoniBoldIt, rootname = GFSBodoniBoldIt, pointsize =
mktexmf: empty or non-existent rootfile!
Cannot find GFSBodoniBoldIt.mf .
I try ps2pk --> gsftopk --> ttf2pk --> hbf2gf.
Cannot find GFSBodoniBoldIt.tfm .
kpathsea: Appending font creation commands to missfont.log.

xdvipdfmx:fatal: Cannot proceed without .vf or "physical" font for
PDF output...


Output file removed.

Error 1 (driver return code) generating output;
file test-utf8.pdf may not be valid.
Transcript written on test-utf8.log.


(That's what started the discussion: Someone tried to use mathdesign
with xelatex which loads fonts in this way and this fails).

[Karl Berry]

kpathsea: Running mktexpk --mfmode / --bdpi 600 --mag 1+0/600 --dpi
600 GFSBodoniBoldIt.otf

Sounds like test-utf8.tex contains
\font\foo = GFSBodoniBoldIt.otf

Is that right? That syntax has never worked. To get XeTeX to look up
an .otf by file name, it's always been necessary to use brackets, most
likely with quotes:
\font\foo = "[GFSBodoniBoldIt.otf]" \foo XYZ
I just tried it with xetex from tl14 on i386-linux; it worked.
FWIW, the xetexref manual has a variety of examples (although personally
I think it could use still more :).

After your initial mail, I submitted an enhancement request for xetex to
pay attention to the .otf extension instead of blindly stripping it off
and looking for a .tfm anyway:
http://sourceforge.net/p/xetex/feature-requests/17/
but that is new functionality not yet implemented.

(That's what started the discussion: Someone tried to use mathdesign
with xelatex which loads fonts in this way and this fails).

Sounds like a bug in mathdesign, then.

thanks,
karl

[Ulrike Fischer]

Post by Ulrike Fischer
kpathsea: Running mktexpk --mfmode / --bdpi 600 --mag 1+0/600 --dpi
600 GFSBodoniBoldIt.otf
Sounds like test-utf8.tex contains
\font\foo = GFSBodoniBoldIt.otf
Is that right?

No test-utf8.tex contains exactly the code of your "tryot.tex" and
that's the output if I run this code with xetex instead of latex +
dvipdfmx.

Try it out in tryot.tex:
\special{pdf:mapline md-grbbi7m ... <XXXX}
works fine with xetex if XXXX is a pfb, but fails if it is a ttf or
a otf font.

Post by Ulrike Fischer
That syntax has never worked. To get XeTeX to look up
an .otf by file name, it's always been necessary to use brackets, most

I know how to call otf by file name. But in your code the font is
called through the tfm: "\font\test = md-grbbi7m" and the question
is how to call the related otf through the "normal"
tfm-vf-tfm-mapline-otf-chain.

If I run your tryot.tex code with "xetex --no-pdf" and then call
xdvipdfmx I get this:

G:\Z-Test>xdvipdfmx test-utf8.xdv
test-utf8.xdv -> test-utf8.pdf
[1
kpathsea: Running mktexpk --mfmode / --bdpi 600 --mag 1+0/600 --dpi
600 GFSBodon
iBoldIt.otf

The command name is D:\texlive\2014\bin\win32\mktexpk

kpathsea: Running mktexmf GFSBodoniBoldIt.mf

The command name is D:\texlive\2014\bin\win32\mktexmf
name = GFSBodoniBoldIt, rootname = GFSBodoniBoldIt, pointsize =
mktexmf: empty or non-existent rootfile!
Cannot find GFSBodoniBoldIt.mf .
I try ps2pk --> gsftopk --> ttf2pk --> hbf2gf.
Cannot find GFSBodoniBoldIt.tfm .
kpathsea: Appending font creation commands to missfont.log.

xdvipdfmx:warning: Could not locate a virtual/physical font for TFM
"md-grbbi7m"
.
xdvipdfmx:warning: >> This font is mapped to a physical font
"GFSBodoniBoldIt.otf".

xdvipdfmx:warning: >> Please check if kpathsea library can find this
font: GFSBodoniBoldIt.otf
xdvipdfmx:fatal: Cannot proceed without .vf or "physical" font for
PDF output...


Output file removed.
================================

So xdvipdfmx did see the map entry, and it did realize that the
"physical font" is GFSBodoniBoldIt.otf but then fails to use it.

Post by Ulrike Fischer
(That's what started the discussion: Someone tried to use mathdesign
with xelatex which loads fonts in this way and this fails).
Sounds like a bug in mathdesign, then.

I don't think so. It looks like a missing feature of xdvipdfmx for
me: Perhaps someone thought that otf will always be loaded directly
by filename and so removed the support for this fonts in the tfm/map
handling.

[Karl Berry]

Ulrike,

(1) I don't think any useful conclusions can be reached from running
xetex without --no-pdf, since it doesn't report why the font loading
fails. FWIW, I also see differences between Windows and Unix.

(2) After further testing, I can agree that there is something awry in
the loading via the map file. I'm not prepared to try to address that
myself now, but I reported it for Khaled's consideration here:
http://sourceforge.net/p/xetex/bugs/97/

(3) For compatibility if nothing else, I feel that the mathdesign author
should change the loading of the gfs* fonts. I see no benefit to his
current approach. (The proprietary fonts in mathdesign are a different
situation.)

(4) The one thing I did just do was make the tiny tweak to the sources to
accept and ignore << as has been done all along with <[:

fontmap.c (revision 34677)
..
- if (++p < endptr && *p == '[') p++; /*skip */
+ if (++p < endptr && (*p == '[' || *p == '<')) p++; /*skip */

It won't help for this year, but at least for the future ...
I'll also do something to disambiguate various fatal error msgs.

karl

[Ulrike Fischer]

Post by Akira Kakuto
Ulrike,
(1) I don't think any useful conclusions can be reached from running
xetex without --no-pdf, since it doesn't report why the font loading
fails. FWIW, I also see differences between Windows and Unix.

I only wanted to show you some more extended messages. You can also
use
xetex --output-driver="xdvipdfmx -vv"

Post by Akira Kakuto
(2) After further testing, I can agree that there is something awry in
the loading via the map file. I'm not prepared to try to address that
http://sourceforge.net/p/xetex/bugs/97/

Imho the problem is on the xdvipdfmx side.
On the xetex list Akira just wrote too that "In the case of
XeTeX-xdvipdfmx, we cannot use the kind of map file
for OpenType fonts. That is, we cannot load OpenType fonts through
tfm fonts."

Post by Akira Kakuto
(3) For compatibility if nothing else, I feel that the mathdesign author
should change the loading of the gfs* fonts.

The problem is the reencoding. If I got it right the map line is for
an OML-encoded font. I don't know an easy way to reencode a font if
you load it directly by font name -- perhaps it can be done with a
teckit-mapping.

Post by Akira Kakuto
I see no benefit to his current approach.

Well it works with pdftex. I would also try to avoid to write extra
"ifxetex" and "ifluatex" unless it is really necessary.