Discussion:
(no subject)
(too old to reply)
Luiz Gonçalves
2018-09-17 03:43:05 UTC
Permalink
3.4.1 Environment variables for Unix

....

PATH=/usr/local/texlive/2018/bin/*x86_64-linux*:$PATH; export PATH
MANPATH=/usr/local/texlive/2018/texmf-dist/doc/man:$MANPATH; export MANPATH
INFOPATH=/usr/local/texlive/2018/texmf-dist/doc/info:$INFOPATH; export
INFOPATH

and not

PATH=/usr/local/texlive/2018/bin/i386-linux:$PATH; export PATH
MANPATH=/usr/local/texlive/2018/texmf-dist/doc/man:$MANPATH; export MANPATH
INFOPATH=/usr/local/texlive/2018/texmf-dist/doc/info:$INFOPATH; export
INFOPATH

and the same error is reproduced all over the page.

Cheers
--
--------------
Luiz CD Gonçalves, Dr. (lattes <http://lattes.cnpq.br/7258901292233920>)
Mobile +55-11-9xxx87848 (leave your message)
Enrico Gregorio
2018-09-17 14:40:23 UTC
Permalink
Post by Luiz Gonçalves
3.4.1 Environment variables for Unix
....
PATH=/usr/local/texlive/2018/bin/x86_64-linux:$PATH; export PATH
MANPATH=/usr/local/texlive/2018/texmf-dist/doc/man:$MANPATH; export MANPATH
INFOPATH=/usr/local/texlive/2018/texmf-dist/doc/info:$INFOPATH; export INFOPATH
and not
PATH=/usr/local/texlive/2018/bin/i386-linux:$PATH; export PATH
MANPATH=/usr/local/texlive/2018/texmf-dist/doc/man:$MANPATH; export MANPATH
INFOPATH=/usr/local/texlive/2018/texmf-dist/doc/info:$INFOPATH; export INFOPATH
and the same error is reproduced all over the page.
The documentation could be improved by showing the most common
architecture strings and using a placeholder in the code (but one
of the strings could be good as well, provided it’s explained clearly
what’s to be actually substituted for a particular system).

But, for sanity’s sake, PLEASE DON'T send messages formatted
with the font I don’t want to mention the name of! :-D

Ciao
Enrico
Luiz Gonçalves
2018-09-17 21:45:44 UTC
Permalink
Sorry, my error.
I have actually had a hard time installing Texlive 2018 and removing tl2016
(another hard time to install).
Your info about i386 architecture is OK but it seems to me that's not the
most common nowadays, and so can mislead one to make errors. Not in my
case, as I took a good reading in the page and some notes about the painful
tl2016 installation. Everything went smoothly after some 3 workarounds and
some hours; "everything" includes running xelatex form command-line and
from Kile.

Problems I had: as it is my case, not a Linux expert, I had some possibly
trivial problems with manpath.config.
1) As suggested
MANPATH_MAP /usr/local/texlive/2018/bin/x86_64-linux \
/usr/local/texlive/2018/texmf-dist/doc/man
for which I had
for luiz:
***@black10:~$ man apt
man: can't parse directory list `/usr/local/texlive/2018/texmf-dist/doc/man
'
man: can't make sense of the manpath configuration file /etc/manpath.config
I use Ubuntu and the continuation character \ is not recognized, the
problems disappeared after removing it and leaving all info in one line, as
in
MANPATH_MAP /usr/local/texlive/2018/bin/x86_64-linux
/usr/local/texlive/2018/texmf-dist/doc/man

2) /etc/environment. Just placed /usr/local/texlive/2018/bin/x86_64-linux
at PATH
PATH="/usr/local/texlive/2018/bin/x86_64-linux:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games"

3) No need for symlinks.

I know that brevity is important but texlive is a winner among newbies and
experts, at tex.stackexchange and around the web.
Nevertheless, congrats for your inestimable good work and time, and for TUG
team (my companion of some decades).

Cheers
Post by Luiz Gonçalves
Post by Luiz Gonçalves
3.4.1 Environment variables for Unix
....
PATH=/usr/local/texlive/2018/bin/x86_64-linux:$PATH; export PATH
MANPATH=/usr/local/texlive/2018/texmf-dist/doc/man:$MANPATH; export
MANPATH
Post by Luiz Gonçalves
INFOPATH=/usr/local/texlive/2018/texmf-dist/doc/info:$INFOPATH; export
INFOPATH
Post by Luiz Gonçalves
and not
PATH=/usr/local/texlive/2018/bin/i386-linux:$PATH; export PATH
MANPATH=/usr/local/texlive/2018/texmf-dist/doc/man:$MANPATH; export
MANPATH
Post by Luiz Gonçalves
INFOPATH=/usr/local/texlive/2018/texmf-dist/doc/info:$INFOPATH; export
INFOPATH
Post by Luiz Gonçalves
and the same error is reproduced all over the page.
The documentation could be improved by showing the most common
architecture strings and using a placeholder in the code (but one
of the strings could be good as well, provided it’s explained clearly
what’s to be actually substituted for a particular system).
But, for sanity’s sake, PLEASE DON'T send messages formatted
with the font I don’t want to mention the name of! :-D
Ciao
Enrico
--
--------------
Luiz CD Gonçalves, Dr. (lattes <http://lattes.cnpq.br/7258901292233920>)
Mobile +55-11-9xxx87848 (leave your message)
Karl Berry
2018-09-17 21:19:40 UTC
Permalink
It's likely we'll remove all 32-bit machines from the DVD image (not
from TL) next year in order to gain enough space.

I'll change the example platform in the doc when that happens, but
probably not before. I don't want to unnecessarily churn the
documentation, implying all translators need to update, etc.
There is nothing wrong with "i386-linux" as the example, though
certainly I agree that nowadays "x86_64-linux" is more common.

I think metavariables would be more confusing than helpful to newbies in
this case. Non-newbies presumably don't have a problem with this simple
text in any case. Brevity is also important. -k
Michael Shell
2018-09-18 19:45:01 UTC
Permalink
On Mon, 17 Sep 2018 21:19:40 GMT
Post by Karl Berry
It's likely we'll remove all 32-bit machines from the DVD image (not
from TL) next year in order to gain enough space.
IMHO, if we have reached a point where space is that tight, then we've
reached the point where the media capacity needs to be expanded. How
much space will deleting 32 bit support gain and how much time before
that is not enough? And have other features already been removed in
the past to gain needed space?

I for one would be willing to pay a little bit more to get *everything*.
Who knows when an old 32 bit machine has to be brought into service as
a backup system when its modern replacement develops a problem? And
those old machines are exactly the ones where compiling from source is
such a lengthy ordeal, where binaries are of the most help. How about
folks in the less developed countries who are still using older machines
(a major feature of TeX is that it does not require much processing
power).

Tex Live is currently using double layer DVDs with a capacity of about
8.3 GB. And data compression techniques are already being employed.

Sandisk 16GB USB flash drives can now be had for $5.49:
https://www.adorama.com/idscbu16gb.html
A better deal can be obtained at higher quantities and/or off brands.
For example:
https://www.amazon.com/TOPSELL-10PCS-Flash-Pack-Memory-Storage/dp/B016RO18JO
$3.23 each in qty 10.

32GB units are in the $6-$10 range:
https://www.amazon.com/PNY-Attache-Flash-Drive-P-FD32GX5ATT03-MP/dp/B01MQ0RR5O/
https://www.amazon.com/SanDisk-Ultra-Flash-Drive-Performance/dp/B01EG72D1Q

but one has to be careful to only use reputable sellers (not Ebay, not
Amazon third party sellers) to avoid all the fakes out there. Be sure
and check the reviews for reliability. I myself would stick to well known
brands such as Kingston, Sandisk and PNY even if they cost a little
more.

As far as USB drives with logos and/or preloading services go, see:
http://blog.notdot.net/2009/12/DIY-USB-preloading-with-nix
and also do a search for

USB flash drive bulk preloading

But watch out that many of the preloading services don't support
preloads more than 1-2 GB, well, at least not at the $0.50/drive
level.

Also, search EBay for

USB flash drive duplicator

They run about $2000 per 30 drive-at-once capability:
https://www.ebay.com/itm/SySTOR-1-31-USB-Memory-Stick-Duplicator-Jump-Drive-Wiper-Copy-Flash-Media-Data/142942568899
and there are units that can do more than 170 at once:
https://www.ebay.com/itm/SySTOR-1-174-Standalone-Multiple-USB-Port-Duplicator-Flash-Drive-Cloner-Sanitize/192661338502

Once the move is made to an "actively evolving media" such as SD cards
or USB flash drives, then we can ride the wave of Moore's law instead
of being stuck with the now-forever-fixed capacity of DVD.

I assume that bluray is not a viable option here because of the
significant percentage of machines that do not have bluray read
capability.

Another approach would be to go to a 2 DVD package, but we'd be
riding a linear, not exponential, media capacity wave there.


Just my $0.02,

Mike Shell
Zdenek Wagner
2018-09-18 20:04:16 UTC
Permalink
Post by Michael Shell
On Mon, 17 Sep 2018 21:19:40 GMT
Post by Karl Berry
It's likely we'll remove all 32-bit machines from the DVD image (not
from TL) next year in order to gain enough space.
IMHO, if we have reached a point where space is that tight, then we've
reached the point where the media capacity needs to be expanded. How
much space will deleting 32 bit support gain and how much time before
that is not enough? And have other features already been removed in
the past to gain needed space?
I for one would be willing to pay a little bit more to get *everything*.
Who knows when an old 32 bit machine has to be brought into service as
a backup system when its modern replacement develops a problem? And
those old machines are exactly the ones where compiling from source is
such a lengthy ordeal, where binaries are of the most help. How about
folks in the less developed countries who are still using older machines
(a major feature of TeX is that it does not require much processing
power).
This may seem to be a valid idea, I have such an old machine with
linux distro which is 7 years after end of life. A newer distre
requires more RAM but there are no slots for more RAM in this machine.
Will TL contain i386 binaries for after-end-of-life distros with
obsolete versions of glibc? If not, I can either use the historical TL
2011 or compile from sources anyway and hope that I will be able to
compile it with obsolete gcc and obsolete glibc.
Post by Michael Shell
Tex Live is currently using double layer DVDs with a capacity of about
8.3 GB. And data compression techniques are already being employed.
https://www.adorama.com/idscbu16gb.html
A better deal can be obtained at higher quantities and/or off brands.
https://www.amazon.com/TOPSELL-10PCS-Flash-Pack-Memory-Storage/dp/B016RO18JO
$3.23 each in qty 10.
On one conference I received proceedings on a cheap USB stick. I was
never able to read anything from it. I tried to use it just as a USB
stich and found that it does not work at all. I would never recommend
cheap media. (Fortunately the proceedings could be downloaded from the
web of the conference.)
Post by Michael Shell
https://www.amazon.com/PNY-Attache-Flash-Drive-P-FD32GX5ATT03-MP/dp/B01MQ0RR5O/
https://www.amazon.com/SanDisk-Ultra-Flash-Drive-Performance/dp/B01EG72D1Q
but one has to be careful to only use reputable sellers (not Ebay, not
Amazon third party sellers) to avoid all the fakes out there. Be sure
and check the reviews for reliability. I myself would stick to well known
brands such as Kingston, Sandisk and PNY even if they cost a little
more.
http://blog.notdot.net/2009/12/DIY-USB-preloading-with-nix
and also do a search for
USB flash drive bulk preloading
But watch out that many of the preloading services don't support
preloads more than 1-2 GB, well, at least not at the $0.50/drive
level.
Also, search EBay for
USB flash drive duplicator
https://www.ebay.com/itm/SySTOR-1-31-USB-Memory-Stick-Duplicator-Jump-Drive-Wiper-Copy-Flash-Media-Data/142942568899
https://www.ebay.com/itm/SySTOR-1-174-Standalone-Multiple-USB-Port-Duplicator-Flash-Drive-Cloner-Sanitize/192661338502
Once the move is made to an "actively evolving media" such as SD cards
or USB flash drives, then we can ride the wave of Moore's law instead
of being stuck with the now-forever-fixed capacity of DVD.
I assume that bluray is not a viable option here because of the
significant percentage of machines that do not have bluray read
capability.
Another approach would be to go to a 2 DVD package, but we'd be
riding a linear, not exponential, media capacity wave there.
Zdeněk Wagner
http://ttsm.icpf.cas.cz/team/wagner.shtml
http://icebearsoft.euweb.cz
Post by Michael Shell
Just my $0.02,
Mike Shell
Reinhard Kotucha
2018-09-18 22:42:39 UTC
Permalink
Still much too expensive in order to offer them for free to members of
TeX user groups. DVDs are very cheap.

But the main problem is there is no filesystem appropriate for all
operating systems. I've heard that recent versions of Windows allow
to mount ISO9660 images if they are files on an NTFS filesystem but do
not recognize an ISO9660 filesystem on a USB stick.

We already considered UDF, which is supposed to be platform
independent. But it's still not supported by all systems and it seems
that it will never be. I do not see any progress.
Post by Michael Shell
Another approach would be to go to a 2 DVD package, but we'd be
riding a linear, not exponential, media capacity wave there.
Yes, this can only solve the problem in the next two or three years.
It's not really a solution, though removing stuff is also painful.

The current size of MiKTeX is 2.7 GB. I'll never understand why
Christian insists on doing everything himself instead of simply
adapting his system so that MiKTeX can use at least the platform
independent TeX Live packages.

I'm sure that almost everybody is using the network installer
nowadays. For the few users with unsuffient internet access in
developing countries we definitely find individual solutions.

Regards,
Reinhard
--
------------------------------------------------------------------
Reinhard Kotucha Phone: +49-511-3373112
Marschnerstr. 25
D-30167 Hannover mailto:***@web.de
------------------------------------------------------------------
George N. White III
2018-09-19 09:37:00 UTC
Permalink
Post by Reinhard Kotucha
Still much too expensive in order to offer them for free to members of
TeX user groups. DVDs are very cheap.
You have to buy a new DVD every year. USB flash drives could be
returned for updating. Local user groups could do this with disk to
disk copying.
Post by Reinhard Kotucha
But the main problem is there is no filesystem appropriate for all
operating systems. I've heard that recent versions of Windows allow
to mount ISO9660 images if they are files on an NTFS filesystem but do
not recognize an ISO9660 filesystem on a USB stick.
There should be no need for a filesystem that supports huge files.
FAT32 is widely supported but needs large sector sizes for large
media. This may not be a problem for a filesystem populated with
archives. VFAT (long filenames) has been supported in linux for
years.
Post by Reinhard Kotucha
We already considered UDF, which is supposed to be platform
independent. But it's still not supported by all systems and it seems
that it will never be. I do not see any progress.
[...]
I'm sure that almost everybody is using the network installer
nowadays. For the few users with unsuffient internet access in
developing countries we definitely find individual solutions.
Even in some developed countries, internet access for many
people uses cellular data. While there is often internet access
at schools, coffee shops, and libraries, it isn't practical to run
a download that takes an hour.

In remote areas and ships at sea, internet requires expensive
satellite links.
--
George N. White III
Martin Sievers
2018-09-19 09:45:17 UTC
Permalink
Post by George N. White III
Post by Reinhard Kotucha
Still much too expensive in order to offer them for free to members of
TeX user groups. DVDs are very cheap.
You have to buy a new DVD every year. USB flash drives could be
returned for updating. Local user groups could do this with disk to
disk copying.
To be honest, I am not convinced, that 2000 members of DANTE e.V. (or at
least a significant part) will return their USB flash drives. Secondly,
I don't want our secretary or anybody else to copy the data to 2000 USB
flash drives.

And again, we are talking about prices less than 0,50 EUR per DVD and
year for producing and shipping. Just shipping the USB flash drive back
to our members would be more expansive.

Best regards
Martin
Zdenek Wagner
2018-09-19 09:52:54 UTC
Permalink
Post by Martin Sievers
Post by George N. White III
Post by Reinhard Kotucha
Still much too expensive in order to offer them for free to members of
TeX user groups. DVDs are very cheap.
You have to buy a new DVD every year. USB flash drives could be
returned for updating. Local user groups could do this with disk to
disk copying.
To be honest, I am not convinced, that 2000 members of DANTE e.V. (or at
least a significant part) will return their USB flash drives. Secondly,
I don't want our secretary or anybody else to copy the data to 2000 USB
flash drives.
You will have to inform them that they have to return it and how long
will you wait?
When I use "dd" to move and ISO image of Live Linux, it takes 6
minutes for 1.6 GB.
If TL has > 8 GB and DANTE has 2000 members, imagine how long it will take.
In addition, if VFAT is mounted on Linux, all 8.3 names appear in
uppercase while
longer names appear as they are. It might cause problems.
Post by Martin Sievers
And again, we are talking about prices less than 0,50 EUR per DVD and
year for producing and shipping. Just shipping the USB flash drive back
to our members would be more expansive.
Best regards
Martin
Zdeněk Wagner
http://ttsm.icpf.cas.cz/team/wagner.shtml
http://icebearsoft.euweb.cz
Philip Taylor
2018-09-19 10:01:45 UTC
Permalink
Binaries are, IMHO, not the cause of the problem, nor is their removal the solution.  The TeX Live installer offers a number of schemes -- why not pack all binaries and the files needed by as many of the the smaller schemes as possible onto DVD-1 and the additional files needed by the larger schemes onto DVD-2 ?   As the size of individual schemes increases, this solution will scale to accommodate DVD-3, etc.

Philip Taylor
George N. White III
2018-09-19 12:09:49 UTC
Permalink
Post by Zdenek Wagner
Post by Martin Sievers
Post by George N. White III
Post by Reinhard Kotucha
Still much too expensive in order to offer them for free to members of
TeX user groups. DVDs are very cheap.
You have to buy a new DVD every year. USB flash drives could be
returned for updating. Local user groups could do this with disk to
disk copying.
To be honest, I am not convinced, that 2000 members of DANTE e.V. (or at
least a significant part) will return their USB flash drives. Secondly,
I don't want our secretary or anybody else to copy the data to 2000 USB
flash drives.
The economics depends on reducing the 2000 DVD's to the number that are
actually used. Commercial popularity of DVD's for videos and technical manuals,
etc. once made DVD copying businesses practical. Now some techical manuals
are distributed in the form of a tablet that replaces a stack of DVD's
while others
are available online.

Many of the same companies that provide mass DVD copying and mailing are
now doing USB flash drives. See
https://www.newcyberian.com/usb-flash-drive.html
for an example.
Post by Zdenek Wagner
You will have to inform them that they have to return it and how long
will you wait?
When I use "dd" to move and ISO image of Live Linux, it takes 6
minutes for 1.6 GB.
If TL has > 8 GB and DANTE has 2000 members, imagine how long it will take.
In addition, if VFAT is mounted on Linux, all 8.3 names appear in
uppercase while longer names appear as they are. It might cause problems.
Moving to USB will not be a simple project, especially since the goal
is to include
less popular systems so relies on testing by others.
Post by Zdenek Wagner
Post by Martin Sievers
And again, we are talking about prices less than 0,50 EUR per DVD and
year for producing and shipping. Just shipping the USB flash drive back
to our members would be more expansive.
DVD's provide a durable collection of older versions for those who have DVD
readers. USB keys tend to collect malware, so recycling is not something that
can be done by ordinary users, and there will be more failures than for DVD's.



--
George N. White III
Boris Veytsman
2018-09-19 14:52:32 UTC
Permalink
GNW> From: "George N. White III" <***@gmail.com>
GNW> Date: Wed, 19 Sep 2018 06:37:00 -0300


GNW> You have to buy a new DVD every year. USB flash drives could be
GNW> returned for updating. Local user groups could do this with disk to
GNW> disk copying.

The cost of this option is astronomical

I would, however, support making a limited edition of TL USBs with
beautiful logos for special purposes: gifts, prizes, promotional
materials etc. Not as a substitute for our regular mass-produced
DVDs, but as a complement for special purposes. And I would support
sending them to those members who opt to pay for them. Maybe we could
even make a profit if USBs are beautiful enough to be collectible
(= capable to be sold at higher prices).
--
Good luck

-Boris

BOFH excuse #51:

Cosmic ray particles crashed through the hard disk platter
Michael Shell
2018-09-19 18:22:19 UTC
Permalink
On Wed, 19 Sep 2018 00:42:39 +0200
Post by Reinhard Kotucha
Still much too expensive in order to offer them for free to members of
TeX user groups. DVDs are very cheap.
Well, it's not just the cost of blank DVDs, but the cost of burning,
the printed sleeves (which are very nice and well done, BTW. That's
what I would miss most with USB drives). If all that can be done under
the 1 $/EURO level, wow, that really is incredibly cheap.

We could still produce DVDs for free distribution, but those would, and
could, not carry everything. And as time goes on, the fraction of the
full distribution they carry is going to have to become less and less.
Post by Reinhard Kotucha
But the main problem is there is no filesystem appropriate for all
operating systems. I've heard that recent versions of Windows allow
to mount ISO9660 images if they are files on an NTFS filesystem but do
not recognize an ISO9660 filesystem on a USB stick.
Not to suggest the free software world is without quirks, but the
proprietary software world always seems to come with such irksome
baggage and malfeasance.

USB drives can be partitioned just like regular hard drives. So, they
can carry multiple filesystems. However, we would want to do this in a
way that minimizes redundancy.

We could even go with 32GB USB units, and/or expect the unix systems
to be able to mount FAT32 and/or NTFS filesystems.

Alternatively, I would even consider a more radical approach - use the
linux ext2 filesystem to carry the vast bulk of the files and then
include a partition(s) with drivers to allow ext2 to be read under
Windows and Mac:

http://www.ext2fsd.com/
https://sourceforge.net/projects/osxfuse/
https://sourceforge.net/projects/fuse-ext2/

The Windows/Mac user would first install the needed ext2 drivers, then
they could proceed normally from that point on using the distribution
files carried on the ext2 filesystem.

I know there are a lot of potential pitfalls here. We would have to
make sure such drivers actually work, could be redistributed freely,
etc. Tis just another option to consider. By no means am I recommending
such a radical approach until we had more information/testing.

FWIW, this is another area where I think our governments are lacking -
the establishment of a free/open/patentless filesystem standard that
most all consumer devices would be required to support. Cameras and
other data handling devices should not be using file formats owned
and controlled by Microsoft. Ditto for fonts (government agencies
should have developed/used only free fonts for normal business and
standardized their availability and use in consumer devices - as a
base standard to permit uniform information interchange, not mandates
beyond that).
Post by Reinhard Kotucha
Yes, this can only solve the problem in the next two or three years.
It's not really a solution, though removing stuff is also painful.
On Wed, 19 Sep 2018 11:45:17 +0200
Post by Reinhard Kotucha
And again, we are talking about prices less than 0,50 EUR per DVD and
year for producing and shipping. Just shipping the USB flash drive
back to our members would be more expansive.
Of course, we/users would have to be the ones to bear any added costs,
not TUG. I do agree that return shipping costs would be too great
relative to $6 USB drives. Also, be aware that USB drives do *not*
qualify for the cheap USPS media mail rates here in the US. So, they
will cost more to ship than DVDs:

https://www.endicia.com/tools-resources/newsletter/2012-fall/understanding-the-media-mail-classification

We just have a question of a tradeoff between price and features. I would be
willing to pay more to get more. The added space of USB drives could also
allow for additional features - the collection of TUG articles, other
bonus material, etc.

Maybe we could offer a flash drive *option*, for say, $20, - beyond the
break even point so that it is even profitable for TUG to offer this
option. Maybe the USB option could come in different Unix, Windows and
Mac "flavors" (filesystems). It could serve as a test trial for the
future.

Even if the USB drive approach is not viable, I would prefer a 2 DVD set
over deleting material. Once the build system for older hardware is no
regularly longer used, it might not reappear even after the space
limitations have been overcome, which is going to have to happen in 2-3
years anyway. Let's just go to the 2 disk set sooner rather than later.
Charge us more as needed. We are going to have to do this in the near
future anyway, right?

As for the cost of a 30 USB drive duplicator, it amounts to less than
$2 per TUG member. If a mere two dozen of us donate $100, we've got
it. For a 55-at-once unit:

https://www.ebay.com/itm/SySTOR-1-55-Standalone-Multiple-USB-Port-Duplicator-Flash-Drive-Cloner-Sanitizer-/141803513416

we would need 43 members to donate $100. These units can copy the
whole media (block by block) do have a post-write verify option
which would automatically catch any bad drives.

I realize there is going to be an additional labor cost of whoever
operates it, but we have to pay that cost even with DVDs, right?

For the record, count me in as one who is willing to donate in
the hundred dollar level to help out here.

Maybe we should take another online survey of the options and costs
and see what members want and are willing to pay for?


Cheers,

Mike
George N. White III
2018-09-18 21:25:19 UTC
Permalink
Post by Karl Berry
It's likely we'll remove all 32-bit machines from the DVD image (not
from TL) next year in order to gain enough space.
My experience has been that it is increasigly hard to find a working
DVD reader. Many new systems don't have a DVD drive, and when
you do find a DVD drive in an old system it often fails to read your DVD.
I don't know if USB drives fare better in the mails than a DVD (I once
received a DVD in an oversize cardboard envelope that had been
folded to fit in my rural mailbox -- the DVD was neatly split into halves).
DVD's use more power than USB keys so are being left out of systems
being used in remote field sites with limited power.
Post by Karl Berry
I'll change the example platform in the doc when that happens, but
probably not before. I don't want to unnecessarily churn the
documentation, implying all translators need to update, etc.
There is nothing wrong with "i386-linux" as the example, though
certainly I agree that nowadays "x86_64-linux" is more common.
I think metavariables would be more confusing than helpful to newbies in
this case. Non-newbies presumably don't have a problem with this simple
text in any case. Brevity is also important. -k
USB memory keys are replacing DVD's in many use cases. The extra space
available with USB keys would make it easier to distribute files for
less popular
platforms and eliminate some of the discussions over what to include on a DVD.
--
George N. White III
m***@dante.de
2018-09-18 21:52:59 UTC
Permalink
Dear all,

I really appreciate all your suggestions regarding alternative media for the TeX Collection. However, please note, that AFAIK a USB stick or other flash medium can _not_ be produced to be used for all three major platforms Windows, Unix and Mac.

Secondly, for the last years we were able to sell the TeX Collection DVDs to the user groups for 0,45 EUR, which is _much_ less than any flash medium would cost (especially when quality is important).

Best regards
Martin

⁣Gesendet mit TypeApp ​
Post by George N. White III
Post by Karl Berry
It's likely we'll remove all 32-bit machines from the DVD image (not
from TL) next year in order to gain enough space.
My experience has been that it is increasigly hard to find a working
DVD reader. Many new systems don't have a DVD drive, and when
you do find a DVD drive in an old system it often fails to read your DVD.
I don't know if USB drives fare better in the mails than a DVD (I once
received a DVD in an oversize cardboard envelope that had been
folded to fit in my rural mailbox -- the DVD was neatly split into halves).
DVD's use more power than USB keys so are being left out of systems
being used in remote field sites with limited power.
Post by Karl Berry
I'll change the example platform in the doc when that happens, but
probably not before. I don't want to unnecessarily churn the
documentation, implying all translators need to update, etc.
There is nothing wrong with "i386-linux" as the example, though
certainly I agree that nowadays "x86_64-linux" is more common.
I think metavariables would be more confusing than helpful to newbies
in
Post by Karl Berry
this case. Non-newbies presumably don't have a problem with this
simple
Post by Karl Berry
text in any case. Brevity is also important. -k
USB memory keys are replacing DVD's in many use cases. The extra space
available with USB keys would make it easier to distribute files for
less popular
platforms and eliminate some of the discussions over what to include on a DVD.
--
George N. White III
Karl Berry
2018-09-18 21:54:01 UTC
Permalink
Hi Luis - thanks for your further explanations. My immediate reaction is
that it would be better to delete the "Environment variables: Global
configuration" section from the manual than write further explanations
there, because, as already mentioned in that section:

.... there is just too much variation between systems in how and
where these things are configured.

Thus, it would be better not to give the "hints" at all than hints
which proved to be problematic for you and probably others.

In my view, anyone who needs to install the native TL globally (rare
indeed, IMHO), as opposed to for themselves, should have enough
experience with software installation not to need a cookie-cutter recipe
(which we can't provide).

For the majority for whom self-installation suffices,
http://tug.org/texlive/quickinstall.html is my attempt at a minimal
installation. (I just changed i386-linux to x86_64-linux there, BTW. :)
The same information with more explanations is in the manual.

Thanks again,
Karl
Karl Berry
2018-09-20 23:21:22 UTC
Permalink
zw> Will TL contain i386 binaries for after-end-of-life distros with
obsolete versions of glibc?

We're still producing binaries for centos 6 == glibc 2.12. They're
available from tug.org/texlive/bugs.html (along with the Build
invocation to do it). It lacks luatex, xetex, etc., since those can no
longer be compiled on that old of a system. The situation with luatex
might change for the better next year, though.

Older than that, no.

They are not included in TL itself; that's too much trouble for too
little gain. --best, karl.
Karl Berry
2018-09-20 23:21:21 UTC
Permalink
Hi Michael and all -

If you or Boris or anyone wants to produce a distribution on USB, by all
means, go for it. We could sell it through the user groups or make it an
optional extra for members. Whatever.

I wouldn't try to integrate it with the current image creation at all.
(which is basically a script that sets a bunch of options and runs
mkisofs, or rather xorriso in mkisofs mode). Creating something for USB
would surely have to be quite different.

Most platforms are about 150MB. That adds up to a lot of packages. It
might be that we freed up enough space last year for some years to come.
We never know until it comes time to build the final DVD image.

No "features" have ever been removed to make space. The reason that
removing platform binary sets is attractive is that it can be done
without impacting anything else -- the available platform list is
already computed dynamically by the installer. Whereas removing anything
else (say, a huge font package) implies many more infrastructure
changes. I don't recall a single user complaint that the removal of the
less common platforms has caused a problem in practice ...

Archiving for posterity doesn't seem like an issue to me. All TL
distributions are available online and will be forever (as far as I'm
concerned). If someone absolutely needs some old distribution and
doesn't have the bandwidth to download, they can ask someone who does.
TUG also has old DVDs for most years (of zero interest to the general
populace), and I imagine other user groups do too.

The feedback I've gotten is that relatively few members actually use the
DVD. Thus I'm not excited about going to the trouble and expense of
making a second DVD with the removed platform binaries. There would be
no way to integrate it into the installation, for sure.

I heartily agree with your comments about public standards.

Best,
Karl
Michael Shell
2018-09-23 00:20:53 UTC
Permalink
On Thu, 20 Sep 2018 23:21:21 GMT
Creating something for USB would surely have to be quite different.
Thanks for info Karl.

You know, when I first looked into this, I said to myself, "surely I
can come up with an easy suggestion to get some more space on the DVD".

The ideas that quickly came to mind included:

1. Use a double layer DVD.
2. Use compression and/or
3. Use an improved compression algorithm
e.g., .xz instead of .gz or .zip.
4. Increase the compression level setting.

Of course, given the unusually high intelligence and dedication of TUG
members/developers, I quickly realized that all the above had already
been done. Well, I didn't actually check #4 (e.g., xz -9), but by the
time I got to that one, I was willing to bet that already had been done
too (or if not, that it is known the higher RAM requirement of the -9
setting would be problematic on some systems).

Had it been any other group, chances are I would have been able to offer
some quick and useful advice, but no, not in TeX circles. As we say here
in the South, "it ain't that easy 'round them there parts".
LOL!

On Sat, 22 Sep 2018 13:59:43 +0900
There is no filesystem that is uniformly accepted *AND* which
support symbolic links - we need symbolic links because
all the Unix-like systems have links for the biggest part of the
"binaries".
If we had to, I think we could overcome the link issue. Links would
be implemented as text files, maybe with a unique suffix (e.g., .lnk)
whose text contains the path, or rules for the path, to the object
to be linked to. The installer then would detect and interpret these
.lnk files as needed for the given target system during the installation.

But, I have to ask, how does the installer handle the case where the
*target* filesystem of the installation does not support links?
(e.g., FAT32)?


Kudos,

Mike
Norbert Preining
2018-09-23 00:28:36 UTC
Permalink
Hi Mike,
Post by Michael Shell
been done. Well, I didn't actually check #4 (e.g., xz -9), but by the
I think we do not use -9, due to heavy load on build and install
systems.
Post by Michael Shell
If we had to, I think we could overcome the link issue. Links would
be implemented as text files, maybe with a unique suffix (e.g., .lnk)
whose text contains the path, or rules for the path, to the object
to be linked to. The installer then would detect and interpret these
.lnk files as needed for the given target system during the installation.
That could indeed be an option BUT it would defeat the whole idea of
having the ability to use the install media also as live system.

If we go for tlnet (archive/*.xz) then there *are* no symlinks and any
filesystem (maybe even vfat) would be fine.

But if we want a USB stick that can be used as installer as well as
running a live system links on Unix are necessary.
Post by Michael Shell
But, I have to ask, how does the installer handle the case where the
*target* filesystem of the installation does not support links?
(e.g., FAT32)?
Not at all. If you are running Unix and install to a vfat/fat32 system
then you are hosed ;-)

Best

Norbert

--
PREINING Norbert http://www.preining.info
Accelia Inc. + JAIST + TeX Live + Debian Developer
GPG: 0x860CDC13 fp: F7D8 A928 26E3 16A1 9FA0 ACF0 6CAC A448 860C DC13
Michael Shell
2018-09-23 00:59:46 UTC
Permalink
On Sun, 23 Sep 2018 09:28:36 +0900
Post by Norbert Preining
Not at all. If you are running Unix and install to a vfat/fat32 system
then you are hosed ;-)
What I mean is, a user is running MS Windows with a FAT32 filesystem,
and they want to install Tex Live on it using the DVD. They can do
that, right?

But, I do see a big problem with regard to running directly on a USB drive
(The "Live" part of TeX Live) on filesystems without symlinks. I suppose
the linked files would just have to be duplicated, and then there would
have to be a data file that contains all the link information. During
installation to filesystem that does have symlink ability, the duplicate
("link") files are simply replaced with symlinks instead of being copied
as-is. Well, something like that.


Mike
Norbert Preining
2018-09-24 02:02:15 UTC
Permalink
Post by Michael Shell
What I mean is, a user is running MS Windows with a FAT32 filesystem,
and they want to install Tex Live on it using the DVD. They can do
that, right?
Windows part of TeX Live of course doesn't have symlinks! And if you
select *additional* unix archs for installation that will probably break
on FAT32. But normal Windows installation of course works without any
problems.
Post by Michael Shell
But, I do see a big problem with regard to running directly on a USB drive
(The "Live" part of TeX Live) on filesystems without symlinks. I suppose
the linked files would just have to be duplicated, and then there would
have to be a data file that contains all the link information. During
installation to filesystem that does have symlink ability, the duplicate
("link") files are simply replaced with symlinks instead of being copied
as-is. Well, something like that.
Or, on the USB all the symlinks are replaced by the actual files, and on
the next update with tlmgr they are converted to links.

But also this is problematic, because some scripts *might* use readlink
etc to find the position of the real script and use the derived
directory, which would break again.

Best

Norbert

--
PREINING Norbert http://www.preining.info
Accelia Inc. + JAIST + TeX Live + Debian Developer
GPG: 0x860CDC13 fp: F7D8 A928 26E3 16A1 9FA0 ACF0 6CAC A448 860C DC13
Norbert Preining
2018-09-22 04:59:43 UTC
Permalink
Hi everyone,

there have been so many comments, but as the one who made lots of
experiments with USBs on all the big platforms here a few comments:

Tere is no filesystem that is uniformly accepted *AND* which
support symbolic links - we need symbolic links because
all the Unix-like systems have links for the biggest part of the
"binaries".
UDF would be the best option, but has problems on both Windows
and MacOS.

NTFS would be also possible, but it is not supported on MacOS out
of the box in most cases.

(After finishing the email and re-doing research) It seems that more
recent versions of MacOS *do* support *reading* NTFS out of the box
(and read/write can easily established by remounting or other options).

That means we could consider doing USB sticks with NTFS as base file
system, which should work out of the box at least on Win, Mac, Linux.

I can try that out easily, but I don't have a recent Mac to test.
If there is enough interest I can produce a USB disk image that one can
download and flash onto a USB stick for testing.
Post by George N. White III
FAT32 is widely supported but needs large sector sizes for large
media. This may not be a problem for a filesystem populated with
archives. VFAT (long filenames) has been supported in linux for
years.
Both don't support links.
Post by George N. White III
Binaries are, IMHO, not the cause of the problem, nor is their removal the solution.  The TeX Live installer offers a number of schemes -- why not pack all binaries and the files needed by as many of the the smaller schemes as possible onto DVD-1 and the additional files needed by the larger schemes onto DVD-2 ?   As the size of individual schemes increases, this solution will scale to accommodate DVD-3, etc.
This would require considerable change of the installer program - which
nobody has currently energy and interest to implement.
Post by George N. White III
I would, however, support making a limited edition of TL USBs with
beautiful logos for special purposes: gifts, prizes, promotional
materials etc. Not as a substitute for our regular mass-produced
That of course is easily possible. One could have two or three types
(Windows - Mac - Unix) and format it with different file systems.
Post by George N. White III
USB drives can be partitioned just like regular hard drives. So, they
can carry multiple filesystems. However, we would want to do this in a
way that minimizes redundancy.
That is the *only* possible option: partition, copy everything two
times.

One *could* reuse some parts of the vfat (windows) part for the Unix
installation and only take the bin/* part from a dedicated Unix
partition (for the links), but that again needs serious work on the
installer, currently a no-go.
Post by George N. White III
We could even go with 32GB USB units, and/or expect the unix systems
to be able to mount FAT32 and/or NTFS filesystems.
As said, NTFS would be definitely fine, but MacOS does not support it
AFAIR. At least the version I had and tested needed third-party
drivers installed. (See above for new infos)
Post by George N. White III
Alternatively, I would even consider a more radical approach - use the
linux ext2 filesystem to carry the vast bulk of the files and then
include a partition(s) with drivers to allow ext2 to be read under
No no and no ...


Best

Norbert

--
PREINING Norbert http://www.preining.info
Accelia Inc. + JAIST + TeX Live + Debian Developer
GPG: 0x860CDC13 fp: F7D8 A928 26E3 16A1 9FA0 ACF0 6CAC A448 860C DC13
George N. White III
2018-09-22 12:23:58 UTC
Permalink
Post by Norbert Preining
Hi everyone,
there have been so many comments, but as the one who made lots of
Very helpful, but if the extra space is to be used for less popular
platforms then we do need to consider USB support on less popular
platforms.
Post by Norbert Preining
Tere is no filesystem that is uniformly accepted *AND* which
support symbolic links - we need symbolic links because
all the Unix-like systems have links for the biggest part of the
"binaries".
So an all-platforms USB would have to be a "mirror" of tlnet.
The installer (install-tl.{tar.gz,zip}) would have to be extracted
to a native filesystem and run using the USB drive location
in the installer's "-repository" option. This is a bit complex
for many users who have become accustomed to installing
from an App Store.
Post by Norbert Preining
UDF would be the best option, but has problems on both Windows
and MacOS.
NTFS would be also possible, but it is not supported on MacOS out
of the box in most cases.
(After finishing the email and re-doing research) It seems that more
recent versions of MacOS *do* support *reading* NTFS out of the box
(and read/write can easily established by remounting or other options).
Correct, but there are many older, mostly 32-bit, Macs still running Snow
Leopard.
Post by Norbert Preining
That means we could consider doing USB sticks with NTFS as base file
system, which should work out of the box at least on Win, Mac, Linux.
These newer systems don't need the binaries for less popular OS's,
so the audience is users who have a recent system (USB 3 but no
DVD drive), and limited internet.
Post by Norbert Preining
I can try that out easily, but I don't have a recent Mac to test.
If there is enough interest I can produce a USB disk image that one can
download and flash onto a USB stick for testing.
The future seems to be thin and light systems with USB 3 but without
DVD drives, so such tests would be useful. It would help to collect
data on the performance of USB installs using current USB 3 hardware.
https://thewirecutter.com/reviews/the-best-usb-3-0-thumb-drive/ has some
information on the current state of USB flash drives. My experience with
DVD installs is very limited, but as I recall they were very slow. Users
could buy external USB DVD drives, but if they are only going to use a
DVD drive for TL installs then USB flash drives are probably cheaper and
faster.
Post by Norbert Preining
Post by George N. White III
FAT32 is widely supported but needs large sector sizes for large
media. This may not be a problem for a filesystem populated with
archives. VFAT (long filenames) has been supported in linux for
years.
Both don't support links.
Post by George N. White III
Binaries are, IMHO, not the cause of the problem, nor is their removal the solution. The TeX Live installer offers a number of schemes -- why not pack all binaries and the files needed by as many of the the smaller schemes as possible onto DVD-1 and the additional files needed by the larger schemes onto DVD-2 ? As the size of individual schemes increases, this solution will scale to accommodate DVD-3, etc.
This would require considerable change of the installer program - which
nobody has currently energy and interest to implement.
If the number of TL users with access to DVD readers is, as I suspect,
in decline,
investing in a multi-DVD installer won't have long-term benefits. The
older systems
that can use DVD's probably include many of the 32-bit OS's that will
loose out if
the DVD's going forward only support current mainstream 64-bit OS's.

[...]


--
George N. White III
Zdenek Wagner
2018-09-22 18:05:33 UTC
Permalink
Just a note, my El Capitan 10.11.6 does not support NTFS. I bought a
third party driver. I was on a conference this July, there was an
official MacBook for showing the slides and it was unable to read my
NTFS USB stick. Fortunatelly I tried well in advance so I could send
it to the organizers by an e-mail from my MacBook and they transfered
it to the official one.

Zdeněk Wagner
http://ttsm.icpf.cas.cz/team/wagner.shtml
http://icebearsoft.euweb.cz
Norbert Preining
2018-09-23 00:00:58 UTC
Permalink
Post by Zdenek Wagner
Just a note, my El Capitan 10.11.6 does not support NTFS. I bought a
I think it needs to be even newer, maybe 10.15 .. but well, that is all
a pain. Let us drop the idea of NTFS again.

Norbert

--
PREINING Norbert http://www.preining.info
Accelia Inc. + JAIST + TeX Live + Debian Developer
GPG: 0x860CDC13 fp: F7D8 A928 26E3 16A1 9FA0 ACF0 6CAC A448 860C DC13
Karl Berry
2018-09-23 23:04:37 UTC
Permalink
Hi Michael,

xz -9

For the record, I did indeed experiment with xz -9 (and other values).
It took tooooooo many hours to create the image for hardly any
additional compression. Indeed, I don't think there's any easy way
left :( ... --best, karl.
Johannes Hielscher
2018-09-27 20:34:58 UTC
Permalink
Am Sun, 23 Sep 2018 23:04:37 GMT
Post by Michael Shell
xz -9
For the record
44M xz -1 20.05 user
42M xz -2 27.46 user
41M xz -3 30.73 user
38M xz -4 60.63 user
36M xz -5 67.72 user
36M xz -6 81.04 user
34M xz -7 77.91 user
32M xz -8 77.42 user
19M xz -9 70.55 user

texlive_bin_aarch64-linux.tar (uncompressed: 141M)
xz-5.2.2 on a 2009ish Xeon L5520 @ 2.27GHz

(not taking into account the astonishing RAM costs of high compression
ratios, at both compression and decompression)
Michael Shell
2018-09-28 03:02:13 UTC
Permalink
On Thu, 27 Sep 2018 22:34:58 +0200
Post by Johannes Hielscher
36M xz -6 81.04 user
34M xz -7 77.91 user
32M xz -8 77.42 user
19M xz -9 70.55 user
texlive_bin_aarch64-linux.tar (uncompressed: 141M)
Another thing to note is in the man page of xz:

-7 ... -9
These are like -6 but with higher compressor and decompressor memory
requirements. These are useful only when compressing files bigger
than 8 MiB, 16 MiB, and 32 MiB, respectively.


So the -9 gain in your example, which indeed is very significant
(36MB for -6 versus 19MB for -9), can only be obtained with such large
files. This explains Karl's seemingly contradictory result:

On Sun, 23 Sep 2018 23:04:37 GMT
Post by Johannes Hielscher
For the record, I did indeed experiment with xz -9 (and other values).
It took tooooooo many hours to create the image for hardly any
additional compression.
in that, in the TeX Live collection, the number of files over 32MB is
not a high enough percentage of the total to provide much of a
compression advantage for the -9 level. Of course, one could organize
the file collection into a few dozen .tar "parts" compress those and then
decompress/untar each part as needed to the target's filesystem during
install. Tis not pretty and it seems would demand too much of the install
machine in terms of CPU, RAM and drive space, but according to Johannes's
results above, we could potentially get a near halving of the media space
required for the distribution DVD with such an approach.


Cheers,

Mike
t***@schoepfer.info
2018-09-28 11:34:49 UTC
Permalink
As a sidenote, maybe it's worth looking at lzip. It won't compress
better, but it seems to be safer and therefor somehow better, see
https://www.nongnu.org/lzip/xz_inadequate.html

Johannes
Norbert Preining
2018-09-28 14:01:03 UTC
Permalink
Hi
As a sidenote, maybe it's worth looking at lzip. It won't compress better,
but it seems to be safer and therefor somehow better, see
https://www.nongnu.org/lzip/xz_inadequate.html
Lots of statements, but not really convincing. Reading through the
linked Debian thread starting at
https://lists.debian.org/debian-devel/2015/06/msg00173.html
no conclusive answer is found.

The biggest critique is about data checking, which we do anyway after
download with separate sha256 hashes, so this is not an issue.

I agree that in principle a simpler format with only one container and
no nesting of containers and variety of filters would be nice, though.

But, it is c++ and although I don't know what level of it, supporting
lzip would mean building lzip utils on all our supported systems which
might be a pain.

Best

Norbert

--
PREINING Norbert http://www.preining.info
Accelia Inc. + JAIST + TeX Live + Debian Developer
GPG: 0x860CDC13 fp: F7D8 A928 26E3 16A1 9FA0 ACF0 6CAC A448 860C DC13
Arthur Reutenauer
2018-09-28 14:15:36 UTC
Permalink
Post by Norbert Preining
As a sidenote, maybe it's worth looking at lzip. It won't compress better,
but it seems to be safer and therefor somehow better, see
https://www.nongnu.org/lzip/xz_inadequate.html
Lots of statements, but not really convincing.
Indeed not. The point, repeated many times, that the xz format is
inadequate for long-term archiving is not really an issue for TeX Live.
I’m also amused by the statement that the format is “unreasonably
extensible” :-)

And, needless to say, it weakens the argument that the writer is also
the author of the competing program, even if he’s being honest and
discloses it.

Best,

Arthur
Karl Berry
2018-09-28 23:03:30 UTC
Permalink
So the -9 gain in your example, which indeed is very significant

Thanks for analyzing this, Michael.

I might have actually given up on xz -9 before it finished. I don't
remember for sure any more what I tried. I know -9 was not a practical
thing to do on the entire TL distribution.

It's true there are relatively few files that are >8,16,32 mb. I also
worry that the increased memory requirements on the user's side might
end up being a problem.

the file collection into a few dozen .tar "parts" compress those

Yeah. Well, one year at a time ... --thanks again, karl.
Zdenek Wagner
2018-09-29 07:55:26 UTC
Permalink
Post by Michael Shell
So the -9 gain in your example, which indeed is very significant
...
It's true there are relatively few files that are >8,16,32 mb. I also
worry that the increased memory requirements on the user's side might
end up being a problem.
It should not be a problem for modern desktop computers but will definitely
be a problem for computers as Raspberry Pi.
Post by Michael Shell
the file collection into a few dozen .tar "parts" compress those
Yeah. Well, one year at a time ... --thanks again, karl.
Zdeněk Wagner
http://ttsm.icpf.cas.cz/team/wagner.shtml
http://icebearsoft.euweb.cz

Loading...