Problems with makepdf.sh and general suitability of docs for PDF conversion

[JMadgwick]

Since makepdf.sh was last updated, pandoc has deprecated the form of the command used in the script. This is easily fixed but the resulting PDFs are not very good. Defaults are used for everything. The paper size has the large default LaTex margins, tables are crushed with overlapping text, hyperlinks are not colored or missing (if inline HTML was used), among other problems. The makepdf.sh script itself is not documented or mentioned on the getting started page.

Unfortunately it's not possible to fix the above problems with a small update to the script and docs. Pandoc is used for the conversion from markdown to PDF (via LaTex) and this doesn't support the mix of html and markdown which is used in many of the pages. Some pages also have problems with extraneous HTML tags. These don't cause an issue with MkDocs but do with Pandoc - e.g. docs/books/advanced.md has an incorrect </p> tag inside various tables, which prevents the tables from generating.

I can see two ways to fix this:

1. The proper fix - Go through the docs and replace all inline HTML with equivalent markdown (e.g. hyperlinks with markdown links) if the HTML is intended to be used in the PDF. This is required because Pandoc suppresses all HTML when using the LaTex output mode.
2. The quick fix - Change the script to add an extra step where Pandoc produces HTML output first and then converts that to PDF (via LaTex). This would in effect be a conversion of: [markdown -> HTML] plus [HTML -> markdown -> LaTex -> PDF]. This would fix the existing problems of a mix of markdown and HTML formatting. But it would come at the cost of much increased processing time and loss of certain formatting which doesn't convert well between markdown -> HTML.

I've not looked at how much work would be involved for 1. In addition to the above, I would suggest adding Pandoc formatting options to color hyperlinks, use normal margins and I'd also suggest using a san-serif font.

I'm happy to have a go at fixing the problem and submitting a PR. But considering the work involved and the number of changes that might be required, I thought it would be best to raise an issue explaining the problem first. Rather than making a large PR for work that is not wanted - OI might not be interested in having PDF versions of the docs.

[Andreas Wacknitz]

Am 27.05.21 um 15:05 schrieb J Madgwick:

Since |makepdf.sh| was last updated, pandoc has deprecated the form of the command used in the script. This is easily fixed but the resulting PDFs are not very good. Defaults are used for everything. The paper size has the large default LaTex margins, tables are crushed with overlapping text, hyperlinks are not colored or missing (if inline HTML was used), among other problems. The |makepdf.sh| script itself is not documented or mentioned on the getting started page <http://docs.openindiana.org/contrib/getting-started/>. Unfortunately it's not possible to fix the above problems with a small update to the script and docs. Pandoc is used for the conversion from markdown to PDF (via LaTex) and this doesn't support the mix of html and markdown which is used in many of the pages. Some pages also have problems with extraneous HTML tags. These don't cause an issue with MkDocs but do with Pandoc - e.g. |docs/books/advanced.md| has an incorrect |</p>| tag inside various tables, which prevents the tables from generating. I can see two ways to fix this: 1. *The proper fix* - Go through the docs and replace all inline HTML with equivalent markdown (e.g. hyperlinks with markdown links) if the HTML is intended to be used in the PDF. This is required because Pandoc suppresses all HTML when using the LaTex output mode <https://stackoverflow.com/a/50552378>. 2. *The quick fix* - Change the script to add an extra step where Pandoc produces HTML output first and then converts that to PDF (via LaTex). This would in effect be a conversion of: [markdown -> HTML] plus [HTML -> markdown -> LaTex -> PDF]. This would fix the existing problems of a mix of markdown and HTML formatting. But it would come at the cost of much increased processing time and loss of certain formatting which doesn't convert well between markdown -> HTML. I've not looked at how much work would be involved for *1*. In addition to the above, I would suggest adding Pandoc formatting options to color hyperlinks, use normal margins and I'd also suggest using a san-serif font. I'm happy to have a go at fixing the problem and submitting a PR. But considering the work involved and the number of changes that might be required, I thought it would be best to raise an issue explaining the problem first. *Rather than making a large PR for work that is not wanted* - OI might not be interested in having PDF versions of the docs. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#181>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADET4N2ZORQHAUTI4NZ2CCLTPY7TBANCNFSM45UIANBA>.

What you wrote sounds reasonable. Our documentation is lacking due to lack of time or interest. I am happy to integrate whatever looks like an enhancement to the current state. But you should be aware that people who are too active here will get an invitation to join the docs team (which means: get commit rights) ;) Regards, Andreas

[Andreas Wacknitz]

Am 27.05.21 um 15:05 schrieb J Madgwick:

Since |makepdf.sh| was last updated, pandoc has deprecated the form of the command used in the script. This is easily fixed but the resulting PDFs are not very good. Defaults are used for everything. The paper size has the large default LaTex margins, tables are crushed with overlapping text, hyperlinks are not colored or missing (if inline HTML was used), among other problems. The |makepdf.sh| script itself is not documented or mentioned on the getting started page <http://docs.openindiana.org/contrib/getting-started/>. Unfortunately it's not possible to fix the above problems with a small update to the script and docs. Pandoc is used for the conversion from markdown to PDF (via LaTex) and this doesn't support the mix of html and markdown which is used in many of the pages. Some pages also have problems with extraneous HTML tags. These don't cause an issue with MkDocs but do with Pandoc - e.g. |docs/books/advanced.md| has an incorrect |</p>| tag inside various tables, which prevents the tables from generating. I can see two ways to fix this: 1. *The proper fix* - Go through the docs and replace all inline HTML with equivalent markdown (e.g. hyperlinks with markdown links) if the HTML is intended to be used in the PDF. This is required because Pandoc suppresses all HTML when using the LaTex output mode <https://stackoverflow.com/a/50552378>. 2. *The quick fix* - Change the script to add an extra step where Pandoc produces HTML output first and then converts that to PDF (via LaTex). This would in effect be a conversion of: [markdown -> HTML] plus [HTML -> markdown -> LaTex -> PDF]. This would fix the existing problems of a mix of markdown and HTML formatting. But it would come at the cost of much increased processing time and loss of certain formatting which doesn't convert well between markdown -> HTML. I've not looked at how much work would be involved for *1*. In addition to the above, I would suggest adding Pandoc formatting options to color hyperlinks, use normal margins and I'd also suggest using a san-serif font. I'm happy to have a go at fixing the problem and submitting a PR. But considering the work involved and the number of changes that might be required, I thought it would be best to raise an issue explaining the problem first. *Rather than making a large PR for work that is not wanted* - OI might not be interested in having PDF versions of the docs. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#181>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADET4N2ZORQHAUTI4NZ2CCLTPY7TBANCNFSM45UIANBA>.

Are you subscribed to the oi-dev mailing list? There are some people offering to work on the documention. I would like to ask all of you to discuss your ideas and coordinate the work.

[JMadgwick]

I've opened a PR #183 to fix this issue. I've sent some emails out on oi-dev. Hopefully this will stoke some more interest in improving the content - this work is more of a technical change to improve formatting consistency and add PDF generating capabilities.
If this is accepted then I'll update docs/contrib/getting-started.md to document the details of the PDF generation script and the new files added.

[JMadgwick]

Now merged, thanks. Here's an example PDF converted with the new script: getting-started.pdf