Giters

jgm / pandoc

Universal markup converter

Home Page:https://pandoc.org

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

longtable not compatible with 2-column LaTeX documents

nrnrnr opened this issue · comments

commented

For some time I have been using pandoc to create two-column PDF documents via LaTeX. (This can be accomplished with some abuse of the fontsize variable.) However, in the transition from 1.9.x to 1.11.1, all tables appear to have become longtable environments, and when presented with two-column format, longtable falls over:

pandoc: Error producing PDF from TeX source.
! Package longtable Error: longtable not in 1-column mode.

See the longtable package documentation for explanation.
Type  H <return>  for immediate help.
 ...                                              

l.267 \begin{longtable}

I see in past issues that the transition from ctable to longtable was decided on a while back, but I really hate to lose the ability to create two-column documents, which I use heavily. I also understand the issue of wanting to stick with relatively standard LaTeX packages.

Is it possible that the old ctable back end could be resurrected via some kind of command-line option? It sounds horrible, but I can't think of a better solution.

commented

Regarding the possibility of using longtable in 2-column documents, I found this thread on a Stack Exchange site, but it is not promising: http://tex.stackexchange.com/questions/39686/longtable-alternative-for-twocolumn-documents

commented

I have a related issue that probably isn't worth opening another issue ticket for. Rather than a "resurrect ctable" solution, I'd prefer an option that just turns tables into very basic tabular environments. Or maybe a new style of markdown (or some sort of in-text flag) that outputs basic tabulars.

commented

As a workaround, one can pandoc into tex and then use sed (or whatever) to replace all instances of longtable with tabular or ctable or whatever other environment you want...

commented

@scmbradley is it really that simple?

commented

@nrnrnr @scmbradley From what I can tell, if you use tabular, you also have to add table environments. I can do this manually, but I don't see a way to do it with the template.

commented

It would be indeed very useful if ctable, longtable or other similar environments would not be hard-coded, but the user could pass it e.g. via a pandoc option.

commented

I solved this by using this tip: replacing longtable with supertabular made it work for me.

commented

When replacing longtable with supertabular, you may get error about undefined command \endhead. Simply remove that from the LaTeX source.

To center the table, wrap them between \begin{center} and \end{center}.

Here's the code I add to the build process (in Ruby). result is the conversion result from Pandoc (actually generated from ipython nbconvert):

result.gsub!('\begin{longtable}', '\begin{center}\begin{supertabular}')
result.gsub!('\endhead', '')
result.gsub!('\end{longtable}', '\end{supertabular}\end{center}')
commented

Hey everyone, sorry for resurrecting this one -- do we have any way of resolving this beyond using sed? In theory, it should be able to just amend the latex template -- but the latex template does not seem to allow edits to the table structure. Any ideas?

commented

If there were no other differences, you could do a
renewenvironment in the template, and redefine longtable
as table. E.g.

\let\longtable\table
\let\endlongtable\endtable

But there are some differences between the two environments,
e.g. as regards handling of footnotes, so this probably
won't work in general. (It might work sometimes.)

+++ Nikos Vasilakis [Aug 17 15 20:32 ]:

Hey everyone, sorry for resurrecting this one -- do we have any way of
resolving this beyond using sed? In theory, it should be able to just
amend the latex template -- but the latex template does not seem to
allow edits to the table structure. Any ideas?

—
Reply to this email directly or [1]view it on GitHub.

References

  1. #1023 (comment)
commented

Thanks John! This does not work though, I think because the generated table code has a number extra stuff -- for instance: ! LaTeX Error: Unknown float optionc'.`.

However, I am now thinking: wouldn't a solution based on detecting supertabular work? For example, in LaTeX.hs source (intended mostly as pseudocode -- my haskell is rusty):

  return ( if ("supertabular" `isInfixOf` (writerTemplate opts)) then
           ( "\\begin{supertabular}[c]" <>
                braces ("@{}" <> colDescriptors <> "@{}")
           $$ capt
           $$ headers
           $$ vcat rows'
           $$ "\\end{supertabular}"
          )
         else -- normal old behavior
           ( "\\begin{longtable}[c]" <>
                braces ("@{}" <> colDescriptors <> "@{}")
                -- the @{} removes extra space at beginning and end
           $$ capt
           $$ "\\toprule"
           $$ headers
           $$ endhead
           $$ vcat rows'
           $$ "\\bottomrule"
           $$ "\\end{longtable}"
           )
         )
commented

I don't really like making behavior sensitive to what's in
the template in the way you describe.

What about doing what I suggested, but with supertabular
instead of table? Does that work? Note that the midline,
topline, bottomline are from booktabs and should still work
with supertabular, I think.

+++ Nikos Vasilakis [Aug 17 15 23:23 ]:

Thanks John! This does not work though, I think because the generated
table code has a number extra stuff -- for instance: ! LaTeX Error:
Unknown float optionc'.`.

However, I am now thinking: wouldn't a solution based on detecting
supertabular work? For example, in LaTeX.hs source (intended mostly as
pseudocode -- my haskell is rusty):

return ( if ("supertabular" isInfixOf (writerTemplate opts)) then
( "\begin{supertabular}[c]" <>
braces ("@{}" <> colDescriptors <> "@{}")
$$ capt
$$ headers
$$ vcat rows'
$$ "\end{supertabular}"
)
else -- normal old behavior
( "\begin{longtable}[c]" <>
braces ("@{}" <> colDescriptors <> "@{}")
-- the @{} removes extra space at beginning and end
$$ capt
$$ "\toprule"
$$ headers
$$ endhead
$$ vcat rows'
$$ "\bottomrule"
$$ "\end{longtable}"
)
)

—
Reply to this email directly or [1]view it on GitHub.

References

  1. #1023 (comment)
commented

Changing the lines you suggested to:

\let\longtable\supertabular
\let\endlongtable\endsupertabular

while adding:

\usepackage{supertabular}
\usepackage{booktabs}

..still doesn't work:

! Undefined control sequence.
<recently read> \endhead

My goal is to have tables in two-column latex documents. Do you use a different way to achieve a similar result in your documents?

Regarding the psudo-patch I proposed, I know it's not optimal, it's just to get people going until a proper solution gets in. Another simple idea might be to just default to supertabular or xtab.

commented

+++ Nikos Vasilakis [Aug 18 15 10:45 ]:

My goal is to have tables in two-column latex documents. Do you use a different way to achieve a similar result in your documents?

I've never tried this.

Regarding the psudo-patch I proposed, I know it's not optimal, it's just to get people going until a proper solution gets in. Another simple idea might be to just default to supertabular or xtab.

The problem is that if we do support it in some versions,
people will have documents that depend on this behavior,
and we'll have to keep it or make breaking changes.

One possibility would be to check for twocolumn in the
classoption variable, and use a regular table environment
if so. But, as I recall, there are still some issues
relating to different handling of footnotes in table and
longtable -- maybe other things too.

What are the advantages of supertabular or xtab over table?

commented

As a workaround, someone (thanks!) has created a filter for using tabular instead of longtable in twocolumn environment (e.g. IEEEtrans, after some struggle I finally find a solution!! I tried it and it works), check it out here:

https://groups.google.com/forum/#!msg/pandoc-discuss/RUC-tuu_qf0/h-H3RRVt1coJ

To use it:

e.g.
$ pandoc SORUCE -o OUT.tex --filter table-filter.py

commented

I tried this, and there are a number of issues with
this filter, though it may be fine for simple cases:

  1. The filter crashes if you have a table without headers.

  2. Because the filter removes Table elements from the
    AST, the latex writer won't include

    \usepackage{booktabs}

in the header; you'll need to add this yourself through
-H or a custom template, or by doing -V tables.

  1. Relative column widths are not kept, so tables with long
    cell contents will stretch beyond page width.
  2. Finally, footnotes don't work in these tables.

I'd be open to adding an option to pandoc to use regular
tabular, if I could figure out how to make notes work
properly, etc.

+++ Aziz Alto [Nov 29 15 02:52 ]:

As a workaround, someone (thanks!) has created a filter for using
tabular instead of longtable in twocolumn environment (e.g. IEEEtrans,
after some struggle I finally find a solution!! I tried it and it
works), check it out here:

[1]https://groups.google.com/forum/#!msg/pandoc-discuss/RUC-tuu_qf0/h-H
3RRVt1coJ

To use it:
* copy the code from that link into some file table-filter.py
* Install [2]pandocfilters
* use the filter when you [3]convert you doc to latex,

e.g.
$ pandoc SORUCE -o OUT.tex --filter table-filter.py

—
Reply to this email directly or [4]view it on GitHub.

References

  1. https://groups.google.com/forum/#!msg/pandoc-discuss/RUC-tuu_qf0/h-H3RRVt1coJ
  2. https://github.com/jgm/pandocfilters#installing
  3. https://github.com/jgm/pandocfilters#what-are-pandoc-filters
  4. #1023 (comment)
commented

That's right! however in my case the filter does work with IEEEtran class without any significant issue. I do use many simple tables (with headers) in my docs, and it's not feasible to convert them manually.

I add \usepackage{booktabs} to the header of the template I use. Also another option (instead of using booktabs) would be to replace \toprule, \bottomrule, and \midrule with \hline in the filter.py.

The tables with long cells work fine in the IEEEtran, but not in the other twocolumn document classes (e.g. IEEEconf, [twocolumn]{journal} ).

I have not tried the footnotes. Thank you.

commented

I would really like to see this get resolved without resorting to running external scripts and such. Since two-column formats are pretty common for LaTeX documents would it be possible to have a separate output mode for them as part of pandoc itself?

commented

I'd like to solve this too. What I need to know is how to reproduce our current table features using regular tabular. As noted above, two features that are a bit tricky are (a) footnotes in table cells and (b) relative widths in columns.

Once I've got this down, we could do several things. One option would be to have a command-line option to use regular tabular instead of longtable. Another would be to have tables inside a specially marked div treated specially (perhaps the special behavior should be longtable instead of regular tabular).

commented

I'd be willing to take a crack at this, possibly over the break. It looks like the relevant parts in the LaTeX writer are the functions blockToLatex (the Table case), tableRowToLatex and tableCellToLatex. I haven't used footnotes, either in markdown or pandoc, so I'll have to look into that.

commented

I'd have no trouble doing the Haskell bits. I just need to
know exactly what output to produce.

+++ Shrutarshi Basu [Dec 20 15 13:21 ]:

I'd be willing to take a crack at this, possibly over the break. It
looks like the relevant parts in the LaTeX writer are the functions
blockToLatex (the Table case), tableRowToLatex and tableCellToLatex. I
haven't used footnotes, either in markdown or pandoc, so I'll have to
look into that.

—
Reply to this email directly or [1]view it on GitHub.

References

  1. #1023 (comment)
commented

Hi, I have converted a multiline table example from the Pandoc user's guide to use tabular / table, and footnotes (courtesy of http://tex.stackexchange.com/a/35328) as well as relative widths seem to work.

The LaTeX code:

\documentclass{article}
\usepackage{hyperref}
\usepackage{tablefootnote}
\usepackage{booktabs}

\begin{document}
\begin{table}
\caption{Here's a multiline table without headers.}
\begin{tabular}{ c l r l }
\tabularnewline
\toprule
\toprule
\begin{minipage}[t]{0.15\columnwidth}\centering\strut
First
\strut\end{minipage} &
\begin{minipage}[t]{0.10\columnwidth}\raggedright\strut
row
\strut\end{minipage} &
\begin{minipage}[t]{0.20\columnwidth}\raggedleft\strut
12.0
\strut\end{minipage} &
\begin{minipage}[t]{0.31\columnwidth}\raggedright\strut
Example of a row that spans multiple lines.\tablefootnote{Footnotes also work.}
\strut\end{minipage}\tabularnewline
\begin{minipage}[t]{0.15\columnwidth}\centering\strut
Second
\strut\end{minipage} &
\begin{minipage}[t]{0.10\columnwidth}\raggedright\strut
row
\strut\end{minipage} &
\begin{minipage}[t]{0.20\columnwidth}\raggedleft\strut
5.0
\strut\end{minipage} &
\begin{minipage}[t]{0.31\columnwidth}\raggedright\strut
Here's another one. \tablefootnote{Another footnote.}Note the blank line between rows.
\strut\end{minipage}\tabularnewline
\bottomrule
\end{tabular}
\end{table}

\end{document}
commented

Hi, will there be a command option/metadata variable for us to choose which kind of table will be used in LaTeX? From another thread it seems pandoc had changed the package it use for table but the old way wasn't kept. It would be nice if pandoc allow a choice of which conversion to use.

And is the use of table package open for discussion? I know that MultiMarkdown use tabulary, and probably tabularx has better footnote in table support (I once need to manually turn the tabulary table in mmd to a tabularx one). And I heard that tabu is supposed to be the newest and best package for table but haven't tried it yet.

commented

See #2384, #1023, and related discussions in pandoc-discuss.
I'd like to have a better solution here.

+++ ickc [Apr 26 16 22:56 ]:

Hi, will there be a command option/metadata variable for us to choose
which kind of table will be used in LaTeX? From another thread it seems
pandoc had changed the package it use for table but the old way wasn't
kept. It would be nice if pandoc allow a choice of which conversion to
use.

And is the use of table package open for discussion? I know that
MultiMarkdown use tabulary, and probably tabularx has better footnote
in table support (I once need to manually turn the tabulary table in
mmd to a tabularx one). And I heard that tabu is supposed to be the
newest and best package for table but haven't tried it yet.

—
You are receiving this because you commented.
Reply to this email directly or [1]view it on GitHub

References

  1. #1023 (comment)
commented

@jgm, what do you think about my solution? I think it should be fairly easy to adapt the existing table export code in pandoc to the format I proposed, and I do not see any problems with it.

commented

I've long had the goal of trying to stick to packages in the
texlive-latex-recommended subset on debian. (I'd rather
that users not have to download gigabytes of LaTeX just to
use regular tables.) Unfortunately, the tablefootnote
package is in texlive-latex-extra -- so that makes me
reluctant to use this approach.

In the end, it might still be worth considering. Maybe
we could have use of longtable or this approach be
configurable.

+++ Michael Färber [Apr 14 16 10:00 ]:

Hi, I have converted a multiline table example from the Pandoc user's
guide to use tabular / table, and footnotes (courtesy of
[1]http://tex.stackexchange.com/a/35328) as well as relative widths
seem to work.

The LaTeX code:

\documentclass{article}
\usepackage{hyperref}
\usepackage{tablefootnote}
\usepackage{booktabs}

\begin{document}
\begin{table}
\caption{Here's a multiline table without headers.}
\begin{tabular}{ c l r l }
\tabularnewline
\toprule
\toprule
\begin{minipage}[t]{0.15\columnwidth}\centering\strut
First
\strut\end{minipage} &
\begin{minipage}[t]{0.10\columnwidth}\raggedright\strut
row
\strut\end{minipage} &
\begin{minipage}[t]{0.20\columnwidth}\raggedleft\strut
12.0
\strut\end{minipage} &
\begin{minipage}[t]{0.31\columnwidth}\raggedright\strut
Example of a row that spans multiple lines.\tablefootnote{Footnotes also work.}
\strut\end{minipage}\tabularnewline
\begin{minipage}[t]{0.15\columnwidth}\centering\strut
Second
\strut\end{minipage} &
\begin{minipage}[t]{0.10\columnwidth}\raggedright\strut
row
\strut\end{minipage} &
\begin{minipage}[t]{0.20\columnwidth}\raggedleft\strut
5.0
\strut\end{minipage} &
\begin{minipage}[t]{0.31\columnwidth}\raggedright\strut
Here's another one. \tablefootnote{Another footnote.}Note the blank line between
rows.
\strut\end{minipage}\tabularnewline
\bottomrule
\end{tabular}
\end{table}

\end{document}

—
You are receiving this because you commented.
Reply to this email directly or [2]view it on GitHub

References

  1. http://tex.stackexchange.com/a/35328
  2. #1023 (comment)
commented

I understand. How about then leaving longtable as the default and creating a command-line option to use tablefootnote? This would not create a dependency for users that "just want some table", and if they need non-breaking tables desperately enough to use a new command-line option, they are probably also willing to depend on some extra package. :)

(At least I am desperate enough. The table situation is by far my greatest Pandoc problem by far.)

commented

Assuming you're bound to longtable (e.g. by the IEEEtran documentclass), but want to use pandocs' tables, there is a workaround forcing the table into a single column here: http://tex.stackexchange.com/a/224096/90603. It breaks when the content becomes too much (starts to write over the other column), but at least you don't have to convert the table manually to TeX.

That said, a solution to this issue would be great! In the meantime, already a possibility to output a markdown table as simple tabular code would really ease the manual workaround.

commented

I've spent quite a bit of time trying to work around this issue. While the workarounds which can be found in this thread (and in other places around the internet) work in some cases they fail in others (including my own relatively simple case*). As of the time of writing my determination is that if you want to produce simple tables for PDF output with Pandoc in a two-column layout the best option is to write raw Latex tables into your Markdown source.

From my point of view this isn't a great outcome.** We're also coming up for four years since this issue was opened, so I had hoped for some progress by now. Even just a cleaner workaround, as others have suggested, would be awesome.

Is this assessment correct? I'm not familiar with the inner workings of either Pandoc or Latex, so it's entirely possible I've missed something. It's also possible that something has changed in the past year or so since @jgm last chimed in, in which case it would be good to update this thread with the latest information (new workarounds, new official Pandoc-supported options, official decisions regarding this issue etc.)

* If anyone's interested, here's what I'm trying to achieve:

  • A two-column layout document (IEEE).
  • A table written in Pandoc's table syntax, with a caption (as per Pandoc's user guide).
  • PDF output from the Markdown source files.
  • In the output, the table should be rendered in the same place relative to the surrounding text as in the source file (ie: not Latex-floated off somewhere else).

** This doesn't mean I'm not grateful for all the work that's been done on this fantastic project!

commented

You're right that we still use longtable in pandoc; I just
haven't gotten around to exploring workarounds for two
columns, but I'll try to prioritize this more.

The dev version now has a groff ms writer; you can do

pandoc -f ms -o output.pdf input.md

This can give you two-column output with a small tweak
in the default template. It handles both math and
tables (which I assume will work in two-column mode).
Output is not quite as nice as PDF, though, and there are
some other limitations. You might try it.

commented

This can give you two-column output with a small tweak
in the default template. It handles both math and
tables (which I assume will work in two-column mode).
Output is not quite as nice as LaTeX, though, and there are
some other limitations. You might try it.

Since he said "A two-column layout document (IEEE)", probably he'll want to use LaTeX instead.

I agree table in LaTeX is highly non-trivial, e.g. see tables - Which tabular packages do which tasks and which packages conflict? - TeX - LaTeX Stack Exchange.

Among the list, the more popular ones are (i.e. the one I heard of):

  • booktabs
  • tabularx
  • tabulary
  • longtable
  • supertabular
  • tabu

I think before thinking about which package to use, may be we should come up with a list of features (e.g. multi-page, footnotes, 2-columns, etc.) required in pandoc and start explores from there.

Once we have the feature lists we absolutely needed (and also eliminated those we are willing to sacrifice for), then we can start to explore which package might does all of them, and if not, how should it be handled.

And in terms of work, I think this should start as a LaTeX project first (such that we are happy with the feature set and output), and after that it would because a pandoc project (e.g. how to write the writer, templates, etc.

While there aren't a lot of pandoc experts, there are much more LaTeX experts. So I think we should let some LaTeX experts to volunteer on the first part first. What do you think about this plan?

commented

Sounds promising. Thanks @jgm and @ickc!

As a starting point my feature list would be something like:

  • Support for academic output formats (including two-column layout).
  • Table captions.
  • The ability to reference a table (even just dropping a \label{...} into the Latex output so that I can use a \ref{…} would be enough).
  • Nice to have: Table cells which wrap their text content.

Obviously Pandoc already supports most of the above, with various workarounds, but as discussed I've found it tricky to get all of them working at once.

commented
commented

Having a command-line switch that causes table/tabular to be
used instead of longtable would be one solution.

Or if table receives attributes, use attributes to switch the output type. The reason behind this is if the document is long enough, there might not be 1 single type of table works for all of them.

commented
commented

Update, thinking about this a bit more:

  • I'd rather not use supertabular. Apparently it doesn't guarantee the same column widths on different pages of a table. Also, I got very bad results when I tried to use it with twocolumn mode.

  • Quite apart from the need to put tables in two-column documents, some users may prefer to have tables that float (as longtable does not). This allows LaTeX to layout pages more harmoniously.

So, probably what we need is a way to tell pandoc to use regular table+tabular environments for floating tables, either on a per-table basis, or for the whole document.

Since floats really need labels and identifiers, this probably requires changes to the Table constructor in the Pandoc AST. We're planning on making some changes to that anyway, and I think this issue needs to wait for those changes.

commented

I use this filter to generate floating tables if there is a caption. Otherwise, I create non-floating tables. it's only for latex. Maybe someone finds it useful. https://gist.github.com/rriemann/56017da044861f7dd459dc5ab2f25cb9

I wrote this filter to work nicely with the classicthesis latex template: https://github.com/classicthesis/classicthesis

commented

I'm working on a document class that is very likely to encounter this bug in any given document. So, I have the luxury of assuming users will have access to a workaround that's in that class. I came up with this, based on this SE answer. (It's just TeX commands, so it could also go in a template or a header.)

% switch into and out-of twocolumn mode when creating tables
\makeatletter
% copy the definition of longtable into oldlongtable:
\let\oldlongtable\longtable
% copy the definition of endlongtable into oldendlongtable:
\let\oldendlongtable\endlongtable
% if longtable is followed by '[', use the option-having version, else use the non-option-having version:
\def\longtable{\@ifnextchar[\longtable@i \longtable@ii}
% option-having beginning:
\def\longtable@i[#1]{\onecolumn \oldlongtable[#1]}
% non-option-having beginning:
\def\longtable@ii{\onecolumn \oldlongtable}
% redefine the ending of longtable:
\def\endlongtable{\oldendlongtable \twocolumn}
\makeatother

The big disadvantage of this is that it creates a page break before and after every table... but at least it compiles! I kind of wonder if the solution here is just to pester the maintainer of longtable to make that package compatible with twocolumn mode. This really seems to be more of a problem with that package. (Though I'm not saying Pandoc should pass the buck either! It shouldn't be generating broken documents.)

commented
commented

I wonder whether it is really important to support long tables that cross page breaks? Pandoc doesn't support long equations that don't fit in the width of a page, so there is certainly precedent for just leaving it to the user to ensure their document formats properly.

commented
commented

a somewhat belated reply to this now ancient issue, but for anyone who comes across this problem like me...

\let\longtable\supertabular
\let\endlongtable\endsupertabular
\let\endhead\bottomrule

in the default output between pandoc and markdown an \endhead exists, it seems to suffice to \let \endhead become \bottomrule

in any case, it solves the \endhead problem mentioned by @nvasilakis

and importantly, it allows tables in two column layouts

commented

What a great suggestion! I think it's better to use \relax instead of \bottomrule to avoid getting a double line under the headings. So here's the idea. Create a file supertabular.tex containing

\usepackage{supertabular}
\let\longtable\supertabular
\let\endlongtable\endsupertabular
\let\endhead\relax

Then use --include-in-header supertabular.tex when you run pandoc.
I'd like to put this in the FAQs, but I've noticed that it seems to break if your table has a caption.

! LaTeX Error: \caption outside float.

Any thoughts on how to fix that? It looks like supertabular wants the caption (\tablecaption) outside the tabular environment, whereas pandoc puts the longtable caption inside, and that's hard to work around.

commented

Google brought me here trying to find a solution for making a table span over both columns of a two-column layout and I ended up using Joedang's solution. The page breaks are a small price to pay for having a readable, sensibly formatted table.

The technical discussions on how to achieve it aside: would it not make sense to let the user choose whether they want a table in a two-column layout behave like normal text or whether they want it to to behave like normal text?

  • If a two-column layout is detected and the user hasn't done anything to change that, default to oribotic's solution and give a warning that captions had to be disabled.
  • If a two-column layout is detected and some sort of option has been specified by the user, use Joedang's solution and give a warning that tables will have pagebreaks around them.

The option-to-be-specified could look like this:

| My | Pretty | Table |
|----|--------|-------|
| 1  | 2      | 3     |

:  My pretty table
:: spanlayoutcols
commented

If there were no other differences, you could do a
renewenvironment in the template, and redefine longtable
as table. E.g.

\let\longtable\table
\let\endlongtable\endtable

But there are some differences between the two environments,
e.g. as regards handling of footnotes, so this probably
won't work in general. (It might work sometimes.)

+++ Nikos Vasilakis [Aug 17 15 20:32 ]:

Hey everyone, sorry for resurrecting this one -- do we have any way of
resolving this beyond using sed? In theory, it should be able to just
amend the latex template -- but the latex template does not seem to
allow edits to the table structure. Any ideas?
—
Reply to this email directly or [1]view it on GitHub.
References

  1. #1023 (comment)

I ended up adding this at the top of the markdown file and it worked (I'm definitely not an expert on this):

---
header-includes: |
 \usepackage{supertabular}
 \renewenvironment{longtable}{\begin{center}\begin{supertabular}}{\end{supertabular}\end{center}}
 \renewcommand{\endhead}{}
...
commented

Have you managed to make it work with captions?

The supertabular package doc says

image

So we would need to rewrite pandoc's caption command \caption to e.g. \tablecaption, but only for the table environment. Any ideas?

I tried the following, but it did not work for me.:

---
header-includes:
  - \usepackage{supertabular}
  - \renewenvironment{longtable}{\begin{center}\begin{supertabular}\renewcommand{\caption}{\tablecaption}}{\end{supertabular}\end{center}}
  - \renewcommand{\endhead}{}
---
commented

I suspect you need more than
\renewcommand{\caption}{\tablecaption}.
At least
\renewcommand{\caption}[1]{\tablecaption{#1}}
Or maybe
\def\caption\tablecaption.

commented

Found kind of an acceptable workaround for my situation using a custom IEEEtran template.
I added the following to my template:

\usepackage{supertabular}
\newcommand\tcap{}
\renewenvironment{longtable}{\begin{center}\bottomcaption{\tcap}\begin{supertabular}}{\end{supertabular}\end{center}}
\renewcommand{\endhead}{}

And now before every table i just insert this inline latex:

\renewcommand\tcap{My Caption}

EDIT: Because this is kind of an eyesore, i wrote a filter to avoid putting raw latex in my markdown doc:

import pandocfilters as pf

def supertabular(key, value, format, meta):
  """
    NOTE: This filter expects a modification to the default template
  """
  if key == 'Table' and 'processed' not in value[0][1]:
    caption = value[1]
    cap = pf.stringify(caption) 
    cmd = f'\\renewcommand\\tcap{{{cap}}}'

    value[0][1].append('processed')
    value[1] = {'t': 'Caption', 'c': [None, []]}
    mytable = pf.elt('Table', len(value))
    return [pf.RawBlock('latex', cmd), mytable(*value)]

if __name__ == '__main__':
  pf.toJSONFilter(supertabular)
commented

For anyone having trouble with supertabular and footnotes (this may also apply to xtab) in the generated minipages for the tables:

If you want to keep the arabic (instead of alphabetic) numbering in footnotes, and also track the value of the footnote counter (I kept getting the value of 'a' as a marker for all the footnotes), here is the relevant code:

header-includes:
  - |
    ```{=latex}
     \usepackage{supertabular}
     \renewenvironment{longtable}{\begin{center}\begin{mpsupertabular}}{\end{mpsupertabular}\end{center}}
     \renewcommand{\endhead}{}
     \usepackage{xparse}
     \renewcommand{\thefootnote}{\arabic{footnote}}
     \renewcommand{\thempfootnote}{\arabic{mpfootnote}}
     \let\latexminipage\minipage
     \let\latexendminipage\endminipage
     \RenewDocumentEnvironment{minipage}{ooom}
      {
       \IfNoValueTF{#1}
        {\latexminipage{#4}}
        {
         \IfNoValueTF{#2}
          {\latexminipage[#1]{#4}}
          {
           \IfNoValueTF{#3}
            {\latexminipage[#1][#2]{#4}}
            {\latexminipage[#1][#2][#3]{#4}}
          }
        }
       \setcounter{mpfootnote}{\value{footnote}}
      }
      {
       \setcounter{footnote}{\value{mpfootnote}}
       \latexendminipage
      }
     ```

This is a mixture of two great solutions from stack exchange: "Renewenvironment minipage with xparse", and "How to change symbol for footnote in minipage".

I hope this is useful for others in a similar situation.

commented

The solution by @Bustel (without the filter) works for me for the LaTeX output, but putting a caption like :This is the table caption in addition (to also have a caption in other output formats) would still result in an error message (caption outside float).

commented

EDIT: Because this is kind of an eyesore, i wrote a filter to avoid putting raw latex in my markdown doc:

import pandocfilters as pf

def supertabular(key, value, format, meta):
  """
    NOTE: This filter expects a modification to the default template
  """
  if key == 'Table' and 'processed' not in value[0][1]:
    caption = value[1]
    cap = pf.stringify(caption) 
    cmd = f'\\renewcommand\\tcap{{{cap}}}'

    value[0][1].append('processed')
    value[1] = {'t': 'Caption', 'c': [None, []]}
    mytable = pf.elt('Table', len(value))
    return [pf.RawBlock('latex', cmd), mytable(*value)]

if __name__ == '__main__':
  pf.toJSONFilter(supertabular)

Thanks for your solution. It is working well for me. But I was wondering where do you define your filter?

commented

Thanks for your solution. It is working well for me. But I was wondering where do you define your filter?

Have a look at the --filter option in the User Guide. A filter can be any executable in a relative path, in $PATH, or you can put them in $DATADIR/filters, which for me is ~/.local/share/pandoc/filters. You can put custom templates and default settings there, too.

commented

Thanks for your solution. It is working well for me. But I was wondering where do you define your filter?

Have a look at the --filter option in the User Guide. A filter can be any executable in a relative path, in $PATH, or you can put them in $DATADIR/filters, which for me is ~/.local/share/pandoc/filters. You can put custom templates and default settings there, too.

Thanks for your quite reply!

I tried adding your filter in a script named twocolumns_table_fix.py:

""" Pandoc filter to fix the issue with longtable not compatible
with 2-column latex documents

See https://github.com/jgm/pandoc/issues/1023#issuecomment-656769330

"""
import pandocfilters as pf

def supertabular(key, value, format, meta):
  """
    NOTE: This filter expects a modification to the default template
  """
  if key == 'Table' and 'processed' not in value[0][1]:
    caption = value[1]
    cap = pf.stringify(caption)
    cmd = f'\\renewcommand\\tcap{{{cap}}}'

    value[0][1].append('processed')
    value[1] = {'t': 'Caption', 'c': [None, []]}
    mytable = pf.elt('Table', len(value))
    return [pf.RawBlock('latex', cmd), mytable(*value)]

if __name__ == '__main__':
  pf.toJSONFilter(supertabular)

I also added the latex package supertabular in a separate file (supertabular.tex):

\usepackage{supertabular}
\newcommand\tcap{}
\renewenvironment{longtable}{\begin{center}\bottomcaption{\tcap}\begin{supertabular}}{\end{supertabular}\end{center}}
\renewcommand{\endhead}{}

My minimalist markdown file article.md is below:

---

---

### Test Table

| Tables        | Are           | Cool  |
| ------------- |:-------------:| -----:|
| col 3 is      | right-aligned | $1600 |
| col 2 is      | centered      |   $12 |
| zebra stripes | are neat      |    $1 |

<!-- \renewcommand\tcap{Table styles. \label{tab:1}} -->
Table: Table styles. \label{tab:1}

Seen in table \ref{tab:1}, Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat.

But when I run the command below:

pandoc article.md \
--citeproc \
--from=markdown+tex_math_single_backslash+tex_math_dollars+raw_tex \
--to=latex \
--output=build/article-print-doublecols.pdf \
--pdf-engine=xelatex \
-M classoption=twocolumn \
  --include-in-header="layout/supertabular.tex" \
--filter="layout/twocolumns_table_fix.py"

I got this error:

Error running filter layout/twocolumns_table_fix.py:
Error in $.blocks[4]: When parsing the constructor Caption of type Text.Pandoc.Definition.Caption expected Array but got Object.

Any idea why I got this error?

commented

@Bustel thanks for sharing your workaround. As mentioned by @wuffi, when giving the table a caption in markdown, processing the resulting LaTeX results in an error with caption being outside of a float. Do you know if this can be fixed somehow?

commented

@Bustel thanks for sharing your workaround. As mentioned by @wuffi, when giving the table a caption in markdown, processing the resulting LaTeX results in an error with caption being outside of a float. Do you know if this can be fixed somehow?

@SamKacer Did you use the command below to set the caption?

\renewcommand\tcap{My Caption}

This worked for me

commented

@lhoupert oh, right. I can't believe I missed that. I just copy pasted that in without actually reading it. that works fine. thanks for pointing that out

commented

I got this error:

Error running filter layout/twocolumns_table_fix.py:
Error in $.blocks[4]: When parsing the constructor Caption of type Text.Pandoc.Definition.Caption expected Array but got Object.

Any idea why I got this error?

Apparently something has changed in pandoc in the meantime and my filter is not correct anymore.
Luckily the fix to the fix is just one line:

-    value[1] = {'t': 'Caption', 'c': [None, []]}
+    value[1] = [None, []]

Then I had to remove your html comment from your example and then it worked for me. Almost.
pf.stringify just seems to remove raw latex from the caption so the reference does not work anymore but you might work around that with pandoc-crossref.

commented

Thanks very much @Bustel for debugging me on this! I managed to find a solution to my problem without having to install other package :-)

The idea is to save the latex reference before "stringifyïng" the caption, so I created a short function for that (return_latex_ref) following @jgm stringyfy.py function. I added it below:

""" Pandoc filter to fix the issue with longtable not compatible
with 2-column latex documents

See https://github.com/jgm/pandoc/issues/1023#issuecomment-656769330

"""
import pandocfilters as pf
from pandocfilters import walk

def return_latex_ref(x):
    """Walks the tree x and returns the latex reference.
    """
    result = []

    def go(key, val, format, meta):
        if key in 'RawInline':
          if val[0] == 'tex' and '\\label' in val[1]:
            result.append(val[1])

    walk(x, go, "", {})
    return ''.join(result)

def supertabular(key, value, format, meta):
  """
    NOTE: This filter expects a modification to the default template
  """

  if key == 'Table' and 'processed' not in value[0][1]:
    caption = value[1]
    cap = pf.stringify(caption) + ' ' + return_latex_ref(caption)

    cmd = f'\\renewcommand\\tcap{{{cap}}}'

    value[0][1].append('processed')
    value[1] = [None, []]
    mytable = pf.elt('Table', len(value))
    
    return [pf.RawBlock('latex', cmd), mytable(*value)]

if __name__ == '__main__':
  pf.toJSONFilter(supertabular)
commented

@lhoupert solution works for me. The only thing I'd like is the possibility of using default pandoc-cross-ref notation {#tbl:name}. I'm new to these filter stuff so I don't know how to do it 😅

|     | Grosor [mm] | Factor de reducción |
|-----|-------------|---------------------|
| HVL |        10.1 |                   2 |
| QVL |          19 |                   4 |
| TVL |        29.9 |                  10 |

: Valores de HVL, QVL y TVL reportados \label{tbl:name}

This works [@tbl:name].
commented

There is a new error: Missing number, treated as zero

It happens when the size the table is bigger than the width of the column. The next couple of lines appear in the latex output once the above condition is triggered.

\begin{longtable}[]{@{}
  >{\raggedright\arraybackslash}p{(\columnwidth - 2\tabcolsep) * \real{0.35}}
  >{\raggedright\arraybackslash}p{(\columnwidth - 2\tabcolsep) * \real{0.22}}@{}}\real{0.22}}@{}}
commented

Kind of off topic, but for anyone just trying to get a long table from pandoc to span over two columns, wrap it in begin{table*} and end{table*} and change all occurences of \columnwidth to \linewidth. Works with crossref, custom column width etc. Just took me 4hrs+ to find this out.

commented

I just found a solution to this on the TeX stack exchange. If you include a header file that looks like this:

\makeatletter
\let\oldlt\longtable
\let\endoldlt\endlongtable
\def\longtable{\@ifnextchar[\longtable@i \longtable@ii}
\def\longtable@i[#1]{\begin{figure}[t]
\onecolumn
\begin{minipage}{0.5\textwidth}
\oldlt[#1]
}
\def\longtable@ii{\begin{figure}[t]
\onecolumn
\begin{minipage}{0.5\textwidth}
\oldlt
}
\def\endlongtable{\endoldlt
\end{minipage}
\twocolumn
\end{figure}}
\makeatother

You can just redefine the \longtable command, which seems to solve the issue well.

Source: https://tex.stackexchange.com/a/224096

This could have some side effects I don't know about, but everything seems fine from what I can see.

commented

None of the recommended solutions above worked for me. I do not want my tables to float. And mysteriously a solution I had from a few years back is now broken and giving a \noalign error that I cannot sort out. I put this in the preamble:

\renewenvironment{longtable}{\begin{center}\begin{tabular}}{\end{tabular}\end{center}}
\def\endhead{}
\renewcommand{\toprule}[2]{\hline}
\renewcommand{\midrule}[2]{\hline}
\renewcommand{\bottomrule}[2]{\hline}

And it is working again. I suppose my tables will not work across pagebreaks, but I also do not use long tables.

PS. One of the early comments in this thread confuses table with tabular and gets a odd message about a float because of it.

commented

None of the proposals above do work for me though :/ I can't use tables in Pandoc anymore. I have to use a two-column template.

Original error is ! Package longtable Error: longtable not in 1-column mode.

Tryin to use tabular:

\usepackage{longtable}
\usepackage{booktabs}

% fix longtable in 2-col mode from https://tex.stackexchange.com/questions/161431/how-to-solve-longtable-is-not-in-1-column-mode-error
\renewenvironment{longtable}{\begin{center}\begin{tabular}}{\end{tabular}\end{center}}
\renewcommand{\endhead}{}
\renewcommand{\toprule}[2]{\hline}
\renewcommand{\midrule}[2]{\hline}
\renewcommand{\bottomrule}[2]{\hline}

results in ! Undefined control sequence. \LT@echunk ->\crcr \LT@save@row \cr \egroup \global \setbox \LT@gbox \lastbo... l.898 \endlastfoot

BTW, the table is as simple as

| test | haha |
| ---- | ---- |
| 1    | 1    |
| 2    | 3    |

which gets converted to

\begin{longtable}[]{@{}ll@{}}
\toprule\noalign{}
test & haha \\
\midrule\noalign{}
\endhead
\bottomrule\noalign{}
\endlastfoot
1 & 1 \\
2 & 3 \\
\end{longtable}

No idea what I could do :(

commented

I just found a solution to this on the TeX stack exchange. If you include a header file that looks like this:

\makeatletter
\let\oldlt\longtable
\let\endoldlt\endlongtable
\def\longtable{\@ifnextchar[\longtable@i \longtable@ii}
\def\longtable@i[#1]{\begin{figure}[t]
\onecolumn
\begin{minipage}{0.5\textwidth}
\oldlt[#1]
}
\def\longtable@ii{\begin{figure}[t]
\onecolumn
\begin{minipage}{0.5\textwidth}
\oldlt
}
\def\endlongtable{\endoldlt
\end{minipage}
\twocolumn
\end{figure}}
\makeatother

You can just redefine the \longtable command, which seems to solve the issue well.

Source: https://tex.stackexchange.com/a/224096

This could have some side effects I don't know about, but everything seems fine from what I can see.

Yeah I tried that, however it doesn't align properly in my document.
image

EDIT: Fixed by moving the \onecolumn lines down one line each

commented

Found kind of an acceptable workaround for my situation using a custom IEEEtran template. I added the following to my template:

\usepackage{supertabular}
\newcommand\tcap{}
\renewenvironment{longtable}{\begin{center}\bottomcaption{\tcap}\begin{supertabular}}{\end{supertabular}\end{center}}
\renewcommand{\endhead}{}

And now before every table i just insert this inline latex:

\renewcommand\tcap{My Caption}

EDIT: Because this is kind of an eyesore, i wrote a filter to avoid putting raw latex in my markdown doc:

import pandocfilters as pf

def supertabular(key, value, format, meta):
  """
    NOTE: This filter expects a modification to the default template
  """
  if key == 'Table' and 'processed' not in value[0][1]:
    caption = value[1]
    cap = pf.stringify(caption) 
    cmd = f'\\renewcommand\\tcap{{{cap}}}'

    value[0][1].append('processed')
    value[1] = {'t': 'Caption', 'c': [None, []]}
    mytable = pf.elt('Table', len(value))
    return [pf.RawBlock('latex', cmd), mytable(*value)]

if __name__ == '__main__':
  pf.toJSONFilter(supertabular)

Currently trying this method, however I get AttributeError: 'dict' object has no attribute 'append' on the value[0][1] line. Updating to latest version caused same issue as previous people had, however was solved with fix provided. However, now I get Undefined control sequence for \endlastfoot. Removing this line solves the problem, however requires removing it every time I run pandoc. Also the solution for the caption also seems to break pandoc-crossref

commented

I have tried all the options here and various other things (like Lua filters) and nothing seems to work. Would really love to see a fix that is at least compatible with simple tables in a two-column format! :)

commented

None of the proposals above do work for me though :/ I can't use tables in Pandoc anymore. I have to use a two-column template.

Original error is ! Package longtable Error: longtable not in 1-column mode.

Tryin to use tabular:

\usepackage{longtable}
\usepackage{booktabs}

% fix longtable in 2-col mode from https://tex.stackexchange.com/questions/161431/how-to-solve-longtable-is-not-in-1-column-mode-error
\renewenvironment{longtable}{\begin{center}\begin{tabular}}{\end{tabular}\end{center}}
\renewcommand{\endhead}{}
\renewcommand{\toprule}[2]{\hline}
\renewcommand{\midrule}[2]{\hline}
\renewcommand{\bottomrule}[2]{\hline}

results in ! Undefined control sequence. \LT@echunk ->\crcr \LT@save@row \cr \egroup \global \setbox \LT@gbox \lastbo... l.898 \endlastfoot

BTW, the table is as simple as

| test | haha |
| ---- | ---- |
| 1    | 1    |
| 2    | 3    |

which gets converted to

\begin{longtable}[]{@{}ll@{}}
\toprule\noalign{}
test & haha \\
\midrule\noalign{}
\endhead
\bottomrule\noalign{}
\endlastfoot
1 & 1 \\
2 & 3 \\
\end{longtable}

No idea what I could do :(

@jankap I got your code to work by adding to the above:

\renewcommand{\endlastfoot}{}
commented

I'm currently using LaTeX with https://ctan.org/pkg/tabularray in markdown to render my tables in two-col mode...