Add new command-line option --variable-json (#10341). This allows non-string values (booleans, lists, maps) to be given to template variables on the command line.

The --pdf-engine option can now take groff as a value.

Markdown writer:

• Avoid spaces after/before open/close delimiters (#10696). E.g. instead of rendering x<em> space </em>y as x* space *y we render it as x *space* y.
• Handle row/colspans in grid tables, and expand cells when it isn’t possible to lay them out without breaking string of non-whitespace.
• Render a figure with Para caption as implicit figure (#10755).
• When falling back to a Div with class figure for a figure that can’t be represented any other way, include a Div with class caption containing the caption.
• Improve use of implicit figures when possible (#10758). When the alt differs from the caption, but only as regards formatting, we still use an implicit figure.
• Omit initial newlines in gfm math blocks to avoid an ugly blank line.

• Support the four_space_rule extension for plain output (#10813, Manolis Stamatogiannakis).

RST writer:

• Handle row/colspans in grid tables, and expand cells when it isn’t possible to lay them out without breaking string of non-whitespace.

Muse writer:

• Handle row/colspans in grid tables, and expand cells when it isn’t possible to lay them out without breaking string of non-whitespace.

JATS writer:

• Fix escaping for writing-review-editing role (#10744).

HTML writer:

• Remove trailing slash from default revealjs URL (#8749). This avoids a double slash in the URL’s path component.

LaTeX writer:

• Make alignment work within multirow in tables (#10772).

Typst writer:

• Support mark class on spans (#10747).
• Add equation label if math contains \label{..} (#10805).

Roff format writers (man, ms):

• Use the most compatible form for roff escapes (#10716). For example, \(xy instead of \[xy]. This was the original AT&T troff form and is the most widely supported. The bracketed form causes problem for some tools, e.g. makewhatis on macOS. And emit e followed by an escape for a unicode combining accent rather than the form \[e aa], which works for groff but not e.g. on macOS’s man. This change affects Text.Pandoc.RoffChar, Text.Pandoc.Writers.Roff, and the Man and Ms writers.

Docx writer:

• Ensure that figures and tables with custom styles are not dropped (#10705).
• Preserve Relationships for images from reference docx (#10759). This should allow one to include an image in a reference.docx and reference it in an openxml template.
• Don’t renumber rels (#10769). We used to renumber the Relationships so they didn’t conflict with the set of fixed Relationships we imposed. We are now preserving the ids from the reference doc’s document.xml.refs, so we shouldn’t renumber them or references introduced by the user (e.g. in a template) will fail.

Ms writer:

• Improve PDF TOC labels. We now use the plain writer to render these, so that Greek characters etc. will show up properly.
• When no pdf-engine variable is specified, do not use the .pdfhref macros at all (#10738). This gives better results for links in formats other than PDF, since the link text would simply disappear if it exists only in a .pdfhref macro. When a PDF engine is specified, escape the argument of .pdfhref O in a way that is appropriate.

OpenDocument writer:

• Fix character styles in footnotes (#10791). Character styles governing the position of the footnote reference should not be imposed on the footnote text.

Powerpoint writer:

• Use reference-doc font for captions (#9896, R. N. West).

DocBook writer:

• Use literallayout element for LineBlock (#10825).

MediaWiki reader/writer:

• Allow definition on same line as term (#10708).

LaTeX reader:

• Skip at most one argument to LaTeX tabular newline (#7512, Evan Silberman).
• Disable ligatures inside \texttt (#10781).
• Support more symbol commands (#10782).

Commonmark Reader:

• Handle GFM math irregularity with braces (#10631). In GFM, you need to use \\{ rather than \{ for a literal brace.

DocBook reader:

• Improve handling of literallayout (#10825). This is now only made a CodeBlock when there is a monospaced class. Otherwise it is made a LineBlock.

Org reader:

• Add AVIF to Org Reader image extensions (#10736, Christian Christiansen).
• Don’t include newlines in inine code/verbatim (#10730). Convert newlines to spaces as we do in other formats.
• Change handling of inline TeX (#10836). Previously inline TeX was handled in a way that was different from org’s own export, and that could lead to information loss. This was particularly noticeable for inline math environments such as equation. Previously, an equation environment starting at the beginning of a line would create a raw block, splitting up the paragraph containing it (see #10836). On the other hand, an equation environment not at the beginning of a line would be turned into regular inline elements representing the math. (This would cause the equation number to go missing and in some cases degrade the math formatting.) Now, we parse all of these as raw “latex” inlines, which will be omitted when converting to formats other than LaTeX (and other formats like pandoc’s Markdown that allow raw LaTex).

Beamer template: fix regression in 3.6.4, reverting the omission of \date when the document does not have a date. By default, beamer will display a date when no \date is present in the title block, so this was an unintended behavior change. The reverted change was motivated by the desire to include a custom \date in the frontmatter via header-includes. This can be achieved more simply by simply setting the date variable. In markdown you can even use date in metadata and put some raw LaTeX there.

Ms template:

• Use T rather than P as default font family (#10738).
• Put PDF-specific things under a conditional. Don’t include them if pdf-engine isn’t set.

Upgrade reveal.js URL to v5 (#10740, Kolen Cheung). v4 is no longer available on unpkg.com.

Text.Pandoc.PDF: Allow groff to be used as --pdf-engine with ms (#10738). When groff is used as a PDF engine, the groff extension to ms is automatically enabled. Limitations:

• groff currently produces larger PDFs than pdfroff.
• With groff, a table of contents produced with --table-of-contents/--toc will always be placed at the end of the document.
• Certain characters (e.g. Greek characters) may be dropped in the PDF outline.

Text.Pandoc.Writers.Shared:

• Export delimited [API change].
• New version of gridTable (#6344) [API change]. This handles row and colspans. It also ensures that cells won’t wrap text in places where it wouldn’t normally wrap, even if this means making the cells wider than requested by the colspec (#9001, #7641). Because the parameters are different, this is a breaking API change.

Text.Pandoc.App: set pdf-engine variable. If --pdf-engine is specified or if a PDF is being produced, we set the pdf-engine variable. This allows writers and templates to behave differently depending on the PDF engine.

Text.Pandoc.Class and Text.Pandoc.URI:

• Fix parsing of base64 data URIs to allow URI escapes and whitespace (which will be ignored) (#10704).
• Handle percent encoding in pBase64URI instead of unescaping later, for efficiency (#10704).

Text.Pandoc.Citeproc.BibTeX:

• Recognize en as a langid in biblatex bibliographies (#10764).

Text.Pandoc.MIME:

• Add mime type and extension for avif (#10704).
• Handle apng, avif, jxl (#10704).

Text.Pandoc.Readers.LaTeX.Math: export inlineEnvironmentNames. Internal module, not a change to the public API.

reference.docx (Andrew Dunning):

• Remove extra spaces around text placeholders.
• Add footnote block text sample.

Text.Pandoc.Class.Sandbox:

• Add sandboxWithFileTree function [API change] (Albert Krewinkel).

Lua subsystem (Albert Krewinkel):

• pandoc-lua-engine: add all test files to the cabal file.
• Allow pandoc.read to be called in “sandbox” mode for added security (#10831). Readers running in a sandbox will not be able to access the network or file system. The sandbox is enabled if the fourth parameter is a list of files or filename/content pairs. The files are read and then made available in the sandbox via en ersatz file system.

Makefile:

• Add target release-checkist.
• Install @daisy/ace from npm if not present.
• Use pandoc lua instead of lua.
• Fix typo in latex-package-dependencies target.
• Use jq instead of json_reformat in validate-docx-golden-tests2.

NiX infrastructure: new working flake.nix and simpler shell.nix. Removed old default.nix.

Require random >= 1.3 and use splitGen. split has been deprecated.

Use citeproc-0.9. Bump citeproc bounds for pandoc, pandoc-lua-engine.

Use texmath-0.12.10.1.

Use released typst 0.8 (partially supporting typst 0.13).

Use citeproc 0.9.0.1.

MANUAL.txt:

• Fix default URL for revealjs.
• Add note that alerts extension only works with commonmark (#9716).
• Remove “Body Text Char” from list of Word styles that can be customized using a reference.docx (#10646). This doesn’t seem to be present in pandoc-generated docx files, nor is it a Word default.
• For pandoc lua, add note about the environment.
• Improve documentation of --variable option.

doc/typst-property-output.md: Mention that typst:no-figure is a class, not an attribute (#10826, Niklas Eicker).

Change RELEASE-CHECKLIST to RELEASE-CHECKLIST-TEMPLATE.org. Use org-babel to automate many of the steps of the release.

INSTALL.md: update MacPorts information (#10719, Mohamed Akram).

COPYRIGHT: fix link to source code.

CONTRIBUTING.md: Fix link to discussion forum. (#10834, R. N. West).

Add mdoc as input format (Evan Silberman). This change introduces a reader for mdoc, a roff-derived semantic markup language for manual pages. This reader has been developed almost exclusively against mandoc’s documentation and implementation of mdoc as a reference, and the real-world manual pages tested against are those from the OpenBSD base system. Of ~3500 manuals in mdoc format shipped with a fresh OpenBSD install, 17 cause the mdoc reader to exit with a parse error. Any further chasing of edge cases is deferred to future work.

New module: Text.Pandoc.Readers.Mdoc, exporting readMdoc [API change].

Issue warnings for duplicate YAML metadata keys (#10312).

Ensure that --sandbox affects --embed-resources. Previously it did not (contrary to what was implied by the manual), which means that an image with URL /etc/passwd would leak an encoded version of that file to HTML output with --self-contained or --embed-resources, even if --sandbox was used. Thanks to Samuel Mortenson for pointing out the issue.

Text.Pandoc.App.OutputSettings: add sandbox' function. This computes the sandboxed files from Opt and avoids code repetition.

Docx reader:

• Parse index references as empty spans with attributes (#10171). Attributes included are entry, and optionally bold, italic, yomi, see.
• Don’t create multiple paragraphs for title or subtitle (#10359). If there are multiple paragraphs with Title or Subtitle style, use only the first for metadata.
• Handle case where Zotero itemData has different id from the citationItem id. In this case we use the citationItemId in the bibliography as well, overriding the referenceId in the itemData (#10366).

LaTeX reader:

• Put parsed minipage in specially marked Div (#10266).

HTML reader:

• Parse footnotes defined by dpub-aria roles (#5294).

MediaWiki reader:

• Fix indented tables with caption (#10390).
• Fix parsing of col/rowspan (#6992).

Typst reader:

• Avoid generating empty paragraphs.
• Support underparen, overparen.
• Fix #quote attribution. If attribution is not present, don’t print the -- (#10320).
• Fix typo in unicode code point for em dash (see #10320).

Commonmark reader:

• implicit_figures should check for empty caption and not produce an implicit figure in this case (#10429).

RST reader:

• Use a new one-pass parsing strategy. Instead of having an initial pass where we collect reference definitions, we create links with target ##SUBST##something or ##REF##something or ##NOTE##something, and resolve these in a pass over the parsed AST. This allows us to handle link references that are not at the top level (#10281).
• Ignore newlines in URL in explicit link (#10279).
• Handle block level substitutions.
• Support :file: on raw directive (#8584).
• Implement option lists (#10318).
• Avoid putting metadata in Para (#7766). Create MetaInlines when possible, just as with markdown input. MetaBlocks is still used when there are multiple paragraphs or non-paragraph content. This change also affects field lists.
• Fix linked substitutions (#6588). E.g. |Python|_.
• Support inline anchors (#9196).
• Explicit links define references (#5081). For example, Go to `g`_ `g <www.example.com>`_. should produce two links to www.example.com.

EPUB writer:

• Use standardized filename for cover image instead of the original name (#10404). This avoids problems with e.g. filenames containing spaces.

Markdown writer:

• Issue INFO warning when not rendering table, e.g., when raw_html is disabled and the table can’t be fit into a supported markdown table format (#10407).
• Respect empty LineBlock lines in plain output (Evan Silberman). The plain writer behaved as a markdown variant with Ext_line_blocks turned off, and so empty lines in a line block would get eliminated.

LaTeX writer:

Ensure that beamer footnotes go on frame, not column (#5769).

HTML writer:

• Unwrap empty incremental divs (#10328, Albert Krewinkel). Divs are unwrapped if the only purpose of the div seems to be to control whether lists are presented incrementally on slides.

Typst writer:

• Make template sensitive to a page-numbering variable (#10370). This can be set to an empty string (or, in metadata, to false) for no page numbers.
• Make smart extension work (#10271). If smart is not enabled, a command in the default template will disable smartquote substitutions. When smart is enabled, render curly apostrophes as straight and escape straight apostrophes. When smart is disabled, render curly apostrophes as curly and don’t escape straight apostrophes. Similarly for quotes, em and en dashes. This should give more idiomatic typst output, with fewer unnecessary escapes.

ANSI writer:

• Respect empty LineBlock lines (Evan Silberman).

JATS writer:

• Correct spelling of suppress attribute (#10350, Andreas Deininger).

Typst template:

• Remove definitions.typst partial.
• Remove unnecessary definition of endnote.
• Incorporate the one remaining definition into default.typst.
• Use typst 0.12 code for two column layout (#10294, Luis Rivera).
• Note: the new templates presuppose typst 0.12; if you try to use an earlier version of typst, an error will be raised.

LaTeX/Beamer template:

• Split fonts.latex partial into two parts: fonts.latex and font-settings.latex.
• In beamer template, load beamer theme between fonts.latex and font-settings.latex. This allows a theme (such as metropolis) to set its own default font, while still allowing the user to override it. This fixes a regression in pandoc 3.5 (#10297).
• Note: Users who have custom templates based on pandoc 3.5 templates will need to add font-settings.latex() after fonts.latex() in the latex template. In a beamer template, the beamer theme-setting code needs to be moved between these two partials.

ConTeXt template: Ensure that font names don’t wrap (#10305).

epub.css: remove background-color (#10264, Suraj Patil). With this greyish background color, epubs look bad on a Kindle (#10263).

Text.Pandoc.ImageSize: add WebP support (Evan Silberman, #10397). Add Webp constructor on ImageType [API change].

Text.Pandoc.Readers.Roff and a new unexported module Text.Pandoc.Readers.Roff.Escape: parameterize Roff escaping (Evan Silberman) [API change]. This allows code to be reused between the mdoc and man readers, despite the differing Token types.

Text.Pandoc.PDF:

• PDF via LaTeX: always do max runs if toc is present (#10308). The old method (checking to see if toc hash had changed) is not completely reliable.
• Use .source extension, not .html, in toPdfViaTempFile (#10314).

Text.Pandoc.Logging: add YamlWarning constructor to LogMessage [API change] (#10312).

Text.Pandoc.Format: remove duplicate typst entry (#10388, Caleb Mclennan).

Fix a typo in the ua.yaml localization for ‘See’ (Jens).

Lua subsystem (Albert Krewinkel):

• Remove prefixes from Lua type names (#8574). Lua type names were inconsistent with regard to the use of prefixes; all prefixes are removed now, and Lua types now have the same name as the Haskell types. The use of app-specific prefixes is suggested by the Lua manual to avoid collisions. However, this shouldn’t be a problem with pandoc, as it cannot be used as a Lua package.

doc/libraries.md: Add newly developed Haskell packages. Sort list alphabetically (Albert Krewinkel).

doc/lua-filters.md: document pandoc.List:iter method (Albert Krewinkel). List objects have a new function iter that returns an iterator function that returns the next list item on each call.

MANUAL.txt:

• Clarify what the example of YAML EPUB metadata shows (#10405).
• Fix typo in template syntax (#10265, Pascal Wagler).
• Update manual with information on openxml template (#10273).
• Clarify that --variable can only assign string values (#10298).

Fix comments in TEI writer referring to DocBook (#10430, Evan Silberman).

Fix several typos in documentation (#10349, Andreas Deininger).

Allow Diff 1.0.

Add font-settings.latex partial to pandoc.cabal (#10379).

Bump upper bound for data-default.

Use latest typst, texmath, pandoc-lua-marshal, commonmark-pandoc, commonmark-extensions, skylighting, skylighting-format-blaze-html.

New output format: ansi (for formatted console output) (Evan Silberman). Most Pandoc elements are supported and printed in a reasonable way, if not always ideally. This version does no detection of terminal capabilities, nor does it fall back to different output styles for less-capable terminals.

Add command line options --table-caption-position and --figure-caption-position. These allow the user to specify whether to put captions above or below tables and figures, respectively. The following output formats are supported: HTML (and related such as EPUB), LaTeX (and Beamer), Docx, ODT/OpenDocument, Typst.

Change default --pdf-engine via HTML to WeasyPrint (#10142). wkhtmltopdf is deprecated. weasyprint is the easiest-to-install, maintained alternative. For better results, one might prefer pagedjs-cli.

Org reader:

• Fix parsing of src blocks with an -i flag (#10071, Albert Krewinkel). Tabs are now preserved in the contents of src blocks if the the block has the -i flag.

RTF reader:

• Handle images inside shp contexts (#10145).

RST reader:

• Improve simple table support (#10093). Multiline rows occur only when the first cell is empty; we were previously treating lines with any empty cell as row continuations. In addition, we no longer wrap multiline cells in Para if they can be represented as Plain. This is consistent with docutils behavior.

LaTeX reader:

• Math environments don’t have bracketed options (#10160).
• Parse nested tabular environments (#4746).

Typst reader:

• Change how “block” elements are handled. Previously they were always parsed as divs. But actually they can occur in some “inline” contexts. Now we first try to parse them as inlines, and only as blocks if that fails. A surrounding Div or Span element is added only if there is an identifier.

HTML reader:

• Only parse main element’s contents (if present) (#10140). If main has an id or class, we include a div with that id or class; otherwise just the contents.
• Read TeX annotation in MathML content if present (#9971).
• Better handle KaTeX-generated math (#9971). KaTeX emits the mathml followed by a span with an HTML fallback. Previously pandoc was converting both. We now ignore the HTML fallback span, marked with class katex-html.

Docx reader:

• Add “SuppressAuthor” and “AuthorOnly” to citationMode when +citations is used (thomjur).

New module: Text.Pandoc.Writers.ANSI [API change] (Evan Silberman).

Docx writer:

• Support custom-style attribute for docx table (Sebbones).
• Support --number-offsets.
• Make table/figure rendering sensitive to caption position settings.

OpenDocument writer:

• Make table/figure rendering sensitive to caption position settings.

Typst writer/template:

• Implement figure caption positions by triggering a show rule in the default template, which determines caption positions for figures and tables globally.
• Don’t include trailing semicolon after @ style citations with suffixes (#10148).
• Template: move header-includes before show doc (#9996, Gordon Woodhull).

LaTeX writer:

• Make table/figure rendering sensitive to caption position settings (#5116).
• Preserve locator labels with --natbib (#10057).

HTML writer/template:

• Make <figcaption> placement sensitive to caption position settings. For tables, <caption> must be the first element, and positioning is determined by CSS, for here we set a variable which the default template is sensitive to.
• Use makeSectionsWithOffsets for writerNumberOffsets, instead of the old, inefficient code.
• Don’t add doc-biblioref role to every link in a citation; only to links to the bibliography (#10156).
• Add data- when rendering label attribute (#10048).

Markdown writer:

• Avoid emitting markdown caption if table has fallen back to raw HTML, which will then contain a <caption> tag (#10094).

• Make math sensitive to tex_math_gfm extension (#9121). This means that in GFM output, the “new style” math will be used by default, e.g.

  $`x=y`$
  
  ```math
  x = y
  ```
  

  To defeat this and get the older behavior, namely

  $x=y$
  
  $$x=y$$
  

  one could use -t gfm-tex_math_gfm.

AsciiDoc writer:

• Add link: prefix when needed (#10105). AsciiDoc requires it except for http, https, irc, mailto, ftp schemes (#10105).
• Preserve original base level (#10062). We used to normalize so that the base level is always 1, but asciidoc no longer seems to care about that, and the behavior creates difficulties when we are converting fragments.
• Don’t emit empty figure caption (#10047).

ODT writer:

• Add TableCaption to styles.xml (#10058, Ian Max Andolina).

LaTeX template:

• Fix wrong beamer color in (sub)section page (Jonathan).

Text.Pandoc.Options:

• Add CaptionPosition and new WriterOptions fields writerFigureCaptionPosition and writerTableCaptionPosition [API change].

Text.Pandoc.Opt:

• Change default for optNumberOffset to []. This behaves the same as [0,0,0,0,0].
• Add Opt fields optFigureCaptionPosition and optTableCaptionPosition [API change].

Text.Pandoc.Format: change formatFromFilePaths so that it is smarter about URLs. URLs are parsed, and we take the format from the path component, if present (#10141). This means that https://emacs.org/ will be treated as HTML, while https://emacs.org/sample.org will be treated as Org.

Text.Pandoc.URI:

• Add unofficial gemini: to list of URI schemes (Pau RE).

Text.Pandoc.Shared:

• Add makeSectionsWithOffsets [API change].
• Remove `stripEmptyParagraphs [API change] (Albert Krewinkel). This function is no longer used.

Text.Pandoc.Highlighting: Expose formatANSI [API change] (Evan Silberman).

Text.Pandoc.Writers.Shared: export to{Sub,Super}scriptInline [API change] (Evan Silberman).

Remove use of partial functions (e.g. head) in code.

Use latest skylighting-core, skylighting, doclayout, texmath, typst.

pandoc-lua-engine: Add accessors for several writer options, including some that were added in previous releases.

pandoc-server: Initialize some missing fields in WriterOptions: writerEpubTitlePage, writerChunkTemplate, writerListTables, writerFigureCaptionPosition, writerTableCaptionPosition.

CONTRIBUTING.md: Summarize steps for adding a new cli option.

MANUAL.txt:

• Clarify that the --number-offset option should only directly affect numbering of the first section heading in a document; subsequent headings will increment normally.
• Fix asciidoc link (#10039).
• Fix CSL Docs broken link (#10100, Tristano Ajmone).
• Document the use of luatexja when CJKmainfont is used with lualatex (#3873, Kolen Cheung).
• Add a citations (typst) section to the manual (#9127).
• Clarify that citations affects both input and output for org.
• Add note on --citeproc that you may need to disable citations extension on the output format (e.g., -t markdown-citations) to see the rendered citation (#9127, #10012).

INSTALL.md — reorganise info on static binaries and add conda-forge install options (#10098, #10069, Ian Max Andolina).

Link pandoc-cli version to pandoc version. Henceforth pandoc-cli’s version will be synchronized with pandoc’s, and pandoc-cli will depend on an exact pandoc version. This will avoid confusion by ensuring that cabal install pandoc-cli-X.Y.Z installs pandoc version X.Y.Z. It will make things more straightforward for upstream packagers (see #9232). This scheme does not follow the Haskell PVP, but that should cause no harm, because this package does not expose a library.

Add alerts markdown extension. This enables GitHub style markdown alerts as a commonmark extension. This extension is now default for gfm. It can’t be used with markdown, only with commonmark and variants.

Markdown reader:

• Preserve newlines in math instead of changing to spaces. Otherwise we can get unwanted results if there’s a % comment (#9193).
• Make attributes work with reference links (#9171).

HTML reader:

• Improve handling of invalidly nested sublists (#9187, cf. #8150).

MediaWiki reader:

• Allow attribute keys with hyphens (#9178).

ODT reader:

• Support attr text:continue-numbering (#8979, Stephan Meijer).

Typst reader:

• Allow references (e.g. @foo) to become citations if there is no corresponding label in the document.
• Collapse adjacent cite elements.
• Handle supplements in cite.
• Change cite (only one key allowed, a label) (typst 0.9 breaking change).
• Support quote element (typst 0.9).

LaTeX reader:

• Handle otherlanguage environment and language-name environments like \begin{french}...\end{french} (#9202).
• Fix theorem label parsing (#8872, Hikaru Ibayashi).

Docx reader:

• Unwrap content of shaped textboxes (Stephan Meijer, #9214).
• Improve handling of w:sym (#9220). We now look up symbols in symbol fonts using the table defined at Text.Pandoc.Readers.Docx.Symbols.
• Add unexported module Text.Pandoc.Readers.Docx.Symbols. This gives us a table to use to resolve characters included in docx via w:sym element.

Man reader:

• Properly handle .sp macro inside lists and block quotes (#9201).

LaTeX writer:

• Fix bug with big footnotes inside emphasis (#8982, Hikaru Ibayashi).
• Handle identifiers inside heading contents. \phantomsection can’t be used in this case, so we need \hypertarget (#9209).

LaTeX template:

• Include bookmark package unconditionally. This package produces better PDF bookmarks than hyperref and does it on the first pass.

Typst writer:

• Use quote for block quotes.
• Support --toc-depth as in other writers (#9242).
• Put inline image dimensions on enclosing box, not image (#9104).
• Better handling of tables with captions (#9194). We now put these in a figure with a caption argument.
• Update typst writer to typst 0.9 citation format (#9188).

Typst template:

• Remove custom definition of blockquote in default template. (We now use built-in quote.)
• Support table of contents.
• Support csl (#9186, Ian Max Andolina). Typst now supports CSL for its native citation engine, so pandoc should use a specified csl style in the template, falling back to bibliographystyle if csl is not specified.

Docx writer:

• Use different style for block quotes in notes (#9243). Using “Footnote Block Text” for the style name, so it can be given a different font size if footnotes are.
• Allow embedded fonts to be used in reference.docx (#6728).

HTML5 writer:

• To conform to validator’s expectations, doc-footnote role is used with aside and doc-endnotes with section.
• aside is used only for notes at ends of sections or blocks; if all the notes come at the end of the document, section is used so we can have the doc-endnotes role.

JATS writer:

• Handle case where there is material after refs div (#9166). Previously in such cases the references were not being moved to back matter.

Ms writer:

• Don’t do normal escapes in filename arguments for PSPIC etc.

T.P.RoffChar: escape - as \-. The groff_man (7) man page indicates that - characters will be treated as typographic hyphens and are not appropriate for cases where the output should be copy-pasteable as an ASCII hyphen-minus character. (E.g. in command line options.) However, until a recent update groff man did not actually do this; it treated - and \- the same. With the new update (1.23.0) the two are distinguished (see https://lwn.net/Articles/947941/ for background), so now it is important that pandoc escape -.

Text.Pandoc.Extension: add Ext_alerts constructor [API change].

Text.Pandoc.PDF: We now default to running LaTeX only once in producing a PDF (instead of twice). This is made possible by the shift to the bookmark package, which does not require a second pass for PDF bookmarks. If a table of contents is present, we still have to run three times to get the page numbers, and if beamer is used we still do a minimum of two runs.

Text.Pandoc.Shared:

• renderTags': use minimized tag for rect.
• Allow svg path element to be minimized.
• Export combineAttr [API change].
• Improve isTightList so that it recognizes an item containing only a list which is itself tight as potentially an item in a tight list (#9161).

Text.Pandoc.MIME: Ensure we use .svg not .svgz as extension for image/svg+xml mime type. This fixes issues with embedded SVG images in docx output, among other things (#9195).

Text.Pandoc.Class: openURL improvements for data uris. Only treat data URI as base64 if ‘;base64’ is specified. Otherwise treat as UTF-8 (not 100% reliable but should cover most other cases). Strip off ;base64 (or ;charset=... or whatever) from mime type (#9195).

Text.Pandoc.SelfContained: Improve treatment of embedded SVGs (#9206, #8948).

• Ensure unique ids for elements by prefixing SVG id.
• Ensure SVG id attribute except when use element is used.
• Remove width, height attributes from svg element when use element is used. Instead, add width and height 100% to the use element. This seems to get the sizing right.

Text.Pandoc.Citeproc: Don’t link citations if suppress-bibliography specified, for there will be nothing to link to (#9163).

epub.css: add styling for sup and sub (#9160).

Switch from base64 to base64-bytestring (#9233).

Use newest versions of commonmark, commonmark-extensions, commonmark-pandoc, texmath, typst, skylighting, skylighting-core.

Benchmark: use standalone documents for reader tests. Otherwise typst reader benchmark fails. Note: this means that we are now parsing longer documents, so bench results on readers won’t be comparable to before.

MANUAL.txt: update defaults file docs for bibliography fields (#9173). Recommend using top-level bibliography csl, etc. instead of a nested metadata field. Reason: ${USERDATA} and ${HOME} are only expanded in these contexts, not in metadata.

Move man pages to pandoc-cli package (#9245).

Fix CVE-2023-38745, a variant of the vulnerability in CVE-2023-35936. Guilhem Moulin noticed that the fix to CVE-2023-35936 was incomplete. An attacker could get around it by double-encoding the malicious extension to create or override arbitrary files.

--embed-resources: Use inline SVG instead of data uris for SVG images in HTML5 (#8948). Note that SelfContained does not have access to the writer name, so we check for HTML5 by determining whether the document starts with <DOCTYPE! html>. This means that inline SVG won’t be used when generating document fragments.

Fix regression on short boolean arguments (#8956). In 3.1.5 boolean arguments were allowed an optional argument (true|false). This created a regression for uses of fused short arguments, e.g. -somyfile.html, which was equivalent to -s -omyfile.html, but now raised an error because pandoc attempted to parse o as a boolean true or false. This change allows the fused short arguments to be used again. Note that -strue will be interpreted as -s with an argument true, not as -s -t -rue. It is best to use long option names with the optional boolean values, to avoid confusion.

Make --epub-title-page’s argument optional. It takes a boolean argument, and now that all of our boolean flags take such an argument, we can make this one optional for consistency.

Improve errors for illegal output formats. Previously if you did pandoc -s -t bbb, it would give you an error about the missing bbb template instead of saying that bbb is not a supported output format.

Improve errors for incorrect command-line option values (#8879). Always give the name of the relevant argument.

Fix typo on error message for incorrect --preserve-tabs argument. Thanks @fsoedjede

Docx reader: use SVG version of image if present (#7244). Previously the backup PNG was exported even if an SVG was present, but the SVG should be preferred.

Typst reader: fix regression in recognition of display math (#8949). The last release caused all math to be parsed as inline math.

JATS writer: don’t use <code> for inline code (#8889). It is intended for block-level code.

HTML writer: don’t make line blocks sensitive to --wrap (#8952).

RST writer: fix figure handling (#8930, #8871). This fixes a number of regressions from pandoc 2.x. Properly handle caption, alt attribute in figures. No longer treat a paragraph with a single image in it as a figure (we have a dedicated Figure element now).

Docx writer: Copy “mirror margins” property from reference.docx (#8946).

Text.Pandoc.UTF8: Deprecate decodeArg which is now a no-op. This was needed for old base versions which we no longer support.

Use released skylighting, typst.

Allow latest commonmark-extensions. This allows entities in wikilinks.

Switch back to using ghc 9.2 for linux and Windows binary releases (#8947, #8955). With ghc 9.4+, we were getting AVX instructions in the amd64 binary, which aren’t supported on older hardware. For maximum compatibility we switch back to ghc 9.2, which doesn’t cause the problem. (As documented, ghc should not be emitting these instructions, so we aren’t clear on the diagnosis, but the cure has been tested.)

Change Windows release build to use cabal instead of stack.

Fix a security vulnerability in MediaBag and T.P.Class.IO.writeMedia. This vulnerability, discovered by Entroy C, allows users to write arbitrary files to any location by feeding pandoc a specially crafted URL in an image element. The vulnerability is serious for anyone using pandoc to process untrusted input. The vulnerability does not affect pandoc when run with the --sandbox flag. [CVE-2023-35936]

Allow epub-title-page to be used in defaults files (#8908).

Issue Extracting info message (in --verbose mode) when using --extract-media or extracting media temporarily in PDF production.

HTML reader: Update TableBody RowHeadColumns caculation (#8634, Ruqi). This change sets RowHeadColumns to the minimum value of each row, which gives better results in cases where rows have different numbers of leading th tags.

Dokuwiki reader: retain image query parameters as attributes (#8887, echo0).

Textile reader: Add support for link references (#8706, Stephen Altamirano). Textile supports what it calls “link alias”, which are analogous to Markdown’s reference-style links.

LaTeX reader: support alt text on images (#8743, Albert Krewinkel).

Commonmark reader: Make implicit_figures work again. Support for this (introduced in #6350) disappeared when we made an architectural change.

JATS reader:

• Add footer and multiple body parsing to table reader (#8765, Noah Malmed).
• Parse references title from ref-list (#8365).

JATS writer:

• Make --number-sections work.
• Include title in ref-list (#8364). Previously the reference title ended up in a separate section at the back of the body instead of in the ref-list in the back matter.

Mediawiki writer: allow highlighting to work for F# language (Adelar da Silva Queiróz).

LaTeX writer: Fix escaping of & in \href and \url (#8903).

Docx writer:

• Fix localization of “Abstract” title (#8702).
• Allow abstract-title to be specified in docx metadata (#8794).

ChunkedHTML writer: Make math work in top-level page (#8915).

Text.Pandoc.Logging: add new log message type ScriptingWarning [API change] (Albert Krewinkel).

Lua: report warnings from Lua scripts (Albert Krewinkel). Lua’s warning system is plugged into pandoc’s reporting architecture. Warnings that are raised with the Lua warn function are now reported together with other messages.

Use crypton-connection instead of connection (#8896, Felix Yan). Follows the change introduced in tls 1.7.0.

Bump versions for skylighting-core, skylighting.

Include lua/module/sample.svg in cabal extra-source-files (Felix Yan).

Add Nynorsk (New Norwegian) translations (Per Christian Gaustad).

Add tests for fillMediaBag/extractMedia.

INSTALL.md:

• Mention alternatives to LaTeX to generate PDF (Norwid Behrnd).
• Update Linux install links (harabat).

pandoc-extras.md: add to “Academic publishing workflows” (#8696, Vladimir Alexiev).

EPUB reader: Give additional information in error if the epub zip container can’t be unpacked.

TSV reader: don’t gobble tabs as whitespace (#8661).

Org reader: accept empty tables (#8659).

LaTeX reader: fix multiplication syntax for tabular (#8658). We recognized *{6}{...} but not *6{...} or *6c.

Docx reader: parse image alt texts in LibreOffice generated files. LibreOffice tags images slightly differently than Word; this change lets the parses take that difference into account when looking for an image description (alt text).

DocBook reader:

• Fix <xref> references to tables in DocBook files (#8626, Pavol Otto).
• Parse figure as a Figure element in the AST (#8668).

JATS reader: avoid generating duplicate figure captions (#8669).

RST reader: align with spec in syntax for role names (#8653). In particular, we now allow colons in row names.

Add note on converting from .doc format to FAQs (#8654).

Trap error in getAppUserDataDirectory (#8648). This can raise an error if pandoc is run in a non-user environment.

LaTeX writer: do not use longtable foot with Beamer (#8638, Albert Krewinkel). The table foot is made part of the table body, as otherwise it won’t show up in the output. The root cause for this is that longtable cannot detect page breaks in Beamer.

LaTeX template: Add CJKsansfont and CJKmonofont for XeLaTeX (#8656, Yudong Jin). CJKsansfont and CJKmonofont will be set for xelatex only if CJKmainfont is also provided.

URL style in ConTeXt (#8612, Thomas Hodgson). Previously, a URL like this would be in monospace text: \useURL[url1][https://example.com]. Now, it will match the main text unless the linkstyle variable is set, which controls the styling of all links. Closes #8602.

Asciidoc writer: Properly escape | in table cells (#8665).

asciidoc{,tor} template: fix revision date when author is unset (#8637, arcnmx). Revision line syntax is only valid in combination with an author line, so the date attribute must be set explicitly when the author is missing

HTML writer: allow “track” element to be treated as block-level HTML (#8629).

Include needed polyfill when MathJaX is used (#8625).

JATS writer: include alt-text in <graphic>, <inline-graphic> elements (#8631, Albert Krewinkel).

Chunked HTML writer: Retain metadata in processing sections for chunked HTML (#8620). Previously we suppressed metadata in all but the top page, in order to prevent the title block from being printed on every page. This prevented use of custom variables set by metadata fields. This commit moves to a better solution: a conditional in the default template restricts the title block to the top page.

Lua API:

• Add new function pandoc.system.cputime (Albert Krewinkel). The function returns the CPU time consumed by pandoc and can be used to benchmark Lua computations.
• Add module pandoc.json to handle JSON encoding (#8605, Albert Krewinkel).

Use pandoc-lua-marshal 0.2.1 (Albert Krewinkel). All major AST elements now have __tojson metamethods that return the JSON representation of an element. This allows to JSON-encode these elements with libraries that respect the __tojson metamethod, including dkjson.

Use latest zip-archive. This allows pandoc to open certain epubs that it could not open before.

Use commonmark-extensions 0.2.3.4. This fixes some bugs involving definition lists and inline formatting.

Use latest skylighting-format-context

MANUAL.txt:

• Document chunk-template in defaults file.
• Remove obsolete “raw content in a style” section.
• Revise documentation for --mathml to reflect support in all major browsers (#8667).

docs/custom-readers.md: Update JSON parsing example. The example now uses the built-in pandoc.json library to parse the API output.

doc/press.md: Add article on CiTO in J Cheminform by @egonw.

doc/lua-filters.md: fix typo in run_json_filter (Morgan Willcock).

Split pandoc-server, pandoc-cli, and pandoc-lua-engine into separate packages (#8309). Note that installing the pandoc package from Hackage will no longer give you the pandoc executable; for that you need to install pandoc-cli.

Pandoc now behaves like a Lua interpreter when called as pandoc-lua or when pandoc lua is used (#8311, Albert Krewinkel). The Lua API that is available in filters is automatically available to the interpreter. (See the pandoc-lua man page.)

Pandoc behaves like a server when called as pandoc-server or when pandoc server is used. (See the pandoc-server man page.)

A new command-line option --list-tables, causes tables to be formatted as list tables in RST (#4564, with Francesco Occhipinti).

New command line option: --epub-title-page=true|false allows the EPUB title page to be omitted (#6097).

--reference-doc can now accept a URL argument (#8535) and load a remote reference doc.

--version output no longer contains version info for dependent packages. Instead, it contains a “Features” line that indicates whether the binary was compiled with support for acting as a server, and for using Lua filters and Custom writers.

A new option --split-level replaces --epub-chapter-level and affects both EPUB and chunked HTML output. --epub-chapter-level will still work but is deprecated.

Multiple input files with --file-scope: fix case where the links are URL-encoded, e.g. with %20 (#8467).

Produce error if --csl is used more than once (#8195, Prat).

Remove deprecated --atx-headers option.

Remove deprecated option --strip-empty-paragraphs.

In --verbose mode add message when running citeproc (as with other filters).

Add new mark extension for highlighted text in Markdown, using == delimiters (#7743).

Add new extensions wikilinks_title_after_pipe and wikilinks_title_before_pipe for commonmark and markdown. (#2923, Albert Krewinkel). The former enables links of style [[Name of page|Title]] and the latter [[Title|Name of page]]. Titles are optional in both variants, so this works for both: [[https://example.org]], [[Name of page]]. The writer is modified to render links with title wikilink as a wikilink if a respective extension is enabled. Pandoc will use wikilinks_title_after_pipe if both extensions are enabled.

Add prefixes to identifiers with --file-scope (#6384). This change only affects the case where --file-scope is used and more than one file is specified on the command line. In this case, identifiers will be prefixed with a string derived from the file path, to disambiguate them. For example, an identifier foo in contents/file1.txt will become contents__file1.txt__foo. Links will be adjusted accordingly: if file2.txt links to file1.txt#foo, then the link will be changed to point to #file1.txt__foo. Similarly, a link to file1.txt will point to #file1.txt. A Div with an identifier derived from the file path will be added around each file’s content, so that links to files will still work.

New output format: chunkedhtml. This creates a zip file containing multiple HTML files, one for each section, linked with “next,” “previous,” “up,” and “top” links. (If -o is used with an argument without an extension, it is treated as a directory and the zip file is automatically extracted there, unless it already exists.) The top page will contain a table of contents if --toc is used. A sitemap.json file is also included. The option --split-level determines the level at which sections are to be split.

Support complex figures (Albert Krewinkel, Aner Lucero). There is now a dedicate Figure block constructor for figures. The old hack of representing a figure as Para [Image attr [..alt..] (source, "fig:title")] has been dropped. Here is a summary of figure support in different formats:

• Markdown reader: paragraphs containing just an image are treated as figures if the implicit_figures extension is enabled. The identifier is used as the figure’s identifier and the image description is also used as figure caption; all other attributes are treated as belonging to the image.
• Markdown writer: figures are output as implicit figures if possible, via HTML if the raw_html extension is enabled, and as Div elements otherwise.
• HTML reader: <figure> elements are parsed as figures, with the caption taken from the respective <figcaption> elements.
• HTML writer: the alt text is no longer constructed from the caption, as was the case with implicit figures. This reduces duplication, but comes at the risk of images that are missing alt texts. Authors should take care to provide alt texts for all images. Some readers, most notably the Markdown reader with the implicit_figures extension, add a caption that’s identical to the image description. The writer checks for this and adds an aria-hidden attribute to the <figcaption> element in that case.
• JATS reader: The <fig> and <caption> elements are parsed into figure elements, even if the contents is more complex.
• JATS writer: The <fig> and <caption> elements are used write figures.
• LaTeX reader: support for figures with non-image contents and for subfigures.
• LaTeX writer: complex figures, e.g. with non-image contents and subfigures, are supported. The subfigure template variable is set if the document contains subfigures, triggering the conditional loading of the subcaption package. Contants of figures that contain tables are become unwrapped, as longtable environments are not allowed within figures.
• DokuWiki, Haddock, Jira, Man, MediaWiki, Ms, Muse, PPTX, RTF, TEI, ZimWiki writers: Figures are rendered like Div elements.
• Asciidoc writer: The figure contents is unwrapped; each image in the the figure becomes a separate figure.
• Classic custom writers: Figures are passed to the global function Figure(caption, contents, attr), where caption and contents are strings and attr is a table of key-value pairs.
• ConTeXt writer: Figures are wrapped in a “placefigure” environment with \startplacefigure/\endplacefigure, adding the features caption and listing title as properties. Subfigures are place in a single row with the \startfloatcombination environment.
• DocBook writer: Uses mediaobject elements, unless the figure contains subfigures or tables, in which case the figure content is unwrapped.

• Docx writer: figures with multiple content blocks are rendered as tables with style FigureTable; like before, single-image figures are still output as paragraphs with style Figure or Captioned Figure, depending on whether a caption is attached.

• DokuWiki writer: Caption and “alt-text” are no longer combined. The alt text of a figure will now be lost in the conversion.
• FB2 writer: The figure caption is added as alt text to the images in the figure; pre-existing alt texts are kept.
• ICML writer: Only single-image figures are supported. The contents of figures with additional elements gets unwrapped.
• OpenDocument writer: A separate paragraph is generated for each block element in a figure, each with style FigureWithCaption. Behavior for single-image figures therefore remains unchanged.
• Org writer: Only the first element in a figure is given a caption; additional block elements in the figure are appended without any caption being added.
• RST writer: Single-image figures are supported as before; the contents of more complex images become nested in a container of type float.
• Texinfo writer: Figures are rendered as float with type figure.
• Textile writer: Figures are rendered with the help of HTML elements.
• XWiki: Figures are placed in a group.

Changes in custom readers/writers:

• It is now possible to have a custom reader and a custom writer for a format together in the same file. The file may also define a custom template for the writer.
• Pandoc now checks the folder custom in the user’s data directory for a matching script if it can’t find one in the local directory. Previously, the readers and writers data directories were searched for custom readers and writers, respectively. Scripts in those directories must be moved to the custom folder.
• Custom readers used to implement a fallback behavior that allowed to consume just a string value as input to the Reader function. This has been removed, the first argument is now always a list of sources. Use tostring on that argument to get a string.

New module Text.Pandoc.Writers.ChunkedHTML, exporting writeChunkedHtml [API change].

We now set the pandoc-version variable centrally rather than in the writers. One effect is the man writer now emits a comment with the pandoc version.

pandoc-server:

• Add simple CORS support to pandoc-server (#8427).
• Print message to stderr when starting the server.

Docx reader:

• Mark unnumbered headings with class unnumbered (#8148, Albert Krewinkel). This change ensures good conversion results when converting with --number-sections.
• Support parsing of highlighted text.
• Fix handling of oMathPara in w:p with other content (#8483).

ODT reader:

• Fix relative links. ODT adds a ../ to relative links (see #3524); this needs to be removed when converting from ODT.
• Handle “section” elements (#8409).
• Rename Text.Pandoc.Readers.Odt -> Text.Pandoc.Readers.ODT, for consistency with Writers.ODT. Rename readOdt -> readODT. [API change]

DocBook reader:

• Support href on link even in a fragment (#8437). (We now just look for an href attribute without worrying about the namespace.)
• Parse title from imageobject/objectinfo (#8437).

JATS reader:

• Handle uri element in references (#8270).

Ipynb reader:

• Add cell id to attachment filename when storing in MediaBag (#8415). Otherwise attachments with the same name can overwrite each other.

LaTeX reader:

• Skip parenthenized args of toprule, midrule, etc (#8242).
• Handle ## macro arguments properly (#8243).
• Remove unused function toksToString in Parsing module.
• Support more soul commands, including \hl.
• Add unnumbered class for \part* (#8447)
• Fix TEXINPUTS handling (#8392). If TEXINPUTS ends with :, then the system default TEXINPUTS is added. We handle this by just adding the working directory in this case.
• Parse short table caption (see jgm/pandoc-types#103). This is not too useful yet, because writers don’t do anything with the short caption.

MediaWiki reader:

• Parse table cell with attributess, to support rowspan, colspan (#8231, Ruqi).
• Refine “blending” rules for MediaWiki links (#8525, Ruqi). The rules for “blending” characters outside a link into the link are described here: https://en.wikipedia.org/wiki/Help:Wikitext#Blend_link These pose a problem for CJK languages, which generally don’t have spaces after links. However, it turns out that the blending behavior, as implemented on Wikipedia, is (contrary to the documentation) only for ASCII letters. This commit implements that restriction, which fixes the problem for CJK.

HTML reader:

• Fix regression for <tt> (#8330). It was no longer being parsed as Code (Justin Wood).

RST reader:

• Support mark role for round-trip.

Textile reader:

• Support linked images (#8541).
• Fix strong emph ending with link (#8540).
• Adding a Parser to look for ordered list start attribute numbers if any (#2465, vkraven).
• Handle empty paragraphs (#8487). Also, if attributes are added explicitly to a paragraph, put it in a Div with the attributes.

Markdown reader:

Org reader:

• Allow org-ref v2 citations with & prefix (#8302).
• Make #+pandoc-emphasis-pre work as expected (#8360, Amir Dekel).

BibTeX reader:

• Fix handling of % in url field (#7678). % does not function as a comment character inside url (where URL-encoding is common).
• Allow url field in bibtex as well as biblatex (#8287). This field is not officially supported for BibTeX, but many styles can handle it (https://www.bibtex.com/f/url-field/), and others will ignore it.
• Support software type in biblatex <-> CSL conversions (#8504).
• Make sure version field comes through in biblatex (#8504).

BibTeX writer:

• Pass through url even for bibtex (#8287).

Org writer:

• Pass through unknown languages in code blocks (#8278), instead of producing begin_example.
• Use span attributes tag-name in headers as tags (#8513, Albert Krewinkel). This enables round-tripping of tags in Org headings.

EndNote reader:

• Better error when parsing EndNote references fails.

DocBook writer:

• Rename Text.Pandoc.Writers.Docbook -> Text.Pandoc.Writers.DocBook. Rename writeDocbook -> writeDocBook, for consistency with the DocBook reader’s naming. [API change]
• Fix position of textobject (#8437). It is a child of inlinemediaobject, not imageobject.
• Add regression tests for #8437.
• Render image alt text using textobject element (#8437).
• Don’t indent contents of title element.
• Store “unnumbered” class in DocBook role attribute (#1402, lifeunleaded).

ConTeXt writer (Albert Krewinkel):

• Support syntax highlighting for code.
• Always use \type for inline code. Inline codes that contained curly braces where previously rendered with \mono; this led to unexpected results when the presentation of \type was customized, as those changes would not have been applied to code rendered with \mono.
• Add support for unlisted, unnumbered headings (#8486).
• Support tagging extension (Albert Krewinkel). Paragraphs are enclosed by \bpar and \epar commands, and highlight commands are used for emphasis. This results in much better tagging in PDF output.

LaTeX writer:

• Do not repeat caption on headless tables (Albert Krewinkel). The caption of headless tables was repeated on each page that contained part of the table. It is now made part of the “first head”, i.e. the table head that is printed only once.
• Add separator line between table’s body and its foot (Albert Krewinkel).
• Ignore languages with no babel equivalent, instead of generating an invalid command in the preamble (#8325).
• Use \includesvg for SVGs and include the svg package (#8334).
• Use soul instead of ulem for strikeout, underline (#8411). This handles things like hyphenation, line breaks, and nonbreaking spaces better.
• Use \toprule\noalign{} instead of \toprule() in tables, and similarly for \midrule and \bottomrule (#8223). This facilitates redefining \toprule, \midrule, and \bottomrule without needing to gobble the ()s. (Those who redefine these macros on the assumption that they will be followed by () may need to change their definitions.)
• Support highlighted text for Span with class mark.

JATS writer:

• Use <break/> for LineBreak in the limited contexts that accept it (#8344).
• Officially deprecate writeJATS in favor of writeJatsArchiving.

RTF writer:

• Add space after unicode escape commands (#8264). This fixes a bug that caused characters to disappear after unicode escapes.

RST writer:

• Render tables as list tables when the --list-tables option is specified (writerListTables) (#4564, Francesco Occhipinti).
• Improve inline escaping rules (#8380).
• Use special mark role for Span with class mark.

Commonmark writer:

• Ensure that we don’t have blank lines in raw HTML (#8307).

HTML writer:

• Only add role attribute in HTML5 (#8241). It is not valid in HTML4.
• Avoid aria-hidden in code blocks for HTML4 (#8241).
• Only treat . . . as a slide pause in slides, and not in regular HTML output (#8281).
• Properly merge classes for headings of level > 6 (#8363).
• Prevent <a> inside <a> (#7585). If a link text contains a link, we replace it with a span.
• Replace deprecated aria roles for bibliography entries (#8354). doc-biblioentry -> listitem, doc-bibliography -> list.
• Remove obsolete stuff about mathml-script. This was a shim we used to include for mathml support. We don’t do anything with this any more, so this is dead code.
• Include math links if there are raw commands or environments that can be interpreted as math e.g. by MathJax (#8469).
• Add prooftree to list of math environments (#8462). This will cause raw LaTeX prooftree environments to be rendered appropriately when --mathjax is used.

HTML, Markdown writers: filter out empty class attributes (#8251). These should not be generated by any pandoc readers, but they might be produced programmatically.

Markdown writer:

• Avoid HTML fallbacks in the generated TOC (Albert Krewinkel, #8131). The generated table of contents usually has IDs for each TOC link, allowing to link back to specific parts of the TOC. However, this leads to unidiomatic markup in formats like gfm, which do not support attributes on links and hence fall back to HTML. The IDs on TOC items are now removed in that case, leading to more aesthetic TOCs.
• Escape ! before [ (#8254).
• Support mark extension.

AsciiDoc writer:

• In link text, only replace commas with entities when they’re in Str elements. If a link contains an image, it may have attributes, and the commas there should not be converted (see #8437, #8070).

ODT writer:

• Fix relative links (#3524).

Docx writer:

• Better handling of tables in lists (#5947). Previously the content of each list cell was indented when the table belonged to a list item.
• Indent tables in list items (#5947).
• Adjust correct attribute on lang element (#7022). For East Asian languages, we need to adjust w:eastAsia rather than w:val. This allows normal fonts to be used for any Latin-font text. Similarly, for bidi languages, we need to adjust w:bidi rather than w:val. We treat he and ar as bidi languages, zh, ja, ko as East Asian languages.
• Support relative image widths (Albert Krewinkel). Image widths given in percent are interpreted to be relative to the text width. Previously, percent widths were taken relative to the image’s native size, inconsistently with other writers.
• Avoid using ‘error’ for unassigned table cells (#8468). Instead, throw a regular pandoc error.
• Render a Span with class mark as highlighted. Currently yellow is hardcoded.

MediaWiki writer:

• Use the ‘new’ table structure, so that colspan and rowspan are supported (Wout Gevaert).

Man writer:

• Use UTF-8 by default for non-ascii characters (#8507). Only use groff escapes if --ascii has been specified on the command line (writerPreferAscii).

ICML writer:

• Use Contents element for images with raw data instead of a link with a data: uri (#8398).

EPUB writer:

• Refactor to use Text.Pandoc.Chunks.
• Refactored and simplified code.
• Make title page optional (#6097).

Ms writer:

• Properly format display equations (#8308).
• Remove -C option on PSPIC. Some old versions don’t support this option, and since it’s the default it shouldn’t be necessary.

XWiki writer:

• Use template if it is specified (#8296). Previously templates were ignored.

LaTeX template:

• Set fonts after Beamer theme (Jeremie Knuesel). Beamer themes such as metropolis and saintpetersburg change the default fonts. This change gives precedence to the user font settings by moving them after the loading of the Beamer theme.
• Set \babelfont when mainlang and lang are specified and pdflatex is not being used (#8538). This is needed for good results in Arabic.
• Add variable urlstyle (#8429, Amar Al-Zubaidi). This is set to same by default, so users should not experience any change.

HTML template:

• Remove default font size, line height and font family in default inline css (#8423). mainfont, fontsize, and linestretch can still be used as before; the only difference is that we no longer provide opinionated defaults. This commit also adds a maxwidth variable that sets max-width; if not set, 36em is used as a default.
• Add code { hyphens: manual; }.
• Use styles.citations.html partial in styles.html.
• Fix class name hanging -> hanging-indent in styles.citations.html.
• Put Consolas before Lucida Console for code font (#8543). This is to prevent Lucida Console from being used on Windows, where it causes spacing issues in some applications, with boldface glyphs wider than regular ones.

EPUB CSS changes: Reduce the amount of inline CSS used for EPUBs (#8379). Almost everything is now in the default EPUB CSS (data/epub.css), which can be overridden either by putting epub.css in the user data directory or by using --css on the command line. Inline styles are only used for syntax highlighting (which depends on the style specified, and is only included on pages with highlighted code) and for bibliography formatting (which can depend on the CSL style, and is only used in the page containing the bibliography).

Note that, for compatibility with older readers, we don’t use flexbox to style column/columns divs by default, as we do in HTML. Instead, we use an older method which only works when there are two column divs inside a columns div. If you need more than two columns and aren’t worried about support for older EPUB readers, you can modify the default CSS (there is a comment in the CSS telling you what to do).

Reveal.js template: prevent line-wrapping of parallax options (#8503, Albert Krewinkel).

reference.pptx: Remove unsupported element (#8342, #6338, Link Swanson). The default template contained text above the header, which can mislead users into thinking there is a way to put text there using pandoc.

Text.Pandoc.Readers.Metadata:

• Fix metadata parsing corner case (#8465).
• Don’t fail on inline metadata beginning with newline (#8358).

Text.Pandoc.App:

• Move initial input-to-Pandoc code to internal submodule (Albert Krewinkel).
• Change parseOptionsFromArgs and parseOptions (#8406) They now return Either OptInfo Opt. [API change]
• Add OptInfo type [API change].
• Add handleOptInfo function. This performs the IO actions for things like --version that were previously done in parseOptionsFromArgs [API change].
• convertWithOpts: add argument for a ScriptingEngine [API change].
• Unify check for standalone output (Albert Krewinkel).
• New optEpubTitlePage field on Opt [API change] (#6097).
• Remove optEpubChapterLevel, add optSplitLevel [API change].
• Export IpynbOutput(..) [API change].

Text.Pandoc.App.OutputSettings:

• Remove unused field outputWriterName in OutputSettings.

Text.Pandoc.Citeproc:

• Check both extension and mime type to determine bibliography type when the bibliography is fetched remotely (#7151).
• CslJson: allow an object with items property in addition to an array of references. This is what is returned by e.g. https://api.zotero.org/groups/904125/items?v=...&format=csljson
• Require a digit for an implicit “page” locator inside explicit locator syntax {...} (#8288). Previously a locator specified as {} would be rendered as p. with nothing after it.
• Update sub verbo to sub-verbo (#8315). This is a change in the term’s canonical name in citeproc. As a result of this change, sub verbo locators have not worked in pandoc since citeproc 0.7.
• Text.Pandoc.Citeproc.MetaValue: remove unused function metaValueToPath.
• Add internal module Text.Pandoc.Citeproc.Name (#8345). This exports toName, which previously had been part of T.P.Citeproc.BibTeX, and allows for cleaner module dependencies.

Export module Text.Pandoc.Slides [API Change] (Albert Krewinkel).

Add new module Text.Pandoc.Format [API change] (Albert Krewinkel). The module provides functions and types for format spec parsing and processing. The function parseFormatSpec was moved from Text.Pandoc.Extensions to the new module and renamed to parseFlavoredFormat. It now operates in a PandocMonad and is based on the updated types.

Text.Pandoc.Sources:

• Add UpdateSourcePos instances for String and strict and lazy ByteString [API change].

Text.Pandoc.Extensions:

• Fix JSON decoding of Extensions (#8352, Albert Krewinkel).
• Add new exported function readExtension [API change].
• Remove parseFormatSpec [API change]. This has been moved to Text.Pandoc.Format and renamed as parseFlavoredFormat (Albert Krewinkel).
• Simpler implementation of Extensions based on Set (benchmarks show no performance penalty).
• Add CustomExtension constructor to Extension [API change].
• Remove Bounded, Enum instances for Extension.
• Add extensionsToList function.
• Revise readExtension so it can handle CustomExtension, and so that it returns a Text rather than Maybe Text.
• Add showExtension [API change].
• Add Ext_mark extension [API change].
• Add Ext_tagging constructor [API change] (Albert Krewinkel).
• Add Ext_wikilinks_title_after_pipe, Ext_wikilinks_title_before_pipe [API change] (Albert Krewinkel).

Text.Pandoc.PDF:

• Fix papersize on PDF generation via ms (#8403). We need to set an option in pdfroff in addition to including a macro in the ms file. With this fix, -Vpapersize=a4 should be sufficient to produce A4 PDF via ms.
• Change default background color of PDFs generated via HTML (#8422, Marcin Serwin).

Text.Pandoc.MIME:

• Base module on package mime-types, which is already a transitive dependency (#8277, Albert Krewinkel).
• Remove deprecated overrides (#8292).

Text.Pandoc.XML:

• Re-export lookupEntity from commonmark-hs [API change].

Text.Pandoc.Parsing:

• Remove gratuitious renaming of Parsec types. We were exporting Parser, ParserT as synonyms of Parsec, ParsecT. There is no good reason for this and it can cause confusion. Also, when possible, we replace imports of Text.Parsec with Text.Pandoc.Parsing. The idea is to make it easier, at some point, to switch to megaparsec or another parsing engine if we want to. New (re-)exports: Stream(..), updatePosString, SourceName, Parsec, ParsecT. Removed exports: Parser, ParserT [API change].
• Export errorMessages, messageString [API change].
• Export fromParsecError, which can be used to turn a parsec ParseError into a regular PandocParseError (#8382) [API change].
• Remove nested [API change]. It was not being used, and in fact it was a bad idea from the beginning, as it had no hope of solving the problem it was introduced to solve.
• Change characterReference, charsInBalanced. characterReference so they now return a Text (some named references don’t correspond to a single Char). Use the the lookupEntity function from commonmark-hs instead of the slow one from tagsoup [API change].
• charsInBalanced now takes a Text parser rather than a Char parser as argument [API change].

Text.Pandoc.Shared:

• Export textToIdentifier [API change].
• Remove deprecated crFilter. [API change]
• Remove deprecated deLink. [API change]
• Deprecate notElemText.
• Deprecate makeMeta.
• Remove pandocVersion (now available in Text.Pandoc.Version as pandocVersionText).
• Remove findM [API change]. This was only used in one place, and can be replaced with simpler code.
• Remove deprecated makeMeta [API change].
• Remove ordNub [API change]. This is just nubOrd from Data.Containers.ListUtils.
• Remove mapLeft [API change]. This is just a synonym for Bifunctor.first.
• Remove elemText, notElemText [API change].
• Drop export of pandocVersion and pandocVersionText, which are now exported by Text.Pandoc.Version.
• Remove escapeURI, isURI. These are now exported by Text.Pandoc.URI, and removing them from Shared helps make the module structure more straightforward.
• Use LineBreak as default block sep in blocksToInlines. (#8499, Albert Krewinkel). This change also affects the pandoc.utils.blocks_to_inlines Lua function.
• defaultUserDataDir is no longer exported (it has been moved to T.P.Data) [API change].
• New function figureDiv, offering offers a standardized way to convert a figure into a Div element (Albert Krewinkel) [API change].

Text.Pandoc.Writers.Shared:

• Export htmlAddStyle, htmlAlignmentToString and htmlAttrs [API change] (Wout Gevaert).
• Use ‘literal tag’ instead of ‘text (T.unpack tag)’ in tagWithAttrs (Wout Gevaert).
• toTableOfContents: handle nested Divs better (#8402).

Rename Text.Pandoc.Network.HTTP -> Text.Pandoc.URI. This is still an unexported internal module. Export urlEncode, escapeURI, isURI, schemes, uriPathToPath. Drop exports of schemes and uriPathToPath.

Text.Pandoc.URI isURI: don’t require non-ASCII characters to be escaped (#8508).

Rename Text.Pandoc.Readers.LaTeX.Types -> Text.Pandoc.TeX (internal module).

Text.Pandoc.Options:

• WriterOptions now has a field writerListTables, specifying that list tables be used in RST output [API change].
• New writerEpubTitlePage field on WriterOptions (#6097) [API change].
• Remove writerEpubChapterLevel, add writerSplitLevel [API change].

Text.Pandoc.Filter:

• Export applyFilters [API change].
• Export applyJSONFilter [API Change] (Albert Krewinkel).
• Parameterize applyFilters over scripting engine [API change] (Albert Krewinkel).

New exported module Text.Pandoc.Chunks [API change]. This module provides functions to split Pandoc documents into chunks to be rendered in separate files, e.g. one per section. Internal identifiers are rewritten appropriately to point to the new locations (#6122).

Text.Pandoc.Readers:

• Change argument type of getReader, so it takes a FlavoredFormat instead of a Text [API change] (Albert Krewinkel).

Text.Pandoc.Writers:

• Change argument type of getWriter, so it takes a FlavoredFormat instead of a Text [API change] (Albert Krewinkel).

Text.Pandoc.Templates:

• Do not try to normalize input to getDefaultTemplate (Albert Krewinkel). The function getDefaultTemplate no longer splits off extension modifers from the given format, as that conflicts with using custom writers as formats. Haskell library users should use getDefaultTemplate <=< (fmap formatName . parseFlavoredFormat) if the input format can still contain extensions. The same is true for compileDefaultTemplate, which calls getDefaultTemplate internally
• Add Wrapper type documentation (#8490, William Rusnack).

New exported module Text.Pandoc.Scripting (Albert Krewinkel). The module contains the central data structure for scripting engines (e.g., Lua) [API change].

Text.Pandoc.Error:

• Add new PandocError constructor PandocNoScriptingEngine [API change] (Albert Krewinkel).
• Add new PandocError constructor PandocFormatError [API change] (Albert Krewinkel). The new error is used to report problems with input or output format specifications.
• Add new PandocError constructor PandocNoTemplateError (Albert Krewinkel).
• Remove PandocParsecError constructor from PandocError (#8385). Henceforth we just use PandocParseError.

New module Text.Pandoc.Version, exporting pandocVersionText and pandocVersion [API change]. pandocVersion returns a Version instead of a Text, which is consistent with pandocTypesVersion.

Text.Pandoc.Class:

• Make getPOSIXTime, getZonedTime sensitive to SOURCE_DATE_EPOCH environment variable if set (#7093). (getTimestamp was already sensitive.) This ensures that EPUB builds are reproducible.
• Text.Pandoc.Class no longer exports readDataFile, readDefaultDataFile, setTranslations, and translateTerm [API change].
• Text.Pandoc.Class now exports checkUserDataDir [API change].

T.P.Class.IO: export function writeMedia [API change] (Albert Krewinkel). This is useful for the pandoc.mediabag module.

Separate out Text.Pandoc.Data and Text.Pandoc.Translations from Text.Pandoc.Class (#8348). This makes Text.Pandoc.Class more self-contained.

• Text.Pandoc.Data is now an exported module, providing readDataFile and readDefaultDataFile (both formerly provided by Text.Pandoc.Class), and also getDataFileNames (formerly unexported in Text.Pandoc.App.CommandLineOptions) and defaultUSerDataDir (formerly provided by Text.Pandoc.Shared). [API change]
• Text.Pandoc.Translations is now an exported module (along with Text.Pandoc.Translations.Types), providing readTranslations, getTranslations, setTranslations, translateTerm, lookupTerm, readTranslations, Term(..), and Translations [API change].

Text.Pandoc now exports Text.Pandoc.Data and setTranslations and translateTerm {API change].

Export module Text.Pandoc.Class.IO [API change]. The module is useful when defining instances of class PandocMonad for types that are also instances of MonadIO.

Remove modules Text.Pandoc.Writers.Custom and Text.Pandoc.Readers.Custom [API Change] (Albert Krewinkel). The functions writeCustom and readCustom are available from module Text.Pandoc.Lua.

Text.Pandoc.Server:

• Split this module into a separate package, pandoc-server, allowing the pandoc library to be compiled without server support.
• Return object if JSON is accepted. Previously we just returned a JSON-encoded string. Now we return something like:

  {
      "output": "<p>hello</p>"
      "base64": false,
      "messages": [
          {
              "message": "Not rendering RawInline (Format \"tex\") \"\\\\noe\"",
              "verbosity": "INFO"
          }
      ],
  }
  

  This is a change in the pandoc-server JSON API.
• Set translations in the writer based on lang metadata.
• Return error in JSON object if response is JSON.
• Remove parseServerOpts. [API change]

Text.Pandoc.Lua:

Lua subsystem (Albert Krewinkel):

• The whole Lua subsystem has been moved to a separate package, pandoc-lua-engine. pandoc does not depend on it. convertWithOpts has a new parameter that can be used to pass in the scripting engine defined in pandoc-lua-engine (or a different one, in theory).
• Fix the behavior of Lua “Version” objects under equality comparisons (#8267).
• Support running Lua with a GC-collected Lua state.
• Ensure that extensions marshaling is consistent.
• Produce more informative error messages for pandoc errors. Errors are reported in Lua in the same words in which they would be reported in the terminal.
• Add new module pandoc.format. The module provides functions to query the set of extensions supported by formats and the set of extension enabled per default.
• Add function pandoc.template.apply.
• Add function pandoc.template.meta_to_context. The functions converts Meta values to template contexts; the intended use is in combination with pandoc.template.apply.
• Allow Doc values in WriterOptions.variables. The specialized peeker and pusher function for Context Text values does not go via JSON, and thus keeps Doc values unchanged during round-tripping.
• Fix rendering of Lua errors in Lua, so that the Error running Lua message is not prepended multiple times.
• Add new module pandoc.zip.
• Allow strings in place of compiled templates (#8321). This allows to use a string as parameter to pandoc.template.apply and in the WriterOptions template field.
• Rename reader_extensions/writer_extensions globals as Extensions (#8390).
• Add pandoc.scaffolding.Writer (#8377). This can be used to reduce boilerplate in custom writers.
• Fix peeker for PandocError (Albert Krewinkel). String error messages were incorrectly popped of the stack when retrieving a PandocError.
• Add functions pandoc.text.toencoding, pandoc.text.fromencoding (#8512, Albert Krewinkel).
• Add pandoc.cli module. Allow processing of CLI options in Lua.
• Support -D CLI option for custom writers. A new error PandocNoTemplateError (code 87) is thrown if a template is required but cannot be found.
• Allow table structure as format spec. This allows to pass structured values as format specifiers to pandoc.write and pandoc.read.
• Add function pandoc.mediabag.write (Albert Krewinkel).
• Add module pandoc.structure (Albert Krewinkel). The function make_sections has been given a friendlier interface and moved to the new module; the old pandoc.utils.make_sections has been deprecated.

Custom writers:

• The global variables PANDOC_DOCUMENT and PANDOC_WRITER_OPTIONS are no longer set when the writer script is loaded. Both variables are still set in classic writers before the conversion is started, so they can be used when they are wrapped in functions.
• Deprecate classic custom writers.
• Add function pandoc.write_classic. The function can be used to convert a classic writer into a new-style writer by setting it as the value of Writer:

  Writer = pandoc.write_classic
  

  or to fully restore the old behavior:

  function Writer (doc, opts)
    PANDOC_DOCUMENT = doc
    PANDOC_WRITER_OPTIONS = opts
    load(PANDOC_SCRIPT_FILE)()
    return pandoc.write_classic(doc, opts)
  end
  

• Support extensions in custom writers. Custom writers can define the extensions that they support via the global writer_extensions. The variable’s value must be a table with all supported extensions as keys, and their default status as values. For example, the below specifies that the writer supports the extensions smart and sourcepos, but only the smart extension is enabled by default:

  writer_extensions = {
    smart = true,
    sourcepos = false,
  }
  

• Custom writers can define a default template via a global Template function; the data directory is no longer searched for a default template. Writer authors can restore the old lookup behavior with

  Template = function ()
    local template
    return template.compile(template.default(PANDOC_SCRIPT_FILE))
  end
  

Custom readers:

Use latest versions of commonmark-extensions, texmath, citeproc, gridtables, and skylighting.

Use pandoc-types 1.23. This adds the Figure Block constructor and removes the Null Block constructor.

Require aeson >= 2.0.

Use jira-wiki-markup 1.5.0 (#8511, Albert Krewinkel). Fixes issues with icon-like sequences at the beginning of words.

Use doctemplates 0.11, avoiding a transitive dependency on HsYAML.

Use skylighting 0.13.1.2.

Allow mtl 2.3.1 (Alexander Batischev).

Use latest skylighting-format-context.

Allow building with mtl 2.3.

Remove lua53 flag. We now only support Lua 5.4.

Add hie.yaml for haskell language server.

Add tools/latex-package-dependencies.lua.

Update default CSL with latest chicago-author-date.csl.

make_artifacts.sh: various small improvements.

Remove sample.lua from data files (#8356).

Documentation:

• Deprecate PANDOC_WRITER_OPTIONS in custom writers (Albert Krewinkel).
• Document pandoc.write_classic (Albert Krewinkel).
• Document new table features (Albert Krewinkel).
• Clarify what background-image does in reveal.js (#6450).
• Documentation improvements for blank_before_blockquote (#8324, Pranesh Prakash).
• Update grid table documentation (#8346).
• Add note about MathJax fonts to --embed-resources.
• Use cabal’s –package-env more (#8317, Artem Pelenitsyn).
• Modify Zerobrane instructions to use Lua 5.4 (#8353, Ian Max Andolina).
• Fix documentation for highlight-style in pandoc-server.md.
• Fix link to fedora package site (#8246, Akos Marton).
• Rephrase paragraph on format extensions (#8375, Ilona Silverwood).
• Update README.template (#8496, Sven Wick).
• Fix a tiny typo in lua-filters.md (TomBen).
• Clarify that --css should be used with -s.
• Clarify font selection for pdf -t ms (#8421, nbehrnd).
• Clarify docs for --metadata-file (#8459).
• Fix typo in epub.md (Vladimir Alexiev).
• Add missing backtick in filters.md (R. N. West).
• doc/lua-filters.md: add documentation for pandoc.format (Albert Krewinkel).
• Fix epub-embed-font documentation (#8455, Terence Eden).
• Removed obsolete Templates section in CONTRIBUTING.md.
• Add manual section on accessible PDFs, archiving standards (#8312, Albert Krewinkel).

Tests.Command: remove unused runTest.

Add pandoc-lua.1 man page.

Improve shell.nix.

Add tools/moduledeps.lua for inspecting the internal module dependency tree.

Fix macOS zip so pandoc-server is a symlink. This cuts its size by 2x.

CI: Improve CI speed by caching more, eliminating macos builds, and splitting benchmarks into a separate action, run by manual dispatch. (We still test that benchmarks build in the regular CI.) The cache can be expired manually by modifying the secret CACHE_VERSION.

Remove the unnecessary Setup.hs from pandoc. Cabal does not need this with build-type ‘simple’.

Add pandoc-lua and pandoc-server (symlinks) and their man pages to releases.

Use hslua-cli package for pandoc-lua interface (Albert Krewinkel).

Add server flag to pandoc-cli, allowing it to be compiled without server support.

pandoc-cli: Allow building a binary without Lua support (Albert Krewinkel). Disabling the lua cabal flag will result in a binary without Lua.

Move --version handling to pandoc-cli. We need it here in order to print information about whether server and Lua support have been compiled in.

Move nightly flag from pandoc to pandoc-cli (#8339).

Makefile changes:

• make help will now print all the targets and what they do.
• Add targets: coverage, weeder, moduledeps, prerelease, ghcid, repl, linecounts, hie.yaml, binpath.
• Note that you can alias pandoc=`make binpath` for convenient local testing of a build.
• Rename quick-cabal -> build, quick-test -> test.
• Exclude tests from SOURCEFILES.

Factor out xml-light into an internal library.

Add CITATION.cff (#8434).

Move trypandoc to a separate repository, jgm/trypandoc.

Add --embed-resources flag (Elliot Bobrow, #7331). This can be used to embed resources without implying --standalone. Deprecate --self-contained in favor of --embed-resources --standalone.

Allow environment variable interpolation in highlight-style and pdf-engine fields in defaults files (#8061; Jaehwang Jung, #8073).

Allow placing custom readers and writers in user data directory (Albert Krewinkel, #8112) (readers and writers subdirectories).

Add tsv (tab separated values) as an input format (#7974). [API change]: Text.Pandoc.Readers.CSV now exports readTSV. Internal change: In Text.Pandoc.CSV, CSVOptions has changed so that csvQuote takes a Maybe value.

Add tex_math_dollars to gfm default extensions (reflecting gfm’s new support for math).

RST, Org, Markdown readers: support rowspans and colspans in grid tables (#8202, Albert Krewinkel). Note: the writers does not yet support these more complex grid table features, so these complex grid tables will not round-trip.

HTML, LaTeX, and MediaWiki readers: use formatCode (#8162, #8129, Elliot Bobrow). This moves formatting from inside inline code elements to the outside, since pandoc’s Code element only takes string content.

Markdown reader:

• Don’t parse inline notes with blank lines inside (#8028).
• Allow attributes in special spans (e.g. smallcaps, underline) (Albert krewinkel, #4102). These spans are parsed as SmallCaps or Underline elements, but any attributes are included in a wrapping Span.

HTML reader:

• Allow sublists that are not marked as items (Albert Krewinkel, #8150). This is technically invalid HTML, but it can be found in the wild and browsers handle it.

Org reader (Albert Krewinkel):

• Recognize absolute paths on Windows (Albert Krewinkel, #8201).
• Recognize {webp,jxl} files as images (YI).
• Allow attrs for Org tables (Albert Krewinkel, #8049). Tables with attributes are no longer wrapped in Div elements; attributes are added directly to the table element.
• Support line selection in INCLUDE directives (Brian Leung, #8060).
• Fix Post / Pre mixup when setting emphasis chars (Amir Dekel, #8134).

LaTeX reader:

• Support \includesvg (#8027).
• Unescape characters in \lstinline inside \passthrough (#8179).
• Improve mathEnvWith (#8122). When converting e.g. an align environment to an aligned environment inside a Math element, we need to include a newline before the \end{aligned}, since the previous line might end in a comment.
• Fix treatment of extensions for \input in LaTeX reader (#8092). Previously we required a .tex extension, but TeX allows any extension for \input (as opposed to \include).

RTF reader:

• support \nosupersub (#8170).

TikiWiki reader:

DocBook reader:

• Improved reading <xref> elements (Frerich Raabe, #8065).

JATS reader:

• Strip ref- prefix from ref id in xref (#8007).
• Support edition in references (#8087).

RIS reader:

• Make parser more forgiving (#8034). Allow blank lines after entries. Allow entries with no space after the -, provided they just have a newline, e.g. DB -\n.
• Get right order of names (#8055).

MediaWiki reader:

• Allow HTML comment after row start (#8110).

DokuWiki reader:

• The tex_math_dollars extension is now supported for dokuwiki (but off by default) (#8178).
• Content inside <latex>...</latex> is parsed as raw LaTeX inline, and inside <LATEX>..</LATEX> as raw LaTeX block (#8178).
• The behavior of <php>...</php> is changed, so that instead of producing a code block, it produces raw HTML with <?php ... ?>.

LaTeX writer:

• Improve grouping with autocites (#8088).
• Extend list of book documentclasses (Wentau Han, #8053).
• Fix width of multicolumn cells (Albert Krewinkel, #8090). Cells spanning multiple columns must be given an explicit width, calculated from the table properties.
• Beamer: allow containsverbatim as alternative to fragile (#8080).

HTML writer:

• Add ‘footnotes’ identifier to footnotes section (#8043).
• Fix bug with --number-offset. This formerly caused section divs to be produced, even when --section-divs was not specified (#8097).
• Use CSS flexboxes for columns (Albert Krewinkel). This allows an arbitrary number of columns, while the previous approach assumed exactly two columns.
• Allow “spanlike” classes to be combined (see #8194). Previously classes like “underline” and “marked” had to be the first class in a span in order for the span to be interpreted as a “ul” or “mark” element. This commit allows these special classes to be “stacked,” e.g. [test]{.mark .underline}; in addition, the special classes are no longer required to come first in the list of classes.
• Avoid doubled style attribute when height and width are added to style because of an image, but the image already has a style attribute (#8047).
• Do not include the deprecated doc-endnote role (#8030). doc-endnote was deprecated in DPUB-ARIA 1.1.
• Remove extra soft break for tasklist (black-desk, #8142). Browser will display the extra newline character between checkbox and text as a space, which make tasklist items cannot be aligned.

EPUB writer:

• Allow choice of math method for v3 (#8164). Previously we always used MathML for math in EPUB3, because the spec includes MathML. But this is not widely supported by readers, so it seems better to allow users to choose their math method as they can with EPUB2 or HTML. NOTE: Existing workflows that produce EPUBv3 documents including math will be affected by this change. You must add --mathml to your command line if you want to continue producing MathML.

RST writer:

• Fix missing spaces with nested inlines (#8182).
• Always escape literal backslash (#8178).

Ms writer:

• Add comment in preamble stating generator.
• Fix roff ms syntax highlighting definitions (#8175, thanks to Branden Robinson).

ConTeXt writer:

• Support complex table structures (Albert Krewinkel, #8116). The following table feature are now supported in ConTeXt:

  • colspans,
  • rowspans,
  • multiple bodies,
  • row headers, and
  • multi-row table head and foot.

  The wrapping placetable environment is also given a reference option with the table identifier, enabling referencing of the table from within the document.

• Unify link handling (Albert Krewinkel, #8096). Autolinks, i.e. links with content that’s the same as the linked URL, are now marked with the \url command. All other links, both internal and external, are created with the \goto command, leading to shorter, slightly more idiomatic code. As before, autolinks can still be styled via \setupurl, other links via \setupinteraction.

• Use “sectionlevel” environment for headings (Albert Krewinkel, #5539). The document hierarchy is now conveyed using the \startsectionlevel/\stopsectionlevel by default. This makes it easy to include pandoc-generated snippets in documents at arbitrary levels. The more semantic environments “chapter”, “section”, “subsection”, etc. are used if the --top-level-division command line parameter is set to a non-default value.

Docx writer:

• Add w:lang to rPr for Span and Div with lang attribute, so that Word can know that “Apfel” is not a spelling error (#8026).
• Prevent crashing when handling invalid tables (Albert Krewinkel, #8102). Tables with different numbers of cells per row would sometimes crash pandoc. This fix prevents this by cutting off overlong rows.

ICML writer:

• Support custom-style attribute on Table (#8079).

AsciiDoc writer:

FB2 writer:

• Fix handling of non-section Divs (#8123).

Markdown writer:

• Disable soft wrapping when hard_line_breaks enabled (#8035). We were already doing this for markdown; this commit does the same thing for markua and commonmark and gfm.
• Avoid excessive indentation on bullet lists for commonmark, markua, gfm. They are now nested by 2 spaces instead of 4 (#8011).

Text.Pandoc.Class:

• Add new function findFileWithDataFallback [API Change] (Albert Krewinkel).
• fillMediaBag: Keep attributes of original image on Span (Albert Krewinkel, #8099). Images that cannot be fetched are replaced with a Span that contains the image’s description. The span now also retains all original image attributes and inherits all attributes of the image. Furthermore, the classes image and placeholder are added, and path and title are store in attributes original-image-src and original-image-title, respectively.

Text.Pandoc.Shared:

• makeSections: don’t make a section for a div with class “fragments” (#8098).
• Ensure that Nulls are ignored by makeSection and in segmenting slides (#8155).
• Add formatCode function to Text.Pandoc.Shared [API change] (Elliot Bobrow, #8129).
• taskListItemToAscii: handle asciidoctor’s characters (#8011). Asciidoctor uses different unicode characters for task lists; we should recognize them too and be able to convert them to ascii task lists in formats like gfm.
• Deprecate deLink and mark for later removal.

Text.Pandoc.Writers.Shared:

• toTableOfContents: Don’t replace links with empty spans in TOC (#8020).

Text.Pandoc.Readers.Metadata:

• Ensure that metadata values w/o trailing newlines are parsed as inlines, as the manual states. Previously, they were parsed as inlines if they would otherwise have been a single Plain or Para, but otherwise left unchanged. This led to some quirky results (e.g. #8143). We now use the general function blocksToInlines from T.P.Shared.

Text.Pandoc.Parsing:

• Simplify gridTableWith', gridTableWith [API Change] (Albert Krewinkel). The functions gridTableWith and gridTableWith' no longer takes a boolean argument that toggles whether a table head should be parsed: both, tables with heads and without heads, are always accepted now.

Lua subsystem (Albert Krewinkel):

• Extend pandoc.system module (Albert Krewinkel, #8184). The module now has the additional functions list_directory, make_directory, and remove_directory. This makes it easier to write cross-platform scripts that need to inspect or modify the file system.
• Require pandoc-lua-marshal 0.1.7. Adds a clone methods to Pandoc objects and allows to pass Blocks in instead of full Caption elements.
• Add fields pandoc.readers and pandoc.writers (#8177). The set of supported input and output formats is made available to Lua users.
• Ensure that tables marshaled via JSON arrays behave like Lists. This allows to invoke methods like map and includes on lists like PANDOC_WRITER_OPTIONS.extensions.
• Require hslua-2.2.1, unless lua53 flag is set, and do not reset foreign encoding before running Lua. This fixes a problem where the encoding used for Lua filenames would sometimes mismatch the encoding used by the OS.
• Simplify module loading code. Modules are now loaded directly; the special pandoc Lua package searcher is no longer necessary and has been removed.
• Add function pandoc.mediabag.fill (#8104). The function allows to fill the mediabag with all images in a given document. Images that cannot be fetched are replaced with a Span containing the image description.

Populate mediabag after filters have run (Albert Krewinkel, #8099). The mediabag is filled with document resources after the filters have run. This allows, for example, filter authors to modify image paths before pandoc tries to fetch the images. Lua filters that rely on a filled mediabag can use the new pandoc.mediabag.fill function to perform that action in the filter.

Ms template: redefine rather than removing .CH macro (#8175).

JATS template (Albert Krewinkel, except as noted):

• Include particles, prefix, suffix in names.
• Mark authors with cor-id as corresponding authors. Corresponding authors are marked by setting the attribute corresp="yes" in their respective <contrib> element.
• Unconditionally include permissions element (#8040). Fixes a bug that caused license information to be omitted when no copyright information was provided.
• Follow JATS4R recommendation and PudMed Central for license URI (Castedo Ellerman, #8041).

LaTeX template:

• Rename \textormath to \TextOrMath (Hos Es, #8036).
• Fix links-as-notes (Albert Krewinkel, #8077).

HTML template styles:

• Remove span.underline rule. This is superfluous now that we render Underline as <u>.
• Improve CSS for task lists (#8151).

LaTeX template: Add boxlinks variable for LaTeX/PDF output (#8198). If boxlinks is set but colorlinks is not, then boxes will be printed around links (hidelinks will not be set in hypersetup).

--self-contained: Handle url() in <style> elements (#8193).

Text.Pandoc.PDF: use SHA1 hash of filename when converting SVG. The previous code threw away the directory component of the filename in constructing a new one. This led to surprising results if you had e.g. foo/pic.svg and bar/pic.svg; in the final PDF they’d be the same image, because the latter would overwrite the former in the temp directory.

Remove Muse reader round-trip tests. These are nondeterministic and have repeatedly failed on strange edge cases.

Update fonts MIME following IANA recommendation (Gabriel Lewertowski, #8127).

Future compat change for liftA2 export from Prelude (Georgi Lyubenov, #8132).

Update default.csl from the latest chicago-author-date.csl.

Update manfilter.hs for greater portability (#8045). The tables in our man pages were not rendering correctly with mandoc, now used by default with macOS. mandoc doesn’t allow man formatting inside table cells. For maximum portability, we now render the tables in plain format and include them as code blocks in the man page.

CI: update macos container version (#8197, Sukka).

Add nightly flag. This causes a -nightly-COMPILEDATE suffix to be added the the output of --version (#8016). This is used in the nightly CI builds.

Update dependencies (aeson, skylighting, pandoc-lua-marshall, citeproc, texmath).

Documentation improvements (thanks to Jiří Wolker, Castedo Ellerman, Albert Krewinkel, Bastien Dumont, Cezar Drożak, Benjamin Wuethrich, Ivan Panchenko, Sukil Etxenike, Masataka Ogawa).

New input formats: endnotexml (EndNote XML bibliography), ris (RIS bibliography).

A RIS bibliography file may now be used with --citeproc.

Citeproc: Allow a formatted bibliography to be placed in metadata fields via a Div with class refs (#7969, #526). Thus, one can include a metadata field, say refs, whose content is an empty div with id refs, and the formatted bibliography will be put into this metadata field. It may then be interpolated into a template using the variable refs.

Ensure that you don’t get PDF output to terminal. -t pdf now behaves like -t docx and gives an error unless the output is redirected.

--version now prints hslua version (#7929) and Lua version (#7997, Albert Krewinkel).

Change --metadata-file parsing so that, when the input format is not markdown or a markdown variant, pandoc’s markdown is used (#6832, #7926). When the input format is a markdown variant, the same format is used. Reason for the change: it doesn’t make sense to run the markdown parser with a set of extensions designed for a non-markdown format, and this dramatically limits what people can do in metadata files.

Trim whitespace from math in --webtex (#7892). This fixes problems with –webtex and markdown output, when display math starts or ends with a newline.

--self-contained: issue warning rather than failing with an error if a resource can’t be found (#7904).

New exported module Text.Pandoc.Readers.EndNote, exporting readEndNoteXML and readEndNoteXMLCitation [API change].

New exported module, Text.Pandoc.Readers.RIS, exporting readRIS (#7894) [API change].

LaTeX reader:

• Handle subequations as inline math environment (#7883).
• Rudimentary support for vbox (#7939).
• Support \today (#7905).
• Handle \label and \ref for footnotes (#7930).
• Allow inline groups starting with \bgroup (#7953).
• Use custom TokStream that keeps track of whether macros are expanded. This allows us to improve performance a bit by avoiding unnecessary runs of the macro expansion code (e.g. from 24 ms to 20 ms on our standard benchmark).
• Further optimizations for inline parsing.
• Better handling of \usepackage. If the package is local but causes parse errors, parse everything up to the error and skip the rest. Issue a CouldNotParseIncludeFile warning indicating that parsing failed at that point.
• Text.Pandoc.Readers.LaTeX.Parsing: Monoid and Semigroup instances for TokStream.

HTML reader:

• Give warnings and emit empty note when parsing <a epub:type="noteref"> and the identifier doesn’t correspond to anything in the note table (#7884). Previously we just silently skipped these cases.
• Fix parsing of epub footnotes (#7884).

DocBook reader:

• Handle complete set of entities as specified at https://www.w3.org/2003/entities/2007doc/byalpha.html (#7938).
• Handle abstract in info section (#7747).
• Improve info parsing.
• Simplify metadata parsing code (#7747). Handle abstract as block-level content. Report skipped info elements with --verbose.
• Handle address and coyright in metadata (#7747).

DokuWiki reader:

• Add DokuWiki table alignment (#5202, damon-sava-stanley).

RST reader:

• Fix treatment of headerless simple tables (#7902).
• Wrap math in Span to preserve attributes (#7998, Albert Krewinkel). Math elements with a name, classes, or other fields are wrapped in a Span with these attributes.

JATS reader:

• Improve handling of fn-group elements (#6348, Albert Krewinkel). Footnotes in <fn-group> elements are collected and re-inserted into the document as proper footnotes in the place where they are referenced.
• Handle pub-date (#8000).
• Support PMID, DOI, issue in citations (#7995).
• Improve refs parsing. Handle issn and isbn; use simpler form for issued date.
• Strip ‘ref-’ from ref id in constructing CSL id. This allows better round-tripping, because the JATS writer adds the ref- prefix to the citation id to get the ref element’s id.

Org reader:

• Allow “:” in property drawer keys (Lucas V. R). Any non-space character is allowed as property drawer key, including “:” itself (so it is not really a delimiter). The real delimiter is a space character, so in a drawer like

  :PROPERTIES:
  ::k:ey:: value
  :END:
  

  “:k:ey:” is a key with value “value”.

• Allow comments above property drawer.

• More flexible LaTeX environments (Lucas V. R).

• Handle #+bibliography: as metadata so that it can work with --citeproc.

• Parse #+print_bibliography: as Div with id refs.

• Allow multiple #+bibliography:.

Markdown reader:

• Allow one-column pipe tables with pipe on right (#7919).
• Remove restriction on identifiers, so they no longer need to begin with a letter (#7920).

Docx reader:

• Enable citations extension for docx reader (#7840). When enabled, Zotero, Mendeley, and EndNote citations embedded in a docx are parsed as native pandoc citations. (When disabled, the generated citation text and bibliography are passed through as regular text.) The bibliography generated by the plugin is suppressed. Instead, bibliographic data embedded in citation items is added to the references metadata field so that it can be used with --citeproc.

Docbook writer:

• Interpret links without contents as cross-references (#7360, Jan Tojnar). Links without text contents are converted to <xref> elements. DocBook processors will generate appropriate cross-reference text when presented with an xref element.

Docx writer:

• Single numbering ID for examples (#7895, mjfs). This change ensures that example list items all belong to a single number sequence, so that if items are added or deleted in a word processor, the other items will renumber automatically.
• Add bookmark with table id to table (#7989, Nikolai Korobeinikov, #7285). This allows tables with ids to be linked to.

Ipynb writer:

• Handle metadata better (#7928). Previously we used the markdown writer to render metadata. This had some undesirable consequences (e.g. en dash expanded to -- when smart enabled), so now we use the plain writer.

LaTeX writer:

• Avoid extra space before \CSLRightInline (#7932).
• Add scrreport to chaptersClasses (#6168, ivardb).
• Support page,trim,clip attributes on images (#7181).
• Add () after booktabs rules (#8001). These commands take optional arguments with () and [], which can lead to problems if the content of the table cell begins with these characters.

RST writer:

• Support all standard metadata (“bibliographic”) fields.

HTML writer: performance improvements.

Org writer:

• Stop indenting property drawers, quote blocks (#3245, Albert Krewinkel). This follows the current default org-mode behavior.

Markdown writer:

• Move table-related code into submodule (Albert Krewinkel).
• Don’t produce redundant header identifier when the gfm_auto_identifiers extension is set (#7941).
• Update escaping rules for \. We now escape \ only if raw_tex is enabled or it is followed by a non-alphanumeric.

JATS writer:

• Encode author “others” as <etal/> (Albert Krewinkel). Citeproc adopted the BibTeX convention to use the author name “others” when there are additional authors that are not named. JATS uses the <etal> element for this.
• Avoid doubled ref-list element (#7990). Previously when generating JATS with the element_citations extension enabled, the references were put in a doubly-nested ref-list element (<ref-list><ref-list>...).
• Keep edition info in element citations (#7993, Albert Krewinkel).
• Fix handling of CSL variable ‘page’ (not ‘pages’ as we had before). It should go to ‘lpage’ and ‘rpage’, not ‘page-range’.

EPUB writer: refactor for clarity (#7991, Jonathan Dönszelmann, Ola Wolska, Ivar de Bruin, Jaap de Jong).

Custom writer (Albert Krewinkel):

• Support new-style Writer function (Albert Krewinkel). See the documentation for custom writers for details.
• Produce stacktrace if Writer function fails

Text.Pandoc.Logging: add CouldNotParseIncludeFile constructor for LogMessage [API change].

Text.Pandoc.Shared:

• Put id attributes on TOC entries (#7907, damon-sava-stanley). Naming scheme of id is “toc-” + id of linked to header/section. Effects HTML, Markdown, Powerpoint, and RTF.
• Define ordNub as alias for nubOrd from containers package (#7963, Albert Krewinkel).
• Export ensureValidXmlIdentifiers. This function changes identifiers that don’t start with letters, and internal links to these identifiers, making them compatible with XML standards. The change is simple: we add id_ to the front. There is potential for duplication if there are already id_... identifiers defined, but this seems rare enough not to worry too much about.

Ensure that valid XML identifiers are used in Docbook, EPUB, FB2, HTML4, S5, Slidy, Slideous, ICML, ODT, TEI writers. Thus, if you convert [anchor]{#1} and [link to](#1), id_1 will be used instead of 1 for the identifier.

Lua (Albert Krewinkel).

• Add module pandoc.layout to format and layout text.
• Move custom writer code into Lua hierarchy.
• Use pandoc-lua-marshal 0.1.5.
• Allow any type of callable object as argument to List functions filter, map, and find_if. These previously required the argument to be of type function, which was too restrictive.
• Inline: the type of Image captions is now Inlines instead of List.
• Allow passing Sources to pandoc.read (#8002).

Text.Pandoc.PDF:

• Restore wkhtmltopdf as default pdf engine for HTML (Albert Krewinkel).
• Allow custom writer as format if engine is explicitly specified (#7898, Albert Krewinkel). Note that it may be necessary to explicitly specify a template on the command line.

Text.Pandoc.MediaBag: improve detection of absolute paths (#7881).

Text.Pandoc.Extensions:

• Remove raw_tex extension from list of commonmark extensions, and from the commonmark_x defaults. commonmark doesn’t parse raw TeX, and it doesn’t make sense to write it if we don’t parse it.
• Remove native_divs from allowed gfm extensions (#7965). This allows <div> to be suppressed using -raw_html. Previously native_divs was enabled but could not be suppressed, because it was not in the list of available extensions for commonmark-based formats.

Text.Pandoc.Parsing:

• Partition module into (internal) submodules (#7962, Albert Krewinkel).

• Unify grid table parsing (#7971, Albert Krewinkel). Grid table parsing in Markdown and RST are updated use the same functions. Functions are generalized to meet requirements for both formats. This change also lays the ground for further generalizations in table parsers, including support for advanced table features. [API change] in Text.Pandoc.Parsing:

  • Parse results of functions tableWith' and gridTableWith' are now a mf TableComponents instead of a quadruple of alignments, column widths, header rows and body rows.
  • Additional exports from Text.Pandoc.Parsing: tableWith', TableComponents, TableNormalization, toTableComponents, toTableComponents'.

Text.Pandoc.XML.Light: add versions of the parsers that allow specifying a custom entity map. Exports new functions: parseXMLElementWithEntities, parseXMLContentsWithEntities [API change].

Text.Pandoc.Writers.GridTable: improve module documentation (Albert Krewinkel).

Text.Pandoc.Parsing.GridTable: simplify column handling code (Albert Krewinkel).

Text.Pandoc.MIME: Add mime type for mkv extension (#7181).

Text.Pandoc.Asciify: Fix regression with ascii_identifiers and Turkish undotted i (#8003).

Relax upper bound for hslua, allow hslua-2.2. (#7929) Lua 5.4 is used by default after this is merged. Packagers may still include Lua 5.3 instead by building pandoc with --constraint='hslua <2.2'. Differences between 5.3 and 5.4 should not generally affect pandoc Lua filters. See list of incompatible changes here: https://www.lua.org/manual/5.4/manual.html#8.1

JATS template: allow multiple licenses (Albert Krewinkel).

LaTeX template:

• Skip \babelprovide if babel-lang is empty (#7945) to avoid an error.
• Move hyperref near end of preamble (#5811). It now comes after header-includes and right before title, author, date, abstract. Note: Users who presuppose hyperref in their header-includes will now have to add \usepackage{hyperref} to their header-includes to make it available there. (The redundant \usepackage will do no harm in this case.)
• Remove special redefinition of \sout. This used to be necessary to avoid problems with hyperref, when headings contain strikeout text, but it does not seem to be necessary any more (tested).

Tests: improve location reporting of failing tests (Albert Krewinkel).

Add tests for idempotency of makeSections (#7950).

Add armhf support in linux build script (#7944, Tony).

Use latest pandoc-types, so that toJSONFilter will work with Meta and MetaValue.

Use latest doclayout, texmath, commonmark, citeproc.

INSTALL.md: add reference to install via winget (#7951, Guriy Samarin).

MANUAL.txt:

• Document way to get list in block quote in slide shows (#7916).
• Minor changes to security section.
• Note that smart works for html input.
• Fix typos (#7934, Dimitris Apostolou).
• Fix documentation for citations and org mode.
• Remove claim that --self-contained does not work with --mathjax (#682).

doc/org.md: remove obsolete citations section. This mostly described citation formats we no longer support.

doc/lua-filters.md: Fix typo (#7981, Mario Lang).

Makefile: Use cabal for default build

Add lua53 cabal flag. It is false by default. If set to true, compile with hslua 2.1 and Lua 5.3, otherwise hslua 2.2 and Lua 5.4.

Support pagedjs-cli as pdf engine (#7838, Albert Krewinkel). PagedJS is a polyfill and supports the Paged Media standards by the W3C. https://www.pagedjs.org/

CommonMark reader: fix source position after YAML metadata (#7863).

LaTeX reader:

Docx reader: Parse both Zotero citation and bibliography as FieldInfo (#7840).

LaTeX writer:

• Allow arbitrary frameoptions to be passed to a beamer frame, using the frameoptions attribute (#7869).
• Add s and squeeze to recognized beamer frameoptions (#7869).

Markdown writer: handle explicit column widths with pipe tables (#7847). If a table has explicit column width information and the content extends beyond the --columns width, we need to adjust the widths of the pipe separators to encode this width information.

Docx writer: Separate tables even with RawBlocks between (#7724, Michael Hoffmann). Adjacent docx tables need to be separated by an empty paragraph. If there’s a RawBlock between tables which renders to nothing, be sure to still insert the empty paragraph so that they will not collapse together.

Man writer: use custom font V for inline code (#7506). The V font is defined conditionally, so that it renders like CB in output formats that support that, and like B in those that don’t (e.g. the terminal). Aliases also defined for VI, VB, VBI.

Asciidoc writer: Support checklists in asciidoctor writer (#7832, Nikolai Korobeinikov, ricnorr). The checklist syntax (similar to task_list in markdown) seems to be an asciidoctor-only addition.

HTML writer:

• Avoid duplicate “style” attributes on table cells (#7871).
• Don’t break lines inside code elements. With the new (default) line wrapping of HTML, in conjunction with the default CSS which includes code { whitespace: pre-wrap; }, spurious line breaks could be introduced into inline code (#7858).

Custom writer: preserve order of element attributes (#7489, Albert Krewinkel). Attribute key-value pairs are marshaled as AttributeList, i.e., as a userdata type that behaves both like a list and a map. This allows to preserve the order of key-value pairs.

Switch to hslua-2.1 (Albert Krewinkel). This allows for some code simplification and improves stability.

Don’t read files outside of user data directory (Even Brenden). If a file path does not exist relative to the working directory, and it does exist relative to the user data directory, but outside of of the user data directory, do not read it. This applies to readDataFile and readMetadataFile in PandocMonad and, by extension, any module that uses these by passing them relative paths.

Text.Pandoc.Class.makeCanonical: Correctly handle consecutive “..“s at the beginning of a path (Even Brenden). Prior to this commit, ../../file would evaluate to file, when it should be unchanged.

Search for metadata files in $DATADIR/metadata (#7851, Even Brenden). If files specified with --metadata-file are not found in the working directory, look in $DATADIR/metadata (#5876).

Text.Pandoc.Class: export readMetadataFile [API change] (#5876).

Text.Pandoc.Error: export new PandocCouldNotFindMetadataFileError constructor for PandocError [API change] (#5876).

Avoid putting a frame around speaker notes in beamer (#7857). If speaker notes (a Div with class ‘notes’) occur right after a section heading, but above slide level, the resulting \note{..} caommand should not be wrapped in a frame, as that will cause a spurious blank slide.

CSS in HTML template: adjust #TOC and h1 on mobile (#7835, Mauro Bieg).

Text.Pandoc.Readers.LaTeX.Parsing: don’t export totoks. Make the first param of tokenize a SourcePos instead of SourceName, and use it instead of totoks.

Text.Pandoc.Shared: Modify stringify so it ignores [Citation] inside Cite (#7855). Otherwise we’ll sometimes get two copies of things, one from the citationPrefix or citationSuffix and another from the embedded fallback text. When there is no fallback text, we’ll get no content. However, it really isn’t an alternative to just rely on the result of running query on the embedded Citations; this will result in a jumble of text rather than anything structured.

Omit --enable-doc in the cabal haddock invocation in tools/build-and-upload-api-docs.sh.

Text.Pandoc.App.Opt: fix logic bug in fullDefaultsPath. Previously we would (also) search the default user data directory for a defaults file, even if a different user data directory was specified using --data-dir. This was a mistake; if --data-dir is used, the default user data directory should not be searched.

Text.Pandoc.Shared: defaultUserDataDir behavior change (#7842). If the XDG data directory is not defined (e.g. because it’s not supported in the OS or HOME isn’t defined), we return the empty string instead of raising an exception.

Update command tests to distinguish stderr and test exit status.

MANUAL: add that speaker notes can be used with beamer (#7856).

Update build-and-upload-api-docs.sh.

Document --trace option. Document no-check-certificate in defaults files. Document ‘sandbox’ option for defaults files. (#7873).

Fix pattern syntax in sample readability custom reader.

doc/custom-readers.lua: add example for “readable HTML.”

Fix message in man page about where code can be found.

manfilter.lua: remove extra indent in table cells with code blocks.

Fix lua-filters documentation for table column widths (#7864).

epub.doc: Update links to KindleGen (#7846, Benson Muite, Mauro Bieg). KindleGen has been deprecated and we need to link to archived versions.

Use tables in defaults files documentation, so each default option is paired with the corresponding command-line option (Carsten Allefeld).

Use skylighting 0.12.2.

Add pandoc-lua-marshal to Nix shell (#7849, Even Brenden).

Support markua as an output format (#1871, Tim Wisotzki and Saumel Lemmenmeier). Markua is a markdown variant used by Leanpub.

Add text wrapping for HTML output (#7764). Previously the HTML writer was exceptional in not being sensitive to the --wrap option. With this change --wrap now works for HTML. The default (as with other formats) is automatic wrapping. Note that the contents of script, textarea, and pre tags are always laid out with the flush combinator, so that unwanted spaces won’t be introduced if these occur in an indented context in a template.

Don’t read sources until in/out format are verified (#7797).

Issue error with --list-extensions for invalid formats (#7797).

Make --citeproc recognize .yml as well as .yaml extensions as YAML bibliography files (#7707, Jörn Krenzer).

Use latest version of KaTeX with --katex.

Fix parsing of footnotes in --metadata-file (#7813). Previously non-inline footnotes were not being parsed.

ODT reader:

• Parse list-header as a list item (Tuong Nguyen Manh).

Commonmark reader:

• Put sourcepos attribute on header, not enclosing div with -f commonmark+sourcepos (#7769).

Markdown reader:

• Don’t allow ^ at beginning of link or image label (#7723). This is reserved for footnotes. Fixes regression from 0a93acf.
• Fix parsing of “bare locators” after author-in-text citations. Previously @item [p. 12; @item2] was incorrectly parsed as three citations rather than two. This is now fixed by ensuring that prefix doesn’t gobble any semicolons.
• Revert changes to inlinesInBalancedBrackets (commit fa83246), which caused regressions.
• Improve detection of pipe table line widths (#7713). Fixed calculation of maximum column widths in pipe tables. It is now based on the length of the markdown line, rather than a “stringified” version of the parsed line. This should be more predictable for users. In addition, we take into account double-wide characters such as emojis.

Custom (Lua) readers:

• First argument is now a list of sources instead of the concatenated text (Albert Krewinkel). The list structure can easily be converted to a string by applying tostring, but it is also possible to access the elements (each with a text and name). A small example is added to the custom reader documentation, showcasing its use in a reader that creates a syntax-highlighted code block for each source code file passed as input. Existing readers will still work through a fallback mechanism, issuing a deprecation notice.

Org reader:

• Parse official org-cite citations (#7329). We also support the older org-ref style as a fallback. We no longer support the “markdown style” or “Berkeley style” citations.
• Support alphabetical (fancy) lists (Lucas Viana). When the fancy_lists extension is enabled, alphabetical list markers are allowed, mimicking the behaviour of Org Mode when org-list-allow-alphabetical is enabled.
• Support counter cookies in lists (Lucas Viana). Such cookies are used to override the item counter in ordered lists. In org it is possible to set the counter at any list item, but since Pandoc AST does not support this, we restrict the usage to setting an offset for the entire ordered list, by using the cookie in the first list item.
• Allow trailing spaces after key/value pairs in directives (Albert Krewinkel). Ensures that spaces at the end of attribute directives like #+ATTR_HTML: :width 100% (note the trailing spaces) are accepted.

LaTeX reader:

• Omit visible content for \label{...}. Previously we included the text of the label in square brackets, but this is undesirable in many cases. See discussion in https://github.com/jgm/pandoc/issues/813#issuecomment-978232426.
• Improve references (#813). Resolve references to theorem environments. Remove the Span caused by “label” in figure, table, and theorem environments; this had an id that duplicated the environments’ id.
• Fix semantics of \ref. We were including the ams environment type in addition to the number. This is proper behavior for \cref but not for \ref. To support \cref we need to store the environment label separately.
• Add babel mappings for Guajati (gu) and Oriya (or) (#7815).
• Fix typo panjabi -> punjabi in babel mappings (#7814).

HTML reader:

• Parse attributes on links and images (#6970).

Docx reader:

• Handle multiple pic elements inside a drawing (#7786).
• Change elemToParPart to return [ParPart] instead of ParPart. Also remove NullParPart constructor, as it is no longer needed. This will allow us to handle elements that contain multiple ParParts, e.g. w:drawing elements with multiple pic:pic.

DocBook reader:

• Collapse internal spaces in literal and other similar tags (#7821), as the standard docbook toolchain does.
• Be sensitive to spacing=“compact” in lists (#7799). When spacing="compact" is set, Para elements are turned into Plain, so we get a “tight” list.

Markdown writer:

• Add new exported function writeMarkua from Text.Pandoc.Writers.Markdown [API change] (#1871, Tim Wisotzki and Saumel Lemmenmeier).
• Fix indentation issue in footnotes (#7801).
• Avoid extra space before citation suffix if it already starts with a space.
• Ensure semicolon between the locator and the next citation when an author-in-text citation has a locator and following citations.
• Improve escaping for # (#7726).

Custom (Lua) writers:

• Allow variables to be set via second return value of Doc (#6731, Albert Krewinkel). New templates variables can be added by giving variable-value pairs as a second return value of the global function Doc. Example:

  function Doc (body, meta, vars)
    vars.date = vars.date or os.date '%B %e, %Y'
    return body, vars
  end
  

• Provide global PANDOC_WRITER_OPTIONS (#6731, Albert Krewinkel).
• Assign default Pandoc object to global PANDOC_DOCUMENT (Albert Krewinkel). The default Pandoc object is now non-strict, i.e., only the parts of the document that are accessed will be marshaled to Lua. A special type is no longer necessary. This change also makes it possible to use the global variable with library functions such as pandoc.utils.references, or to inspect the document contents with walk().

LaTeX writer:

• Fix typo panjabi -> punjabi in babel mappings (#7814).

MediaWiki writer:

• Remove redundant display text for wiki links (Jesse Hathaway).

Docx writer:

• Handle bullets correctly in lists by not reusing numIds (#7689, Michael Hoffmann). This fixes a bug in which a Div in a list item would receive bullets on its contained paragraphs.

Org writer:

• Fix list items starting with a code block or other non-paragraph content (#7810).
• Avoid blank lines after tight sublists (#7810).
• Fix extra blank line inserted after empty list item (#7810).
• Don’t add blank line before lists (#7810).
• Support starting number cookies (Lucas Viana). This is necessary for lists that start at a number other than 1.
• Support the new org-cite syntax (#7329).

Haddock writer:

• Avoid blank lines after tight sublists (#7810).

Ipynb writer:

• Ensure deterministic order of keys.
• Handle cell output with raw block of markdown (#7563, Kolen Cheung). Write RawBlock of markdown in code-cell output. This is designed to fit the behavior of #7561, which makes the ipynb reader parse code-cell output with mime “text/markdown” to a RawBlock of markdown. This commit makes the ipynb writer writes this RawBlock of markdown back inside a code-cell output with the same mime, preserving this information in round-trip.
• In choosing between multiple output options, always favor those marked with the output format over images (Kolen Cheung). Previously, both fmt == f case and Image have a rank of 1.

Ipynb reader & writer: properly handle cell “id” (#7728). This is passed through if it exists (in Nb4); otherwise the writer will add a random one so that all cells have an “id”.

Ms writer:

• Properly encode strings for PDF contents (#7731).

JATS writer:

• Keep quotes in element-citations (Albert Krewinkel). Fixed a bug that lead to quote characters being lost in element-citations.

RTF writer:

• Properly handle images in data URIs (#7771).

Commonmark writer:

• Allow ‘)’ delimiters on ordered lists.

RST writer:

• Avoid extra blank line after empty list item (#7810).

HTML writer:

• Make line breaks more consistent. With --wrap=none, we now output line breaks between block-level elements. Previously they were omitted entirely, so the whole document was on one line, unless there were literal line breaks in pre sections. This makes the HTML writer’s behavior more consistent with that of other writers. Also, regardless of wrap settings, put newline after <dd> and after block-level elements in the footnotes section. And add a line break between an img tag and the associated figcaption.
• reveal.js: Make sure images with r-stretch are not in p tags. They must be direct children of the section. There was previously code to make this work with the older class name stretch, but the name has changed in reveal.js.
• reveal.js: don’t add r-fit-text class to section. It must go on the header only.

AsciiDoc writer:

• Improve detection of intraword emphasis (#7803).

OpenDocument writer:

• Fix vertical alignment bug with display math (#7777). Previously some displayed formulas would be floated above a preceding text line. This is fixed by setting vertical-rel to text rather than paragraph-content.

JATS template (Albert Krewinkel):

• Fix position of contrib affiliations in authoring set. Any <aff> element must come before any <email> element.
• Fix affiliation tagging in articleauthoring output. Affiliations were xlinked even in the articleauthoring tag set, but <aff> are not allowed as children of contrib-group elements in that tag set. Each affiliation must be listed directly in the contrib element.
• Add support for article subtitles.

EPUB template:

• Include abstract in default template.
• Ensure that the essential styles needed by pandoc (styles.html partial) are included in the templates. This is important for correct formatting of CSL bibliographies. Note that much of the styling in styles.html will be ignored for EPUB, because of the conditional on document-css. Setting the document-css variable will cause it not to be ignored.

HTML template: Add abstract (#7588, Jannik Buhr, John MacFarlane). By default, a localized title (the word “Abstract”) will be used, unless the variable abstract-title is set.

ConTeXt template: Make title appear in PDF title bar. This is recommended for accessibility reasons. Note: doesn’t work with macOS Preview.app.

reference.pptx: change to use 16:9 aspect ratio, Powerpoint’s default.

Text.Pandoc.Writers:

• Do not export writeCustom [API change]. This ensures that all writers exported in T.P.Writers are parameterized and work with any PandocMonad type. This is consistent with T.P.Readers, as readCustom is not exported from that module either.

Text.Pandoc.Writers.Shared:

• endsWithPlain now returns True if the list ends with a list that ends with a Plain, and so on recursively (#7810).

Text.Pandoc.Class.IO:

• writeMedia: unescape percent-encoding in creating file path. This addresses a problem with spaces in image filenames when creating PDFs (#7819); it also affects --extract-media.

New internal module Text.Pandoc.Writers.Blaze, exporting layoutMarkup. This converts a blaze Html structure into a doclayout Doc Text.

Text.Pandoc.Extensions:

• parseFormatSpec: cleaner error message for invalid extensions.

Text.Pandoc.MediaBag:

• Fix bug in extractMedia, which caused the test for .. in paths to fail, with the result that images with .. in the path name could be extracted outside of the directory specified by extractMedia. It also led a check for media in resource paths to fail in the docx reader.

Text.Pandoc.Citeproc:

• Avoid adding comma before an author-in-text citation in a note if it begins with a title (no author) (#7761).
• Text.Pandoc.Citeproc.Locator now exports toLocatorMap, LocatorInfo, and LocatorMap. The type of parseLocator has changed, so it now takes a LocatorMap rather than a Locale as parameter, and returns a LocatorInfo instead of a tuple.
• Fix citation locator detection for German. toLocatorMap now stores keys case-folded. We want to do a case-insensitive comparison when parsing locators, so that e.g. both Chap. and chap. work. Previously we case-folded terms when doing the lookup, but they weren’t case-folded in the map itself, which led to locator-detection breaking for German (where the terms have uppercase letters).

Lua (Albert Krewinkel):

• Allow single elements as singleton MetaBlocks/MetaInlines. Single elements should always be treated as singleton lists in the Lua subsystem.
• Add pandoc.template module. The module provides a compile function to use strings as templates.
• Add pandoc.WriterOptions constructor.
• Add function pandoc.write.
• Provide global PANDOC_WRITER_OPTIONS (#5221).
• The function Text.Pandoc.Filter.applyFilters now takes a filter environment of type Environment, instead of a ReaderOptions value [API change].
• The Environment type is exported from Text.Pandoc.Filter and allows to combine ReaderOptions and WriterOptions in a single value [API change].
• Global, exported from Text.Pandoc.Lua, has a new type constructor PANDOC_WRITER_OPTIONS [API change].
• Add constructors pandoc.Blocks and pandoc.Inlines. The functions convert their argument into a list of Block and Inline values, respectively. When applied to a string, they split the string into Str elements separated by Space or SoftBreak (#7712).
• Support topdown traversals The traversal order of filters can now be selected by setting the key traverse of the filter to either 'topdown' or 'typewise'; the default remains 'typewise'. Topdown traversals can be cut short by returning false as a second value from the filter function. No child-element of the returned element is processed in that case.
• Marshal ReaderOptions field extensions, track_changes via JSON. Extensions are now available as a list of strings; the track-changes settings are given as the kebab-case representation used in JSON.
• Allow binary (byte string) readers to be used with pandoc.read.
• Use global state when parsing documents in pandoc.read. The function pandoc.read is updated to use the same state that was used while parsing the main input files. This ensures that log messages are preserved and that images embedded in the input are added to the mediabag.
• Cleanup stack in peekReadOptionsTable. A ReaderOptions element was left on top of the stack when the peekReadOptionsTable function was invoked.
• walk methods are added to Pandoc, Block, Inline, Blocks, Inlines values; the methods are similar to pandoc.utils.walk_block and pandoc.utils.walk_inline, but apply to filter to the element’s contents.
• Functions of name Doc are no longer accepted as alternatives for Pandoc filter functions. This functionality was undocumented.
• Improve handling of empty caption, body by from_simple_table #7776). Create truly empty table caption and body when these are empty in the simple table.
• Change representation of TableHead, TableFoot, and Row values (#7718). The objects now also follow the principle that element attributes are accessible through the .attr field. Rows in TableHead and TableFoot are available via the .rows field. Row objects have a .cells field, containing the list of table cells.
• Simplify code of pandoc.utils.stringify. Minor behavior change: plain strings nested in tables are now included in the result string.
• Simplify and deprecate function pandoc.utils.equals. The function is no longer required for element comparisons; it is now an alias for the == operator.
• Add function pandoc.utils.references (#7752).
• Add new library function pandoc.utils.type. The function behaves like the default type function from Lua’s standard library, but is aware of pandoc userdata types. A typical use-case would be to determine the type of a metadata value.
• Fix return types of blocks_to_inlines, make_sections. Ensures the returned lists have the correct type (Inlines and Blocks, respectively).
• Use more natural representation for Reference values Omit false boolean values, push integers as numbers.

Lua: use package pandoc-lua-marshal (#7719, Albert Krewinkel). The marshaling functions for pandoc’s AST are extracted into a separate package. The package comes with a number of changes:

• Pandoc’s List module was rewritten in C, and error messages were improved.
• Lists of Block and Inline elements are marshaled using the new list types Blocks and Inlines, respectively. These types currently behave identical to the generic List type, but give better error messages. This also opens up the possibility of adding element-specific methods to these lists in the future.
• Elements of type MetaValue are no longer pushed as values which have .t and .tag properties. This was already true for MetaString and MetaBool values, which are still marshaled as Lua strings and booleans, respectively. Affected values:
  • MetaBlocks values are marshaled as a Blocks list;
  • MetaInlines values are marshaled as a Inlines list;
  • MetaList values are marshaled as a generic pandoc Lists.
  • MetaMap values are marshaled as plain tables and no longer given any metatable.
• Cell values are now marshaled as userdata objects; a constructor function for table cells is provided as pandoc.Cell.
• The test suite for marshaled objects and their constructors has been extended and improved.
• A bug in Citation objects, where setting a citation’s suffix modified it’s prefix, has been fixed.
• Inlines, Blocks, and List objects now have an __eq metamethod, testing equality by comparing two lists element-wise.

Powerpoint tests: shorten lines by grouping tests (Albert Krewinkel). This makes the test output more pleasant to read in narrow terminal windows.

make check: check for unreleased dependencies.

Add tools/build-and-upload-api-docs.sh.

Update cabal description.

MANUAL.txt: Add section on EPUB styling.

MANUAL.txt: clarify “standard Markdown” as “original Markdown” (#7802, Martin Fischer).

doc/custom-writers.md: use filter to include source of example.

Add an example to doc/custom-readers.md.

Fix typo in custom-readers.md (#7722, Mauro Bieg).

doc/jats.md: add link to JATS documentation (Martin Fischer).

doc/lua-filters.md: many improvements (Albert Krewinkel, John MacFarlane).

Use commonmark-extensions 0.2.3. This allows a bare-word class attribute on fenced divs.

Use ipynb 0.2.

Use citeproc 0.6.0.1.

Use texmath 0.12.4.

Use doctemplates 0.10.0.1.

Add interface for custom readers written in Lua (#7669). Users can now do -f myreader.lua and pandoc will treat the script myreader.lua as a custom reader, which parses an input string to a pandoc AST, using the pandoc module defined for Lua filters. A sample custom reader can be found in data/creole.lua. Also see documentation in doc/custom-readers.md.

New module Text.Pandoc.Readers.Custom, exporting readCustom [API change].

Allow plain to be used in raw attribute syntax.

Accept empty --metadata-file (#7675). This was a regression from 2.15 behavior.

Markdown reader: Improve inlinesInBalancedBrackets. This is just a small improvement in terms of performance, but it’s simpler and more direct code. Also, we avoid parsing interparagraph spaces in balanced brackets, as the original did.

BibTeX reader: Properly handle commented lines in BibTeX/BibLaTeX (#7668).

RST reader: handle class attribute for for custom roles (#7699, willj-dev). Previously the class attribute was ignored, and the name of the role used as the class.

DocBook reader:

• Add <titleabbr> support (Rowan Rodrik van der Molen).
• Support for <indexterm> (#7607, Rowan Rodrik van der Molen).

LaTeX reader:

• Add rudimentary support for \autoref (#7693).
• Add ‘uri’ class when parsing \url, for consistency with treatment of autolinks in other formats (#7672).

JATS reader: Capture alt-text in figures (#7703, Aner Lucero).

MediaWiki writer: use HTML spans for anchors when header has id (#7697). We need to generate a span when the header’s ID doesn’t match the one MediaWiki would generate automatically. Note that MediaWiki’s generation scheme is different from pandoc’s (it uses uppercase letters, and _ instead of -, for example). This means that in going from markdown to mediawiki, we’ll now get spans before almost every heading, unless explicit identifiers are used that correspond to the ones MediaWiki auto-generates. This is uglier output but it’s necessary for internal links to work properly.

Markdown writer: don’t create autolinks when this loses information (#7692). Previously we sometimes lost attributes when rendering links as autolinks.

Text.Pandoc.Readers.Metadata: allow multiple YAML documents when parsing YAML for yamlBsToRefs. Some people use --- as the end delimiter in YAML bibliography files, which causes the yaml library to emit an error unless we explicitly allow multiple YAML documents (and just consider the first).

JATS writer:

• Ensure figures are wrapped with <p> in list items (Albert Krewinkel). This prevents the generation of invalid output.
• Add URL to element citation entries (Albert Krewinkel). The URL of a reference, if present, is added in tag <uri> to element-citation entries.

HTML writer: Don’t create invalid data- attribute for empty attribute key (#7546).

LaTeX writer:

• Babel mappings: use ancientgreek for grc.
• With -t latex-smart, don’t generate \ldots from ellipsis (#7674). Instead just use unicode ellipsis.

JATS template: fix equal-contrib attribute (Albert Krewinkel). The standard requires the value to be either yes or no, but is was set to true for authors who contributed equally.

reveal.js template: Add disableLayout variable (Christophe Dervieux).

Text.Pandoc.Error: sort errors in handleError by exit code (Albert Krewinkel).

Text.Pandoc.Writers.Shared: Improve toLegacyTable (#7683, Christian Despres).

Lua subsystem:

• Include lpeg module (#7649, Albert Krewinkel). Compiles the lpeg library (Parsing Expression Grammars For Lua) into the program. Package maintainers may choose to rely on package dependencies to make lpeg available, in which case they can compile the with the constraint lpeg +rely-on-shared-lpeg-library. lpeg and re are always made available in global variables, without the need for a require.

• Set lpeg and re as globals; allow shared lib access via require. The lpeg and re modules are loaded into globals of the respective name, but they are not necessarily registered as loaded packages. This ensures that

  • the built-in library versions are preferred when setting the globals,
  • a shared library is used if pandoc has been compiled without lpeg, and
  • the require mechanism can be used to load the shared library if available, falling back to the internal version if possible and necessary.

• Fix argument order in constructor pandoc.Cite (Albert Krewinkel). This restores the old behavior; argument order had been switched accidentally in pandoc 2.15.

• Add Pushable instance for ReaderOptions (Albert Krewinkel).

• Allow to pass custom reader options to pandoc.read as an optional third argument (#7656, Albert Krewinkel). The object can either be a table or a ReaderOptions value like PANDOC_READER_OPTIONS. Creating new ReaderOptions objects is possible through the new constructor pandoc.ReaderOptions.

• Display Pandoc values using their native Haskell representation (Albert Krewinkel).

• Require latest hslua (2.0.1) (#7661, #7657, Albert Krewinkel). This fixes issues with

  • misleading error messages when a required function parameter is omitted;
  • absent properties still being listed in the output of pairs; and
  • alias accessing leading to errors instead of returning nil, e.g. with (pandoc.Str '').identifier.

• Add missing space in “package not found” message (#7658, Albert Krewinkel).

Update build files (#7696, Fabián Heredia Montiel). Drop old windows 32-bit constraints. Update cabal tested-with field to correspond to ci.yml matrix

Remove unneeded package dependencies from benchmark target.

Require ghc >= 8.6, base >= 4.12. This allows us to get rid of the old custom prelude and some crufty cpp. But the primary reason for this is that conduit has bumped its base lower bound to 4.12, making it impossible for us to support lower base versions.

Require Cabal 2.4. Use wildcards to ensure that all pptx tests are included (#7677).

Update bash_completion.tpl (S.P.H.).

Add data/creole.lua as sample custom reader.

Add doc/custom-readers.md and doc/custom-writers.md.

doc/lua-filters.md: add section on global modules, including lpeg (Albert Krewinkel).

MANUAL.txt: update table of exit codes and corresponding errors (Albert Krewinkel).

Use latest texmath.

Powerpoint writer (Emily Bourke):

• Add support for more layouts (#5097). Up til now, four layouts were supported: “Title Slide” (used for the automatically generated metadata slide), “Section Header” (used for headings above slide level), “Two Column” (used when there’s a columns div), “Title and Content” (used for all other slides). We now support three additional layouts: “Comparison”, “Content with Caption”, and “Blank”. The manual describes the logic that determines which layout is used for a slide. Layouts may be customized in the reference doc.
• Support specifying slide background images using a background-image attribute on the slide’s heading. Only the “stretch” mode is supported, and the background image is centred around the slide in the image’s larger axis, matching the observed default behaviour of PowerPoint.
• Add support for incremental lists (through same methods as in other slide writers) (#5689).
• Copy embedded fonts from reference doc.
• Include all themes in output archive.
• Fix list level numbering (#4828, #4663). In PowerPoint, the content of a top-level list is at the same level as the content of a top-level paragraph: the only difference is that a list style has been applied. Previously, the writer incremented the paragrap h level on each list, turning what should be top-level lists into second-level lists.
• Line up list continuation paragraphs. This commit changes the marL and indent values used for plain paragraphs and numbered lists, and changes the spacing defined in the reference doc master for bulleted lists. For paragraphs, there is now a left-indent taken from the otherStyle in the master. For numbered lists, the number is positioned where the text would be if this were a plain paragraph, and the text is indented to the next level. This means that continuation paragraphs line up nicely with numbered lists. Existing reference docs may need to be modified so that otherStyle and bodyStyle indent levels match, for this feature to work with them.
• Consolidate text runs when possible (jgm). This slims down the output files by avoiding unnecessary text run elements.
• Support footers in the reference doc. There is one behaviour which may not be immediately obvious: if the reference doc specifies a fixed date (i.e. not automatically updating), and there’s a date specified in the metadata for the document, the footer date is replaced by the metadata date.
• Fix presentation rel numbering. Before now, the numbering of rIds was inconsistent when making the presentation XML and when making the presentation relationships XML.
• Don’t add relationships unnecessarily. Before now, for any layouts added to the output from the default reference doc, the relationships were unconditionally added to the output. However, if there was already a layout in slideMaster1 at the same index then that results in duplicate relationships.
• If slide level is 0, don’t insert a slide break between a heading and a following table, “columns” div, or paragraph starting with an image.
• Fix capitalisation of notesMasterId.
• Restructure tests.

Asciidoc writer:

• Translate numberLines attribute to linesnum switch (Samuel Tardieu).
• Improve escaping for -- in URLs (#7529).

LaTeX writer:

• Make babel use more idiomatic (#7604, hseg). Use babel’s bidi implementation. Import babel languages individually instead of as package options. Move header-includes to after babel setup so it can be modified.
• Use babel, not polyglossia, with xelatex. Previously polyglossia worked better with xelatex, but that is no longer the case, so we simplify the code so that babel is used with all latex engines. This involves a change to the default LaTeX template.

Markdown writer:

• Avoid bad wraps at the Doc Text level. Previously we tried to do this at the Inline list level, but it makes more sense to intervene on breaking spaces at the Doc Text level.
• Use underline class rather than ul for underline. This only affects output with bracketed_spans enabled. The markdown reader parses spans with either .ul or .underline as Underline elements, but we’re moving towards preferring the latter.

RST writer:

HTML writer:

• Render \ref and \eqref as inline math, not display (see #7589).
• Pass through \ref and \eqref if MathJax is used (#7587).
• Pass through inline math environments with KaTeX.
• Support --reference-location for HTML output (#7461, Francesco Mazzoli).
• Set “hash” to True by default (for reveal.js) (#7574). See #6968 where the motivation for setting “hash” to True is explained.

Native writer: Use pretty-show to format native output (#7580). Previously we used our own homespun formatting. But this produces over-long lines that aren’t ideal for diffs in tests. Performance is slower by about a factor of 10, but this isn’t really a problem because native isn’t suitable as a serialization format. (For serialization you should use json, because the reader is so much faster than native.)

Org writer:

• Don’t indent contents of code blocks. We previously indented them by two spaces, following a common convention. Since the convention is fading, and the indentation is inconvenient for copy/paste, we are discontinuing this practice.
• Update list of supported source languages in org writer (#5440).

Ipynb writer (Kolen Cheung):

• Improve round trip identity for raw cell output. See jupyter/nbformat#229. The Jupyter ecosystem, including nbconvert, lab and notebook, deviated from their own spec in nbformat, where they used the key raw_mimetype instead of format. Moreover, the mime-type of rst used in Jupyter deviated from that suggested by https://docutils.sourceforge.io/FAQ.html and is defined as text/restructuredtext when chosen from “Raw NBConvert Format” in Jupyter. The new behavior should matche the real world usage better, hence improving the round-trip “identity” in raw-cell.
• Add more formats that can be handled as “raw” cells.

EPUB writer:

• Add EPUB3 subject metadata (authority/term) (nuew). This adds the ability to specify EPUB 3 authority and term specific refinements to the subject tag. Specifying a plain subject tag in metadata will function as before.
• Treat epub:type “frontispiece” as front matter (#7600).

reveal.js template: Fix line numbers in source code (#7634). We need overflow: visible for these to work, and reveal’s default css disables this. So we re-enable it in the default template.

Text.Pandoc.Writers.Shared:

• Export splitSentences as a Doc Text transform [API change]. Use this in man/ms. We used to attempt automatic sentence splitting in man and ms output, since sentence-ending periods need to be followed by two spaces or a newline in these formats. But it’s difficult to do this reliably at the level of [Inline].

Text.Pandoc.Translations: small revisions for compatibility with aeson 2.

Don’t prepend file:// to --syntax-definition on Windows (#6374). This was a fix for a problem in skylighting, but this problem doesn’t exist now that we’ve moved from HXT to xml-conduit.

Text.Pandoc.Extensions:

• Add Ext_footnotes to default gfm etxensions. Now gfm supports footnotes.
• Alphabetize Extension constructors (also affects --list-extensions).

Text.Pandoc.Citeproc.Util: Better implementation of splitStrWhen. Previously the citeproc code had two less efficient implementations.

Update documentation for definition_list extension (#7594). In 2015, we relaxed indentation requirements for the first line of a definition (see commit d3544dc and issue #2087), but the documnentation wasn’t updated to reflect the change.

Text.Pandoc.Citeproc.BibTeX: Fix expansion of special strings in series e.g. newseries or library (#7591). Expansion should not happen when these strings are protected in braces, or when they’re capitalized.

Text.Pandoc.Logging: add NotUTF8Encoded constructor to LogMessage [API change].

Text.Pandoc.App.FormatHeuristics: remove .tei.xml extension for TEI (#7630). This never worked, because takeExtension only returns .xml. So it won’t be missed if we remove it.

Text.Pandoc.Image:

• Generalize svgToPng to MonadIO.
• svgToPng, change first parameter from WriterOptions to Int.

Text.Pandoc.Class:

• Add readStdinStrict method to PandocMonad [API change].
• Generalize type of extractMedia [API change]. It was uselessly restricted to PandocIO, instead of any instance of PandocMonad and MonadIO.

Text.Pandoc.Class.PandocIO: derive MonadCatch, MonadThrow, MonadMask. This allows us to use withTempDir [API change].

Add module Text.Pandoc.Class.Sandbox, defining sandbox. Exported via Text.Pandoc.Class. [API change]

Text.Pandoc.Filter: Generalize type of applyFilters from PandocIO to any instance of MonadIO and PandocMonad [API change].

Text.Pandoc.PDF: generalize type of makePDF: instead of PandocIO, it can be used in any instance of PandocMonad, MonadIO, and MonadMask [API change].

Lua subsystem and custom writers: generalize types from PandocIO to any instance of PandocMonad and MonadIO [API change]. The type of runLua is now

(PandocMonad m, MonadIO m) =>
LuaE PandocError a -> m (Either PandocError a)

The change from Lua to LuaE PandocError is due to the switch to hslua-2.0; see next item.

Lua modules (Albert Krewinkel):

• Switch to hslua-2.0. The new HsLua version takes a somewhat different approach to marshalling and unmarshalling, relying less on typeclasses and more on specialized types. This allows for better performance and improved error messages. Furthermore, new abstractions allow to document the code and exposed functions.
• Marshal Version values, Inline elements, Attr elements, and Pandoc elements as userdata.
• Remove deprecated inline constructors DoubleQuoted, SingleQuoted, DisplayMath, and InlineMath.
• Attr values are no longer normalized when assigned to an Inline element property.
• It’s no longer possible to access parts of Inline elements via numerical indexes. E.g., pandoc.Span('test')[2] used to give pandoc.Str 'test', but yields nil now. This was undocumented behavior not intended to be used in user scripts. Use named properties instead.
• Accessing .c to get a JSON-like tuple of all components no longer works. This was undocumented behavior.
• Only known properties can be set on an element value. Trying to set a different property will now raise an error.
• Adds a new pandoc.AttributeList() constructor, which creates the associative attribute list that is used as the third component of Attr values. Values of this type can often be passed to constructors instead of Attr values.
• Convert IOErrors to PandocErrors in pandoc.pipe function (#7523).

Text.Pandoc.PDF: Previously we had to run runIOorExplode inside withTempDir. Now that PandocIO is an instance of MonadMask, this is no longer necessary.

Text.Pandoc.App:

• Reorganize to make it easier to limit IO in main loop. Previously we used liftIO fairly liberally. The code has been restructured to avoid this.
• Move output-file writing out of PandocMonad action.

Text.Pandoc.App.OutputSettings: Generalize some types so we can run this with any instance of PandocMonad and MonadIO, not just PandocIO.

Use simpleFigure builder in readers and SimpleFigure pattern synonym in writers (Aner Lucero).

Allow time 1.12.

Use skylighting-0.12, skylighting-core-0.12. This fixes highlighting issues with typescript, scala, and other syntaxes that include keyword lists from different syntaxes.

Use citeproc 0.6, commonmark 0.2.2.1, commonmark-extensions 0.2.2, texmath 0.12.3.2, ipynb 0.1.0.2. (These changes also allow building with aeson >= 2.)

Require doclayout >= 0.3.1.1. This fixes recognition of “real widths” of emoji characters, which is important for tabular layout.

Cut out over 100K of fat in epub test golden files.

Make test/epub/wasteland.epub valid.

Add missing % on some command tests. This prevented --accept from working properly.

Command tests: raise error if command doesn’t begin with %.

OOXML tests: use pretty-printed form to display diffs. Otherwise everything is on one line and the diff is uninformative.

Fix compareXML helper in Tests.Writers.OOXML. Given how it is used, we were getting “mine” and “good” flipped in the test results.

MANUAL.txt:

• Clarify attributes extension support (William Lupton).
• Document formats affected by --reference-location.
• Document error code 25
• Add some more info regarding --slide-level=0 (Salim B).
• Add more to security section of manual.
• Mention support of title-toc (#7171, Christophe Dervieux).

doc/lua-filters.md:

• Add missing type for Image title (Quinn).
• Improve order of Image fields (Quinn).
• Rephrase pandoc.path docs (#7548, Quinn).
• Do not leak working directory in TikZ filter (Jeroen de Haas).

Allow --slide-level=0 (#7476). When the slide level is set to 0, headings won’t be used at all in splitting the document into slides. Horizontal rules must be used to separate slides.

Add RTF reader (#3982). rtf is now supported as an input format as well as an output format. New module Text.Pandoc.Readers.RTF (exporting readRTF). [API change]

HTML reader: treat comments as blank when parsing (#7482).

Markdown reader:

• Fix raw LaTeX injection issue (#7497). Using a code block containing \end{verbatim}, one could inject raw TeX into a LaTeX document even when raw_tex is disabled. Thanks to Augustin Laville for noticing the bug.
• Multimarkdown sub- and superscripts (#5512, OCzarnecki). Added an extension short_subsuperscripts which modifies the behavior of subscript and superscript, allowing subscripts or superscripts containing only alphanumerics to end with a space character (eg. x^2 = 4 or H~2 is combustible). This improves support for multimarkdown.

RST reader: Fix :literal: includes (#7513). These should create code blocks, not insert raw RST.

LaTeX reader:

• Proper implicit grouping around environment macros.
• Support \global before \def, \let, etc. (#7494).
• Fix scope for LaTeX macros (#7494). They should by default scope over the group in which they are defined (except \gdef and \xdef, which are global). In addition, environments must be treated as groups.
• Improve handling of plain TeX macro primitives (#7474). Fixed semantics for \let.
• Implement \edef, \gdef, and \xdef.

Docx reader: Improve docx reader’s robustness in extracting images (#7511). The docx reader made some assumptions about how docx containers were laid out that were not always true, with the result that some images in documents did not get extracted.

LaTeX writer: Increase table column width precision (#7466, Peter Fabinski). In some cases, the rounding performed by the LaTeX table writer would introduce visible overrun outside the text area. This adds two more decimal places to the width values.

Powerpoint writer:

• Include image title in description (#7352, Emily Bourke). The image title (i.e. ![alt text](link "title")) was previously ignored when writing to pptx. This commit includes it in PowerPoint’s description of the image, along with the link.
• Select layouts from reference doc by name (Emily Bourke). Until now, users had to make sure that their reference doc contains layouts in a specific order: the first four layouts in the file had to have a specific structure. Now the layout selection uses the layout names rather than order: users must make sure their reference doc contains four layouts with specific names, and if a layout with the right name isn’t found pandoc will emit a warning and use the corresponding layout from the default reference doc as a fallback.

Docx writer: be sensitive to the native_numbering extension (#7499). Figure and table numbers are now only included if native_numbering is enabled. (By default it is disabled.) This is a behavior change with respect to 2.14.1, but the default behavior is now that of previous versions. The change was necessary to avoid incompatibilities between pandoc’s native numbering and third-party cross reference filters like pandoc-crossref.

RTF writer:

• Omit \bin in \pict. According to the spec, this is not needed or wanted when the data is in hexadecimal format, as here.
• Emit `\outlinelevel`` for section headings.

RTF template: specify font family for fixed-width font f1. According to the spec, this is mandatory.

LaTeX writer: Use ulem for underline (#7351). ulem is conditionally included already when the strikeout variable is set, so we set this when there is underlined text, and use \uline instead of \underline. This fixes wrapping for underlined text.

Text.Pandoc.Citeproc:

• Revise citeproc code to fit new citeproc 0.5 API (thanks to Benjamin Bray). Linkification of URLs in the bibliography is now done in the citeproc library, depending on the setting of an option. We set that option depending on the value of the metadata field link-bibliography (defaulting to true, for consistency with earlier behavior). If a DOI, PMID, PMCID, or URL field is present but not explicitly rendered, the title (or if no title, the whole entry) is hyperlinked. These changes implement the recommendations from the draft CSL v1.0.2 spec (Appendix VI): https://github.com/citation-style-language/documentation/blob/master/specification.rst#appendix-vi-links
• Avoid odd handling of quotes. Recent citeproc changes allow us to ignore Quoted elements; citeproc now uses its own method for represented quoted things, and only localizes and flipflops quotes it adds itself. Convert Quoted in bib entries to special Spans before passing them off to citeproc. This ensures that we get proper localization and flipflopping if, e.g., quotes are used in titles (jgm/citeproc#87).
• Removed quote localization from citeproc processing. This is now done in citeproc itself.

Text.Pandoc.Logging: Add PowerpointTemplateWarning log message type [API change] (Emily Bourke).

Text.Pandoc.Extension: Add Ext_short_subsuperscripts constructor to Extension [API change] (OCzarnecki).

Various sample.lua editorial fixes (#7493, #7487, William Lupton).

Bump base-compat version so we get compatibility with base 4.12.

Use Prelude from base-compat for ghc 8.4 too.

Add haskell-language-server to shell.nix (#7496, Emily Bourke).

Tests.Helpers: export testGolden and use it in RTF reader. This gives a diff output on failure.

Remove obsolete and incorrect sentence in --slide-level docs.

Add internal module Text.Pandoc.Network.HTTP, exporting urlEncode.

Text.Pandoc.Parsing: parseFromString: preserve at least the source directory (#7464). Previously we just set the source name to “chunk” when parsing from strings, to avoid misleading source positions. This had the side effect that rebase_relative_paths would break inside sections that were parsed as strings. So, now we use “ORIGINAL_SOURCE_PATH_chunk” instead of just “chunk”.

Text.Pandoc.MIME: use image/x-xcf instead of application/x-xcf (#7454).

Don’t compare cdLine in OOXML golden tests (Emily Bourke). The cdLine field gives the line of the file some CData was found on, which reflects irrelevant formatting differences.

Provide more detailed XML diff in tests (Emily Bourke).

OOXML tests: silence warnings. These can make the test output confusing, making people think tests are failing when they’re passing.

INSTALL.md: Add GitLab CI/CD example (#7448, Veratyr).

MANUAL.txt

• Clarifications (William Lupton).
• Add a note on security risks of include directives.

Document use of the ‘underline’ class (#7492, #7484, William Lupton).

Add a FAQ about the “Cannot allocate memory” error on M1 macs.

Use texmath 0.12.3.1.

Use released citeproc 0.5.

Remove dependency on HTTP package (#7456, mt_caret).

Text.Pandoc.ImageSize: Add Tiff constructor for ImageType (#7405) [Minor API change]. This allows pandoc to get size information from tiff images.

Markdown reader: don’t try to read contents in self-closing HTML tag. Previously we had problems parsing raw HTML with self-closing tags like <col/>. The problem was that pandoc would look for a closing tag to close the markdown contents, but the closing tag had, in effect, already been parsed by htmlTag.

LaTeX reader:

• Avoid trailing hyphen in translating languages (#7447). Previously \foreignlanguage{english} turned into <span lang="en-">. The same issue affected Arabic.
• Support \cline in LaTeX tables (#7442).
• Improved parsing of raw LaTeX from Text streams (rawLaTeXParser, used to read LaTeX in Markdown files, #7434). We now use source positions from the token stream to tell us how much of the text stream to consume. Getting this to work required a few other changes to make token source positions accurate.

DocBook reader:

• Handle images with imageobjectco elements (#7440).
• Add support for citerefentry (#7437, Jan Tojnar).

RST reader: fix regression with code includes (#7436). With the recent changes to include infrastructure, included code blocks were getting an extra newline.

HTML reader:

• Recognize data-external when reading HTML img tags (#7429, Michael Hoffmann). Preserve all attributes in img tags. If attributes have a data- prefix, it will be stripped. In particular, this preserves a data-external attribute as an external attribute in the pandoc AST.
• Add col, colgroup to ‘closes’ definitions

HTML writer:

• Remove duplicated alt text in HTML output (Aner Lucero).
• Remove aria-hidden when explicit alt text is provided (Aner Lucero).
• Set boolean values for reveal.js variables.

Docx writer:

• Add table numbering for captioned tables. The numbers are added using fields, so that Word can create a list of tables that will update automatically.
• Support figure numbers. These are set up in such a way that they will work with Word’s automatic table of figures (#7392).

Markdown writer: put space between Plain and following fenced Div (#4465).

EPUB writer: Don’t incorporate externally linked images in EPUB documents (#7430, Michael Hoffmann). Just as it is possible to avoid incorporating an image in EPUB by passing data-external="1" to a raw HTML snippet, this makes the same possible for native Images, by looking for an associated external attribute.

Text.Pandoc.PDF:

• Fix svgIn path error (#7431). We were duplicating the temp directory; this didn’t cause problems on macOS or linux because there we use absolute paths for the temp directory. But on Windows it caused errors converting SVG files.
• convertImage: normalize paths (#7431). This will avoid paths on Windows with mixed path separators.

Text.Pandoc.Class: Always use / when adding directory to image destination with extractMedia, even on Windows.

Text.Pandoc.Citeproc:

• Allow $ characters in bibtex keys (#7409).
• Set proper initial source name in parsing BibTeX (for better error messages.)
• Revamp note citation handling (#7394). Use latest citeproc, which uses a Span with a class rather than a Note for notes. This helps us distinguish between user notes and citation notes. Don’t put citations at the beginning of a note in parentheses. Fix small bug in handling of citations in notes, which led to commas at the end of sentences in some cases.
• Cleanup and efficiency improvement in deNote.
• Improve punctuation moving with --citeproc. Previously, using --citeproc could cause punctuation to move in quotes even when there aer no citations. This has been changed; punctuation moving is now limited to citations. In addition, we only move footnotes around punctuation if the style is a note style, even if notes-after-punctuation is true.

Use citeproc 0.10. This helps improve note citations (see above) and eliminates double hyperlinks in author-in-text citations. Author-only citations are no longer hyperlinked. See jgm/citeproc#77. It also fixes moving of punctuation inside quotes to conform to the CSL spec: only comma and period are moved, not question mark or exclamation point.

Text.Pandoc.Error: fix line calculations in reporting parsec errors. Also remove a spurious initial newline in the error report.

Use doctemplates 0.4.1, which gives us better support for boolean variable values. Previously $if(foo)$ would evaluate to true for variables with boolean false values, because it cared only about the string rendering (#7402).

Require commonmark-pandoc >= 0.2.2.1. This fixes task lists with multiple paragraphs.

Use skylighting 0.11.

CSS in HTML template: reset overflow-wrap on code blocks (Mauro Bieg, #7423).

LaTeX template: Revert change in PR #7295: “move title, author, date up to top of preamble.” The change caused problem for people who used LaTeX commands defined defined later in the preamble in the title or author fields (#7422).

Add doc/faqs.md. This is imported from the website; in the future the website version will be drawn from here. Added a FAQ on the use of \AtEndPreamble for cases when the contents of header-includes need to refer to definitions that come later in the preamble. See #7422.

Upgrade Debian 10 AMI for build-arm.sh.

CircleCI: change to using xcode 11.1.0 (macOS 10.14.4). We previously built on 10.13, but 10.13 no longer gets security updates and CirclCI is deprecating.

Fix MediaBag regressions (#7345). iIn the 2.14 release --extract-media stopped working as before; there could be mismatches between the paths in the rendered document and the extracted media. This patch makes several changes that restore the earlier behavior (while keeping the same API). The mediaPath in 2.14 was always constructed from the SHA1 hash of the media contents. Now, we preserve the original path unless it’s an absolute path or contains .. segments (in that case we use a path based on the SHA1 hash of the contents).

In Text.Pandoc.MediaBag, mediaDirectory and mediaItems now use the mediaPath, rather than the mediabag key, for the first component of the tuple. This makes more sense, I think, and fits with the documentation of these functions; eventually, though, we should rework the API so that mediaItems returns both the keys and the MediaItems.

In Text.Pandoc.Class.IO, rewriting of source paths in extractMedia has been fixed.

In Text.Pandoc.Class.PandocMonad, fillMediaBag has been modified so that it doesn’t modify image paths (that was part of the problem in #7345).

We now do path normalization (e.g. \ separators on Windows) in writing the media.

Text.Pandoc.PDF:

• Text.Pandoc.PDF: Fix regression in 2.14 for generation of PDFs with SVGs (#7344).
• Only print relevant part of environment on --verbose. Since --verbose output might be put in an issue, we want to avoid spilling out secrets in environment variables.

Markdown reader: fix pipe table regression in 2.11.4 (#7343). Previously pipe tables with empty headers (that is, a header line with all empty cells) would be rendered as headerless tables. This broke in 2.11.4. The fix here is to produce an AST with an empty table head when a pipe table has all empty header cells.

LaTeX reader: don’t allow optional * on symbol control sequences (#7340). Generally we allow optional starred variants of LaTeX commands (since many allow them, and if we don’t accept these explicitly, ignoring the star usually gives acceptable results). But we don’t want to do this for \(*\) and similar cases.

Docx reader: handle absolute URIs in Relationship Target (#7374).

Docx writer: fix handling of empty table headers (Albert Krewinkel, #7369). A table header which does not contain any cells is now treated as an empty header.

LaTeX writer: Fix regression in table header position (#7347). In recent versions the table headers were no longer bottom-aligned (if more than one line). This patch fixes that by using minipages for table headers in non-simple tables.

CommonMark writer:

• Do not use simple class for fenced-divs (Jan Tojnar, amends #7242.)
• Do not throw away attributes when Ext_attributes is enabled. Ext_attributes covers at least the following: Ext_fenced_code_attributes, Ext_header_attributes, Ext_inline_code_attributes, Ext_link_attributes.

Markdown writer:

• Allow pipe_tables to be disabled for commonmark formats (commonmark_x, gfm) (#7375).
• Re-use functions from Text.Pandoc.Markdown.Inline (Jan Tojnar).

DocBook writer: Remove non-existent admonitions (Jan Tojnar). attention, error and hint are reStructuredText specific.

HTML writer: Don’t omit width attribute on div (#7342).

Text.Pandoc.MIME, extensionFromMimeType: add a few special cases. When we do a reverse lookup in the MIME table, we just get the last match, so when the same mime type is associated with several different extensions, we sometimes got weird results, e.g. .vs for text/plain. These special cases help us get the most standard extensions for mime types like text/plain.

Lua utils: fix handling of table headers in from_simple_table (Albert Krewinkel, #7369). Passing an empty list of header cells now results in an empty table header.

Text.Pandoc.Citeproc:

• Avoid duplicate classes and attributes on references div.
• Fix regression in citeproc processing (#7376). If inline references are used (in the metadata references field), we should still only include in the bibliography items that are actually cited (unless nocite is used).

Require citeproc 0.4.0.1. This fixes a bug which led to doubled “et al.” in some (rare) circumstances.

MANUAL.txt:

• Mention GladTeX for EPUB export (Sebastian Humenda). This updates the manual and the web site about the GladTeX usage.
• More details and a useful link for YAML syntax.

CONTRIBUTING.md: update modules overview (Albert Krewinkel).

using-the-pandoc-api.md: switch from String to Text (Albert Krewinkel).

Change reader types, allowing better tracking of source positions [API change]. Previously, when multiple file arguments were provided, pandoc simply concatenated them and passed the contents to the readers, which took a Text argument. As a result, the readers had no way of knowing which file was the source of any particular bit of text. This meant that we couldn’t report accurate source positions on errors or include accurate source positions as attributes in the AST. More seriously, it meant that we couldn’t resolve resource paths relative to the files containing them (see e.g. #5501, #6632, #6384, #3752).

Add rebase_relative_paths extension (#3752). When enabled, this extension rewrites relative image and link paths by prepending the (relative) directory of the containing file. This behavior is useful when your input sources are split into multiple files, across several directories, with files referring to images stored in the same directory. The extension can be enabled for all markdown and commonmark-based formats.

Add Text.Pandoc.Sources (exported module), with a Sources type and a ToSources class. A Sources wraps a list of (SourcePos, Text) pairs [API change]. A parsec Stream instance is provided for Sources. The module also exports versions of parsec’s satisfy and other Char parsers that track source positions accurately from a Sources stream (or any instance of the new UpdateSourcePos class).

Text.Pandoc.Parsing

• Export the modified Char parsers defined in Text.Pandoc.Sources instead of the ones parsec provides. Modified parsers to use a Sources as stream [API change].
• Improve include file functions [API change]. Remove old insertIncludedFileF. Give insertIncludedFile a more general type, allowing it to be used where insertIncludedFileF was.
• Add parameter to the citeKey parser from Text.Pandoc.Parsing, which controls whether the @{..} syntax is allowed [API change].

Text.Pandoc.Error: Modified the constructor PandocParsecError to take a Sources rather than a Text as first argument, so parse error locations can be accurately reported.

Fix source position reporting for YAML bibliographies (#7273).

Issue error message when reader or writer format is malformed (#7231). Previously we exited with an error status but (due to a bug) no message.

Smarter smart quotes (#7216, #2103). Treat a leading " with no closing " as a left curly quote. This supports the practice, in fiction, of continuing paragraphs quoting the same speaker without an end quote. It also helps with quotes that break over lines in line blocks.

Markdown reader:

• Use MetaInlines not MetaBlocks for multimarkdown metadata fields. This gives better results in converting to e.g. pandoc markdown.
• Implement curly-brace syntax for Markdown citation keys (#6026). The change provides a way to use citation keys that contain special characters not usable with the standard citation key syntax. Example: @{foo_bar{x}'} for the key foo_bar{x}. It also allows separating citation keys from immediately following text, e.g. @{foo}A.

RST reader:

• Seek include files in the directory of the file containing the include directive, as RST requires (#6632).
• Use insertIncludedFile from Text.Pandoc.Parsing instead of reproducing much of its code.

Org reader: Resolve org includes relative to the directory containing the file containing the INCLUDE directive (#5501).

ODT reader: Treat tabs as spaces (#7185, niszet).

Docx reader:

• Add handling of vml image objects (#7257, mbrackeantidot).
• Support new table features (Emily Bourke, #6316): column spans, row spans, multiple header rows, table description (parsed as a simple caption), captions, column widths.

LaTeX reader:

• Improved siunitx support (#6658, #6620).
• Better support for \xspace (#7299).
• Improve parsing of \def macros. We previously set “verbatim mode” even for parsing the initial \def; this caused problems for \def nested inside another \def.
• Implement \newif.

ConTeXt writer: improve ordered lists (#5016, Denis Maier). Change ordered list from itemize to enumerate. Add new itemgroup for ordered lists. Remove manual insertion of width attributes. Use tabular figures in ordered list enumerators.

HTML reader:

• Don’t fail on unmatched closing “script” tag (Albert Krenkel, #7282).
• Keep h1 tags as normal headers (#2293, Albert Krewinkel). The tags <title> and <h1 class="title"> often contain the same information, so the latter was dropped from the document. However, as this can lead to loss of information, the heading is now always retained. Use --shift-heading-level-by=-1 to turn the <h1> into the document title, or a filter to restore the previous behavior.
• Handle relative lengths (e.g. 2*) in HTML column widths (#4063). See https://www.w3.org/TR/html4/types.html#h-6.6.

DocBook/JATS readers:

• Fix mathml regression caused by the switch in XML libraries (#7173).
• Fix “phrase” in DocBook: take classes from “role” not “class” (#7195).

DocBook reader: ensure that first and last names are separated (#6541).

Jira reader (Albert Krewinkel, #7218):

• Support “smart” links: [alias|https://example.com|smart-card] syntax.
• Allow spaces and most unicode characters in attachment links.
• No longer require a newline character after {noformat}.
• Only allow URI path segment characters in bare links.
• The file: schema is no longer allowed in bare links; these rarely make sense.

Plain writer: handle superscript unicode minus (#7276).

LaTeX writer:

• Better handling of line breaks in simple tables (#7272). Now we also handle the case where they’re embedded in other elements, e.g. spans.
• For beamer output, support exampleblock and alertblock (#7278). A block will be rendered as an exampleblock if the heading has class example and an alertblock if it has class alert.
• Separate successive quote chars with thin space (#6958, Albert Krewinkel). Successive quote characters are separated with a thin space to improve readability and to prevent unwanted ligatures. Detection of these quotes sometimes had failed if the second quote was nested in a span element.
• Separate successive quote chars with thin space (#6958, Albert Krewinkel).

EPUB Writer: Fix belongs-to-collection XML id choice (#7267, nuew). The epub writer previously used the same XML id for both the book identifier and the epub collection. This causes an error on epubcheck.

BibTeX/BibLaTeX writer: Handle annote field (#7266).

ZimWiki writer: allow links and emphasis in headers (#6605, Albert Krewinkel).

ConTeXt writer:

• Support blank lines in line blocks (#6564, Albert Krewinkel, thanks to @denismaier).
• Use span identifiers as reference anchors (#7246, Albert Krewinkel).

HTML writer:

• Keep attributes from code nested below pre tag (#7221, Albert Krewinkel). If a code block is defined with <pre><code class="language-x">…</code></pre>, where the <pre> element has no attributes, then the attributes from the <code> element are used instead. Any leading language- prefix is dropped in the code’s class attribute are dropped to improve syntax highlighting.
• Ensure headings only have valid attribs in HTML4 (#5944, Albert Krewinkel).
• Parse <header> as a Div (Albert Krewinkel).

Org writer:

• Inline latex envs need newlines (#7252, tecosaur). As specified in https://orgmode.org/manual/LaTeX-fragments.html, an inline \begin{}…\end{} LaTeX block must start on a new line.
• Use LaTeX style maths deliminators (#7196, tecosaur).

JATS writer (Albert Krewinkel):

• Use either styled-content or named-content for spans (#7211). If the element has a content-type attribute, or at least one class, then that value is used as content-type and the span is put inside a <named-content> element. Otherwise a <styled-content> element is used instead.
• Reduce unnecessary use of <p> elements for wrapping (#7227). The <p> element is used for wrapping in cases were the contents would otherwise not be allowed in a certain context. Unnecessary wrapping is avoided, especially around quotes (<disp-quote> elements).
• Convert spans to <named-content> elements (#7211). Spans with attributes are converted to <named-content> elements instead of being wrapped with <milestone-start/> and <milestone-end> elements. Milestone elements are not allowed in documents using the articleauthoring tag set, so this change ensures the creation of valid documents.
• Add footnote number as label in backmatter (#7210). Footnotes in the backmatter are given the footnote’s number as a label. The articleauthoring output is unaffected from this change, as footnotes are placed inline there.
• Escape disallows chars in identifiers. XML identifiers must start with an underscore or letter, and can contain only a limited set of punctuation characters. Any IDs not adhering to these rules are rewritten by writing the offending characters as Uxxxx, where xxxx is the character’s hex code.

Jira writer: use {color} when span has a color attribute (Albert Krewinkel, tarleb/jira-wiki-markup#10).

Docx writer:

• Autoset table width if no column has an explicit width (Albert Krewinkel).
• Extract Table handling into separate module (Albert Krewinkel).
• Support colspans and rowspans in tables (Albert Krewinkel, #6315).
• Support multirow table headers (Albert Krewinkel).
• Improve integration of settings from reference.docx (#1209). This change allows users to create a reference.docx that sets w:proofState for spelling or grammar to dirty, so that spell/grammar checking will be triggered on the generated docx.
• Copy over more settings from reference.docx (#7240). From settings.xml in the reference-doc, we now include: zoom, embedSystemFonts, doNotTrackMoves, defaultTabStop, drawingGridHorizontalSpacing, drawingGridVerticalSpacing, displayHorizontalDrawingGridEvery, displayVerticalDrawingGridEvery, characterSpacingControl, savePreviewPicture, mathPr, themeFontLang, decimalSymbol, listSeparator, autoHyphenation, compat.
• Set zoom to 100% by default in settings.xml.
• Align math options more with current Word defaults (e.g. Cambria Math font).
• Remove rsids from default settings.xml. Word will add these when revisions are made.

Ms writer: Handle tables with multiple paragraphs (#7288). Previously they overflowed the table cell width. We now set line lengths per-cell and restore them after the table has been written.

Markdown writer:

• Use cleaner braceless syntax for code blocks with a single class (#7242, Jan Tojnar).
• Add quotes properly in markdown YAML metadata fields (#7245). This fixes a bug, which caused the writer to look at the last rather than the first character in determining whether quotes were needed. So we got spurious quotes in some cases and didn’t get necessary quotes in others.
• Use @{..} syntax for citations when needed.
• Use fewer unneeded escapes for # (see #6259).
• Improve escaping of @. We need to escape literal @ before { because of the new citation syntax.

Commonmark writer: Use backslash escapes for < and |… instead of entities (#7208).

Powerpoint writer: allow monofont to be specified in metadata (#7187).

LaTeX template:

• Use non-starred names for xcolor color names (#6109). This should make svgnames and x11names work properly.
• Fix bad vertical spacing after bibliography (#7234, badumont).
• List of figures before list of tables (#7235, Julien Dutant).
• Move CSL macro definitions before header-includes so they can be overridden (#7286).
• Improve treatment of CSL entry-spacing (#7296). Previously with the default template settings (indent variable not set), we would get interparagraph spaces separating bib entries even with entry-spacing="0". On the other hand, setting entry-spacing="2" gave ridiculously large spacing. This change makes the spacing caused by entry-spacing a multiple of \parskip by default, which gives aesthetically reasonable output. Those who want a larger or smaller unit (e.g. because they use indent which sets \parskip to 0) may \setlength{\cslentryspacingunit}{10pt} in header-includes to override the defaults.
• Move title, author, date up to top of preamble (#7295). This allows header-includes to use them, and puts them in a position where you can see them immediately.
• Define commands for zero width non-joiner character (#6639, Albert Krewinkel). The zero-width non-joiner character is used to avoid ligatures (e.g. in German).

ConTeXt template:

• Define enumerate itemgroup (#5016, Denis Maier).
• List of figures before list of tables (#7235, Julien Dutant).

reveal.js template:

• Support toc-title (#7171, Florian Kohrt).
• Use hash: true by default rather than history: true (#6968).

HTML-based slide shows: add support for institute (#7289, Thomas Hodgson).

Text.Pandoc.Extensions: Add constructor Ext_rebase_relative_paths to Extensions [API change].

Text.Pandoc.XML.Light: add Eq, Ord instances for Content, Element, Attr, CDataKind [API change].

Text.Pandoc.MediaBag:

• Change type to use a Text key instead of [FilePath]. We normalize the path and use / separators for consistency.
• Export MediaItem type [API change].
• Change MediaBag type to a map from Text to MediaItem [API change].
• lookupMedia now returns a MediaItem [API change].
• Change insertMedia so it sets the mediaPath to a filename based on the SHA1 hash of the contents. This will be used when contents are extracted.

Text.Pandoc.Class.PandocMonad:

• Remove fetchMediaResource [API change]. Use fetchItem to get resources in fillMediaBag.
• Add informational message in downloadOrRead indicating what path local resources have been loaded from.

Text.Pandoc.Logging:

• Remove single quotes around paths in messages.
• Add LoadedResource constructor to LogMessage [API change]. This is for INFO-level messages telling where image data has been loaded from. (This can vary because of the resource path.)

Text.Pandoc.Asciify: simplify code and export toAsciiText [API change]. Instead of encoding a giant (and incomplete) map, we now just use unicode-transforms to normalize the text to a canonical decomposition, and manipulate the result.

App: allow tabs expansion even if file-scope is used (Albert Krewinkel, #6709). Tabs in plain-text inputs are now handled correctly, even if the --file-scope flag is used.

Add new internal module Text.Pandoc.Writers.GridTable (Albert Krewinkel).

Text.Pandoc.Highlighting: Change type of languagesByExtension, adding a parameter for a SyntaxMap [API change] (Jan Tojnar, #7241). Languages defined using --syntax-definition were not recognized by languagesByExtension. This patch corrects that, allowing the writers to see all custom definitions. The LaTeX writer still uses the default syntax map, but that’s okay in that context, since --syntax-definition won’t create new listings styles.

Text.Pandoc.Citeproc:

• Ensure that CSL-related attributes are passed on to a Div with id ‘refs’. Otherwise things like entry-spacing won’t work when such Divs are used.
• Use metadata’s lang for the lang parameter of citeproc, overriding localeLanguage.
• Recognize locators spelled with a capital letter (#7323).
• Add a comma and a space in front of the suffix if it doesn’t start with space or punctuation (#7324).
• Don’t detect math elements as locators (#7321).

Remove Text.Pandoc.BCP47 module [API change]. Use types and functions from UnicodeCollation.Lang instead. This is a richer implementation of BCP 47.

Text.Pandoc.Shared:

• Fix regression in grid tables for wide characters (#7214). In the translation from String to Text, a char-width-sensitive splitAt' was dropped. This commit reinstates it and uses it to make splitTextByInstances char-width sensitive.
• Add getLang (formerly in the now-removed BCP47) [API change].

Text.Pandoc.SelfContained: use application/octet-stream for unknown mime types instead of halting with an error (#7202).

Lua filters: respect Inlines/Blocks filter functions in pandoc.walk_* (Albert Krewinkel).

Add text as build-depend for trypandoc (#7193, Roman Beránek).

Bump upper-bounds for network-uri, time, attoparsec.

Use citeproc 0.4.

Use texmath 0.12.3.

Use jira-wiki-markup 1.3.5 (Albert Krewinkel).

Require latest skylighting (fixes a bug in XML syntax highlighting).

Use latest xml-conduit.

Use latest commonmark, commonmark-extensions, commonmark-pandoc.

Use haddock-library-1.10.0 (Albert Krewinkel).

Allow compilation with base 4.15 (Albert Krewinkel).

MANUAL:

• Add information about lang and bibliography sorting.
• Add info about YAML escape sequences, link to spec (#7152, Albert Krewinkel).
• Note that institute variable works for HTML-based slides.
• Update documentation on citation syntax.
• Add citation example for locators and suffixes (Tristan Stenner)

Updated and fixed typos in documentation (Charanjit Singh, Anti-Distinctlyminty, Tatiana Porras, obcat).

Add instructions for installing pandoc-types before compiling filter.

INSTALL: add note that parallel installations should be avoided (#6865).

Remove biblatex-nussbaum.md test. It is basically the same as biblaetx-quotes.md.

Command tests: fail if a file contains no tests—and fix a test that failed in that way!

Use smaller images in tests, reducing the size of the source tarball by 8 MB.

Support yaml_metadata_block extension for commonmark, gfm (#6537). This supported is a bit more limited than with pandoc’s markdown. The YAML block must be the first thing in the input, and the leaf notes are parsed in isolation from the rest of the document. So, for example, you can’t use reference links if the references are defined later in the document.

Fix fallback to default partials when custom templates are used. If the directory containing a template does not contain the partial, it should be sought in the default templates, but this was not working properly (#7164).

Handle nocite better with --biblatex and --natbib (#4585). Previously the nocite metadata field was ignored with these formats. Now it populates a nocite-ids template variable and causes a \nocite command to be issued.

Text.Pandoc.Citeproc: apply fixLinks correctly (#7130). This is code that incorporates a prefix like https://doi.org/ into a following link when appropriate.

Text.Pandoc.Shared:

• Remove backslashEscapes, escapeStringUsing [API change]. Replace these inefficient association list lookups with more efficient escaping functions in the writers that used them (for a 10-25% performance boost in org, haddock, rtf, texinfo writers).
• Remove ToString, ToText typeclasses [API change]. These were needed for the transition from String to Text, but they are no longer used and may clash with other things.
• Simplify compactDL.

Text.Pandoc.Parsing:

• Change type of readWithM so that it is no longer polymorphic [API change]. The ToText class has been removed, and now that we’ve completed the transition to Text we no longer need this to operate on Strings.
• Remove F type synonym [API change]. Muse and Org were defining their own F anyway.

Text.Pandoc.Readers.Metadata:

• Export yamlMetaBlock [API change].
• Make yamlBsToMeta, yamlBsToRefs polymorphic on the parser state [API change].

Markdown reader: Fix regression with tex_math_backslash (#7155).

MediaWiki reader: Allow block-level content in notes (ref) (#7145).

Jira reader (Albert Krewinkel):

• Fixed parsing of autolinks (i.e., of bare URLs in the text). Previously an autolink would take up the rest of a line, as spaces were allowed characters in these items.
• Emoji character sequences no longer cause parsing failures. This was due to missing backtracking when emoji parsing fails.
• Mark divs created from panels with class “panel”.

RST reader: fix logic for ending comments (#7134). Previously comments sometimes got extended too far.

DocBook writer: include Header attributes as XML attributes on section (Erik Rask). Attributes with key names that are not allowed as XML attributes are dropped, as are attributes with invalid values and xml:id (DocBook 5) and id (DocBook 4).

Docx writer:

• Make nsid in abstractNum deterministic. Previously we assigned a random number, but we don’t need random values, so now we just assign a value based on the list marker.
• Use integral values for w:tblW (#7141).

Jira writer (Albert Krewinkel):

• Block quotes are only rendered as bq. if they do not contain a linebreak.
• Jira writer: improve div/panel handling. Include div attributes in panels, always render divs with class panel as panels, and avoid nesting of panels.

HTML writer: Add warnings on duplicate attribute values. This prevents emitting invalid HTML. Ultimately it would be good to prevent this in the types themselves, but this is better for now.

Org writer: Prevent unintended creation of ordered list items (#7132, Albert Krewinkel). Adjust line wrapping if default wrapping would cause a line to be read as an ordered list item.

JATS templates: support ‘equal-contrib’ attrib for authors (Albert Krewinkel). Authors who contributed equally to a paper may be marked with equal-contrib.

reveal.js template: replace JS comment with HTML (#7154, Florian Kohrt).

Text.Pandoc.Logging: Add DuplicateAttribute constructor to LogMessage. [API change]

Use -j4 for linux release build. This speeds up the build dramatically on arm.

cabal.project: remove ghcoptions. Move flags to top level, so they can be set differently on the command line.

Require latest texmath, skylighting, citeproc, jira-wiki-markup. (The latest skylighting fixes a bad bug with Haskell syntax highlighting.) Narrow version bounds for texmath, skylighting, and citeproc, since the test output depend on them.

Use doclayout 0.3.0.2. This significantly reduces the time and memory needed to compile pandoc.

Use foldl' instead of foldl everywhere.

Update bounds for random (#7156, Alexey Kuleshevich).

Remove uses of some partial functions.

Don’t bake in a larger stack size for the executable.

Test improvements:

• Use getExecutablePath from base, avoiding the dependency on executable-path.
• Factor out setupEnvironment in Helpers, to avoid code duplication.
• Fix finding of data files by setting teh pandoc_datadir environment variable when we shell out to pandoc. This avoids the need to use --data-dir for the tests, which caused problems finding pandoc.lua when compiling without the embed_data_files flag (#7163).

Benchmark improvements:

• Build +RTS -A8m -RTS into default ghc-options for benchmark. This is necessary to get accurate benchmark results; otherwise we are largely measuring garbage collecting, some not related to the current benchmark.
• Allow specifying BASELINE file in ‘make bench’ for comparison (otherwise the latest benchmark is chosen by default).
• Force readFile in benchmarks early (Bodigrim).

CONTRIBUTING: suggest using a cabal.project.local file (#7153, Albert Krewinkel).

Add ghcid-test to Makefile. This loads the test suite in ghcid.

--resource-path now accumulates if specified multiple times (#6152). Resource paths specified later on the command line are prepended to those specified earlier. Thus, --resource-path foo --resource-path bar:baz is equivalent to --resource-path bar:bas:foo. (The previous behavior was for the last --resource-path to replace all the rest.) resource-path in defaults files behaves the same way: it will be prepended to the resource path set by earlier command line options or defaults files. This change facilitates the use of multiple defaults files: each can specify a directory containing resources it refers to without clobbering the resource paths set by the others.

Allow defaults files to refer to the home directory, the user data directory, and the directory containing the defaults file itself (#5871, #5982, #5977). In fields that expect file paths (and only in these fields),

• ${VARIABLE} will expand to the value of the environment variable VARIABLE (and in particular ${HOME} will expand to the path of the home directory). A warning will be raised for undefined variables.
• ${USERDATA} will expand to the path of the user data directory in force when the defaults file is being processed.
• ${.} will expand to the directory containing the defaults file. (This allows default files to be placed in a directory containing resources they make use of.)

When downloading content from URL arguments, be sensitive to the character encoding (#5600). We can properly handle UTF-8 and latin1 (ISO-8859-1); for others we raise an error. Fall back to latin1 if no charset is given in the mime type and UTF-8 decoding fails.

Allow abbreviations that don’t end in a period to be specified using --abbreviations (#7124).

Add new unexported module Text.Pandoc.XML.Light, as well as Text.Pandoc.XML.Light.Types, Text.Pantoc.XML.Light.Proc, Text.Pandoc.XML.Light.Output. (Closes #6001, #6565, #7091).

This module exports definitions of Element and Content that are isomorphic to xml-light’s, but with Text instead of String. This allows us to keep most of the code in existing readers that use xml-light, but avoid lots of unnecessary allocation.

We also add versions of the functions from xml-light’s Text.XML.Light.Output and Text.XML.Light.Proc that operate on our modified XML types, and functions that convert xml-light types to our types (since some of our dependencies, like texmath, use xml-light).

We export functions that use xml-conduit’s parser to produce an Element or [Content]. This allows existing pandoc code to use a better parser without much modification.

The new parser is used in all places where xml-light’s parser was previously used. Benchmarks show a significant performance improvement in parsing XML-based formats (with docbook, opml, jats, and docx almost twice as fast, odt and fb2 more than twice as fast).

In addition, the new parser gives us better error reporting than xml-light. We report XML errors, when possible, using the new PandocXMLError constructor in PandocError.

These changes revealed the need for some changes in the tests. The docbook-reader.docbook test lacked definitions for the entities it used; these have been added. And the docx golden tests have been updated, because the new parser does not preserve the order of attributes.

DocBook reader:

• Avoid expensive tree normalization step, as it is not necessary with the new XML parser.
• Support informalfigure (#7079) (Nils Carlson).

Docx reader:

• Use Map instead of list for Namespaces. This gives a speedup of about 5-10%. With this and the XML parsing changes, the docx reader is now about twice as fast as in the previous release.

HTML reader:

• Small performance tweaks.
• Also, remove exported class NamedTag(..) [API change]. This was just intended to smooth over the transition from String to Text and is no longer needed.
• As a result, the functions isInlineTag and isBlockTag are no longer polymorphic; they apply to a Tag Text [API change].
• Do a lookahead to find the right parser to use. This takes benchmarks from 34ms to 23ms, with less allocation.
• Fix bad handling of empty src attribute in iframe (#7099). If src is empty, we simply skip the iframe. If src is invalid or cannot be fetched, we issue a warning nd skip instead of failing with an error.

JATS reader:

• Avoid tree normalization, which is no longer necessary given the new XML parser.

LaTeX reader:

• Don’t export tokenize, untokenize [API change]. These are internal implementation details, which were only exported for testing. They don’t belong in the public API.
• Improved efficiency of the parser. With these changes the reader is almost twice as fast as in the last release in our benchmarks.
• Code cleanup, removing some unnecessary things.
• Rewrite withRaw so it doesn’t rely on fragile assumptions about token positions (which break when macros are expanded) (#7092). This requires the addition of sEnableWithRaw and sRawTokens in LaTeXState, and a new combinator disablingWithRaw to disable collecting of raw tokens in certain contexts. Add parseFromToks to Text.Pandoc.Readers.LaTeX.Parsing. Fix parsing of single character tokens so it doesn’t mess up the new raw token collecting. These changes slightly increase allocations and have a small performance impact.
• Handle some bibtex/biblatex-specific commands that used to be dealt with in pandoc-citeproc (#7049).
• Optimize satisfyTok, avoiding unnecessary macro expansion steps. Benchmarks after this change show 2/3 of the run time and 2/3 of the allocation of the Feb. 10 benchmarks.
• Removed sExpanded in state. This isn’t actually needed and checking it doesn’t change anything.
• Improve braced'. Remove the parameter, have it parse the opening brace, and make it more efficient.
• Factor out pieces of the LaTeX reader to make the module smaller. This reduces memory demands when compiling. Created Text.Pandoc.Readers.{LaTeX,Math,Citation,Table,Macro,Inline}. Changed Text.Pandoc.Readers.LaTeX.SIunitx to export a command map instead of individual commands.
• Handle table cells containing & in \verb (#7129).

Make Text.Pandoc.Readers.LaTeX.Types an unexported module [API change].

Markdown reader:

• Improved handling of mmd link attributes in references (#7080). Previously they only worked for links that had titles.
• Improved efficiency of the parser (benchmarks show a 15% speedup).

OPML reader:

• Avoid tree normalization, which is no longer necessary with the new XML parser.

ODT reader:

• Finer-grained errors on parse failure (#7091).
• Give more information if the zip container can’t be unpacked.

Org reader:

• Support task_lists extension (Albert Krewinkel, #6336).
• Fix bug in org-ref citation parsing (Albert Krewinkel, #7101). The org-ref syntax allows to list multiple citations separated by comma. Previously commas were accepted as part of the citation id, so all citation lists were parsed as one single citation.

RST reader:

• Use getTimestamp instead of getCurrentTime to fetch timestamp. Setting SOURCE_DATE_EPOCH will allow reproducible builds.
• RST reader: fix handling of header in CSV tables (#7064). The interpretation of this line is not affected by the delim option.

Jira reader:

• Modified the Doc parser to skip leading blank lines. This fixes parsing of documents which start with multiple blank lines (Albert Krewinkel, #7095).
• Prevent URLs within link aliases to be treated as autolinks (Albert Krewinkel, #6944).

Text.Pandoc.Shared

• Remove formerly exported functions that are no longer used in the code base: splitByIndices, splitStringByIndicies, substitute, and underlineSpan (which had been deprecated in April 2020) [API change].
• Export handleTaskListItem (Albert Krewinkel) [API change].
• Change defaultUserDataDirs to defaultUserDataDir [API change]. We determine what is the default user data directory by seeing whether the XDG directory and/or legacy directory exist.

BibTeX writer:

• BibTeX writer: use doclayout and doctemplate. This change allows bibtex/biblatex output to wrap as other formats do, depending on the settings of --wrap and --columns (#7068).

CSL JSON writer:

• Output [] if no references in input, instead of raising a PandocAppError as before.

Docx writer:

• Use getTimestamp instead of getCurrentTime for timestamp. Setting SOURCE_DATE_EPOCH will allow reproducible builds.

EPUB writer:

• Use getTimestamp instead of getCurrentTime for timestamp. Setting SOURCE_DATE_EPOCH will allow reproducible builds (#7093). This does not suffice to fully enable reproducible in EPUB, since a unique id is still being generated for each build.
• Support belongs-to-collection metadata (#7063) (Nick Berendsen).

JATS writer:

• Escape special chars in reference elements (Albert Krewinkel). Prevents the generation of invalid markup if a citation element contains an ampersand or another character with a special meaning in XML.

Jira writer:

• Use Span identifiers as anchors (Albert Krewinkel).
• Use {noformat} instead of {code} for unknown languages (Albert Krewinkel). Code blocks which are not marked as a language supported by Jira are rendered as preformatted text via {noformat} blocks.

LaTeX writer:

• Adjust hypertargets to beginnings of paragraphs (#7078). Use \vadjust pre so that the hypertarget takes you to the beginning of the paragraph rather than one line down. This makes a particular difference for links to citations using --citeproc and link-citations: true.
• Change BCP47 lang tag from jp to ja (Mauro Bieg, #7047).
• Use function instead of map for accent lookup (should be more efficient).
• Split the module to make it easier to compile on low-memory systems: added Text.Pandoc.Writers.LaTeX.{Util,Citation,Lang}.

Markdown writer:

• Handle math right before digit. We insert an HTML comment to avoid a $ right before a digit, which pandoc will not recognize as a math delimiter.
• Split the module to make it easier to compile on low-memory systems: added Text.Pandoc.Writers.Markdown.{Types,Inline}.

ODT writer:

• Use getTimestamp instead of getCurrentTime for timestamp. Setting SOURCE_DATE_EPOCH will allow reproducible builds.
• Update default ODT style (Lorenzo). Previously, the “First paragraph” style inherited from “Standard” but not from “Text body.” Now it is adjusted to inherit from “Text body”, to avoid some ugly spacing issues. It may be necessary to update a custom reference.odt in light of this change.

Org writer:

• Support task_lists extension (Albert Krewinkel, #6336).

Pptx writer:

• Use getTimestamp instead of getCurrentTime for timestamp. Setting SOURCE_DATE_EPOCH will allow reproducible builds.

JATS templates: tag author.name as string-name (Albert Krewinkel). The partitioning the components of a name into surname, given names, etc. is not always possible or not available. Using author.name allows to give the full name as a fallback to be used when author.surname is not available.

Add default templates for bibtex and biblatex, so that the variables header-include, include-before, include-after (or alternatively the command line options --include-in-header, --include-before-body, --include-after-body) may be used.

LaTeX template:

• Update to iftex package (#7073) (Andrew Dunning)
• Wrap url colours in braces (#7121) (Loïc Grobol).

revealjs template: Add ‘center’ option for vertical slide centering. (maurerle, #7104).

Text.Pandoc.XML: Improve efficiency of fromEntities.

Text.Pandoc.MIME

• Add exported function getCharset [API change].

Text.Pandoc.UTF8: change IO functions to return Text, not String [API change]. This affects readFile, getContents, writeFileWith, writeFile, putStrWith, putStr, putStrLnWith, putStrLn. hPutStrWith, hPutStr, hPutStrLnWith, hPutStrLn, hGetContents. This avoids the need to uselessly create a linked list of characters when emiting output.

Text.Pandoc.App

• Add parseOptionsFromArgs [API change, new exported function].
• Add fields for CSL options to Opt [API change]: optCSL, optbibliography, optCitationAbbreviations.

Text.Pandoc.Citeproc.BibTeX

• Text.Pandoc.Citeproc.writeBibTeXString now returns Doc Text instead of Text (#7068).
• Correctly handle pages (= page in CSL) (#7067).
• Correctly handle BibLaTeX langid (= language in CSL, #7067).
• In BibTeX output, protect foreign titles since there’s no language field (#7067).
• Clean up BibTeX parsing (#7049). Previously there was a messy code path that gave strange results in some cases, not passing through raw tex but trying to extract a string content. This was an artefact of trying to handle some special bibtex-specific commands in the BibTeX reader. Now we just handle these in the LaTeX reader and simplify parsing in the BibTeX reader. This does mean that more raw tex will be passed through (and currently this is not sensitive to the raw_tex extension; this should be fixed).

Text.Pandoc.Citeproc.MetaValue

• Correctly parse “raw” date value in markdown references metadata. (See jgm/citeproc#53.)

Text.Pandoc.Citeproc

• Use https URLs for links (Salim B, #7122).

Text.Pandoc.Class

• Add getTimestamp [API change]. This attempts to read the SOURCE_DATE_EPOCH environment variable and parse a UTC time from it (treating it as a unix date stamp, see https://reproducible-builds.org/specs/source-date-epoch/). If the variable is not set or can’t be parsed as a unix date stamp, then the function returns the current date.

Text.Pandoc.Error

• Add PandocUnsupportedCharsetError constructor for PandocError [API change].
• Export renderError [API change].
• Refactor handleError to use renderError. This allows us render error messages without exiting.

Text.Pandoc.Extensions

• Ext_task_lists is now supported by org (and turned on by default) (Albert Krewinkel, #6336).
• Remove Ext_fenced_code_attributes from allowed commonmark attributes (#7097). This attribute was listed as allowed, but it didn’t actually do anything. Use attributes for code attributes and more.

Lua subsystem:

• Always load built-in Lua scripts from default data-dir (Albert Krewinkel). The Lua modules pandoc and pandoc.List are now always loaded from the system’s default data directory. Loading from a different directory by overriding the default path, e.g. via --data-dir, is no longer supported to avoid unexpected behavior and to address security concerns.
• Add module “pandoc.path” (Albert Krewinkel, #6001, #6565). The module allows to work with file paths in a convenient and platform-independent manner.
• Use strict evaluation when retrieving AST value from the stack (Albert Krewinkel, #6674).

Text.Pandoc.PDF

• Disable smart extension when building PDF via LaTeX. This is to prevent accidental creation of ligatures like ?` and !` (especially in languages with quotations like German), and similar ligature issues. (See jgm/citeproc#54.)

Text.Pandoc.CSV:

• Fix parsing of unquoted values (#7112). Previously we didn’t allow unescaped quotes in unquoted values, but they are allowed in CSV.

Test suite:

• Use a more robust method for testing the executable. Many of our tests require running the pandoc executable. This is problematic for a few different reasons. First, cabal-install will sometimes run the test suite after building the library but before building the executable, which means the executable isn’t in place for the tests. One can work around that by first building, then building and running the tests, but that’s fragile. Second, we have to find the executable. So far, we’ve done that using a function findPandoc that attempts to locate it relative to the test executable (which can be located using findExecutablePath). But the logic here is delicate and work with every combination of options. To solve both problems, we add an --emulate option to the test-pandoc executable. When --emulate occurs as the first argument passed to test-pandoc, the program simply emulates the regular pandoc executable, using the rest of the arguments (after --emulate). Thus, test-pandoc --emulate -f markdown -t latex is just like pandoc -f markdown -t latex. Since all the work is done by library functions, implementing this emulation just takes a couple lines of code and should be entirely reliable. With this change, we can test the pandoc executable by running the test program itself (locatable using findExecutablePath) with the --emulate option. This removes the need for the fragile findPandoc step, and it means we can run our integration tests even when we’re just building the library, not the executable. [Note: part of this change involved simplifying some complex handling to set environment variables for dynamic library paths. I have tested a build with --enable-dynamic-executable, and it works, but further testing may be needed.]
• Print accurate location if a test fails (Albert Krewinkel). Ensures that tasty-hunit reports the location of the failing test instead of the location of the helper test function.

Documentation: Update URLs and use https where possible (#7122, Salim B).

Add doc/libraries.md, a description of libraries that support pandoc.

MANUAL.txt

• MANUAL: block-level formatting is not allowed in line blocks (#7107).
• Clarify tex_math_dollars extension. Note that no blank lines are allowed between the delimiters in display math.
• Add MANUAL section on reproducible builds.
• Document no template fallback for absolute path (#7077, Nixon Enraght-Moony.)
• Improve docs for cite-method.
• Update README and man page.

Makefile: in make bench, create CSV files for comparison and compare against previous benchmark run. Add timestamp to CSV filenames.

cabal.project: don’t explicitly set -trypandoc. If we do, this can’t be overridden on the cabal command line.

doc/lua-filters.md: improve documentation for pandoc.mediabag.insert, pandoc.mediabag.fetch, directory, normalize (Albert Krewinkel).

Allow base64-bytestring-1.2.* (Dmitrii Kovanikov)

Require jira-wiki-markup 1.3.3 (Albert Krewinkel)

Require citeproc 0.3.0.8, which correctly titlecases when titles contain non-ASCII characters.

Use skylighting 0.10.4. This version of skylighting uses xml-conduit rather than hxt. This speeds up parsing of XML syntax definitions fourfold, and removes four packages from pandoc’s dependency graph: hxt-charproperties, hxt-unicode, hxt-regex-xmlschema, hxt.

Add script tools/parseTimings.pl to help pin down which modules take the most time and memory to compile.

Avoid unnecessary use of NoImplicitPrelude pragma (#7089) (Albert Krewinkel)

Benchmarks

• Use the lighter-weight tasty-bench instead of criterion.
• Run writer benchmarks for binary formats too.
• Alphabetize benchmarks.
• Don’t run benchmarks for bibliography formats (yet; we need a special input for them).
• Show allocation data
• Clean up benchmark code.
• Allow specifying patterns using `-p blah’.

trypandoc: add 2 second timeout.

Use -split-sections in creating linux release binary. This reduces executable size significantly (by about 30%).

Remove weigh-pandoc. It’s not really useful any more, now that our regular benchmarks include data on allocation.

Improve linux package build process and add script to automate building an arm64 binary package.

With --bibliography (or bibliography in metadata), a URL may now be provided, and pandoc will fetch the resource. In addition, if a file path is provided and it is not found relative to the working directory, the resource path will be searched (#6940).

Add sourcepos extension for commonmark, gfm, commonmark_x (#4565). With the sourcepos extension set set, data-pos attributes are added to the AST by the commonmark reader. No other readers are affected. The data-pos attributes are put on elements that accept attributes; for other elements, an enlosing Div or Span is added to hold the attributes.

Change extensions for commonmark_x: replace auto_identifiers with gfm_auto_identifiers (#6863). commonmark_x never actually supported auto_identifiers (it didn’t do anything), because the underlying library implements gfm-style identifiers only. Attempts to add the auto_identifiers extension to commonmark will now fail with an error.

HTML reader:

• Split module into several submodules (Albert Krewinkel). Reducing module size should reduce memory use during compilation.
• Support advanced table features (Albert Krewinkel): block level content in captions, row and colspans, body headers, row head columns, footers, attributes.
• Disable round-trip testing for tables. Information for cell alignment in a column is not preserved during round-trips (Albert Krewinkel).
• Allow finer grained options for tag omission (Albert Krewinkel).
• Simplify list attribute handling (Albert Krewinkel).
• Pay attention to lang attributes on body element (#6938). These (as well as lang attributes on the html element) should update lang in metadata.
• Retain attribute prefixes and avoid duplicates (#6938). Previously we stripped attribute prefixes, reading xml:lang as lang for example. This resulted in two duplicate lang attributes when xml:lang and lang were both used. This commit causes the prefixes to be retained, and also avoids invald duplicate attributes.

Commonmark reader:

• Refactor specFor.
• Set input name to "" to avoid clutter in sourcepos output.

Org reader:

• Parse #+LANGUAGE into lang metadata field (#6845, Albert Krewinkel).
• Preserve targets of spurious links (#6916, Albert Krewinkel). Links with (internal) targets that the reader doesn’t know about are converted into emphasized text. Information on the link target is now preserved by wrapping the text in a Span of class spurious-link, with an attribute target set to the link’s original target. This allows to recover and fix broken or unknown links with filters.

DocBook reader:

• Table text width support (#6791, Nils Carlson). Table width in relation to text width is not natively supported by docbook but is by the docbook fo stylesheets through an XML processing instruction, <?dbfo table-width="50%"?>.

LaTeX reader:

• Improve parsing of command options (#6869, #6873). In cases where we run into trouble parsing inlines til the closing ], e.g. quotes, we return a plain string with the option contents. Previously we mistakenly included the brackets in this string.
• Preserve center environment (#6852, Igor Pashev). The contents of the center environment are put in a Div with class center.
• Don’t parse \rule with width 0 as horizontal rule. These are sometimes used as spacers in LaTeX.
• Don’t apply theorem default styling to a figure inside (#6925). If we put an image in italics, then when rendering to Markdown we no longer get an implicit figure.

Dokuwiki reader:

• Handle unknown interwiki links better (#6932). DokuWiki lets the user define his own Interwiki links. Previously pandoc reacted to these by emitting a google search link, which is not helpful. Instead, we now just emit the full URL including the wikilink prefix, e.g. faquk>FAQ-mathml. This at least gives users the ability to modify the links using filters.

Markdown writer:

• Properly handle boolean values in writing YAML metadata (#6388).
• Ensure that a new csl-block begins on a new line (#6921). This just looks better and doesn’t affect the semantics.

RST writer:

• Better image handling (#6948). An image alone in its paragraph (but not a figure) is now rendered as an independent image, with an alt attribute if a description is supplied. An inline image that is not alone in its paragraph will be rendered, as before, using a substitution. Such an image cannot have a “center”, “left”, or “right” alignment, so the classes align-center, align-left, or align-right are ignored. However, align-top, align-middle, align-bottom will generate a corresponding align attribute.

Docx writer:

• Keep raw openxml strings verbatim (#6933, Albert Krewinkel).
• Use Content instead of Element. This allows us to inject raw OpenXML into the document without reparsing it into an Element, which is necessary if you want to inject an open tag or close tag.
• Fix bullets/lists indentation, so that the first level is slightly indented to the right instead of right on the margin (cholonam).
• Support bold and italic in “complex script” (#6911). Previously bold and italics didn’t work properly in LTR text. This commit causes the w:bCs and w:iCs attributes to be used, in addition to w:b and w:i, for bold and italics respectively.

ICML writer:

• Fix image bounding box for custom widths/heighta (Mauro Bieg, #6936).

LaTeX writer:

• Improve table spacing (#6842, #6860). Remove the \strut that was added at the end of minipage environments in cells. Replace \tabularnewline with \\ \addlinespace.
• Improve calculation of column spacing (#6883).
• Extract table handling into separate module (Albert Krewinkel).
• Fix bug with nested csl- display Spans (#6921).
• Improve longtable output (#6883). Don’t create minipages for regular paragraphs. Put width and alignment information in the longtable column descriptors.

OpenDocument writer:

• Support for table width as a percentage of text width (#6792, Nils Carson).
• Implement Div and Span ident support (#6755, Nils Carson). Spans and Divs containing an ident in the Attr will become bookmarks or sections with idents in OpenDocument format.
• Add two extensions, xrefs_name and xrefs_number (#6774, Nils Carlson). Links to headings, figures and tables inside the document are substituted with cross-references that will use the name or caption of the referenced item for xrefs_name or the number for xrefs_number. For the xrefs_number to be useful heading numbers must be enabled in the generated document and table and figure captions must be enabled using for example the native_numbering extension. In order for numbers and reference text to be updated the generated document must be refreshed.

JATS writer:

• Support advanced table features (Albert Krewinkel).
• Support author affiliations (#6687, Albert Krewinkel).

Docbook writer:

• Use correct id attribute consistently (Jan Tojnar). DocBook5 should always use xml:id instead of id.
• Handle admonition titles better (Jan Tojnar). Docbook reader produces a Div with title class for <title> element within an “admonition” element. Markdown writer then turns this into a fenced div with title class attribute. Since fenced divs are block elements, their content is recognized as a paragraph by the Markdown reader. This is an issue for Docbook writer because it would produce an invalid DocBook document from such AST – the <title> element can only contain “inline” elements. Handle this special case separately by unwrapping the paragraph before creating the <title> element.
• Add XML namespaces to top-level elements (#6923, Jan Tojnar). Previously, we only added xmlns attributes to chapter elements, even when running with --top-level-division=section. These namespaces are now added to part and section elements too, when they are the selected top-level divisions. We do not need to add namespaces to documents produced with --standalone flag, since those will already have xmlns attribute on the root element in the template.

HTML writer:

• Fix handling of nested csl- display spans (#6921). Previously inner Spans used to represent CSL display attributes were not rendered as div tags as intended.

EPUB writer:

• Include title page in landmarks (#6919). Note that the toc is also included if --toc is specified.
• Add frontmatter type on body element for nav.xhtml (#6918).

EPUB templates: use preserveAspectRatio=“xMidYMid” for cover image (#6895, Shin Sang-jae). This change affects both the epub2 and the epub3 templates. It avoids distortion of the cover image by requiring that the aspect ratio be preserved.

LaTeX template:

• Include csquotes package if csquotes variable set.
• Put back amssymb. We need it for checkboxes in todo lists, and maybe for other things. In this location it seems compatible with the cases that prompted #6469 and PR #6762.
• Disable language-specific shorthands in babel (#6817, #6887). Babel defines “shorthands” for some languages, and these can produce unexpected results. For example, in Spanish, 1.22 gets rendered as 122, and et~al. as etal. One would think that babel’s shorthands=off option (which we were using) would disable these, but it doesn’t. So we remove shorthands=off and add some code that redefines the shorthands macro. Eventually this will be fixed in babel, I hope, and we can revert to something simpler.

JATS template: allow array of persistent institute ids in pid (Albert Krewinkel).

Text.Pandoc.Parsing: minor code and efficiency improvements.

Text.Pandoc.Extension:

• Add Ext_sourcepos constructor for Extension [API change].
• Add Ext_xrefs_name and Ext_xrefs_number constructors for Extension (Nils Carson) [API change].

Text.Pandoc.Citeproc:

• Fix truncation of [Citation] list in Cite inside footnotes (#6890). This affected author-in-text citations in footnotes. It didn’t cause problems for the printed output, but for filters that expected the citation id and other information.
• Allow the use of both inline and external references (#6951), as with pandoc-citeproc. References defined in the document’s metadata take priority over references with the same id defined in an external bibliography.
• Use fetchItem to get external bibliography (#6940).
• Ensure that BCP47 lang codes can be used. We ignore the variants and just use the base lang code and country code when passing off to citeproc.
• Citeproc BibTeX parser: revert change in getRawField which was made (for reasons forgotten) when transferring this code from pandoc-citeproc. The change led to -- in URLs being interpreted as en-dashes, which is unwanted (#6874).

Text.Pandoc.ImageSize:

• Default to DPI 72 if the format specifies DPI of 0 (#6880). This shouldn’t happen, in general, but it can happen with JPEGs that don’t conform to the spec. Having a DPI of 0 will blow up size calculations (division by 0).
• ImageSize: use JuicyPixels to determine size for png, jpeg, and gif, instead of doing our own binary parsing (#6936). This gives more reliable results.

Text.Pandoc.CSS:

• Remove foldOrElse (internal module) (Albert Krewinkel).

Use skylighting 0.10.2 (#6625).

Use citeproc 0.3. This fixes issues with references with duplicate ids (jgm/citeproc#36).

Use doctemplates 0.9. This fixes issues with boolean metadata values in the Markdown writer (#6388) and in meta-json (#6650). It also fixes issues with nested for loops in templates.

Add translations zh-Hans.yaml and zh-Hant.yaml (#6904, #6909, Kolen Cheung, taotieren).

Add tests: True to cabal.project. This fixes some CI failures for cabal.

Normalize test/tables/*.native (#6888, Kolen Cheung).

Move executable to app directory to avoid problems with cabal repl.

CONTRIBUTING: add section “How can I help?” (#6892, Albert Krewinkel). Also adds a paragraph aimed at highlighting the importance of feature maintenance.

MANUAL: Document that –number-sections works in ms (#6935).

Default to using ATX (##-style) headings for Markdown output (#6662, Aner Lucero). Previously we used Setext (underlined) headings by default for levels 1–2.

Add option --markdown-headings=atx|setext, and deprecate --atx-headers (#6662, Aner Lucero).

Support markdown-headings in defaults files.

Fix corner case in YAML metadata parsing (#6823). Previously YAML metadata would sometimes not get recognized if a field ended with a newline followed by spaces.

--self-contained: increase coverage (#6854). Previously we only self-contained attributes for certain tag names (img, embed, video, input, audio, source, track, section). Now we self-contain any occurrence of src, data-src, poster, or data-background-image, on any tag; and also href on link tags.

Markdown reader:

• Fix detection of locators following in-text citations. Prevously, if we had @foo [p. 33; @bar], the p. 33 would be incorrectly parsed as a prefix of @bar rather than a suffix of @foo.
• Improve period suppression algorithm for citations in notes in note citation styles (#6835).
• Don’t increment stateNoteNumber for example list references. This helps with #6836 (a bug in which example list references disturb calculation of citation note number and affect when ibid is triggered).

LaTeX reader:

• Move getNextNumber from Readers.LaTeX to Readers.LaTeX.Parsing.
• Fix negative numbers in siunitx commands. A change in pandoc 2.11 broke negative numbers, e.g. \SI{-33}{\celcius} or \num{-3}. This fixes the regression.

DocBook reader: drop period in formalpara title and put it in a div with class formalpara-title, so that people can reformat with filters (#6562).

Man reader: improve handling of .IP (#6858). We now better handle .IP when it is used with non-bullet, non-numbered lists, creating a definition list. We also skip blank lines like groff itself.

Bibtex reader: fall back on en-US if locale for LANG not found. This reproduces earlier pandoc-citeproc behavior (jgm/citeproc#26).

JATS writer:

• Wrap all tables (Albert Krewinkel). All <table> elements are put inside <table-wrap> elements, as the former are not valid as immediate child elements of <body>.
• Move Table handling to separate module (Albert Krewinkel). Adds two new unexported modules: Text.Pandoc.Writers.JATS.Types, Text.Pandoc.Writers.JATS.Table.

Org writer:

• Replace org #+KEYWORDS with #+keywords (TEC). As of ~2 years ago, lower case keywords became the standard (though they are handled case insensitive, as always).
• Update org supported languages and identifiers according to the current list contained in https://orgmode.org/worg/org-contrib/babel/languages/index.html (TEC).

Only use filterIpynbOutput if input format is ipynb (#6841). Before this change content could go missing from divs with class output, even when non-ipynb was being converted.

When checking reader/writer name, check base name now that we permit extensions on formats other than markdown.

Text.Pandoc.PDF: Fix changePathSeparators for Windows (#6173). Previously a path beginning with a drive, like C:\foo\bar, was translated to C:\/foo/bar, which caused problems. With this fix, the backslashes are removed.

Text.Pandoc.Logging:

• Add constructor ATXHeadingInLHS to LogMessage [API change].
• Add constructor EnvironmentVariableUndefined to LogMessage [API change].

Fix error that is given when people specify doc output (#6834, gison93).

LaTeX template: add a \break after parbox in CSLRightInline. This should fix spacing problems between entries with numeric styles. Also fix number of params on CSLReferences.

reveal.js template: Put quotes around controlsLayout, controlsBackArrows, and display, since these require strings. Add showSlideNumber, hashOneBasedIndex, pause.

Use citeproc 0.2. This fixes a bug with title case around parentheses.

pandoc.cabal: remove ‘static’ flag. This isn’t really necessary and can be misleading (e.g. on macOS, where a fully static build isn’t possible). cabal’s new option --enable-executable-static does the same. On stack you can add something like this to the options for your executable in package.yaml:

ld-options: -static -pthread

Remove obsolete bibutils flag setting in linux/make_artifacts.sh.

Manual:

• Correct link-citation -> link-citations.
• Add a sentence about pagetitle for HTML (#6843, Alex Toldaiev).

INSTALL.md: Remove references to pandoc-citeproc (#6857).

CONTRIBUTING: describe hlint and how it’s used (#6840, Albert Krewinkel).

DocBook Reader: fix duplicate bibliography bug (#6773, Nils Carlson).

HTML reader:

• Parse contents of iframes (#6770).
• Parse inline svg as image unless raw_html is set in the reader (in which case the svg is passed through as raw HTML) (#6770).

LaTeX reader:

• Fix bug parsing macro arguments (#6796). If \cL is defined as \mathcal{L}, and \til as \tilde{#1}, then \til\cL should expand to \tilde{\mathcal{L}}, but pandoc was expanding it to \tilde\mathcal{L}. This is fixed by parsing the arguments in “verbatim mode” when the macro expands arguments at the point of use.
• Properly support optional (cite) argument for \blockquote from csquotes (#6802).

LaTeX writer: Improved calculation of table column widths. We now have LaTeX do the calculation, using \tabcolsep. So we should now have accurate relative column widths no matter what the text width. The default template has been modified to load the calc package if tables are used.

HTML writer: Fix duplicate “class” attribute for table rows (Andy Morris).

Text.Pandoc.Filter: allow shorter YAML representation of Citeproc (Albert Krewinkel). The map-based YAML representation of filters expects type and path fields. The path field had to be present for all filter types, but is not used for citeproc filters. The field can now be omitted when type is “citeproc”, as described in the MANUAL.

Text.Pandoc.Error: Add PandocBibliographyError constructor for PandocError [API change]. This ensures that bibliography parsing errors generate messages that include the bibliography file name – otherwise it can be quite mysterious where it is coming from.

Citeproc: properly handle csl field with data: URI (#6783). This is used with the JATS writer, so this fixes a regression in pandoc 2.11 with JATS output and citeproc.

Allow citation-abbreviations in defaults file.

JATS templates: ensure jats_publishing output is valid (Albert Krewinkel).

LaTeX template: Fix CSLRightInline, so that it does not run over the right margin.

HTML template: default CSS tweaks (Mauro Bieg and John MacFarlane).

• Fix margin before codeblock
• Add monobackgroundcolor variable, making the background color and padding of code optional.
• Ensure that backgrounds from highlighting styles take precedence over monobackgroundcolor
• Remove list markers from TOC
• Add margin-bottom where needed
• Remove italics from blockquote styling
• Change borders and spacing in tables to be more consistent with other output formats
• Style h5, h6
• Set font-size for print media to 12pt.
• Reduce interline space.
• Reduce interparagraph space.
• Reduce line width.
• Remove the special line-height: 1 for table cells.
• Remove the special line-height for pre.
• Ensure that there is a bit more space before a heading than after.
• Slightly reduced space after title header.
• Add CSS example to MANUAL

man template: Change comment that triggers tbl from .\"t to '\" t, as specified in groff_man(7) (#6803).

Use latest commonmark, commonmark-extensions. This fixes a bug with nested blocks in footnotes with the footnote extension to commonmark. See jgm/commonmark-hs#63.

Citeproc: use comma for in-text citations inside footnotes. When an author-in-text citation like @foo occurs in a footnote, we now render it with: AUTHOR NAME + COMMA + SPACE + REST. Previously we rendered: AUTHOR NAME + SPACE + "(" + REST + ")". This gives better results. Note that normal citations are still rendered in parentheses.

Use latest citeproc:

• citeproc no longer capitalizes notes, so we do it in pandoc when appropriate.
• Closes #6783.

Clarify manual on --track-changes (#6801).

Add doc/jats.md to document pandoc’s handling of JATS (#6794, Albert Krewinkel).

Fix code example in lua-filters.md (#6795).

Add --citeproc (-C) option to trigger built-in citation processing. It is no longer necessary to use the external pandoc-citeproc filter. --citeproc behaves like a filter and can be positioned relative to other filters as they appear on the command line.

The new built-in citation processing uses the citeproc library, a reimplementation of CSL citation processing that fixes many of the shortcomings of pandoc-citeproc. In general, citation processing should work much the same as it did with pandoc-citeproc, but with greater fidelity to CSL styles and better performance. (The tests from the pandoc-citeproc package have been carried over to pandoc.) The following differences should be noted:

• At this point, only some of the writers (HTML, ms, LaTeX) properly interpret CSL display styles. You should get decent output in all formats (at least as good as with pandoc-citeproc), but indentation and block-alignment may not be right.

• pandoc-citeproc searches the ~/.csl directory for .csl styles. Pandoc instead searches the csl subdirectory of the pandoc user data directory (e.g., ~/.pandoc/csl or ~/.local/share/pandoc/csl). Users who already keep CSL styles in ~/.csl may find it convenient to add a symlink.

• Some of the bibliography formats supported by pandoc-citeproc (via hs-bibutils) are no longer supported: Copac, EndNote, ISI, MEDLINE, MODS, and RIS. If you use one of these formats, you may use the bibutils utility to convert to BibLaTeX. We now support only BibTeX, BibLaTeX, CSL JSON, and pandoc’s YAML/Markdown analogue of CSL JSON.

• pandoc-citeproc would always retrieve the independent parent of a dependent style by doing an HTTP request. pandoc will now first seek the independent parent locally (in the resource path or in the csl subdirectory of the pandoc user data directory) before resorting to HTTP. In addition, you may omit the .csl extension, e.g. --csl zoology.

• Using the --bibliography option (or including bibliography in YAML metadata) no longer triggers citation processing implicitly: one must always use the --citeproc option if citation processing is wanted.

Add csljson as and input and output format. This allows pandoc to convert between csljson and other bibliography formats (e.g. -f csljson -t markdown -s or -f bibtex -t csljson), and to generate formatted versions of CSL JSON bibliographies (e.g., pandoc -f csljson --citeproc pl.json -o pl.pdf).

Added bibtex, biblatex as input formats. This allows pandoc to convert between BibLaTeX and BibTeX and other bibliography formats, and to generated formatted versions of BibTeX/BibLaTeX bibliographies (e.g., pandoc -f biblatex --citeproc pl.bib -o pl.pdf).

Raise informative errors when YAML metadata parsing fails (#6730). Previously the command would succeed, returning empty metadata, with no errors or warnings.

Sort languages in --list-highlight-languages output (#6718, Albert Krewinkel). Languages were previously sorted by their long name, which leads to unexpected results).

Add CSS to default HTML template (#6601, Mauro Bieg). This greatly improves the default typography in pandoc’s HTML output. The CSS is sensitive to a number of variables (e.g. mainfont, fontsize, linestretch): see the manual for details. To restore the earlier, more spartan output, you can disable this with -M document-css=false.

Support --toc-depth option for ODT writer (#6696, niszet).

Fix issues with Windows UNC paths with some options (#5127).

Remove fenced_code_blocks and backtick_code_blocks from allowed commonmark and gfm extensions. These shouldn’t really be counted as extensions, because they can’t be disabled in commonmark. Adjust markdown writer to check for the commonmark variant in addition to extensions.

Add these extensions to gfm and commonmark: fenced_code_blocks, backtick_code_blocks, fenced_code_attributes. These can’t really be disabled in the reader, but they need to be enabled in the writer or we just get indented code.

Make sure proper set of extensions is recognized for commonmark_x.

Allow gfm_auto_identifiers, ascii_identifiers extensions for docx.

Markdown reader:

• Add Maybe FilePath parameter to yamlToMeta [API change].
• Export yamlToRefs [API change], a version of yamlToMeta specialized to references.
• Set citationNoteNum accurately in citations.
• Revise abbreviation support. Don’t insert a nonbreaking space after a potential abbreviation if it comes right before a note or citation. This causes problems for citeproc’s moving of note citations.

LaTeX reader:

• Support missing siunitx commands (#6658).
• Support squared, cubed, tothe in siunitx (#6657).
• Improved uncertainty handling in slunitx.
• Factored out siunitx stuff into separate unexposed module.
• Fix improper empty cell filtering (#6689, Christian Despres).
• Fix parsing of “show name” in \newtheorem (#6734). Previously we were just treating it as a string and ignoring accents and formatting.
• Prevent wrong nesting of \multirow and \multicolumn table cells (#6603, Laurent P. René de Cotret).
• Table cell parser not consuming spaces correctly (#6596, Laurent P. René de Cotret).
• Change SIRange to SIrange (#6617, Emerson Harkin).
• Allow blank lines inside \author (#6324).

DocBook reader:

• Don’t squelch space at end of emphasis and other inline elements; instead, move it outside the element (#6719).
• Implement table cell alignment (#6698, Nils Carlson).
• Implement column span support for tables (#6492, Nils Carlson).
• Update list of block level tags (#6610).

JATS reader:

• Don’t squelch space at end of emphasis and other inline elements; instead, move it outside the element (#6719).

RST reader:

• Apply .. class:: directly to following Header rather than creating a surrounding Div (#6699).

Docx reader:

• Allow empty dates in comments and tracked changes (#6726, Diego Balseiro).

Markdown writer:

• Be less aggressive about using quotes for YAML values, allowing e.g. a quotation mark or bracket as long as it’s not at the beginning of the line.
• Use double quotes for YAML metadata (#6727).
• Sort YAML metadata keys in Markdown output case-insensitive.

Asciidoc writer:

• Support asciidoctor’s block figures (#6538, argent0).

LaTeX writer:

• Fix spacing issue with list in definition list. When a list occurs at the beginning of a definition list definition, it can start on the same line as the label, which looks bad. Fix that by starting such lists with an \item[].

HTML writer:

• Support intermediate table headers (#5314, Albert Krewinkel).
• Support attributes on all table elements (Albert Krewinkel).
• Render table footers if present (#6314, Albert Krewinkel).
• Fix addition of doc-biblioentry role.
• Support colspans and rowspans in HTML tables (#6312, Albert Krewinkel).

ICML writer:

• Support internal document links (#5541, Leonard Rosenthol).
• Changed default link state to invisible (#6676, Leonard Rosenthol).

Docx writer:

• Better handle list items whose contents are lists (#5948, Michael Hoffmann). If the first element of a bulleted or ordered list is another list, then that first item would previously disappear if the target format is docx.
• Separate adjacent tables (#4315). Word combines adjacent tables, so to prevent this we insert an empty paragraph between two adjacent tables.

Org writer:

• Don’t force blank line after headings (#6554).

OpenDocument writer:

• Implement table cell alignment (#6700 Nils Carson, Mauro Bieg).
• New table cell support with row and column spans (#6682, Nils Carson).
• Syntax highlighting for inline code (#6711, niszet).

Add Text.Pandoc.Citeproc module, exporting processCitations [API change]. This depends on several other, unexported modules under Text.Pandoc.Citeproc.

Add module Text.Pandoc.Writers.CslJson, exporting writeCslJson. [API change]

Add module Text.Pandoc.Readers.CslJson, exporting readCslJson. [API change]

Add module Text.Pandoc.Readers.BibTeX, exporting readBibTeX and readBibLaTeX. [API change]

Text.Pandoc.Filter: Add CiteprocFilter constructor to Filter. [API change] This runs the processCitations transformation. We need to treat it like a filter so it can be placed in the sequence of filter runs (after some, before others). In FromYAML, this is parsed from citeproc or {type: citeproc}, so this special filter may be specified either way in a defaults file (or by citeproc: true, though this gives no control of positioning relative to other filters).

Add new exported module Text.Pandoc.Writers.AnnotatedTable [API change] (#6655, Christian Despres). This module (which should generally be imported qualified to avoid name conflicts) provides a Table type that mirrors the structure of a pandoc Table, but with added inferred information so that the writers do not have to lay out tables themselves. The toTable and fromTable functions convert between an annotated Table and a regular pandoc Table. In addition to producing a Table with coherent and well-formed annotations, the toTable function also normalizes its input table like the table builder does. Tests ensure that toTable normalizes tables exactly like the table builder, and that its annotations are coherent.

Text.Pandoc.Logging:

• Remove unused CouldNotParseYamlMetadata constructor for LogMessage [API change].
• Add CiteprocWarning constructor to LogMessage [API change].

Text.Pandoc.Readers.Metadata: export yamlBsToRefs [API change]. These allow specifying an id filter so we parse only references that are used in the document.

Text.Pandoc.Parsing:

• Export ParseError [API change].
• Add stateInNote and stateNoteNumber to ParserState [API change]. These are used to populate note numbers for citation processing.

Fix apparent typos in sample.lua (#6729, William Lupton). Also make the writer less aggressive in escaping quotes.

Text.Pandoc.Options:

• defaultMathJaxURL: use tex-chtml-full instead of tex-mml-chtml (#6599, Kolen Cheung). This drops the MathML support (which we don’t need for HTML math rendering) and includes the full JavaScript, which makes it possible to use --self-contained (though there may still be issues if the required math fonts aren’t available). This change should also reduce latency in pages with lots of formulas.
• Add /tex-chtml-full.js to defaultMathJaxURL (#6593) Previously we added this in processing command line options, but not in processing defaults files, which was inconsistent.

epub.css: Fix cover page selectors and add note explaining their use (#6649, a-vrma).

Add data files needed for Text.Pandoc.Citeproc: these include default.csl in the data directory and a citeproc directory that is only used at compile-time for biblatex localizations. Note that we’ve added file-embed as a mandatory rather than a conditional dependency, because of the biblatex localization files.

Lua filters:

• Add SimpleTable for backwards compatibility (#6575, Albert Krewinkel). A new type SimpleTable is made available to Lua filters. It is similar to the Table type in pandoc versions before 2.10; conversion functions from and to the new Table type are provided. Old filters using tables now require minimal changes and can use, e.g.,

  if PANDOC_VERSION > {2,10,1} then
    pandoc.Table = pandoc.SimpleTable
  end
  

  and

  function Table (tbl)
    tbl = pandoc.utils.to_simple_table(tbl)
    …
    return pandoc.utils.from_simple_table(tbl)
  end
  

  to work with the current pandoc version.

• Make attr argument optional in Table constructor (Albert Krewinkel). This changes the Lua API. It is highly unlikely for this change to affect existing filters, since the documentation for the new Table constructor (and type) was incomplete and partly wrong before. The Lua API is now more consistent, as all constructors for elements with attributes now take attributes as the last parameter.

MANUAL.txt:

• Add a dedicated Citations section which consolidates the information the manual used to contain about citation processing, and incorporates some information formerly found in the pandoc-citeproc man page.

• Add note about lualatex using selnolig.
• Remove duplicate seriespage (#6568, Blake Eryx).
• Remove lists of support extensions for markdown variants (#6604). Instead, offer the advice to use --list-extensions=FORMAT.
• Fix position of attributes in header (Albert Krewinkel).
• Delete obsolete section on compact and loose lists (#6684).

doc/lua-filters.md:

• Add info on how to debug Lua filters (#6732, Ian Max Andolina).
• Document Underline type and constructor (Albert Krewinkel).
• Document body field (Albert Krewinkel).
• Add missing header attribute
• Add missing Link.title field (Albert Krewinkel).
• Make the setting-the-date example conditional (the-solipsist). This makes the example a bit more realistic.
• Remove outdated link table example.

doc/org.md:

• Add section on tables (Albert Krewinkel).
• Add section on handling of unknown directives (Albert Krewinkel).

CONTRIBUTING.md: fix typo (#6584, Dmitry Volodin).

Use golden test framework for command tests. This means that --accept can be used to update expected output.

Use the smart extension when generating pandoc’s man page (#6613).

Release-candidate: don’t build windows i386. So far we haven’t been able to figure out how to get stack to use a 32-bit ghc.

Use null instead of deprecated Builder.isNull.

Makefile:

• Fix macospkg target to fetch target from S3 artifacts.
• Fix pandoc-templates target to include all partials.

Remove duplicated dependency in pandoc.cabal (#6591, Felix Yan).

Sort build depends in pandoc.cabal alphabetically (#6691, Albert Krewinkel).

Add .travis.yml for macos release candidate build (#6622). We need to build the release candidate on Travis rather than GitHub actions, because GH has macos 10.15, and binaries compiled on that OS will not work with 10.13. This build is only triggered on rc/* branches.

Remove instructions for building pandoc-citeproc from CI and release binary build instructions. We will no longer distribute pandoc-citeproc.

Fix math rendering in trypandoc (this broke after commit d8ad766d17603784b86fc5c2e1b22864125d04d1).

Use latest versions of skylighting, commonmark (#6589), comonmark-extensions, commonmark-pandoc, texmath.

Relax version bounds for hslua, hslua-module-text, bytestring.

Use released pandoc-types 1.22. This changes the JSON encoding slightly for the new table types introduced in 1.21, so they’re more consistent with the rest. Developers of libraries for pandoc filters will want to take note.

Fix hlint suggestions, update hlint.yaml (#6680, Christian Despres).

Code cleanup (#6678, Joseph C. Sible).

Add haddocks to functions in Text.Pandoc.Writers.Shared (Albert Krewinkel).

Remove duplicate tshow definition.

Linux release candidate build: use ghc-musl container. This simplifies our build process (over using a customized alpine container).

Use pandoc-types 1.21. This adds two things:

• A native Underline constructor for Inline (#6277, Vaibhav Sagar).
• More expressive types for tables (#1024, Christian Despres). Tables can now take attributes; and rowspans and colspans, column headers, multiple row headers, table head and foot can all be represented. (Note, however, that reader and writer support for these features is still lacking, so most users won’t see any differences in table conversion yet. These changes just lay the foundation for further improvements.)

Support new Underline element in readers and writers (#6277, Vaibhav Sagar).

Support new Table type (Christian Despres). The Builder.simpleTable now only adds a row to the TableHead when the given header row is not null. This uncovered an inconsistency in the readers: some would unconditionally emit a header filled with empty cells, even if the header was not present. Now every reader has the conditional behaviour. Only the XWiki writer depended on the header row being always present; it now pads its head as necessary.

Add an option to disable certificate validation (#6156, Cédric Couralet, Cécile Chemin, Juliette Fourcot). This commit adds the option --no-check-certificate, which disables certificate checking when resources are fetched by HTTP.

Unify defaults and markdown metadata parsers (#6328, Nikolay Yakimov). Clean up code in Text.Pandoc.Readers.Metadata and properly handle errors in yamlToMeta. This fixes parsing of Boolean fields in metadata withinin defaults files and reduces code duplication.

Docbook reader:

• Implement <procedure> (#6442, Mathieu Boespflug).
• Implement <phrase> (#6438, Mathieu Boespflug).
• Treat envar and systemitem like code (#6435, Mathieu Boespflug).
• Implement <replaceable> (#6437, Mathieu Boespflug)
• Map <simplesect> to unnumbered section (#6436, Mathieu Boespflug).

JATS reader:

• Handle “label” element in section title (#6288).
• Parse abstract element into metadata field of same name (#6480, Albert Krewinkel).

Jira reader (Albert Krewinkel):

• Resolve multiple parsing problems, including issues with empty table cells, faulty recognition of closing emphasis characters, and parsing of image attributes (#6212, #6219, #6220).
• Two consecutive markup chars are now parsed verbatim (#6343); styled text must not be empty.
• Newlines are no longer allowed within styled text (#6325).
• Links to anchors are now parsed as links (#6407).
• Retain image attributes (#6234). Jira images attributes as in !image.jpg|align=right! are retained as key-value pairs. Thumbnail images, such as !example.gif|thumbnail!, are marked by a thumbnail class in their attributes.
• Use Underline for inserted text (#6237). Previously, the span was marked with the non-standard class inserted.
• Improve icon conversion for (/), (x), (!), (?) (+), (-), (off), (*). (#6236, #6264).
• Support citations, attachment links, and user links (#6231, #6238, #6239).
• Resolve parsing issues of blockquote, color (#6233, #6235).

HTML reader:

• Parse attributes into table attributes.
• Support <bdo> (#5794, Tristan de Cacqueray).
• Add summary to list of block-level HTML tags (#6385). This improves support for summary/details inside Markdown. NOTE: you need to include a blank line before the closing </details>, if you want the last part of the content to be parsed as a paragraph.
• Fix parsing unclosed th elements in a table (#6247).

Commonmark reader: Implement implicit_figures extension (#6350).

Markdown Reader:

• Fix inline code in lists (#6284, Nikolay Yakimov). Previously inline code containing list markers was sometimes parsed incorrectly.
• Don’t require blank line after grid table (#6481). This allows grid tables to be enclosed in fenced divs with no intervening blank lines.

LaTeX reader:

• Don’t parse beyond \end{document} (#6380). This required some internal changes to \subfile handling.
• Better handling of \lettrine. SmallCaps instead of Span for the part after the initial capital. Ensure that both arguments are parsed, so that in Markdown both are treated as raw LateX. (Closes #6258.)

Org reader (Albert Krewinkel):

• Recognize images with uppercase extensions (#6472).
• Keep unknown keyword lines as raw org. The lines of unknown keywords, like #+SOMEWORD: value are no longer read as metadata, but kept as raw org blocks. This ensures that more information is retained when round-tripping org-mode files; additionally, this change makes it possible to support non-standard org extensions via filters.
• Unify keyword handling. Handling of export settings and other keywords (like #+LINK) has been combined and unified.
• Support LATEX_HEADER_EXTRA and HTML_HEAD_EXTRA settings. These export settings are treated like their non-extra counterparts, i.e., the values are added to the header-includes metadata list.
• Allow multiple #+SUBTITLE export settings. The values of all lines are read as inlines and collected in the subtitle metadata field.
• Read #+INSTITUTE values as text with markup. The value is stored in the institute metadata field and used in the default beamer presentation template.
• The behavior of the #+AUTHOR and #+KEYWORD export settings has changed: Org now allows multiple such lines and adds a space between the contents of each line. Pandoc now always parses these settings as meta inlines; setting values are no longer treated as comma-separated lists. Note that a Lua filter can be used to restore the previous behavior.
• Read description lines as inlines (#6485). #+DESCRIPTION lines are now treated as text with markup. If multiple such lines are given, then all lines are read and separated by soft linebreaks.
• Honor tex export option (#4070). The tex export option can be set with #+OPTION: tex:nil and allows three settings: t (the default) causes LaTeX fragments to be parsed as TeX or added as raw TeX. nil removes all LaTeX fragments from the document. verbatim treats LaTeX as text.

RST reader:

• Pass arbitrary attributes through in code blocks (#6465). Exceptions: name (which becomes the id), class (which becomes the classes), and number-lines (which is treated specially to fit with pandoc highlighting).
• Handle date:: directive (#6276).

Textile reader: support pre. for code blocks (#6454).

Ipynb reader:

• Implement raw_markdown extension (#5408). Specifying -f ipynb+raw_markdown will cause Markdown cells to be represented as raw Markdown blocks, instead of being parsed. This is not what you want when going from ipynb to other formats, but it may be useful when going from ipynb to Markdown or to ipynb, to avoid semantically insignificant changes in the contents of the Markdown cells that might otherwise be introduced.
• Handle application/pdf output as image (#6430).
• Properly handle image/svg+xml as an image (#6430).

Docx reader:

• Distinguish between docx parsing and docx container unpacking errors.

MediaWiki reader:

• Fix gfm_auto_identifiers so that - is not replaced by _ (#6335).

Vimwiki reader:

Jira writer (Albert Krewinkel):

• Always escape braces (#6478). Braces are now always escaped, even within words or when surrounded by whitespace. Jira and Confluence treat braces specially.
• Convert Underline to inserted text (+inserted+).
• Add image attributes (#6234). Image attributes are added to the output as image parameters. If the image has a class “thumbnail”, then a thumbnail image is generated; all other attributes are discarded in this case.

LaTeX writer:

• Ensure that -M csquotes works even in fragment mode (#6265).
• Escape ^ specially for listings (#6460).
• Create hypertarget for links with identifier (#6360).
• Distinguish between single and double quotes when using enquote package (#6457, dbecher-ito).
• Add support for customizable alignment of columns in beamer (#6331, andrebauer).
• Add support for customizable alignment of columns in beamer (#4805, #4150, andrebauer).

HTML writer:

• Use CSS in favor of <br> for display math (#6372) Some CSS to ensure that display math is displayed centered and on a new line is now included in the default HTML-based templates; this may be overridden if the user wants a different behavior.

Org writer:

• Clean-up Div handling (Albert Krewinkel).

Docx writer:

• Enable column and row bands for tables (#6371). This change will not have any effect with the default style. However, it enables users to use a style (via a reference.docx) that turns on row and/or column bands.

OpenDocument (and ODT) writer:

• Add custom-style “Abstract” in metadata abstract. This ensures that the abstract is rendered with style Abstract.
• Enable custom-style attribute on a Div. This allows you to apply a custom style to contained paragraphs.

DocBook writer:

• Add id of figure to enclosed image.
• Add personname element to docbook author (#6244).

FB2 writer:

• Properly handle cover-image containing spaces (#6391).

Markdown writer:

• Ensure consistent padding for pipe tables (#6240).
• Avoid unnecessary escapes before intraword _ when intraword_underscores extension is enabled (#6296).

RST writer:

• Properly handle images with same alt text (#6194). Previously we created duplicate references for these in rendering RST.

AsciiDoc writer:

• Add blank line after Div (#6308).

Haddock Writer:

PowerPoint writer (Jesse Rosenthal):

• Write math input verbatim in speaker notes (#6301). OMML in speaker notes would lead to corrupt PowerPoint output. We now output the OMML verbatim as LaTeX in the speaker notes.

LaTeX template: Make polyglossia package options list-aware (#6444, Frederik Elwert).

Reveal.js template:

• Update template for reveal.js 4.0.0 (#6390, Salim B).
• Update template with newly available options (#6347, Jake Zimmerman).
• Use CDN version of revealjs v4 by default (#6408).

opendocument template: Add abstract and subtitle to opendocument template (#6369).

reference.odt: clean up styles. Add Abstract. Change Author, Date to centered paragraphs with no character styling.

epub.css: wrap overlong lines in highlighted code blocks (#6242). This fixes a problem in iBooks v2.4 with our earlier horizontally scrolling code blocks. The problem seems to be a bug in iBooks, not pandoc, but since iBooks is a major target we’re changing pandoc’s default behavior so that pandoc-produced epubs work on that platform.

Text.Pandoc.PDF:

• Use --enable-local-file-access in invoking wkhtmltopdf (#6474). wkhtmltopdf changed in recent versions to require this for access to local files. This fixes PDF via HTML5 with --css.
• Send verbose output to stderr, not stdout (#6483).

Text.Pandoc.MIME: Fix MIME type for TrueType fonts in EPUBs (#6464, Michael Reed).

Text.Pandoc.Shared:

• makeSections: omit number attribute when unnumbered class is present (#6339). Previously the attribute was included but given an empty value, and this caused the table of contents creation functions in Text.Pandoc.Writers.Shared to think these items had numbers, which meant that they were included in the TOC even if the unlisted class was used.
• Deprecate underlineSpan in Shared in favor of Text.Pandoc.Builder.underline (Vaibhav Sagar).
• renderTags': use self-closing tag for col element (#6295).

Text.Pandoc.UUID: Fix getRandomUUID, which previously would return the same value twice in a row. Make getRandomUUID polymorphic in PandocMonad. Remove getUUID (#6228, Joseph C. Sible).

Text.Pandoc.Class: Generalize PandocIO functions to MonadIO.

Fixed Katex standalone script (#6399, Lucas Escot). Global macros are now persistent when using the HTML Writer with the --katex option.

Lua subsystem (Albert Krewinkel):

• Use new type PandocLua for all pandoc Lua operations (API change). The new type PandocLua is an instance of the PandocMonad typeclass and can thus be used in a way similar to PandocIO.
• Use PandocError for exceptions in Lua subsystem (API change). The PandocError type is used throughout the Lua subsystem. All Lua functions throw an exception of this type if an error occurs. The LuaException type is removed and no longer exported from Text.Pandoc.Lua. In its place, a new constructor PandocLuaError is added to PandocError.

Lua filters: improve error messages for failing filters (#6332, Albert Krewinkel). Print the Lua error properly instead of displaying their show string.

Use latest skylighting. This fixes a bug with lua multiline comments (and may improve handling of other syntaxes as well). IT also adds aria-hidden="true" to the empty a elements, which helps people who use screen readers.

Use latest texmath.

Require latest doctemplates 0.8.2. This adds support for template pipes first, rest, last, allbutlast.

Revert 0e48a02 and dependency on base-noprelude, which hasn’t been updated for ghc 8.10 (see #6187).

Dependency adjustments:

• Allow haddock-library 1.9.x.
• Allow hslua 1.1 (#6243, Felix Yan).
• Allow base64-bytestring 1.1.
• Use latest jira-wiki-markup.
• Allow http-client 0.7.
• Allow tasty 1.3.x.
• Allow aeson 1.5 (#6400, Felix Yan).
• Remove unused dependency vector (#6462, Laurent P. René de Cotret).
• Bump QuickCheck upper bound.

Significant code cleanup and simplification (Joseph C. Sible, #6223, #6209, #6225, #6229, #6226, #6340).

Remove unnecessary hlint ignores (#6341, Joseph C. Sible).

Remove obsolete RelaxedPolyRec extension (#6487, Nikolay Yakimov).

trypandoc improvements (Mike Tzou):

• Add standalone option to the command text (#6210).
• Update third party libraries.

MANUAL.txt:

• Clarify template partial naming (#6476, Mauro Bieg).
• Describe jira as “Jira/Confluence wiki markup” (#6351, Albert Krewinkel). In the past, Jira’s wiki markup was also used by – and could be imported into – Atlassian Confluence.
• Add link to print-css.rocks (#6272, Mauro Bieg).
• Clarify pipe table column width adjustment (#6254).
• Fix ATX header syntax.
• Fix misleading note about image size conversions (#6353).
• Update links to reveal.js documentation (#6386, Salim B).
• Separate adjacent verbatim code blocks (#6307, tom-audm).

org.md:

• Document behavior of smart extension (#4387, Albert Krewinkel).
• Describe all supported export options in detail.

lua-filters.md:

• Fix description of BulletList Lua type (Levi Gruspe).
• Use pandoc.system module in TikZ example (Albert Krewinkel). Showcase temporary directory handling with with_temporary_directory and with_working_directory.

INSTALL.md: fix FreeBSD port link (#6422, Mo). The FreeBSD port was renamed from pandoc to hs-pandoc in 2010. The old pandoc port is still at version 1.5.1.1

Propagate (DY)LD_LIBRARY_PATH in tests (#6376, Lila).

Bump cabal-version to 2.2 (#6377).

Make it possible to compile using Stack on NixOS (#6439, Mathieu Boespflug).

CI action to check for commit message length (Nikolay Yakimov, #6398).

Markdown reader: Fix table alignment when heading begins with t (#6153). Due to a typo (t instead of \t) we were center aligning column headings that begin with a lowercase t!

Text.Pandoc.Readers.Roff:

• Fix parsing of \. in man/ms readers (#6175). Previously due to a typo it was being parsed as `.
• Fix parsing of \' in man/ms readers (#6175). It was being parsed as a backtick.

Jira reader (Albert Krewinkel):

• Fix parsing of tables without preceding blankline (#6198). A bug was fixed which caused faulty parsing if a table was not preceded by a newline and the first table cell had no space after the initial | characters.
• Fix parsing of strikeout, emphasis (#6196). A bug was fixed which caused non-emphasized text containing digits and/or non-special symbols (like dots) to sometimes be parsed incorrectly.
• Support colored inline text, indented lists (#6183, #6184).

Ms writer:

• Fix definition lists so indent even when paragraph indent is set to 0 (as is the default). Also ensure indent for display math that falls back to TeX.
• Use .QS/.QE instead of .RS/.RE for block quotes.

EPUB writer: fix regression on detection of front/back/bodymatter (#6170). This bug caused sections with epub:type dedication to be misplaced in bodymatter instead of frontmatter as specified in the manual. The same problem would affect other epub:types. The pattern matching needed to be changed with the use of makeSection.

AsciiDoc writer: remove redundant otherwise guard in inlineToAsciiDoc (#6146, Ryan Scott).

Text.Pandoc.Class:

• Fix missing import when data files are not embedded (Albert Krewinkel).
• Subdivide Text.Pandoc.Class into small unexported modules and ensure that all functions have Haddock documentation (#6106, Albert Krewinkel).
• Finer grained imports of Text.Pandoc.Class submodules (#6203, Albert Krewinkel).

Text.Pandoc.XML: Add to list of HTML5 attributes: allow, autocapitalize, decoding, enterkeyhint, imagesizes, imagesrcset, loading.

Use implicit Prelude (#6187, Albert Krewinkel). The previous behavior was introduced as a fix for #4464. It seems that this change alone did not fix the issue, and stack ghci and cabal repl only work with GHC 8.4.1 or newer, as no custom Prelude is loaded for these versions. Given this, it seems cleaner to revert to the implicit Prelude.

Always use custom prelude (#6187, Albert Krewinkel). Previously, the custom prelude was used only with older GHC versions, as a workaround for problems with ghci. The ghci problems are resolved by replacing package base with base-noprelude, allowing for consistent use of the custom prelude across all GHC versions.

Remove outdated checks for no longer supported base versions (Albert Krewinkel).

PDF via wkhtmltopdf: put user-specified options last (#6171). Certain options (e.g. cover) need to come after flags on the command line.

Text.Pandoc.App: set resource path at the beginning so it can affect things like include-in-header (#5982).

Change macOS release candidate CI process so that notarized packages can be produced (#6169).

Make MANUAL more explicit about nonbreaking space handling by all_symbols_escapable (#6154, Fabien Schurter).

trypandoc (Mike Tzou):

• Add checkbox for standalone option (#6189).
• Use strict mode for JavaScript code (#6188).
• Fetch resources over https (#6188).
• Remove unnecessary attributes on style, script elements (#6188).

Use details tag to make GitHub releases changelog collapsible.

Update filter code in doc/filters.md so it works with latest pandoc (#6185).

linux/Dockerfile: upgrade to alpine 3.11 (#6180, Albert Krewinkel). This is used to build the static linux binaries.

Add csv as an input format (#6100). The CSV table is converted into a pandoc simple table. A new module Text.Pandoc.Readers.CSV exports readCSV [API change].

Introduce new format variants for JATS writer (#6014, Albert Krewinkel):

• jats_archiving for the “Archiving and Interchange Tag Set”,
• jats_publishing for the “Journal Publishing Tag Set”, and
• jats_articleauthoring for the “Article Authoring Tag Set.”

The jats output format is now an alias for jats_archiving. The module Text.Pandoc.Writers.JATS now exports writeJatsArchiving, writeJatsPublishing, and writeJatsArticleAuthoring, as well as the legacy writeJATS [API change].

--defaults: Support bibliography and csl fields. Move addMeta from Text.Pandoc.App.CommandLineOptions to Text.Pandoc.App.Opt (internal change).

Add timing info for filters in --verbose mode (#6112). When verbose mode is specified (verbosity == INFO), print a notice when running a filter and when a filter completes (including timing).

LaTeX reader:

• Allow & in LaTeX citation keys (#6110).
• Improve caption and label parsing.
• Don’t emit empty Span elements for labels.
• Put tables with labels in a surrounding Div.
• Resolve \ref to table numbers (#6137).
• Skip comments in more places where this is needed (#6114).
• Allow beamer overlays for all commands in all raw tex (#6043). This affects parsing of raw tex in LaTeX and in Markdown and other formats.
• Improve parsing of raw environments (#6034). If parsing fails in a raw environment (e.g. due to special characters like unescaped _), try again as a verbatim environment, which is less sensitive to special characters. This allows us to capture special environments that change catcodes as raw tex when -f latex+raw_tex is used.

RST reader:

• Add highlight directive (#6140, Lucas Escot).

MediaWiki writer:

• Prevent triple [[[ which confuses MediaWiki (#6119).

HTML reader:

• Don’t parse data-id as id attribute. And similarly don’t parse any data-X as X when X is a valid HTML attribute.

Org reader:

• Simplify parsing of sub- and superscripts (#6127, Albert Krewinkel). Speeds up parsing of single-word, markup-less sub- and superscripts.

LaTeX writer:

• Group biblatex citations even with prefix and suffix (#5849, Ethan Riley). Previously biblatex citations were only grouped if there was no prefix. This patch allows them to be grouped in subgroups split by prefixes and suffixes, which allows better citation sorting.
• Fix regression in handling of columns in beamer slides (#6033). Columns in title slides were causing problems with slide division.
• Fix duplicate frame classes in LaTeX/Beamer output (#6107).

HTML writer:

• Fix duplicate attributes on headings (#6062), regression from 2.7.x.
• Fix --number-offset with HTML TOC. Eventually it would be worth adding a parameter to makeSections so this could be done at that level; then it would also affect other writers that construct TOC manually.
• reveal.js: restore old behavior for 2D nesting (#6032). The fix to #6030 actually changed behavior, so that the 2D nesting occurred at slide level N-1 and N, instead of at the top-level section. This commit restores the v2.7.3 behavior. If there are more than 2 levels, the top level is horizontal and the rest are collapsed to vertical.
• reveal.js: ensure that pauses work even in title slides (#5819).

Markdown writer:

• Fix regression: spurious dots in markdown_mmd metadata output (#6133).

Docx writer:

• Fix regression with Compact style on tight lists (#6072). Starting in 2.8, the docx writer no longer distinguishes between tight and loose lists, since the Compact style is omitted. This is a side-effect of the fix to #5670, as explained in the changelog. This patch fixes the problem by extending the exception currently offered to Plain blocks inside tables to Plain blocks inside list items.

Jira writer:

• Fix output of table headers (Albert Krewinkel, #6035).

Add Text.Pandoc.Image with unexported svgToPng.

Text.Pandoc.XML: Export html5Attributes, html4Attributes, rdfaAttributes (formerly unexported in Text.Pandoc.Writers.HTML). [API change]

Text.Pandoc.Shared: Export a new function findM (#6125, Joseph C. Sible).

Text.Pandoc.Logging: Add RunningFilter, FilterCompleted constructors to LogMessage [API change].

Text.Pandoc.CSV: fix bug in CSV parser; previously an extra blank record would sometimes be inserted at the end.

LaTeX template: add space option to xeCJK with PassOptionsToPackage (#6002). Otherwise we can get a clash with documentclasses that already load the package.

Lua filters:

• Allow filtering of element lists (#6038, Albert Krewinkel). Lists of Inline and Block elements can now be filtered via Inlines and Blocks functions, respectively. This is helpful if a filter conversion depends on the order of elements rather than a single element. For example, the following filter can be used to remove all spaces before a citation:

  function isSpaceBeforeCite (spc, cite)
    return spc and spc.t == 'Space'
     and cite and cite.t == 'Cite'
  end
  
  function Inlines (inlines)
    for i = #inlines-1,1,-1 do
      if isSpaceBeforeCite(inlines[i], inlines[i+1]) then
        inlines:remove(i)
      end
    end
    return inlines
  end
  

• Add methods insert, remove, and sort to pandoc.List (Albert Krewinkel). Example of use:

  local numbers = pandoc.List {2, 3, 1}
  numbers:sort()     -- numbers is now {1, 2, 3}
  

• Make pandoc.List a callable constructor (Albert Krewinkel). It is now possible to construct a new List via pandoc.List() instead of pandoc.List:new().

• Add tests for pandoc.List module (Albert Krewinkel).

Text.Pandoc.App.CommandLineOptions: Change setVariable to use Text instead of String. This avoids some unnecessary unpacking.

Use versioned directory for windows release zipfile. Also remove old make-windows-installer.bat, superseded by GitHub actions workflow, and modify pandoc.wxs for new paths.

Extensive code cleanup (#6141, #6128, #6129, #6130, #6123, #6105, 6102, #6117, #6124, #6115, #6116, #6111, Joseph C. Sible).

Fix hlint warnings (Albert Krewinkel).

Use latest doclayout, doctemplates (#6031). The new version of doclayout fixes a memory leak that affected --include-in-header with large files (and possibly other cases involving extremely long lines).

Use latest texmath.

Use latest skylighting and fix test suite (#6086).

sample.lua: Fix typo in descriptive comments (#6136, Caleb Maclennan). Fix typo in error message (#6135).

Add Docker and GH Actions instructions/links to INSTALL.md.

Update filter documentation (#6065). Improve cabal v2 instructions. Remove example using pandoc API directly (we have other docs for that and it was outdated).

Lua filter docs:

• Cross-link constructors and types (Albert Krewinkel). Thanks to @bpj for the idea.
• Sort pandoc.List methods alphabetically (Albert Krewinkel).
• Unify, fix anchors and internal links (#6061, Albert Krewinkel). Links and anchors now follow consistent conventions, like lowercase-only anchor names. This breaks some links to specific sections in the document, but will make it much easier to link documentation in the future.
• Clarify filter function execution order (#6059, Albert Krewinkel).

In docs, update URLs and use https: wherever possible (#6090, Salim B).

Improvements in templates system (from doctemplates):

• Pandoc templates now support a number of new features that have been added in doctemplates: notably, elseif, it, partials, filters, and syntax to control nesting and reflowing of text. These changes make pandoc more suitable out of the box for generating plain-text documents from data in YAML metadata. It can create enumerated lists and even tabular structures.
• We now used templates parameterized on doclayout Doc types. The main impact of this change is better reflowing of content interpolated into templates. Previously, interpolated variables were rendered independently and interpolated as strings, which could lead to overly long lines. Now the templates interpolated as Doc values which may include breaking spaces, and reflowing occurs after template interpolation rather than before.
• Remove code from the LaTeX, Docbook, and JATS writers that looked in the template for strings to determine whether it is a book or an article, or whether csquotes is used. This was always kludgy and unreliable.
• Change template code to use new API for doctemplates.

Add --defaults/-d option. This adds the ability to specify a collection of default values for options in a YAML file. For example, one might define a set of defaults for letters, and then do pandoc -d letter myletter.md -o myletter.pdf. See the documentation of this feature in MANUAL.txt.

Raise error on unsupported extensions (#4338).

The --list-extensions[=FORMAT] option now lists only extensions that affect the given FORMAT.

Add -L option as shortcut for --lua-filter.

Add --shift-heading-level-by option and deprecate --base-heading-level (#5615). The new option does everything the old one does, but also allows negative shifts. It also promotes the document metadata (if not null) to a level-1 heading with a +1 shift, and demotes an initial level-1 heading to document metadata with a -1 shift. This supports converting documents that use an initial level-1 heading for the document title.

Allow --metadata-file to be used repeatedly to include multiple metadata files (Owen McGrath, #5702). Values in files specified first will be overridden by those in later files.

--ascii now uses numerical hex character references (#5718).

Allow PDF output to stdout (#5751). PDF output now behaves like other binary formats: it will not be output to the terminal, but can be sent to stdout using either -o - or a pipe. The intermediate format will be determined based on the setting of --pdf-engine.

Make some writers sensitive to ‘unlisted’ class on headings (#1762). If this is present on a heading with the ‘unnumbered’ class, the heading won’t appear in the TOC. This class has no effect if ‘unnumbered’ is not also specified. This affects HTML-based writers (including slide shows and EPUB), LateX (including beamer), RTF, and PowerPoint. Other writers do not yet support unlisted.

Fix gfm_auto_identifiers behavior with emojis (#5813). Note that we also now use emoji names for emojis when ascii_identifiers is enabled.

When --ipynb-output is used with the default “best” format, strip ANSI escape codes for non-ipynb output (#5633). These cause problems in many formats, including LaTeX.

Don’t look for template files remotely for remote input (#5579). Previously pandoc would look for the template at a remote URL when a URL was used for the input file, instead of taking it from the data directory.

Allow combining -Vheader-includes and --include-in-header (#5904). Previously header-includes set as a variable would be clobbered by material included using --include-in-header.

Change merge behavior for metadata. Previously, if a document contained two YAML metadata blocks that set the same field, the conflict would be resolved in favor of the first. Now it is resolved in favor of the second (due to a change in pandoc-types). This makes the behavior more uniform with other things in pandoc (such as reference links and --metadata-file).

Don’t add a newline to fragment output if there’s already one.

Change exit codes and document in MANUAL.txt:

• PandocAppError was 1, is now 4
• PandocOptionError was 2, is now 6
• PandocMakePDFError was 65, is now 66

Switch to new pandoc-types and use Text instead of String [API change]. (Christian Despres, #5884).

HTML reader:

• Better handling of <q> with cite attribute (#5798, Ole Martin Ruud). If a <q> tag has a cite attribute, we interpret it as a Quoted element with an inner Span.
• Add support for HTML <samp> element (#5792, Amogh Rathore). The <samp> element is parsed as Code with class sample.
• Add support for HTML <var> element (#5799, Amogh Rathore). The <var> element is parsed as Code with class variable.
• Add support for <mark> elements (Florian B, #5797). Parse <mark> elements from HTML as Spans with class mark.
• Add support for <kbd> elements, parsing them as Span with class kbd (Daniele D’Orazio, #5796).
• Add support for <dfn>, parsing this as a Span with class dfn (#5882, Florian Beeres).

Markdown reader:

• Headers: don’t parse content over newline boundary (#5714).
• Handle inline code more eagerly within lists (Brian Leung, #5627).
• Removed some needless lookaheads.
• Don’t parse footnote body unless extension enabled.
• Fix small super/subscript issue (#5878). Superscripts and subscripts cannot contain spaces, but newlines were previously allowed (unintentionally). This led to bad interactions in some cases with footnotes. With this change newlines are also not allowed inside super/subscripts.
• Use take1WhileP for str, table row. This yields a small but measurable performance improvement.

LaTeX reader:

• Fix parsing of optional arguments that contain braced text (#5740).
• Don’t try to parse includes if raw_tex is set (#5673). When the raw_tex extension is set, we just carry through \usepackage, \input, etc. verbatim as raw LaTeX.
• Properly handle optional arguments for macros (#5682).
• Fix \\ in \parbox inside a table cell (#5711).
• Improve withRaw so it can handle cases where the token string is modified by a parser (e.g. accent when it only takes part of a Word token) (#5686). This fixes a bug that caused the ends of certain documents to be dropped.
• Handle \passthrough macro used by latex writer (#5659).
• Support tex \tt command (#5654).
• Search for image with list of extensions like latex does, if an extension is not provided (#4933).
• Handle \looseness command values better (#4439).
• Add mbox and hbox handling (Vasily Alferov, #5586). When +raw_tex is enabled, these are passed through literally. Otherwise, they are handled in a way that emulates LaTeX’s behavior.
• Properly handle \providecommand and \provideenvironment (#5635). They are now ignored if the corresponding command or environment is already defined.
• Support epigraph command in LaTeX Reader (oquechy, #3523).
• Ensure that expanded macros in raw LaTeX end with a space if the original did (#4442).
• Treat ly environment from lilypond as verbatim (Urs Liska, #5671).
• Add tikzcd to list of special environments (Eigil Rischel). This allows it to be processed by filters, in the same way that one can do for tikzpicture.

Roff reader:

• Better support for while.
• More improvements in parsing conditionals.
• Fix problem parsing comments before macro.
• Improve handling of groups.
• Better parsing of groups (#5410). We now allow groups where the closing \\} isn’t at the beginning of a line.

RST reader:

• Keep name property in imgAttr (Brian Leung, #5619).
• Fixed parsing of indented blocks (#5753). We were requiring consistent indentation, but this isn’t required by RST.
• Use title, not admonition-title, for admonition title. This puts RST reader into alignment with docbook reader.
• Don’t strip final underscore from absolute URI (#5763).
• Avoid spurious warning when resolving links to internal anchors ending with _ (#5763).

Org reader:

• Accept ATTR_LATEX in block attributes (Albert Krewinkel, #5648). Attributes for LaTeX output are accepted as valid block attributes; however, their values are ignored.
• Modify handling of example blocks (Brian Leung, #5717).
• Allow the -i switch to ignore leading spaces (Brian Leung).
• Handle awkwardly-aligned code blocks within lists (Brian Leung). Code blocks in Org lists must have their #+BEGIN_ aligned in a reasonable way, but their other components can be positioned otherwise.
• Fix parsing of empty comment lines (#5856, Albert Krewinkel). Comment lines in Org-mode can be completely empty.

Muse reader (Alexander Krotov):

• Add RTL support (#5551).
• Do not allow closing asterisks to be followed by *.
• Do not split series of asterisks into symbols and emphasis (#5821).
• Do not terminate emphasis on * not followed by space.

DokuWiki reader:

• Parse markup inside monospace (’’) (#5916, Alexander Krotov).

Docx reader:

• Move style-parsing-specific code to a new unexported module, Text.Pandoc.Readers.Docx.Parse.Styles (Nikolay Yakimov).
• Move StyleMap to docx writer (Nikolay Yakimov).
• Only use LTR when it is overriding BiDi setting (#5723, Jesse Rosenthal). The left-to-right direction setting in docx is used in the spec only for overriding an explicit right-to-left setting. We only process it when it happens in a paragraph set with BiDi. This is especially important for docs exported from Google Docs, which explicitly (and unnecessarily) set rtl=0 for every paragraph.
• Fix list number resumption for sublists (#4324). The first list item of a sublist should not resume numbering from the number of the last sublist item of the same level, if that sublist was a sublist of a different list item.

Docbook reader:

• Richer parse for admonitions (Michael Peyton Jones, #1234). Instead of parsing admonitions as blockquotes, we now parse them as Divs with an appropriate class. We also handle titles for admonitions as a nested Div with the “title” class.
• Fix nesting of chapters and sections (#5864, Florian Klink, Félix Baylac-Jacqué).
• Fix bug with entities in mathphrase element (#5885).

MediaWiki reader:

• Skip optional {{table}} template (#5757).

LaTeX reader:

• Fix dollar-math parsing to ensure that space is left between a control sequence and a following letter (#5836).
• In untokenize, ensure space between control sequence and following letter (#5836).
• Don’t omit macro definitions defined in the preamble. These were formerly omitted (though they still affected macro resolution if latex_macros was set). Now they are included in the document body.
• Parse macro definitions as raw LaTeX when latex_macros is disabled. (When latex_macros is enabled, we omit them, since pandoc is applying the macros itself.)
• Fix a hang/memory leak in certain circumstances (#5845).
• Text.Pandoc.Readers.LaTeX.Parsing: add [Tok] parameter to rawLaTeXParser. This allows us to repeat retokenizing unnecessarily in e.g. rawLaTeXBlock.
• Add KOMA-Script metadata commands (#5910, Andrew Dunning). Add all titling commands to existing definition for \dedication.
• Parse \micro siunitx unit command (#5921, Jose Luis Duran).

Markdown writer:

• Ensure proper nesting when we have long ordered list markers (#5705).
• Make plain output plainer (#5741). Previously we used the following Project Gutenberg conventions for plain output: extra space before and after level 1 and 2 headings, all-caps for strong emphasis, underscores surrounding regular emphasis. Now these conventions are used only when the gutenberg extension is enabled. By default, Strong and Emph are rendered without special formatting, and headings are rendered without special formatting, and with only one blank line following. To restore the former behavior, use -t plain+gutenberg.
• Prefer using raw_attribute when enabled (#4311). The raw_attribute will be used to mark raw bits, even HTML and LaTeX, and even when raw_html and raw_tex are enabled, as they are by default. To get the old behavior, disable raw_attribute in the writer.
• Prefer pipe_tables to raw HTML even when we must lose width information (#2608, #4497).
• Improve escaping in list items (#5918).

AsciiDoc writer:

• Don’t include + in code blocks for regular asciidoc. This is asciidoctor-specific.
• Handle admonitions (#5690).

LaTeX writer:

• Add thin space when needed in LaTeX quote ligatures (#5684).
• Use \hspace{0pt} for 0-width space U+200B (#5756).
• Use cslreferences environment for csl bibliographies. This allows bibliographies to receive special formatting. The template now contains definition of this environment (enabled only when CSL is used). It also defines a \cslhangindent length. This is set to 2em by default when the bibliography style specifies a hanging indent. To override the length, you can use e.g. \setlength{\cslhangindent}{7em} in header-includes. See jgm/pandoc-citeproc#410.
• Strip off {} around locator for biblatex/natbib output (#5722).
• Fix line breaks at start of paragraph (#3324). Previously we just omitted these. Now we render them using \hfill\break instead of \\. This is a revision of a PR by @sabine (#5591) who should be credited with the idea.
• We no longer look in the template or header-includes to see if a book or article documentclass is used, or to see whether the csquotes package is used. To use csquotes for LaTeX, set csquotes in your variables or metadata. To specify a book style, use the documentclass variable or --top-level-division.
• Fix horizontal rule (#5801). We change to use 0.5pt rather than \linethickness, which apparently only ever worked “by accident” and no longer works with recent updates to texlive.

ConTeXt writer:

• Add option to include source files in ConTeXt PDFs (Tristan Stenner, #5578). The metadata field or variable (includesource) can be set to attach the source documents to the resulting PDF.
• Customizable type of PDF/A for the ConTeXt writer (Karl Pettersson, #5608). The pdfa variable may now be set in metadata. Also updated color profile settings in accordance with ConTeXt wiki, and made ICC profile and output intent for PDF/A customizable using pdfaiccprofile and pdfaintent.
• Unit tests: adjust code property to avoid an irrelevant failure involving inline code with two consecutive newlines.
• Set csl-hanging-indent variable if needed.
• Use special environment for CSL references.
• Use braces, not start/stop, for inline language tags. This prevents unwanted gobbling of spaces.

HTML writer:

• Use numeric character references with --ascii (#5718). Previously we used named character references with html5 output. But these aren’t valid XML, and we aim to produce html5 that is also valid XHTML (polyglot markup). (This is also needed for epub3.)
• Ensure that line numbers in code blocks get id-prefix (#5650).
• Ensure TeX formulas are rendered correctly (Philip Pesca, #5658). The web service passed in to --webtex may render formulas using inline or display style by default. Prefixing formulas with the appropriate command ensures they are rendered correctly.
• Render inline formulas correctly with --webtex (Philip Pesca, #5655). We add \textstyle to the beginning of the formula to ensure it will be rendered in inline style.
• Pass through aria- attributes to HTML5 (#5642).
• Render a Quoted element with an inner Span with cite attribute using a <q> tag (#5798, Ole Martin Ruud).
• Render a Span with class mark using the <mark> element (Florian Beeres, #5797).
• Render Span with class dfn using <dfn> element (Florian Beeres, #5882).
• Render Span with class kbd using <kbd> element (Daniele D’Orazio, #5796).
• Render Code with class variable using <var> element (Amogh Rathore, #5799).
• Render Code with class sample using <samp> element (Amogh Rathore, #5799).

EPUB writer:

• Improve splitting into chapters (#5761), using makeSection.
• Avoid issuing warning multiple times when title not set (see #5760).
• Use svg tag wrapper for cover image (#5638). In addition, the code generating the image has been moved to the template, to make it more customizable. NOTE: Those who use custom EPUB templates will need to adjust their templates, adding the code to generate the cover image. (Previously this was just inserted into ‘body’.)
• Improve toChapters, making it work better if there are Divs around sections.
• Add support for EPUB2 covers (blmage, #3992).
• Do not override existing “fileN” medias when writing to EPUB format (blmage, #4206).
• Ensure that lang variable is set on all chapters (so that it will add an xml:lang attribute on the html element).

RST writer:

• Removed remnants of admonition-title.
• Fix handling of :align: on figures and images (#4420). When the image has the align-right (etc.) class, we now use an :align: attribute.
• Improve spacing for tables with no width information (#5899). If a simple table would be too wide, we use a grid table.
• Fix backslash escaping after strings (Albert Krewinkel, #5906). The check whether a complex inline element following a string must be escaped, now depends on the last character of the string instead of the first.
• Ensure there’s a blank line before tables (#5898).

Dokuwiki writer:

• Handle mixed lists without HTML fallback (#5107).

XWiki writer:

• Fix multiline table (Zihang Chen, #5683).

Muse writer:

• Add RTL support (Alexander Krotov, #5551).

Man writer:

• Suppress non-absolute link URLs (#5770). Absolute URLs are still printed in parentheses following the link text, but relative URLs are suppressed (just as internal links starting with ‘#’ always have been).
• Improved definition list term output. Now we boldface code but not other things. This matches the most common style in man pages (particularly option lists).

Ms writer:

• Use .LP instead of .PP for line block (#5588).
• Use boldface for definition terms in DefinitionLists.

JATS writer:

• Do not emit empty <fn-group> (Mauro Bieg, #5595).
• Update template to v1.1dtd (#5632, Arfon Smith).
• Update data/jats.csl to avoid commas between editor name-part elements. (#5629)
• Add abstract to template (Mauro Bieg).

TEI writer:

• Don’t strip hash from internal links (#5922).

Jira writer:

• Remove escapeStringForJira for code blocks (Jan-Otto Kröpke).
• Remove extraneous newline after single-line block quotes (#5858, Albert Krewinkel).
• Improve escaping of special characters, using backslash escapes instead of HTML entities (#5858, Albert Krewinkel).

OpenDocument writer:

• Avoid duplicate attributes (#4634). We use the innermost attribute in nested cases.
• If native_numbering extension is set, use native OpenDocument enumeration for figures and tables (Nils Carlson).
• Place caption before table (#5681, Dmitry Pogodin).

ODT writer:

• Add a test for MathML formulas in ODT documents (blmage).
• Improve the parsing of frames in ODT documents (blmage).

Docx writer:

• Make handling of styles more robust in localized versions of Word (Nikolay Yakimov, #5523, #5052, #5074). We now use style names, not ids, for assigning semantic meaning, since the ids can change depending on the locale. Style name comparisons are case-insensitive, since those are case-insensitive in Word. Since docx style names can have spaces in them, and pandoc-markdown classes can’t, anywhere when style name is used as a class name, spaces are replaced with ASCII dashes -. Code styles, i.e. “Source Code” and “Verbatim Char” now honor style inheritance. Docx Reader now honours “Compact” style (used in Pandoc-generated docx). The side-effect is that “Compact” style no longer shows up in docx+styles output. Styles inherited from “Compact” will still show up.
• Re-use Readers.Docx.Parse for StyleMap (#5766, Nikolay Yakimov).
• Internal improvements and code simplification (Nikolay Yakimov).
• Preserve built-in styles in DOCX with custom style (Ben Steinberg, #5670). This change prevents custom styles on divs and spans from overriding styles on certain elements inside them, like headings, blockquotes, and links. On those elements, the “native” style is required for the element to display correctly. This change also allows nesting of custom styles; in order to do so, it removes the default “Compact” style applied to Plain blocks, except when inside a table.
• Add proofState to list of elements carried over from settings.xml in the reference.docx (Krystof Beuermann, #5703).
• Change order of ilvl and numId in document.xml (Agustín Martín Barbero, #5645). Also, make list para properties go first. This reordering of properties shouldn’t be necessary but it seems Word Online does not understand the docx correctly otherwise.

PowerPoint writer:

• Code formatting is now context dependent (Jeroen de Haas, #5573). This commit alters the way in which the Powerpoint writer treats inline code and code blocks. Inline code is now formatted at the same size as the surrounding text. Code blocks are now given a margin and font size according to their level. Furthermore this commit allows changing the font with which code is formatted via the monofont option.
• Start numbering at appropriate numbers (Jesse Rosenthal, #5709). Starting numbers for ordered lists were previously ignored. Now we specify the number if it is something other than 1.

Text.Pandoc.Parsing:

• Add manyChar, many1Char, manyTillChar, many1TillChar, many1Till, manyUntil, mantyUntilChar: these are like their unsuffixed counterparts but pack some or all of their output (Christian Despres, #5884).
• Add stateAllowLineBreaks to ParserState [API change].
• Fix inline parsing in grid table cells (#5708).
• Change type of setLastStrPos so it takes a Maybe SourcePos rather than a SourcePos [API change].
• Make parseFromString' and gridTableWith and gridTableWith' polymorphic in the parser state, constraining it with HasLastStrPosition [API change].
• parseFromString': reset stateLastStrPos to Nothing before parse.
• Rename takeWhileP -> take1WhileP and clean it up. (It doesn’t match the empty sequence.)

Text.Pandoc.PDF:

• For PDFs via HTML, ensure temp file is deleted even if the pdf program is not found (#5720).
• Better detection of a Cygwin environment (#5451).
• Don’t assume tex log file is UTF8-encoded (#5872). Fall back to latin1 if it can’t be read as UTF-8.

Text.Pandoc.Extensions:

• Export new function getAllExtensions, which returns the extensions that affect a given format (whether enabled by default or not) [API change].
• Change type of parseFormatSpec from Either ParseError (String, Extensions -> Extensions) to Either ParseError (String, [Extension], [Extension]) [API change].
• Add Ext_gutenberg constructor to Extension [API change].
• Add Ext_native_numbering constructor to Extension [API change] (Nils Carlson).

Text.Pandoc.Readers, Text.Pandoc.Writers:

• Change type of getReader and getWriter so they return a value in the PandocMonad instance rather than an Either [API change]. Exceptions for unknown formats and unsupported extensions are now raised by these functions.

Text.Pandoc.App

• Change optMetadataFile type from Maybe FilePath to [FilePath] (Owen McGrath, #5702) [API change].

Text.Pandoc.Logging:

• Add CouldNotDeduceFormat constructor to LogMessage [API change]. Issue this warning when we’re falling back to markdown or html because we don’t recognize the extension of the input or output files.
• Clarify warning for missing title (#5760).
• Add UnusualConversion constructor to LogMessage [API change] (Mauro Bieg, #5736). Emit warning on -f latex -o out.pdf.

Lua filters:

• Improve function documentation (Albert Krewkinkel).
• Traverse nested blocks and inlines in correct order (Albert Krewinkel, #5667). Traversal methods are updated to use the new Walk module so that sequences with nested Inline (or Block) elements are traversed in the order in which they appear in the linearized document.
• New unexported module Text.Pandoc.Lua.Walk (Albert Krewinkel). Lua filters must be able to traverse sequences of AST elements and to replace elements by splicing sequences back in their place. Special Walkable instances can be used for this; those are provided in a new module Text.Pandoc.Lua.Walk.
• Attr values can now be given as normal Lua tables (Albert Krewinkel, #5744). This can be used as a convenient alternative to constructing Attr values with pandoc.Attr. Identifiers are taken from the id field, classes must be given as space separated words in the class field. All remaining fields are included as attributes. With this change, the following lines now create equal elements:

  pandoc.Span('test', {id = 'test', class = 'a b', check = 1})
  pandoc.Span('test', pandoc.Attr('test', {'a','b'}, {check = 1}))
  

  This also works when using the attr setter:

  local span = pandoc.Span 'text'
  span.attr = {id = 'test', class = 'a b', check = 1}
  

  Furthermore, the attributes field of AST elements can now be a plain key-value table even when using the attributes accessor:

  local span = pandoc.Span 'test'
  span.attributes = {check = 1}   -- works as expected now
  

• Export make_sections, remove hierarchicalize. Lua filters that use hierarchicalize will need to be rewritten to use make_sections.
• Add a clone() method to all AST elements (Albert Krewinkel, #5568).
• Fix Lua function names in pandoc.system (niszet). Change get_current_directory to get_working_directory and with_temp_directory to with_temporary_directory, to conform to the manual.

Text.Pandoc.Error:

• Add constructors PandocUnknownReaderError, PandocUnknownWriterError, PandocUnsupportedExtensionError. [API change].
• Better message for PandocShouldNeverHappenError.
• Better message for PandocTemplateError.

Text.Pandoc.Emoji:

• Update emoji list (#5666). Done using new tools/emojis.hs, which uses the list from the gem GitHub uses. Future updates can be done with this tool.

Text.Pandoc.PDF:

• Pass value of --dpi to rsvg-convert when converting SVG to PDF in the process of creating a PDF (#5721).

Text.Pandoc.Shared:

• Replace Element and makeHierarchical with makeSections. Now that we have Divs, we can use them to represent the structure of sections, and we don’t need a special Element type. makeSections reorganizes a block list, adding Divs with class section around sections, and adding numbering if needed. This change also fixes some longstanding issues recognizing section structure when the document contains Divs (#3057, see also #997).
• Remove Element type [API change]
• Remove makeHierarchicalize [API change]
• Add makeSections [API change]
• Export deLink [API change]
• Make filterIpynbOutput strip ANSI escapes from code in output for non-ipynb formats, when the default “best” option is used with --ipynb-output (#5633).
• Fix camelCaseToHyphenated so it handles ABCDef better.
• Improve isTight (#5857). If a list has an empty item, this should not count against its being a tight list.
• Export htmlSpanLikeElements [API change] (Daniele D’Orazio, #5796). This is a mapping of HTML span-like elements that are internally represented as a Span with a single class.
• Change the implementation of htmlSpanLikeElements to retain classes and attributes (#5882, Florian Beeres).

Text.Pandoc.Slides: recognize content in Divs when determining slide level.

Text.Pandoc.SelfContained:

• Omit content-type on type attribute for <style> (#5725). It doesn’t seem to be valid for HTML5, and as a result Chrome ignores the style element.

Text.Pandoc.Pretty has been removed [API change]. We now use the new external doclayout module instead.

Text.Pandoc.Writers.Shared:

• Remove metaToJSON, metaToJSON' [API change].
• Modify addVariablesToContext, defField, setField, getField, resetField to work with Context rather than JSON values. [API change]
• Export new function endsWithPlain [API change].
• Change gridTables so it does better at keeping the widths of columns (#4320) and does better at figuring out column widths when no widths are given (#5899).

Text.Pandoc.Options

• Change type of writerTemplate to Maybe Template instead of Maybe String.
• Change To/FromJSON instances for HTMLMathMethod, CiteMethod, ObfuscationMethod, TrackChanges, WrapOption, TopLevelDivision, ReferenceLocation, HTMLSlideVariant (#5790). In each case we use lowercase (or hyphenated lowercase) for constructors to line up more closely with command-line option values. This is a breaking change for those who manually decode or encode JSON for these data types (e.g. for ReaderOptions or WriterOptions).

Text.Pandoc.Filters:

• Add FromYAML instance for Filter.
• applyFilters: Add and apply filters in order (not reversed) This changes applyFilters from Text.Pandoc.Filter so that it does a left fold rather than a right fold, applying the filters in the order listed.

Text.Pandoc.XML:

• Change toEntities to emit numerical hex character references (#5718).

Text.Pandoc.Highlighting:

• Add additional listings languages (Wandmalfarbe).

Text.Pandoc.MediaBag:

• Some of the types using Strings were switched to use FilePath instead (Christian Despres, #5884).

Text.Pandoc.Templates:

• Don’t import/export varListToJSON [API change]. It is removed in doctemplates >= 0.3.
• Change type of renderTemplate' [API change]. Return value is now Text rather than being polymorphic. This makes room for upcoming removal of the TemplateTarget class from doctemplates.

Text.Pandoc.App.Opt [API changes]:

• More convenient To/FromJSON instances. Make the field names like strip-empty-paragraphs rather than optStripEmptyParagraphs. Anyone who is using JSON serialization of Opt will need to adjust things accordingly.
• Change optHighlightStyle to a Maybe String instead of Maybe Style. Do the parsing/loading of themes later, after option parsing.
• Remove optBaseHeaderLevel from Opt. We now just use optShiftHeadingLevelBy, to avoid redundancy.
• Change optShiftHeadingLevel to optShiftHeadingLevelBy to match the option.
• Custom FromJSON instance for LineEnding, so either CRLF or crlf will work.
• Change optVariables from [(String, String)] to Context Text.
• Change optMetadata to Meta, to allow structured values. The current behavior of the --metadata option stays the same.
• Rename optReader, optWriter as optFrom, optTo.
• Add FromYAML instances to Opt and to all subsidiary types.
• Rename optMetadataFile to optMetadataFiles.
• Rename optPDFEngineArgs to optPDFEngineOpts.
• Rename optWrapText to optWrap.
• Add IpynbOutput enumerated type: use this instead of a string for optIpynbOutput.
• Change optInputFiles to a Maybe [FilePath] (#5888) [API change]. Nothing means: nothing specified. Just [] means: an empty list specified (e.g. in defaults).
• List fields in Opt so they aren’t reversed (#5881) [API change]. Previously optIncludeInHeader, etc. were in reverse order.
• The sourcefile variable is now always a list. It used to be sometimes a string, sometimes a list (when there was more than one).

Template changes:

• default.latex: added code for cslreferences environment, to be used for pandoc-citeproc references. A csl-hanging-indent variable (set automatically if there is a hanging-ident class on the references Div) controls whether contents of this environment receive a hanging indent.
• default.latex: Add space as default option for xeCJK, so that spaces between words are preserved (#5855, jeongminkim-islab). This is necessary for Korean.
• default.latex: Remove include of grffile (#5848). This package used to be needed for proper handling of image filenames containing periods (in addition to the period before the extension). It no longer works with the latest LaTeX kernel and graphicx, so we have removed it. Future versions of graphicx will handle these filenames without the need for grffile.
• default.context: add a saner default for page numbers. Previously they appeared centered at the top of the page; now we put them centered at the bottom, unless the pagenumbering variable is set.
• default.context: define a start-stop-pair cslreferences to allow for hanging indents in the bibliography (#5875, Denis Maier).
• default.ms: update defaults. Use Palatino font, use slightly wider interparagraph space, don’t indent paragraphs, and put page numbers on the bottom. This brings ms output closer to default LaTeX output.
• default.revealjs: add navigationMode (Mauro Bieg, #5657).
• default.muse: handle multiple authors better.
• docbook4, docbook5 templates: add indentation to body.
• HTML-based templates: use styles.html partial to avoid code duplication.
• HTML-based templates: change indentation of styles in template.

reference.docx (#5820):

• Change Block Text (block quote) style so that the same font is used as in the body text, and the block text is indented left and right.
• All headings now have a uniform color.
• Level-1 headings no longer set w:themeShade="B5".
• Level-2 headings are now 14 point rather than 16 point.
• Level-3 headings are now 12 point rather than 14 point.
• Level-4 headings are italic rather than bold.

epub.css: Add CSS for hanging-indent div to support pandoc-citeproc’s new hanging indents.

pandoc.cabal:

• Repeat ghc-options in all stanzas.
• Remove conditionals for ghc < 8 (Albert Krewinkel, #5693).
• Compile with -Wcpp-undef and -fhide-source-paths when possible (Albert Krewinkel).
• Add cabal.project to extra-source-files (Albert Krewinkel).
• Add dependency on skylighting-core (#5729). Even though it shouldn’t be needed, some tools require it.
• Require latest pandoc-types, texmath, skylighting, haddock-library.
• Ensure TemplateHaskell is added to other-extensions when it is used (Vanessa McHale, #5728).
• Remove derive_json_via_th flag; always use TH. This cuts down on code duplication and reduces the chance for errors. See #4083.

Makefile:

• Add ghci target.
• Add ghcid target.
• Remove references to obsolete flag (#5694).

Benchmarks: fix failure on ipynb.

Use MathJax 3 (zorkow).

KaTeX math: respect classoption=fleqn variable, bump KaTeX version to 0.11.1 (#5815, Mauro Bieg).

Fix redundant constraint compiler warnings (Pete Ryland, #5625).

Use throwError instead of fail when appropriate.

Use Prelude.fail to avoid ambiguity with fail from GHC.Base.

Add diff-zip.sh to tools (John MacFarlane, Agustín Martín Barbero). This is intended to make it easier to test differences in zip containers, like epub, docx, or odt.

Add .gitattributes (#5747). This ensures that the golden files in test/fb2/reader/ don’t have newlines converted. This should fix a test failure on GitHub CI with Windows.

Reorder options in --help.

Revise code for HsYAML-0.2.0.0 (@vijayphoenix, #5704).

Remove blank line in code example in Haddocks (Leif Metcalf, #5679).

Fix trypandoc with getReader/getWriter changes.

Allow building pandoc with GHC 8.8.

linux tarball: add architecture -amd64 to filename. Now it will be: pandoc-VERSION-linux-ARCH.tar.gz.

MANUAL.txt:

• Add section for exit codes.
• Document some pptx limitations in slide show section: No incremental display (#5689). No pause with . . . (#5701).
• reveal.js flags (Mauro Bieg, #5653).
• Document addition of data- prefix to unknown attributes in HTML5.
• Link to YAML spec (Mauro Bieg, #5687).
• Fix capitalization of “Linux” (#5859, Marcus Stollsteimer).
• Use a table for exit codes.
• Put all template variable docs into one section.
• Use ATX headers consistently.
• Add fuller documentation of templates (#5779), including new template syntax, partials, etc.
• Add documentation for the variable hyperrefoptions (Wandmalfarbe).
• Clarify when macro definitions are passed as raw latex. In Markdown input, they are always passed through. In LaTeX, only if latex_macros is disabled.
• Clarify that --dpi provides a default and doesn’t override dpi values specified in the images themselves (#5721).
• Document how to use custom writers with --standalone (#5866).
• Clarify --preserve-tabs default.

INSTALL.md:

• Fix instructions for libicu.
• Add Void Linux instructions (Volodymyr Kozachnyskyi).

CONTRIBUTING.md:

• Add information on tests (Agustín Martín Barbero, #5652).
• Add information about command test naming to CONTRIBUTING (Florian B).

Fix typos in changelog and comments (#5896, Brian Wignall).

doc/lua-filters.md:

• Fix mistakes in mediabag module docs (#5851, Albert Krewinkel).
• Improve metadata replacement example in lua-filters doc (#5851).
• Mention which Lua version is shipped with pandoc (Albert Krewinkel, #5892).

Add jira (Atlassian’s Jira wiki markup) as output format (#2497, Albert Krewinkel).

Add tex_math_dollars to multimarkdownExtensions (#5512). This form is now supported in multimarkdown, in addition to tex_math_double_backslash.

Fix --self-contained so it works when output format has extensions. Previously if you used --self-contained with html-smart or html+smart, it wouldn’t work.

Add template variable curdir with working directory from which pandoc is run (#5464).

Markdown reader: don’t create implicit reference for empty header (#5549).

Muse reader: allow images inside link descriptions (Alexander Krotov).

HTML reader: epub related fixes.

• With epub extensions, check for epub:type in addition to type.
• Fix problem with noteref parsing which caused block-level content to be eaten with the noteref.
• Rename pAnyTag to pAny.
• Refactor note resolution.
• Trim definition list terms (Alexander Krotov).

LaTeX reader:

• Add braces when resolving \DeclareMathOperator (#5441). These seem to be needed for xelatex but not pdflatex.
• Allow newlines in \mintinline.
• Pass through unknown listings language as class (#5540). Previously if the language was not in the list of languages supported by listings, it would not be added as a class, so highlighting would not be triggered.
• rawLaTeXInline: Include trailing {}s in raw latex commands (#5439). This change affects the markdown reader and other readers that allow raw LaTeX. Previously, trailing {} would be included for unknown commands, but not for known commands. However, they are sometimes used to avoid a trailing space after the command. The chances that a {} after a LaTeX command is not part of the command are very small.

MediaWiki reader: handle multiple attributes in table row (#5471, chinapedia).

Docx reader: Add support for w:rtl (#5545). Elements with this property are put into Span inlines with dir="rtl".

DocBook reader: Issue IgnoredElement warnings.

Org reader (Albert Krewinkel):

• Fix planning elements in headers level 3 and higher (#5494). Planning info is now always placed before the subtree contents. Previously, the planning info was placed after the content if the header’s subtree was converted to a list, which happens with headers of level 3 and higher per default.
• Omit, but warn about unknown export options. Unknown export options are properly ignored and omitted from the output.
• Prefer plain symbols over math symbols (#5483). Symbols like \alpha are output plain and unemphasized, not as math.
• Recognize emphasis after TODO/DONE keyword (#5484).

FB2 reader:

• Skip unknown elements rather than throwing errors (#5560). Sometimes custom elements are used (e.g. id element inside author); previously the reader would halt with an error. Now it skips the element and issues an IgnoredElement warning.
• Parse notes (#5493, Alexander Krotov).
• Internal improvements (Alexander Krotov).

OpenDocument writer: Roll back automatic figure/table numbering (#5474). This was added in pandoc 2.7.2, but it makes it impossible to use pandoc-crossref. So this has been rolled back for now, until we find a good solution to make this behavior optional (or a creative way to let pandoc-crossref and this feature to coexist).

New module Text.Pandoc.Writers.Jira, exporting writeJira [API change] (Albert Krewinkel).

EPUB writer:

• Don’t include ‘landmarks’ if there aren’t any. Previously we could get an empty ol element, which caused validation errors with epubcheck.
• Ensure unique ids for styleesheets in content.opf (#5463).
• Make stylesheet link compatible with kindlegen (#5466, Eric Schrijver). Pandoc omitted type="text/css" from both <style> and <rel="stylesheet"> elements in all templates, which is valid according to the spec. However, Amazon’s kindlegen software relies on this attribute on <link> elements when detecting stylesheets to include.

HTML writer:

• Output video and audio elements depending on file extension of the image path (Mauro Bieg).
• Emit empty alt tag in figures (#5518, Mauro Bieg). The same text is already in the and screen-readers would read it twice, see #4737.
• Don’t add variation selector if it’s already there. This fixes round-trip failures.
• Prevent gratuitous emojification on iOS (#5469). iOS chooses to render a number of Unicode entities, including ‘↩’, as big colorful emoji. This can be defeated by appending Unicode VARIATION SELECTOR-15’/‘VARIATION SELECTOR-16’. So we now append this character when escaping strings, for both ‘↩’ and ‘↔’. If other characters prove problematic, they can simply be added to needsVariationSelector. + Add class="heading" to level 7+ Headers rendered as <p> elements (#5457).

RST writer: treat Span with no attributes as transparent (#5446). Previously an Emph inside a Span was being treated as nested markup and ignored. With this patch, the Span is just ignored.

LaTeX writer:

• Include inline code attributes with --listings (#5420).
• Don’t produce columns environment unless beamer (#5485).
• Fix footnote in image caption. Regression: the fix for #4683 broke this case.
• Don’t highlight code in headings (#5574). This causes compilation errors.
• Use \mbox to get proper behavior inside \sout (#5529).

EPUB writer: Fix document section assignments (#5546). For example, introduction should go in bodymatter, not frontmatter, and epigraph, conclusion, and afterward should go in bodymatter, not backmatter. For the full list of assignments, see the manual.

Markdown writer:

• Add backslashes to avoid unwanted interpretation of definition list terms as other kinds of block (#554).
• Ensure the code fence is long enough (#5519). Previously too few backticks were used when the code block contained an indented line of backticks. (Ditto tildes.)
• Handle labels with integer names (Jesse Rosenthal, #5495). Previously if labels had integer names, it could produce a conflict with auto-labeled reference links. Now we test for a conflict and find the next available integer. This involves adding a new state variable stPrevRefs to keep track of refs used in other document parts when using --reference-location=block|section

Textile writer: fix closing tag for math output (Albert Krewinkel). Opening and closing tag for math output match now.

Org writer: always indent src blocks content by 2 spaces (#5440, Albert Krewinkel). Emacs always uses two spaces when indenting the content of src blocks, e.g., when exiting a C-c ' edit-buffer. Pandoc used to indent contents by the space-equivalent of one tab, but now always uses two spaces, too.

Asciidoc writer:

• Use `+...+` form for inline code. The old `a__b__c` yields emphasis inside code in asciidoc. To get a pure literal code span, use `+a__b__c+`.
• Use proper smart quotes with asciidoctor (#5487). Asciidoctor has a different format for smart quotes.
• Use doubled ## when necessary for spans (#5566).
• Ensure correct nesting of strong/emph (#5565): strong must be the outer element.

JATS writer:

• Wrap elements with p when needed (#5570). The JATS spec restricts what elements can go inside fn and list-item. So we wrap other elements inside <p specific-use="wrapper"> when needed.
• Properly handle footnotes (#5511) according to “best practice.” (Group them at the end in <fn-group> and use <xref> elements to link them.)
• Fix citations with PMID so they validate (#5481). This includes an update to data/jats.csl.
• Ensure validity of <pub-date> by parsing the date and extracting year, month, and day, as expected. Also add an iso-8601-date attribute automatically.
• Don’t use <break> element for LineBreak. It is only allowed in a few special contexts, and not in <p> elements.
• Don’t make <string-name> a child of <string>, which is illegal.

FB2 writer:

• Do not wrap note references into <sup> and brackets (Alexander Krotov). Existing FB2 readers, such as FBReader, already display links with type=“note” as a superscript.
• Use genre metadata field (#5478).

Muse writer: do not escape empty line after <br> (Alexander Krotov).

Add unicode code point in “Missing character” warning (#5538). If the character isn’t in the console font, the message is pretty useless, so we show the code point for anything non-ASCII.

Lua: add Version type to simplify comparisons (Albert Krewinkel). Version specifiers like PANDOC_VERSION and PANDOC_API_VERSION are turned into Version objects. The objects simplify version-appropriate comparisons while maintaining backward-compatibility. A function pandoc.types.Version is added as part of the newly introduced module pandoc.types, allowing users to create version objects in scripts.

pandoc lua module (Albert Krewinkel):

• Fix deletion of nonexistent attributes (#5569).
• Better tests for Attr and AttributeList.

pandoc.mediabag lua module (Albert Krewinkel):

• Add function delete for deleting a single item.
• Add function empty for removing all entries.
• Add function items for iterating over mediabag.

Text.Pandoc.Class: Fix handling of file: URL scheme in downloadOrRead (#5517, Mauro Bieg). Previously file:/ URLs were handled wrongly and pandoc attempted to make HTTP requests, which failed.

Text.Pandoc.MIME: add mediaCategory [API change] (Mauro Bieg).

Text.Pandoc.Shared:

• Add onlySimpleTableCells [API change] (Mauro Bieg) and use this to consolidate simple-table detection (#5524). This fixes an inconsistency in the HTML reader, which did not treat tables with <p> inside cells as simple.
• metaToJSON: treat digits starting with 0 as a string, not a number (#5479). This fixes a regression in YAML metadata in pandoc 2.7.2.

Text.Pandoc.Logging: Add IgnoredElement constructor for LogMessage. SkippedConetnt doesn’t work for some XML-based readers which don’t have access to parsec source positions.

Text.Pandoc.Asciify: Add Turkish undotted-i (#5433, Mauro Bieg).

Improve output of Lua tests (#5499, Albert Krewinkel). This makes use of tasty-lua, a package to write tests in Lua and integrate the results into Tasty output. Test output becomes more informative: individual tests and test groups become visible in test output. Failures are reported with helpful error messages.

Lua: add pandoc.system module (#5468, Albert Krewinkel). The system Lua module provides utility functions to interact with the operating- and file system. E.g. print(pandoc.system.get_current_directory()) or

    pandoc.system.with_temporary_directory('tikz', function (dir)
      -- write and compile a TikZ file with pdflatex
    end)

LaTeX template: Add pdflang to hypersetup if lang is set (#5443).

beamer template: Fix using Beamer with geometry (#5505, Daniel Maslowski). Beamer already loads geometry, so we need to use the \geometry command to set geometry options.

EPUB2/3 templates: Move inline styles to default epub.css (#5466). NOTE: Those who use a custom CSS stylesheet with EPUB should add these lines:

code{ white-space: pre-wrap; }
span.smallcaps{ font-variant: small-caps; }
span.underline{ text-decoration: underline; }
q { quotes: "“" "”" "‘" "’"; }
div.column{ display: inline-block; vertical-align: top; width: 50%; }

reveal.js template:

• Updates for revealjs 3.8.0 (#5435, ebiim).
• Remove reference to head.min.js (#5448, Winnie Hellmann). NOTE: users will need to update reveal.js to at least 3.8.0 for their presentations generated with this version of pandoc to work correctly.

Text.Pandoc.PDF:

• Replace </> with literal / (#5451). We use forward-slash for a directory separator in tmpDir, even on Windows (because that’s what tex likes). So we should not put a backslash between the tmpDir and the filename on Windows. This is harmless enough in normal Windows setups, but it breaks on Cygwin. Thanks to @cc2x for noticing and diagnosing the problem.
• Allow use of -output-directory in --pdf-engine-opt (#5462). This is currently possible with mklatex and -outdir, but was not yet possible with xelatex and -output-directory.
• For PDF via ms/pdfroff, make TOC appear at beginning and in PDF bookmarks (#5553). Previously the TOC appeared at the end of the document, and was not bookmarked. To keep it at the end, add --pdf-engine-opt=--no-toc-relocation to your command line.

Fix broken links in documents (#5473, Shim Myeongseob). Also, use absolute links to pandoc.org when possible, so that the links can be followed by people viewing these documents on GitHub.

Improved sample lua tikz filter in lua-filters docs (#5445, Matthew Doty). There are three changes:

• It only processes elements which begin with \begin{tikzpicture}
• It uses pdf2svg instead of imagemagick to preserve fidelity
• The images produced have transparent backgrounds

MANUAL.txt:

• Add note about title-meta, author-meta, date-meta (#5486).
• Fix typo (#5489, Christian Krause).

add test/tables.xwiki to git and pandoc.cabal (#5498, Mauro Bieg).

Disable missing-home-modules warning in stack.yaml. Otherwise stack ghci fails.

Add XWiki writer (#1800, Derek Chen-Becker). Add Text.Pandoc.Writers.XWiki, exporting writeXWiki [API change].

Dokuwiki Reader: parse single curly brace (#5416, Mauro Bieg).

Vimwiki reader: improve handling of internal links (#5414). We no longer append .html to link targets, and we add a title wikilink. This mirrors behavior of other wiki readers. Generally the .html extension is not wanted. It may be important for output to HTML in certain circumstances, but it can always be added using a filter that matches on links with title wikilink.

If your workflow requires the current behavior, here is a lua filter that will add the .html extension:

function Link(el)
  if el.title == 'wikilink' then
    el.target = el.target .. ".html"
  end
  return el
end

ipynb reader:

• Use format ipynb for raw cell where no format given.
• Avoid introducing spurious .0 on integers in metadata.

Markdown reader: fenced div takes priority over setext header.

HTML reader: read data-foo attribute into foo (#5392). The HTML writer adds the data- prefix for HTML5 for nonstandard attributes. But the attributes are represented in the AST without the data- prefix, so we should strip this when reading HTML.

LaTeX reader: Improve autolink detection (#5340).

PowerPoint writer (Jesse Rosenthal):

• Expand builtin reference doc to model all layouts. The previous built-in reference doc had only title and content layouts. Add in a section-header slide and a two-content slide, so users can more easily modify it to build their own templates.
• Always open up in slide view. When editing a template/reference-doc, the user might be in Master view, but when producing a slide show, it is assumed that slide view will be desired.
• Remove handoutsMasterList from template presentation.xml
• Fix numerous errors in templating (#5402). Previously, some templates produced by Office 365 (MacOS) would not render with --reference-doc correctly. We now apply correct shapes for content, and build shape trees correctly.
• Make default placeholder type for template lookup.
• Apply speaker notes to metadata slide if applicable.
• Test for speaker notes after breaking header.
• Correctly handle notes after section-title header. Previously, if notes came after a section-title header (ie, a level-1 header in a slide-level=2 presentation), they would go on the next slide. This keeps them on the slide with the header.
• Internal improvements.

ipynb writer:

• Use format ipynb for raw cell where no format given. According to nbformat docs, this is supposed to render in every format. We don’t do that, but we at least preserve it as a raw block in markdown, so you can round-trip.
• Consolidate adjacent raw blocks. Sometimes pandoc creates two HTML blocks, e.g. one for the open tag and one for a close tag. If these aren’t consolidated, only one will show up in output cell.
• Fixed carry-over of nbformat from metadata.
• Preserve nbformat_minor if it’s given. This helps with round-tripping.

LaTeX writer:

• Avoid inadvertently creating ? or ! ligatures (#5407). These are upside down ? and !, resp.
• Fix footnotes in table caption and cells (#5367). This fixes a bug wherein footnotes appeared in the wrong order, and with duplicate numbers, when in table captions and cells. We now use regular \footnote commands, even in the table caption and the minipages containing cells. Apparently longtable knows how to handle this.

HTML writer: Don’t add data- prefix to RDFa attributes (#5403).

JATS writer: Ensure that plain strings go inside <pub-id> tag (#5397).

Markdown writer:

• Better rendering of numbers (#5398). If the number is integral, we render it as an integral not a float.
• Proper rendering of empty map in YAML metadata (#5398). Should be {}, not empty string.
• Properly escape attributes in Markdown writer (#5369).
• Be sure implicit figures work in list contexts (#5368). Previously they would sometimes not work: e.g., when they occurred in final paragraphs in lists that were originally parsed as Plain and converted later using PlainToPara.

Docx writer: Use w:br without attributes for line breaks (#5377). We previously added the attribute type="textWrapping", but this causes problems on Word Online.

LaTeX template (Andrew Dunning):

• Ensure correct heading/table order (#5365). Improve workaround (#1658) for tables following headings. The new solution works whether or not the indent variable is enabled.
• Remove subparagraph variable. The default is now to use run-in style for level 4 and 5 headings (\paragraph and \subparagraph). To get the previous default behavior (where these were formatted as blocks, like \subsubsection), set the block-headings variable.
• Add pandoc to PDF metadata (#5388).
• Group graphics-related code (#5389).
• Move \setstretch after front matter (#5179). Ensures that \maketitle, \tableofcontents, and so forth are not affected by changes to line spacing.

Update data/jats.csl to avoid commas between name-part elements (#5397).

Add support for golang (go) with --listings (#5427).

Text.Pandoc.Shared - improve metaToJSON behavior with numbers. We now do a better job marshalling numbers from MetaString or MetaInlines into JSON Number.

Text.Pandoc.Writers.Shared: metaValueToJSON: use Number Values for integers. Pandoc’s MetaValue doesn’t have a distinguished number type, so numbers are put in MetaStrings. If the MetaString consists entirely of digits, we convert it to a Number. We should probably consider adding a MetaNumber constructor to MetaValue, for better round-tripping with JSON etc. This change aids round-tripping in ipynb metadata fields, like toc_depth.

Text.Pandoc.Class: fetchItem: don’t treat UNC paths as protocol-relative URLs (#5127). These are paths beginning //?/UNC/....

Text.Pandoc.ImageSize: Improve pdfSize so it handles a wider range of PDFs (#4322, with help from Richard Davis).

Text.Pandoc.Pretty: avoid stack overflow by using strict sum (#5401).

Fix harmless error in file-scope code (#5422).

MANUAL.txt:

• Improve ‘header’ and ‘heading’ usage (#5423, Andrew Dunning). The term ‘header’ was being used where ‘heading’ is more appropriate.
• Add paragraph on options affecting markdown in ipynb.

stack.yaml - remove -Wmissing-home-modules This seems to cause problems with stack ghci. Remove RTS options.

Add ghc-options to cabal.project.

appveyor.yml - use ghc 8.6.4. Fixes segfault issues on Windows (#5037).

linux build process: Remove clone of pandoc-citeproc (#5366). It wasn’t being used; cabal.project specifies the version to use.

Use XDG data directory for user data directory (#3582). Instead of $HOME/.pandoc, the default user data directory is now $XDG_DATA_HOME/pandoc, where XDG_DATA_HOME defaults to $HOME/.local/share but can be overridden by setting the environment variable. If this directory is missing, then $HOME/.pandoc is searched instead, for backwards compatibility. However, we recommend moving local pandoc data files from $HOME/.pandoc to $HOME/.local/share/pandoc. On Windows the default user data directory remains the same.

Slide show formats behavior change: content under headers less than slide level is no longer ignored, but included in the title slide (for HTML slide shows) or in a slide after the title slide (for beamer). This change makes possible 2D reveal.js slideshows with content in the top slide on each stack (#4317, #5237).

Add command line option --ipynb-output=all|none|best (#5339). Output cells in ipynb notebooks often contain several different versions of an output, with different MIME types, e.g. an HTML table and a plain-text fallback. Specifying --ipynb-output=best (the default) ensures that the best version for the output format is used. all includes all versions, and none suppresses them all, leaving output cells empty.

asciidoctor is now an output format separate from asciidoc, to accommodate some minor implementation-specific differences (currently just in the treatment of display math).

Add latexmk as an option for --pdf-engine (#3195). Note that you can use --pdf-engine-opt=-outdir=bar to specify a persistent temp directory.

Markdown reader:

• Improve tight/loose list handling (#5285). Previously the algorithm allowed list items with a mix of Para and Plain, which is never wanted.
• Add newline when parsing blocks in YAML (#5271). Otherwise last block gets parsed as a Plain rather than a Para. This is a regression in pandoc 2.x. This patch restores pandoc 1.19 behavior.
• Make yamlToMeta respect extensions (#5272, Mauro Bieg). This adds a ReaderOptions parameter to yamlToMeta [API change].
• Fix bug parsing fenced code blocks (#5304). Previously parsing would break if the code block contained a string of backticks of sufficient length followed by something other than end of line.

LaTeX reader: don’t let \egroup match {. braced now actually requires nested braces. Otherwise some legitimate command and environment definitions can break.

Docx reader (Jesse Rosenthal):

• Rename getDocumentPath as getDocumentXmlPath.
• Use field notation for setting ReaderEnv.
• Figure out document.xml path once at the beginning of parsing, and add it to the environment, so we can avoid repeated lookups.
• Dynamically determine main document xml path (#5277). The desktop Word program places the main document file in word/document.xml, but the online word places it in word/document2.xml. This file path is actually stated in the root _rels/.rels file, in the Relationship element with an http://../officedocument type.
• Fix paths in archive to prevent Windows failure (#5277). Some paths in archives are absolute (have an opening slash) which, for reasons unknown, produces a failure in the test suite on MS Windows. This fixes that by removing the leading slash if it exists.
• Add comments to aid code readability.
• Trim space inside the last inline (#5273).
• Unwrap sdt elements in footnotes and comments (#5302).

Muse reader (Alexander Krotov):

• Test that block level markup does not break <verbatim>.
• Add secondary note support.

ipynb reader: handle images referring to attachments. Previously we didn’t strip off the attachment: prefix, so even though the attachment was available in the mediabag, pandoc couldn’t find it.

JATS reader:

• Fix parsing of figures (#5321). This ensures that a figure containing a single image is parsed as a pandoc “implicit figure” (i.e., a Para with a single Image whose title attribute begins with fig:). More complex figures will still be parsed as divs.
• Support fig-group block element (#5317).
• Handle citations with multiple references (#5310). The rid attribute can have a space-separated list of ids.

AsciiDoc Writer: Add writeAsciiDoctor [API change, Tarik Graba]. Handle display math appropriately for Asciidoctor.

JATS writer: wrap figure caption in <p> to fix validation (#5290, Mauro Bieg).

HTML writer:

• Implement WAI-ARIA roles for (end)notes, citations, and bibliography (#4213). Note that doc-biblioref is only used when link-citations produces links, since it belongs on links.
• Include content (including speaker notes) in title slides (#4317, #5237).

ipynb writer:

• Ensure final newline.
• Only include metadata under jupyter field.
• Don’t create attachments for images with absolute URIs, including data: URIs (#5303).
• Keep plain text fallbacks in output even if a richer format is included (#5293). We don’t know what output format will be needed. See the --ipynb-output command line option for a way to control what formats are included in the output.

Markdown writer: use markdown="1" when appropriate for Divs: when native_divs and markdown_in_html_blocks are disabled but raw_html and markdown_attribute are enabled.

LaTeX writer:

• Use right fold for escapeString. This is more elegant than the explicit recursive code we were using.
• Avoid {} after control sequences when escaping. \ldots{}. doesn’t behave as well as \ldots. with the latex ellipsis package. This patch causes pandoc to avoid emitting the {} when it is not necessary. Now \ldots and other control sequences used in escaping will be followed by either a {}, a space, or nothing, depending on context.
• For beamer, include contents under headers superordinate to slidelevel (#4317). Currently we keep the fancy title slide, and add a new slide with the same title and whatever content was under the header.

Powerpoint writer (Jesse Rosenthal): support underlines. Use span with single class “underline” as in docx writer.

Muse writer: escape secondary notes (Alexander Krotov).

FB2 writer: add section identifiers support (#5229, John KetzerX).

Make --fail-if-warnings work for PDF output (#5343).

Lua filters (Albert Krewinkel):

• Load module pandoc before calling init.lua (#5287). The file init.lua in pandoc’s data directory is run as part of pandoc’s Lua initialization process. Previously, the pandoc module was loaded in init.lua, and the structure for marshaling was set up after. This allowed simple patching of element marshaling, but made using init.lua more difficult. Now, all required modules are now loaded before calling init.lua. The file can be used entirely for user customization. Patching marshaling functions, while discouraged, is still possible via the debug module.
• All Lua modules bundled with pandoc, i.e., pandoc.List, pandoc.mediabag, pandoc.utils, and text are re-exported from the pandoc module. They are assigned to the fields List, mediabag, utils, and text, respectively.

Text.Pandoc.Lua (Albert Krewinkel):

• Split StackInstances into smaller Marshaling modules.
• Get CommonState from Lua global. This allows more control over the common state from within Lua scripts.

LaTeX template:

• Support the subject metadata variable (#5289, Pascal Wagler).
• Add \frontmatter, \mainmatter, \backmatter for book classes (#5306).

epub3 template: Add titlepage class to section (#5269).

HTML5 template: Add ARIA role doc-toc for table of contents (#4213).

Make --metadata-file use selected extensions (#5279, #5272, Mauro Bieg).

Text.Pandoc.Shared:

• Remove withTempDir [API change].
• Add new exported function defaultUserDataDirs [API change].
• Add filterIpynbOutput [API change].
• compactify: Avoid lists with a mix of Plain and Para elements (#5285).

Text.Pandoc.Translations: reorder alphabetically and remove Author (#5334, Mauro Bieg).

Text.Pandoc.Extensions:

• More carefully groom ipynb default extensions.
• Add all_symbols_escapable to githubMarkdownExtensions.

Text.Pandoc.PDF:

• Use system temp directory when possible (#1192). Previously we created temp dirs in the working directory, partly (a) because there were problems using the system temp directory on Windows, when their pathnames included tildes, and partly (b) because programs like epstopdf.pl would not be allowed to write to directories outside the working directory in restricted mode. We now (a) use the system temp dir except when the path includes tildes, and (b) set TEXMFOUTPUT when creating the PDF, so that subsidiary programs can use the system temp directory. This addresses problems that occurred when pandoc was used in a synced directory (such as Dropbox).
• Change types of subsidiary functions to PandocIO, to allow warnings to be threaded through (#5343).

Text.Pandoc.MIME: add WebP (#5267, Mauro Bieg).

Tests: avoid calling findPandoc multiple times.

Old tests: remove need for temp files by using pipeProcess.

Added simple ipynb reader/writer tests (#5274).

Rearrange --help output in a more rational way, with common options at the beginning and options grouped by function (#5336).

trypandoc: Add JATS and other missing formats (Arfon Smith, #5291).

Add missing copyright notices and remove license boilerplate (#4592, Albert Krewinkel).

Use latest basement/foundation on 32bit windows.

Use latest skylighting (#5328). Custom syntax definitions no longer try to load language.dtd.

Require texmath 0.11.2.1

Use latest pandoc-citeproc (0.16.1.1).

MANUAL.txt:

• Clarify variable substitution indentation in templates (#5338, Agustín Martín Barbero).
• Reorder custom-styles section (#5324, Mauro Bieg).

Support ipynb (Jupyter notebook) as input and output format.

• Add ipynb as input and output format (extension .ipynb).
• Added Text.Pandoc.Readers.Ipynb [API change].
• Added Text.Pandoc.Writers.Ipynb [API change].
• Add PandocIpynbDecodingError constructor to Text.Pandoc.Error.Error [API change].
• Depend on ipynb library.
• Note: there is no template for ipynb.

Add DokuWiki reader (#1792, Alexander Krotov). This adds Text.Pandoc.Readers.DokuWiki [API change], and adds dokuwiki as an input format.

Implement task lists (#3051, Mauro Bieg). Added task_lists extension. Task lists are supported from markdown and gfm input. They should work, to some degree, in all output formats, though in most formats you’ll get a bullet list with a unicode character for the box. In HTML, you get checkboxes and in LaTeX/PDF output, a box is used as the list marker. API changes:

• Added constructor Ext_task_lists to Extension.
• Added taskListItemFromAscii and taskListItemToAscii to Text.Pandoc.Shared.

Allow some command line options to take URL in addition to FILE. --include-in-header, --include-before-body, --include-after-body.

HTML reader:

• Handle empty start attribute (see #5162).
• Treat textarea as a verbatim environment (#5241) and preserve spacing.

RST reader:

• Change treatment of number-lines directive (Brian Leung, #5207). Directives of this type without numeric inputs should not have a startFrom attribute; with a blank value, the writers can produce extra whitespace.
• Removed superfluous sourceCode class on code blocks (#5047).
• Handle sourcecode directive as synonym for code (#5204).

Markdown reader:

• Remove sourceCode class for literate Haskell code blocks (#5047). Reverse order of literate and haskell classes on code blocks when parsing literate Haskell, so haskell is first.
• Treat <textarea> as a verbatim environment (#5241).