Extra Reference

July 2, 2026 · View on GitHub

The reference documentation for extra and deprecated headers.

Preamble
Extras
Deprecated
Index

Preamble

The extra headers contain classes that are experimental or otherwise not ready to be made part of the "official" public interface.

The deprecated headers contain functionality from previous versions that was not deemed useful or replaced by something better.

Extras

The extras can be found in include/tao/pegtl/extra/.

Provides string-to-integer conversion based on std::from_chars() from the standard <charconv> header. The classes defined in this header implement both the match() function for use as rule and the apply() function for use as action.

Conversion classes for decimal and hexadecimal are provided, other bases can be selected via the appropriate template parameter.

enum class overflow_mode : bool;

template< typename Integral, std::uint8_t Base, overflow_mode Over > struct from_chars_combo;

template< typename Integral, std::uint8_t Base = 10 > struct from_chars_throws;
template< typename Integral, std::uint8_t Base = 10 > struct from_chars_nothrow;

template< typename Integral > struct from_xchars_throws : from_chars_throws< Integral, 16 > {};
template< typename Integral > struct from_xchars_nothrow : from_chars_nothrow< Integral, 16 > {};

The Base parameter is passed to std::from_chars() as fourth function argument. It can be anything between 2 and 36, inclusive.

Note

Remember that std::from_chars() does not allow a leading + and does not recognize the 0x prefix for hexadecimal numbers.

In both rules the functions accept arbitrary state arguments, and both return bool (as is mandatory for match() but optional for apply()).

If the first state argument is a mutable reference to Integral then it will be assigned the result of the string-to-integer conversion.
If the first state argument is of any other type it will be ignored -- together with any additional states.

The _throws vs. _nothrow suffix chooses between failing locally, i.e. returning false, when std::from_chars() returns std::errc::result_out_of_range, or failing globally, i.e. throwing an exception derived from tao::pegtl::parse_error_base as described in Errors and Exceptions.

The template parameter Integral can also be void, which changes the behavior slightly. In this case

the first state must be a mutable reference to any integer type supported by std::from_chars(),
and when used as rule the apply_mode is ignored and the conversion result always assigned to the first state.

`dispatch.hpp`

An experimental approach to attaching actions to multiple rules with less code. Instead of specializing one action class template for every rule, a list of clauses maps rules to actions. Rules matched by no clause use nothing, while rules matched by more than one clause cause a compile-time error.

template< typename Action, typename... Rules >
struct clause1;

template< template< typename... > class Action, typename... Rules >
struct clause2;

template< typename Action, typename... Clauses >
struct default1;

template< template< typename... > class Action, typename... Clauses >
struct default2;

template< typename... Clauses >
struct dispatch;

The clause1 template attaches the same concrete Action class to each rule in Rules.... The clause2 template attaches Action< Rule > to each selected rule and is useful when the action needs to depend on the matched rule type.

The default1 and default2 templates are enabled for rules not enabled by any of the clauses passed as their remaining template arguments. As with the explicit clauses, default1 uses a concrete action class while default2 uses an action class template.

The action class template created by a dispatcher is available as dispatch< Clauses... >::action. The parse() member function is a convenience wrapper around match() using that action class template.

template< typename... Clauses >
struct dispatch
{
   template< typename Rule >
   using action = /* selected action, or nothing< Rule > */;

   template< typename Rule,
             apply_mode A = apply_mode::enabled,
             rewind_mode M = rewind_mode::optional,
             template< typename... > class Control = normal,
             typename ParseInput,
             typename... States >
   [[nodiscard]] static bool parse( ParseInput&&, States&&... );
};

Both dispatch and the default clauses also accept a single type_list in place of a parameter pack. See src/example/dispatch.cpp for a small example that dispatches different actions for selected JSON rules.

`nested_exceptions.hpp`

The visit_nested functions call the visitor once for every exception, including the first/outer one. The visitor is called first with the inner-most exception, the one that was thrown first from furthest down/inside the call stack, and a nesting count corresponding to the total number of exceptions minus 1. The visitor is called last with the outer-most exception, the one that was caught in the try-catch that is calling visit_nested(), and a nesting count of 0.

For example, when an exception E is caught, and E has exception F as nested exception, then visit_nested( E, V ) for some visitor V will first call V( F, 1 ) and then V( E, 0 ).

Exceptions in an inheritance hierarchy must be listed from more general to more specific in the template parameter list. For example visit_nested< std::exception, std::runtime_error >() and not vice versa.

template< typename... Exceptions, typename Visitor >
void visit_nested( Visitor&& );

template< typename... Exceptions, typename Visitor >
void visit_nested( const std::exception_ptr&, Visitor&& );

template< typename... Exceptions, typename Exception, typename Visitor >
void visit_nested( const Exception&, Visitor&& );

The following convenience functions call visit_nested() and produce a flattened vector derived from the nested exceptions. The flatten_type() and flatten_base() functions return type sliced copies of the exceptions; flatten_what() calls what() on the std::exceptions and returns a vector of strings.

template< typename Exception >
[[nodiscard]] std::vector< Exception > flatten_type( const std::exception_ptr& = std::current_exception() );
template< typename Exception >
[[nodiscard]] std::vector< Exception > flatten_type( const Exception& );

[[nodiscard]] std::vector< parse_error_base > flatten_base( const std::exception_ptr& = std::current_exception() )
[[nodiscard]] std::vector< parse_error_base > flatten_base( const parse_error_base& );

[[nodiscard]] std::vector< std::string > flatten_what( const std::exception_ptr& = std::current_exception() )
[[nodiscard]] std::vector< std::string > flatten_what( const std::exception& );

`parse_tree.hpp`

`parse_tree_to_dot.hpp`

The parse tree has its own dedicated page.

`overflow_mode.hpp`

Defines enum class overflow_mode with values local_failure and global_failure. This enum is used by charconv.hpp to choose whether integer conversion overflow causes a local failure or a global failure.

`partial_apply1.hpp`

An experimental rule that calls actions in an unusual way.

Based on partial< R... > but with major differences:
First, R must contain at least two rules.
Second, only succeeds when (at least) the first sub-rule succeeds, i.e.
the "partial" matching behavior only starts at the second rule.
Equivalent to seq< R1, partial< R2, ... > > when R is R1, R2, ...
Third, the R... are matched with actions disabled, however:
Fourth, on success exactly one action is applied
- for the last rule in R... that matched, and
- with an action input spanning from the start of partial_apply1 to the current input position.
Fifth, if actions are enabled for partial_apply1 and
- Action< T > is not exactly tao::pegtl::nothing< T > then
- Action< T > must implement apply() or apply0() for all T in R...
- (assuming that Action is the current action in the parsing run).
Meta data and implementation mapping:
- partial_apply1< R... >::rule_t is internal::partial_apply1< R... >
- partial_apply1< R... >::subs_t is type_list< R... >

`raw_string.hpp`

Rules for parsing Lua-style long string literals, also called raw literals because they do not support any escape sequences.

The following description of these kinds of string literals is copied from the Lua documentation.

An "opening long bracket of level n" is defined as an opening square bracket followed by n equal signs followed by another opening square bracket. So, an opening long bracket of level 0 is written as [[, an opening long bracket of level 1 is written as [=[, and so on.
A "closing long bracket" is defined similarly; for instance, a closing long bracket of level 4 is written as ]====].
A "long literal" starts with an opening long bracket of any level and ends at the first closing long bracket of the same level. It can contain any text except a closing bracket of the same level.
Literals in this bracketed form can run for several lines, do not interpret any escape sequences, and ignore long brackets of any other level.
For convenience, when the opening long bracket is eagerly followed by a newline, the newline is not included in the string.

Unlike Lua's long literal the raw_string rule is customizable to use other characters than [, = and ] for matching the opening and closing long brackets. Also note that Lua introduced newline-specific replacements in Lua 5.2, which are not supported at the grammar level by the raw_string rule.

The raw_string rule has a sub-type called content that actions should usually be bound to instead of binding to raw_string itself. Binding to content will call the action with the matched string excluding the opening and closing long brackets.

template< char Open, char Marker, char Close, typename... Contents >
struct raw_string
   // base is implementation detail
{
   // struct content; -- inherited
};

template< char Open, char Marker, char Close >
struct raw_string< Open, Marker, Close >
   // base is implementation detail
{
   // struct content; -- inherited
};

`record.hpp`

Builds on dispatch.hpp to build a linear record of a parsing run. For each successful match of a selected rule the vector contains an entry with a kind of view_input to represent the matched input, and the name of the rule that matched. Successful sub-rules of a failed rule will not show up in the record.

template< typename ParseInput >
using record_input = input_with_offset< internal::text_view_alias< typename ParseInput::eol_rule, typename ParseInput::data_t, void, void > >;

template< typename ParseInput >
struct record_value
{
   using data_t = typename ParseInput::data_t;
   using position_t = typename ParseInput::error_position_t;

   record_value( const position_t&, const data_t*, const std::size_t, const std::string_view );

   record_input< ParseInput > input;
   std::string_view rule;
};

template< typename ParseInput >
using record_vector = std::vector< record_value< ParseInput > >;

template< typename... Rules >
struct record
{
   using clause = // clauses to produce the record for Rules...

   template< typename Rule >
   using action = // clauses converted to an action

   template< typename Rule,
             template< typename... > class Control = normal,
             typename ParseInput >
   [[nodiscard]] static auto parse( ParseInput&& )
   {
      // parse with action and return a record_vector< ParseInput >
   }
};

`type_to_string.hpp`

Functions to convert compile-time sequences of char into a std::string or std::string_view.

template< typename T >
[[nodiscard]] std::string type_to_string();

template< char... Cs >
[[nodiscard]] std::string type_to_string();

template< typename T >
[[nodiscard]] constexpr std::string_view type_to_string_view() noexcept;

template< char... Cs >
[[nodiscard]] constexpr std::string_view type_to_string_view() noexcept;

The overloads that take a single type can be used with tao::pegtl::string<> or any other type that takes a sequence of char as template parameters.

`unescape.hpp`

A slightly refactored version of the previous unescape header which is now moved to deprecated/unescape.hpp.

It defines individual action classes for common unescape operations as well as an action class template that attaches these actions to corresponding grammar rules.

All actions output to their first state (second argument) which must be a mutable std::string.

struct all_of_these_actions
{
   template< typename ActionInput >
   static void apply( const ActionInput& in, std::string& out );
};

unescape_append is for non-escaped characters that need to just be copied from the input unchanged by calling out.append( in.string_view() );

unescape_char<> is for single character escape sequences like \n. It takes any number of template parameters of type char and has to be attached to a rule of type (ascii::)one<> with the same number of template parameters.

During unescape the n-th template parameter of the one is mapped to the n-th template parameter of unescape_char.

unescape_byte is for bytes represented by two hex digits and must be attached to a rule that matches exactly that.

unescape_unicode<> is for hexadecimal Unicode code points, excluding surrogates. The template parameter of type std::size_t must match the fixed (!) number of hex digits the rule it is attached to matches, usually 4 or 8.

json_unescape_unicode is for sequences of hexadecimal UTF-16 code points with 4 hex digits each. It unescapes each individual code point while checking for and merging consecutive surrogate pairs as required by RFC 8259.

Warning

The actions do not double-check that the matched portion of the input they are called with has the expected size and data!

The action template unescape<> is set up with the usual default

template< typename Rule > unescape< Rule > : nothing< Rule > {};

and some useful specializations for some generic rules, and some specific rules defined in include/tao/pegtl/example/escaped.hpp, that might be part of a rule for strings with escape sequences. Please check out include/tao/pegtl/extra/unescape.hpp to see which individual unescape action is attached to which rule by unescape<>.

alphabet.hpp ^(deprecated)
charconv.hpp ^(extra)
dispatch.hpp ^(extra)
if_then.hpp ^(deprecated)
integer.hpp ^(deprecated)
nested_exceptions.hpp ^(extra)
overflow_mode.hpp ^(extra)
parse_tree.hpp ^(extra)
parse_tree_to_dot.hpp ^(extra)
partial_apply1.hpp ^(extra)
raw_string.hpp ^(extra)
record.hpp ^(extra)
rep_one_min_max.hpp ^(deprecated)
rep_string.hpp ^(deprecated)
type_to_string.hpp ^(extra)
unescape.hpp ^(extra)
unescape.hpp ^(deprecated)

This page is part of the PEGTL and its documentation.

Copyright (c) 2014-2026 Dr. Colin Hirsch and Daniel Frey
Distributed under the Boost Software License, Version 1.0
See accompanying file LICENSE_1_0.txt or copy at https://www.boost.org/LICENSE_1_0.txt

Extra Reference

Contents

Preamble

Extras

`charconv.hpp`

`dispatch.hpp`

`nested_exceptions.hpp`

`parse_tree.hpp`

`parse_tree_to_dot.hpp`

`overflow_mode.hpp`

`partial_apply1.hpp`

`raw_string.hpp`

`record.hpp`

`type_to_string.hpp`

`unescape.hpp`

Deprecated

`alphabet.hpp`

`if_then.hpp`

`integer.hpp`

`rep_one_min_max.hpp`

`rep_string.hpp`

`unescape.hpp`

Index