Custom parsing
For many users, the data provided by the simple API is enough. In some advanced cases you may find it necessary to use this more customizable parsing mechanism.
First, define a visitor that implements the CxxVisitor
protocol. Then
you can create an instance of it and pass it to the CxxParser
.
visitor = MyVisitor()
parser = CxxParser(filename, content, visitor)
parser.parse()
# do something with the data collected by the visitor
Your visitor should do something with the data as the various callbacks are
called. See the SimpleCxxVisitor
for inspiration.
API
- class cxxheaderparser.parser.CxxParser(filename, content, visitor, options=None, encoding=None)
Single-use parser object
- Parameters:
filename (
str
) –content (
Optional
[str
]) –visitor (
CxxVisitor
) –options (
Optional
[ParserOptions
]) –encoding (
Optional
[str
]) –
- parse()
Parse the header contents
- Return type:
None
- class cxxheaderparser.visitor.CxxVisitor(*args, **kwargs)
Defines the interface used by the parser to emit events
- on_class_end(state)
Called when the end of a class/struct/union is encountered.
When a variable like this is declared:
struct X { } x;
Then
on_class_start
, ..on_class_end
are emitted, along withon_variable
for each instance declared.- Parameters:
state (
ClassBlockState
) –- Return type:
None
- on_class_field(state, f)
Called when a field of a class is encountered
- Parameters:
state (
ClassBlockState
) –f (
Field
) –
- Return type:
None
- on_class_friend(state, friend)
Called when a friend declaration is encountered
- Parameters:
state (
ClassBlockState
) –friend (
FriendDecl
) –
- Return type:
None
- on_class_method(state, method)
Called when a method of a class is encountered inside of a class
- Parameters:
state (
ClassBlockState
) –method (
Method
) –
- Return type:
None
- on_class_start(state)
Called when a class/struct/union is encountered
When part of a typedef:
typedef struct { } X;
This is called first, followed by on_typedef for each typedef instance encountered. The compound type object is passed as the type to the typedef.
If this function returns False, the visitor will not be called for any items inside this class (including on_class_end)
- Parameters:
state (
ClassBlockState
) –- Return type:
Optional
[bool
]
- on_concept(state, concept)
template <class T> concept Meowable = is_meowable<T>;
- Parameters:
state (
Union
[ExternBlockState
[TypeVar
(T
),TypeVar
(PT
)],NamespaceBlockState
[TypeVar
(T
),TypeVar
(PT
)]]) –concept (
Concept
) –
- Return type:
None
- on_deduction_guide(state, guide)
Called when a deduction guide is encountered
- Parameters:
state (
Union
[ExternBlockState
[TypeVar
(T
),TypeVar
(PT
)],NamespaceBlockState
[TypeVar
(T
),TypeVar
(PT
)]]) –guide (
DeductionGuide
) –
- Return type:
None
- on_enum(state, enum)
Called after an enum is encountered
- Parameters:
state (
Union
[NamespaceBlockState
[TypeVar
(T
),TypeVar
(PT
)],ExternBlockState
[TypeVar
(T
),TypeVar
(PT
)],ClassBlockState
[TypeVar
(T
),TypeVar
(PT
)]]) –enum (
EnumDecl
) –
- Return type:
None
- on_extern_block_end(state)
Called when an extern block ends
- Parameters:
state (
ExternBlockState
) –- Return type:
None
- on_extern_block_start(state)
extern "C" { }
If this function returns False, the visitor will not be called for any items inside this block (including on_extern_block_end)
- Parameters:
state (
ExternBlockState
) –- Return type:
Optional
[bool
]
- on_forward_decl(state, fdecl)
Called when a forward declaration is encountered
- Parameters:
state (
Union
[NamespaceBlockState
[TypeVar
(T
),TypeVar
(PT
)],ExternBlockState
[TypeVar
(T
),TypeVar
(PT
)],ClassBlockState
[TypeVar
(T
),TypeVar
(PT
)]]) –fdecl (
ForwardDecl
) –
- Return type:
None
- on_function(state, fn)
Called when a function is encountered that isn’t part of a class
- Parameters:
state (
Union
[ExternBlockState
[TypeVar
(T
),TypeVar
(PT
)],NamespaceBlockState
[TypeVar
(T
),TypeVar
(PT
)]]) –fn (
Function
) –
- Return type:
None
- on_include(state, filename)
Called once for each
#include
directive encountered- Parameters:
state (
Union
[NamespaceBlockState
[TypeVar
(T
),TypeVar
(PT
)],ExternBlockState
[TypeVar
(T
),TypeVar
(PT
)],ClassBlockState
[TypeVar
(T
),TypeVar
(PT
)]]) –filename (
str
) –
- Return type:
None
- on_method_impl(state, method)
Called when a method implementation is encountered outside of a class declaration. For example:
void MyClass::fn() { // does something }
Note
The above implementation is ambiguous, as it technically could be a function in a namespace. We emit this instead as it’s more likely to be the case in common code.
- Parameters:
state (
Union
[ExternBlockState
[TypeVar
(T
),TypeVar
(PT
)],NamespaceBlockState
[TypeVar
(T
),TypeVar
(PT
)]]) –method (
Method
) –
- Return type:
None
- on_namespace_alias(state, alias)
Called when a
namespace
alias is encountered- Parameters:
state (
Union
[ExternBlockState
[TypeVar
(T
),TypeVar
(PT
)],NamespaceBlockState
[TypeVar
(T
),TypeVar
(PT
)]]) –alias (
NamespaceAlias
) –
- Return type:
None
- on_namespace_end(state)
Called at the end of a
namespace
block- Parameters:
state (
NamespaceBlockState
) –- Return type:
None
- on_namespace_start(state)
Called when a
namespace
directive is encounteredIf this function returns False, the visitor will not be called for any items inside this namespace (including on_namespace_end)
- Parameters:
state (
NamespaceBlockState
) –- Return type:
Optional
[bool
]
- on_parse_start(state)
Called when parsing begins
- Parameters:
state (
NamespaceBlockState
) –- Return type:
None
- on_pragma(state, content)
Called once for each
#pragma
directive encountered- Parameters:
state (
Union
[NamespaceBlockState
[TypeVar
(T
),TypeVar
(PT
)],ExternBlockState
[TypeVar
(T
),TypeVar
(PT
)],ClassBlockState
[TypeVar
(T
),TypeVar
(PT
)]]) –content (
Value
) –
- Return type:
None
- on_template_inst(state, inst)
Called when an explicit template instantiation is encountered
- Parameters:
state (
Union
[NamespaceBlockState
[TypeVar
(T
),TypeVar
(PT
)],ExternBlockState
[TypeVar
(T
),TypeVar
(PT
)],ClassBlockState
[TypeVar
(T
),TypeVar
(PT
)]]) –inst (
TemplateInst
) –
- Return type:
None
- on_typedef(state, typedef)
Called for each typedef instance encountered. For example:
typedef int T, *PT;
Will result in
on_typedef
being called twice, once forT
and once for*PT
- Parameters:
state (
Union
[NamespaceBlockState
[TypeVar
(T
),TypeVar
(PT
)],ExternBlockState
[TypeVar
(T
),TypeVar
(PT
)],ClassBlockState
[TypeVar
(T
),TypeVar
(PT
)]]) –typedef (
Typedef
) –
- Return type:
None
- on_using_alias(state, using)
using foo = int; template <typename T> using VectorT = std::vector<T>;
- Parameters:
state (
Union
[NamespaceBlockState
[TypeVar
(T
),TypeVar
(PT
)],ExternBlockState
[TypeVar
(T
),TypeVar
(PT
)],ClassBlockState
[TypeVar
(T
),TypeVar
(PT
)]]) –using (
UsingAlias
) –
- Return type:
None
- on_using_declaration(state, using)
using NS::ClassName;
- Parameters:
state (
Union
[NamespaceBlockState
[TypeVar
(T
),TypeVar
(PT
)],ExternBlockState
[TypeVar
(T
),TypeVar
(PT
)],ClassBlockState
[TypeVar
(T
),TypeVar
(PT
)]]) –using (
UsingDecl
) –
- Return type:
None
- on_using_namespace(state, namespace)
using namespace std;
- Parameters:
state (
Union
[ExternBlockState
[TypeVar
(T
),TypeVar
(PT
)],NamespaceBlockState
[TypeVar
(T
),TypeVar
(PT
)]]) –namespace (
List
[str
]) –
- Return type:
None
- on_variable(state, v)
Called when a global variable is encountered
- Parameters:
state (
Union
[NamespaceBlockState
[TypeVar
(T
),TypeVar
(PT
)],ExternBlockState
[TypeVar
(T
),TypeVar
(PT
)],ClassBlockState
[TypeVar
(T
),TypeVar
(PT
)]]) –v (
Variable
) –
- Return type:
None
- class cxxheaderparser.visitor.NullVisitor
This visitor does nothing
- on_class_end(state)
- Parameters:
state (
ClassBlockState
) –- Return type:
None
- on_class_field(state, f)
- Parameters:
state (
ClassBlockState
) –f (
Field
) –
- Return type:
None
- on_class_friend(state, friend)
- Parameters:
state (
ClassBlockState
) –friend (
FriendDecl
) –
- Return type:
None
- on_class_method(state, method)
- Parameters:
state (
ClassBlockState
) –method (
Method
) –
- Return type:
None
- on_class_start(state)
- Parameters:
state (
ClassBlockState
) –- Return type:
Optional
[bool
]
- on_concept(state, concept)
- Parameters:
state (
Union
[ExternBlockState
[TypeVar
(T
),TypeVar
(PT
)],NamespaceBlockState
[TypeVar
(T
),TypeVar
(PT
)]]) –concept (
Concept
) –
- Return type:
None
- on_deduction_guide(state, guide)
- Parameters:
state (
Union
[ExternBlockState
[TypeVar
(T
),TypeVar
(PT
)],NamespaceBlockState
[TypeVar
(T
),TypeVar
(PT
)]]) –guide (
DeductionGuide
) –
- Return type:
None
- on_enum(state, enum)
- Parameters:
state (
Union
[NamespaceBlockState
[TypeVar
(T
),TypeVar
(PT
)],ExternBlockState
[TypeVar
(T
),TypeVar
(PT
)],ClassBlockState
[TypeVar
(T
),TypeVar
(PT
)]]) –enum (
EnumDecl
) –
- Return type:
None
- on_extern_block_end(state)
- Parameters:
state (
ExternBlockState
) –- Return type:
None
- on_extern_block_start(state)
- Parameters:
state (
ExternBlockState
) –- Return type:
Optional
[bool
]
- on_forward_decl(state, fdecl)
- Parameters:
state (
Union
[NamespaceBlockState
[TypeVar
(T
),TypeVar
(PT
)],ExternBlockState
[TypeVar
(T
),TypeVar
(PT
)],ClassBlockState
[TypeVar
(T
),TypeVar
(PT
)]]) –fdecl (
ForwardDecl
) –
- Return type:
None
- on_function(state, fn)
- Parameters:
state (
Union
[ExternBlockState
[TypeVar
(T
),TypeVar
(PT
)],NamespaceBlockState
[TypeVar
(T
),TypeVar
(PT
)]]) –fn (
Function
) –
- Return type:
None
- on_include(state, filename)
- Parameters:
state (
Union
[NamespaceBlockState
[TypeVar
(T
),TypeVar
(PT
)],ExternBlockState
[TypeVar
(T
),TypeVar
(PT
)],ClassBlockState
[TypeVar
(T
),TypeVar
(PT
)]]) –filename (
str
) –
- Return type:
None
- on_method_impl(state, method)
- Parameters:
state (
Union
[ExternBlockState
[TypeVar
(T
),TypeVar
(PT
)],NamespaceBlockState
[TypeVar
(T
),TypeVar
(PT
)]]) –method (
Method
) –
- Return type:
None
- on_namespace_alias(state, alias)
- Parameters:
state (
Union
[ExternBlockState
[TypeVar
(T
),TypeVar
(PT
)],NamespaceBlockState
[TypeVar
(T
),TypeVar
(PT
)]]) –alias (
NamespaceAlias
) –
- Return type:
None
- on_namespace_end(state)
- Parameters:
state (
NamespaceBlockState
) –- Return type:
None
- on_namespace_start(state)
- Parameters:
state (
NamespaceBlockState
) –- Return type:
Optional
[bool
]
- on_parse_start(state)
- Parameters:
state (
NamespaceBlockState
) –- Return type:
None
- on_pragma(state, content)
- Parameters:
state (
Union
[NamespaceBlockState
[TypeVar
(T
),TypeVar
(PT
)],ExternBlockState
[TypeVar
(T
),TypeVar
(PT
)],ClassBlockState
[TypeVar
(T
),TypeVar
(PT
)]]) –content (
Value
) –
- Return type:
None
- on_template_inst(state, inst)
- Parameters:
state (
Union
[NamespaceBlockState
[TypeVar
(T
),TypeVar
(PT
)],ExternBlockState
[TypeVar
(T
),TypeVar
(PT
)],ClassBlockState
[TypeVar
(T
),TypeVar
(PT
)]]) –inst (
TemplateInst
) –
- Return type:
None
- on_typedef(state, typedef)
- Parameters:
state (
Union
[NamespaceBlockState
[TypeVar
(T
),TypeVar
(PT
)],ExternBlockState
[TypeVar
(T
),TypeVar
(PT
)],ClassBlockState
[TypeVar
(T
),TypeVar
(PT
)]]) –typedef (
Typedef
) –
- Return type:
None
- on_using_alias(state, using)
- Parameters:
state (
Union
[NamespaceBlockState
[TypeVar
(T
),TypeVar
(PT
)],ExternBlockState
[TypeVar
(T
),TypeVar
(PT
)],ClassBlockState
[TypeVar
(T
),TypeVar
(PT
)]]) –using (
UsingAlias
) –
- Return type:
None
- on_using_declaration(state, using)
- Parameters:
state (
Union
[NamespaceBlockState
[TypeVar
(T
),TypeVar
(PT
)],ExternBlockState
[TypeVar
(T
),TypeVar
(PT
)],ClassBlockState
[TypeVar
(T
),TypeVar
(PT
)]]) –using (
UsingDecl
) –
- Return type:
None
- on_using_namespace(state, namespace)
- Parameters:
state (
Union
[ExternBlockState
[TypeVar
(T
),TypeVar
(PT
)],NamespaceBlockState
[TypeVar
(T
),TypeVar
(PT
)]]) –namespace (
List
[str
]) –
- Return type:
None
- on_variable(state, v)
- Parameters:
state (
Union
[NamespaceBlockState
[TypeVar
(T
),TypeVar
(PT
)],ExternBlockState
[TypeVar
(T
),TypeVar
(PT
)],ClassBlockState
[TypeVar
(T
),TypeVar
(PT
)]]) –v (
Variable
) –
- Return type:
None
Parser state
- class cxxheaderparser.parserstate.BaseState(parent, location)
- Parameters:
parent (
Union
[NamespaceBlockState
[TypeVar
(T
),TypeVar
(PT
)],ExternBlockState
[TypeVar
(T
),TypeVar
(PT
)],ClassBlockState
[TypeVar
(T
),TypeVar
(PT
)],None
]) –location (
Location
) –
-
location:
Location
Approximate location that the parsed element was found at
-
parent:
Union
[NamespaceBlockState
[TypeVar
(T
),TypeVar
(PT
)],ExternBlockState
[TypeVar
(T
),TypeVar
(PT
)],ClassBlockState
[TypeVar
(T
),TypeVar
(PT
)],None
] parent state
-
user_data:
TypeVar
(T
) Uninitialized user data available for use by visitor implementations. You should set this in a
*_start
method.
- class cxxheaderparser.parserstate.ClassBlockState(parent, location, class_decl, access, typedef, mods)
- Parameters:
parent (
Union
[NamespaceBlockState
[TypeVar
(T
),TypeVar
(PT
)],ExternBlockState
[TypeVar
(T
),TypeVar
(PT
)],ClassBlockState
[TypeVar
(T
),TypeVar
(PT
)],None
]) –location (
Location
) –class_decl (
ClassDecl
) –access (
str
) –typedef (
bool
) –mods (
ParsedTypeModifiers
) –
-
access:
str
Current access level for items encountered
-
mods:
ParsedTypeModifiers
modifiers to apply to following variables
-
parent:
Union
[NamespaceBlockState
[TypeVar
(T
),TypeVar
(PT
)],ExternBlockState
[TypeVar
(T
),TypeVar
(PT
)],ClassBlockState
[TypeVar
(T
),TypeVar
(PT
)]] parent state
-
typedef:
bool
Currently parsing as a typedef
- class cxxheaderparser.parserstate.ExternBlockState(parent, location, linkage)
- Parameters:
parent (
Union
[ExternBlockState
[TypeVar
(T
),TypeVar
(PT
)],NamespaceBlockState
[TypeVar
(T
),TypeVar
(PT
)]]) –location (
Location
) –linkage (
str
) –
-
linkage:
str
The linkage for this extern block
-
parent:
Union
[ExternBlockState
[TypeVar
(T
),TypeVar
(PT
)],NamespaceBlockState
[TypeVar
(T
),TypeVar
(PT
)]] parent state
- class cxxheaderparser.parserstate.NamespaceBlockState(parent, location, namespace)
- Parameters:
parent (
Union
[ExternBlockState
[TypeVar
(T
),TypeVar
(PT
)],NamespaceBlockState
[TypeVar
(T
),TypeVar
(PT
)],None
]) –location (
Location
) –namespace (
NamespaceDecl
) –
-
namespace:
NamespaceDecl
The incremental namespace for this block
-
parent:
Union
[ExternBlockState
[TypeVar
(T
),TypeVar
(PT
)],NamespaceBlockState
[TypeVar
(T
),TypeVar
(PT
)]] parent state
- class cxxheaderparser.parserstate.PT
type of custom user data for a parent state
alias of TypeVar(‘PT’)
- class cxxheaderparser.parserstate.ParsedTypeModifiers(vars, both, meths)
-
both:
Dict
[str
,LexToken
] Alias for field number 1
-
meths:
Dict
[str
,LexToken
] Alias for field number 2
- validate(*, var_ok, meth_ok, msg)
- Parameters:
var_ok (
bool
) –meth_ok (
bool
) –msg (
str
) –
- Return type:
None
-
vars:
Dict
[str
,LexToken
] Alias for field number 0
-
both:
- class cxxheaderparser.parserstate.T
custom user data for this state type
alias of TypeVar(‘T’)
Preprocessor
Contains optional preprocessor support functions
- exception cxxheaderparser.preprocessor.PreprocessorError
- cxxheaderparser.preprocessor.make_gcc_preprocessor(*, defines=[], include_paths=[], retain_all_content=False, encoding=None, gcc_args=['g++'], print_cmd=True)
Creates a preprocessor function that uses g++ to preprocess the input text.
gcc is a high performance and accurate precompiler, but if an #include directive can’t be resolved or other oddity exists in your input it will throw an error.
- Parameters:
defines (
List
[str
]) – list of #define macros specified as “key value”include_paths (
List
[str
]) – list of directories to search for included filesretain_all_content (
bool
) – If False, only the parsed file content will be retainedencoding (
Optional
[str
]) – If specified any include files are opened with this encodinggcc_args (
List
[str
]) – This is the path to G++ and any extra args you might wantprint_cmd (
bool
) – Prints the gcc command as its executed
- Return type:
Callable
[[str
,Optional
[str
]],str
]
pp = make_gcc_preprocessor() options = ParserOptions(preprocessor=pp) parse_file(content, options=options)
- cxxheaderparser.preprocessor.make_msvc_preprocessor(*, defines=[], include_paths=[], retain_all_content=False, encoding=None, msvc_args=['cl.exe'], print_cmd=True)
Creates a preprocessor function that uses cl.exe from Microsoft Visual Studio to preprocess the input text. cl.exe is not typically on the path, so you may need to open the correct developer tools shell or pass in the correct path to cl.exe in the msvc_args parameter.
cl.exe will throw an error if a file referenced by an #include directive is not found.
- Parameters:
defines (
List
[str
]) – list of #define macros specified as “key value”include_paths (
List
[str
]) – list of directories to search for included filesretain_all_content (
bool
) – If False, only the parsed file content will be retainedencoding (
Optional
[str
]) – If specified any include files are opened with this encodingmsvc_args (
List
[str
]) – This is the path to cl.exe and any extra args you might wantprint_cmd (
bool
) – Prints the command as its executed
- Return type:
Callable
[[str
,Optional
[str
]],str
]
pp = make_msvc_preprocessor() options = ParserOptions(preprocessor=pp) parse_file(content, options=options)
- cxxheaderparser.preprocessor.make_pcpp_preprocessor(*, defines=[], include_paths=[], retain_all_content=False, encoding=None, passthru_includes=None)
Creates a preprocessor function that uses pcpp (which must be installed separately) to preprocess the input text.
If missing #include files are encountered, this preprocessor will ignore the error. This preprocessor is pure python so it’s very portable, and is a good choice if performance isn’t critical.
- Parameters:
defines (
List
[str
]) – list of #define macros specified as “key value”include_paths (
List
[str
]) – list of directories to search for included filesretain_all_content (
bool
) – If False, only the parsed file content will be retainedencoding (
Optional
[str
]) – If specified any include files are opened with this encodingpassthru_includes (
Optional
[Pattern
]) – If specified any #include directives that match the compiled regex pattern will be part of the output.
- Return type:
Callable
[[str
,Optional
[str
]],str
]
pp = make_pcpp_preprocessor() options = ParserOptions(preprocessor=pp) parse_file(content, options=options)