Visitors

Foundatio.Parsers uses the visitor pattern for AST traversal and transformation. Visitors allow you to inspect, modify, or extract information from parsed queries.

Visitor Pattern Overview

When a query is parsed, it creates an Abstract Syntax Tree (AST) of nodes. Visitors traverse this tree, performing operations on each node type.

Traversal Order

Visitors traverse the AST depth-first, left-to-right. The GroupNode.Children property yields Left first, then Right. The base class QueryNodeVisitorBase.VisitAsync(GroupNode) iterates these children in order but does not act on the GroupNode itself -- it only recurses into children.

Whether a visitor processes a GroupNode before or after its children depends on where the subclass places its logic relative to the base.VisitAsync() call:

• Pre-order (process node, then children): FieldResolverQueryVisitor, NestedVisitor
• Post-order (process children, then node): CombineQueriesVisitor, CombineAggregationsVisitor
• Wrap-around (before and after children): ValidationVisitor
• Short-circuit (skip children): InvertQueryVisitor (when the entire group can be inverted at once)

When multiple visitors are chained via ChainedQueryVisitor, each visitor completes a full traversal of the entire tree before the next visitor begins, in priority order (lower priority number runs first).

For detailed coverage of how visitors handle nested and hierarchical query structures, field scoping, and the @field:(-@field:(...)) pattern, see Nested Queries and Visitor Traversal.

Built-in Visitors

GenerateQueryVisitor

Converts an AST back to a query string:

using Foundatio.Parsers.LuceneQueries;
using Foundatio.Parsers.LuceneQueries.Visitors;

var parser = new LuceneQueryParser();
var result = parser.Parse("status:active AND created:>2024-01-01");

// Generate query string from AST
string query = GenerateQueryVisitor.Run(result);
// Output: "status:active AND created:>2024-01-01"

// Async version
query = await GenerateQueryVisitor.RunAsync(result);

With Default Operator

var context = new QueryVisitorContext {
    DefaultOperator = GroupOperator.And
};

var result = parser.Parse("value1 value2");
string query = await GenerateQueryVisitor.RunAsync(result, context);
// Output: "value1 AND value2"

DebugQueryVisitor

Outputs the AST structure for debugging:

var parser = new LuceneQueryParser();
var result = parser.Parse("(status:active OR status:pending) AND type:user");

string debug = DebugQueryVisitor.Run(result);
Console.WriteLine(debug);

Output:

Group:
  Operator: And
  Left - Group:
    HasParens: True
    Operator: Or
    Left - Term:
        Field: status
        Term: active
    Right - Term:
        Field: status
        Term: pending
  Right - Term:
      Field: type
      Term: user

FieldResolverQueryVisitor

Resolves field aliases:

var parser = new LuceneQueryParser();
var result = await parser.ParseAsync("user:john");

var fieldMap = new FieldMap {
    { "user", "data.user.identity" }
};

var resolved = await FieldResolverQueryVisitor.RunAsync(result, fieldMap);
// Result: data.user.identity:john

IncludeVisitor

Expands query includes/macros:

var parser = new LuceneQueryParser();
var result = await parser.ParseAsync("@include:active");

var includes = new Dictionary<string, string> {
    { "active", "status:active AND deleted:false" }
};

var expanded = await IncludeVisitor.RunAsync(result, includes);
// Result: (status:active AND deleted:false)

ValidationVisitor

Validates queries against rules:

var parser = new LuceneQueryParser();
var result = await parser.ParseAsync("status:active");

var context = new QueryVisitorContext();
context.SetValidationOptions(new QueryValidationOptions {
    AllowedFields = { "status", "name" }
});

await ValidationVisitor.RunAsync(result, context);

var validation = context.GetValidationResult();
Console.WriteLine($"Valid: {validation.IsValid}");

InvertQueryVisitor

Inverts query negation:

var parser = new LuceneQueryParser();
var result = await parser.ParseAsync("status:active");

var invertVisitor = new InvertQueryVisitor();
var inverted = await invertVisitor.AcceptAsync(result, new QueryVisitorContext());

string query = inverted.ToString();
// Output: "(NOT status:active)"

With Non-Inverted Fields

// Some fields should not be inverted
var invertVisitor = new InvertQueryVisitor(
    nonInvertedFields: new[] { "organization_id", "tenant_id" });

var result = await parser.ParseAsync("status:active organization_id:123");
var inverted = await invertVisitor.AcceptAsync(result, context);
// Output: "(NOT status:active) organization_id:123"

RemoveFieldsQueryVisitor

Removes specific fields from queries:

var parser = new LuceneQueryParser();
var result = await parser.ParseAsync("status:active AND secret:value AND name:john");

var removeVisitor = new RemoveFieldsQueryVisitor(new[] { "secret" });
var cleaned = await removeVisitor.AcceptAsync(result, new QueryVisitorContext());

string query = cleaned.ToString();
// Output: "status:active AND name:john"

CleanupQueryVisitor

Simplifies and cleans up the AST:

var parser = new LuceneQueryParser();
var result = await parser.ParseAsync("((status:active))");

// Returns the cleaned query string
string cleaned = await CleanupQueryVisitor.RunAsync(result);
// Removes unnecessary nesting

TermToFieldVisitor

Converts standalone terms to field queries (used for sort expressions):

var parser = new LuceneQueryParser();
var result = await parser.ParseAsync("-created +name");

await TermToFieldVisitor.RunAsync(result);
// Converts terms to field nodes for sort processing

Chained Visitors

Multiple visitors can be chained together with priority ordering:

using System;
using System.Threading.Tasks;
using Foundatio.Parsers.LuceneQueries;
using Foundatio.Parsers.LuceneQueries.Visitors;

var ast = await new LuceneQueryParser().ParseAsync("@include:active")
    ?? throw new InvalidOperationException("Could not parse the example query.");
var context = new QueryVisitorContext
{
    IncludeResolver = name => Task.FromResult<string?>(name is "active" ? "alias:active" : null)
};
QueryFieldResolver fieldResolver = (field, _) => Task.FromResult<string?>(field is "alias" ? "status" : field);
var chainedVisitor = new ChainedQueryVisitor();

// Add visitors with priority (lower runs first)
chainedVisitor.AddVisitor(new IncludeVisitor(), priority: 0);
chainedVisitor.AddVisitor(new FieldResolverQueryVisitor(fieldResolver), priority: 10);
chainedVisitor.AddVisitor(new ValidationVisitor(), priority: 30);

// Run all visitors
var result = await chainedVisitor.AcceptAsync(ast, context);
var query = await GenerateQueryVisitor.RunAsync(result!); // (status:active)

Expanding includes first lets field resolution and validation process terms from the included query. At priority 20, included terms would miss the field resolver at priority 10.

Managing Chained Visitors

var chain = new ChainedQueryVisitor();

// Add visitor
chain.AddVisitor(new MyVisitor(), priority: 100);

// Remove visitor by type
chain.RemoveVisitor<MyVisitor>();

// Replace visitor
chain.ReplaceVisitor<OldVisitor>(new NewVisitor(), newPriority: 50);

// Add before/after specific visitor
chain.AddVisitorBefore<ValidationVisitor>(new MyVisitor());
chain.AddVisitorAfter<FieldResolverQueryVisitor>(new MyVisitor());

Visitor Context

Visitors receive a context object for sharing data:

var context = new QueryVisitorContext();

// Set values
context.SetValue("UserId", "123");
context.SetValue("IsAdmin", true);

// Get values
string userId = context.GetValue<string>("UserId");
bool isAdmin = context.GetValue<bool>("IsAdmin");

// Default operator
context.DefaultOperator = GroupOperator.And;

// Default fields for unqualified terms
context.DefaultFields = new[] { "title", "description" };

// Query type
context.QueryType = QueryTypes.Query; // or Aggregation, Sort

Context Extensions

// Field resolver
context.SetFieldResolver(async (field, ctx) => {
    return fieldMap.GetValueOrDefault(field);
});
var resolver = context.GetFieldResolver();

// Include resolver
context.SetIncludeResolver(async name => includes.GetValueOrDefault(name));
var includeResolver = context.GetIncludeResolver();

// Validation
context.SetValidationOptions(options);
var validation = context.GetValidationResult();
context.AddValidationError("Error message", position: 5);
context.ThrowIfInvalid();

AST Node Types

Visitors can handle different node types:

Node Type	Description	Key Properties
GroupNode	Boolean group	Left, Right, Operator, HasParens
TermNode	Single term	Field, Term, IsQuotedTerm, Boost, Proximity
TermRangeNode	Range query	Field, Min, Max, MinInclusive, MaxInclusive
ExistsNode	Existence check	Field
MissingNode	Missing check	Field

Node Properties

// Common to all field nodes
string field = node.Field;
string unescapedField = node.UnescapedField;
bool? isNegated = node.IsNegated; // the NOT keyword; also set by tree-rewriting visitors
string prefix = node.Prefix; // +, -, !, or null

// TermNode specific
string term = termNode.Term;
bool isQuoted = termNode.IsQuotedTerm;
bool isRegex = termNode.IsRegexTerm;
string boost = termNode.Boost;
string proximity = termNode.Proximity;

// TermRangeNode specific
string min = rangeNode.Min;
string max = rangeNode.Max;
bool minInclusive = rangeNode.MinInclusive;
bool maxInclusive = rangeNode.MaxInclusive;

// GroupNode specific
IQueryNode left = groupNode.Left;
IQueryNode right = groupNode.Right;
GroupOperator op = groupNode.Operator; // And, Or, Default
bool hasParens = groupNode.HasParens;

Negation and Prefix Operators

Negation is stored in two different places depending on the syntax used:

Query	IsNegated	Prefix
NOT field:value	true	null
NOT [1 TO 2]	true	null
-field:value	null	"-"
!field:value	null	"!"
+field:value	null	"+"
field:value	null	null
field:[1 TO 2]	false	null
_exists_:field	false	null

Why IsNegated is a bool? and not a bool

The name reads like a boolean, so the nullability invites the question of whether null means something false does not. For query rendering it does not: flipping any non-negated node between null and false across a corpus of nested, grouped and negated queries produces byte-identical output from both CleanupQueryVisitor and GenerateQueryVisitor, including through the group-collapse and double-negative folding. Only true changes the result. Every consumer in the repo tests HasValue && Value or is true, so none of them distinguishes the two.

Two places do observe the difference, so null and false are not strictly interchangeable:

• DebugQueryVisitor prints an IsNegated line only when the value is set, so false adds a line that null omits.
• CopyTo copies the value only when it is set, so copying a null source over a true target leaves true, while copying a false source overwrites it with false.

Neither is a semantic distinction worth relying on, and which value a non-negated node receives varies by grammar rule (see the table above) with no meaning attached. The practical rules are:

• Compare against true (node.IsNegated is true); never treat it as a plain boolean and never branch on false versus null.
• Never call .Value without checking HasValue.
• Better still, call IsExcluded(), which handles all of this along with the - and ! prefixes.

Narrowing the property to a non-nullable bool would be a reasonable cleanup, but it is a breaking change to the public IFieldQueryNode interface and to every node type and custom visitor implementing it, so it belongs in a major version rather than here.

Visitors that rewrite the tree (InvertNegation, CleanupQueryVisitor) also set this value, including to false.

The split between the two properties is intentional: keeping the operator that was actually written means GenerateQueryVisitor and ToString() re-emit -value as -value rather than rewriting it to NOT value. As a result, IsNegated alone is never a complete negation check.

Write operators before the field name: -field:(value) and NOT field:(value). Post-colon operators such as field:-(value) and field:NOT (value) are parse errors; see the breaking syntax correction. Operators inside a scoped group, such as field:(-value), still belong to the inner clause and remain valid.

In query contexts, use the extension methods instead of inspecting the properties directly:

• IsExcluded() - node-local negation, covering NOT, -, and !
• IsRequired() - the + prefix
• IsNodeOrGroupNegated() - IsExcluded() plus negation on the nearest enclosing parenthesized group, falling back to the root group. Its limits are described below.

// Check only the node's own NOT, - or ! operator.
bool isLocallyExcluded = node.IsExcluded();

// Also inspect the nearest enclosing group (or root), unless the node has +.
// For example, the value term in -field:(value).
bool isNodeOrGroupExcluded = node.IsNodeOrGroupNegated();

What IsNodeOrGroupNegated() checks

It checks exactly two things:

• IsExcluded() on the node itself, covering the NOT keyword and the - and ! prefixes.
• IsExcluded() on the nearest enclosing parenthesized group, or the root group when no enclosing parenthesized group exists.

The ancestor search starts at the node's parent and skips non-parenthesized intermediate groups. A node without a parent is checked only for its own exclusion.

It returns false for a null node or when the node carries a + prefix, even if NOT is also present. This override applies only to the node's own prefix, not an ancestor's +. Otherwise, the two exclusion checks are combined with logical OR: an excluded node inside an excluded group still reports true; double negation is not canceled.

It does not look past that nearest parenthesized group, so two nodes in the same query can disagree. In -(field:(value)) both the outer and inner groups report true, but the term inside field:(value) reports false, because its nearest group is parenthesized without being excluded. In the flatter -(field:value) that same term reports true.

It is also not a check of a term's effective negation, even inside its own group. In NOT (status:active AND region:us) the status:active term reports true, but the query only excludes records that are both active and in the US, so an active record outside the US still matches. A per-node answer describes the term's syntactic context; it does not describe how that term participates in the overall condition.

Use it for node-local and immediate-group negation - its one production use is choosing a sort direction - not to ask whether a node is negated anywhere up the tree, and not to reason about what a query will match. The depth limit is intentional, decided in #279: making it parity-correct over the full ancestor chain would be a behavior change, since -(-x) would then have to cancel, and it will only be revisited if a real query consumer needs those semantics.

Compatibility

The fix in #277 changes the result for the inner parenthesized GroupNode in -(field:(value)) from false to true. Custom visitors using this public helper should account for enclosing-group exclusion; its signature, null handling, + override, and term-node behavior are unchanged.

Ordering contexts interpret these operators differently

Outside of query contexts these operators are interpreted as ordering, not negation, and only + (ascending) and - (descending) are accepted:

• Sort: DefaultSortNodeExtensions calls IsNodeOrGroupNegated(). For standalone fields, -field sorts descending and field / +field sort ascending. Unprefixed terms can inherit group direction; -(price name +rank) sorts price and name descending, with rank ascending because of its own +.
• Aggregations: CombineAggregationsVisitor reads Prefix directly and honors - (descending) and + (ascending) on a sub-aggregation.
• Boolean negation is rejected: the built-in string-based sort and aggregation APIs reject ! and NOT, including NOT +field, before returning generated output. Previously, !field and NOT field sorted descending, NOT +field sorted ascending, and aggregation negation could be silently ignored. The lower-level AST build overloads do not rerun validation; see Ordering Operators.

Do not use IsExcluded() to interpret sort or aggregation direction.

Node Data Dictionary

Nodes have a data dictionary for storing metadata:

// Store custom data
node.Data["CustomKey"] = "CustomValue";

// Retrieve data
var value = node.Data.GetValueOrDefault("CustomKey");

// Extension methods for common data
node.SetOriginalField("alias");
string original = node.GetOriginalField();

node.SetTimeZone("America/New_York");
string tz = node.GetTimeZone();

Visitor Base Classes

Base Class	Description
QueryNodeVisitorBase	Non-mutating visitor
MutatingQueryNodeVisitorBase	Can modify nodes
QueryNodeVisitorWithResultBase<T>	Returns a result
ChainableQueryVisitor	Can be chained
ChainableMutatingQueryVisitor	Chainable and mutating

Next Steps

• Nested Queries and Visitor Traversal - Nested query handling and traversal details
• Custom Visitors - Create your own visitors
• Elasticsearch Integration - Elasticsearch-specific visitors
• Field Aliases - Field resolution