Visitors

Foundatio.Parsers uses the visitor pattern for AST traversal and transformation. Visitors allow you to inspect, modify, or extract information from parsed queries.

Visitor Pattern Overview

When a query is parsed, it creates an Abstract Syntax Tree (AST) of nodes. Visitors traverse this tree, performing operations on each node type.

Traversal Order

Visitors traverse the AST depth-first, left-to-right. The GroupNode.Children property yields Left first, then Right. The base class QueryNodeVisitorBase.VisitAsync(GroupNode) iterates these children in order but does not act on the GroupNode itself -- it only recurses into children.

Whether a visitor processes a GroupNode before or after its children depends on where the subclass places its logic relative to the base.VisitAsync() call:

• Pre-order (process node, then children): FieldResolverQueryVisitor, NestedVisitor
• Post-order (process children, then node): CombineQueriesVisitor, CombineAggregationsVisitor
• Wrap-around (before and after children): ValidationVisitor
• Short-circuit (skip children): InvertQueryVisitor (when the entire group can be inverted at once)

When multiple visitors are chained via ChainedQueryVisitor, each visitor completes a full traversal of the entire tree before the next visitor begins, in priority order (lower priority number runs first).

For detailed coverage of how visitors handle nested and hierarchical query structures, field scoping, and the @field:(-@field:(...)) pattern, see Nested Queries and Visitor Traversal.

Built-in Visitors

GenerateQueryVisitor

Converts an AST back to a query string:

using Foundatio.Parsers.LuceneQueries;
using Foundatio.Parsers.LuceneQueries.Visitors;

var parser = new LuceneQueryParser();
var result = parser.Parse("status:active AND created:>2024-01-01");

// Generate query string from AST
string query = GenerateQueryVisitor.Run(result);
// Output: "status:active AND created:>2024-01-01"

// Async version
query = await GenerateQueryVisitor.RunAsync(result);

With Default Operator

var context = new QueryVisitorContext {
    DefaultOperator = GroupOperator.And
};

var result = parser.Parse("value1 value2");
string query = await GenerateQueryVisitor.RunAsync(result, context);
// Output: "value1 AND value2"

DebugQueryVisitor

Outputs the AST structure for debugging:

var parser = new LuceneQueryParser();
var result = parser.Parse("(status:active OR status:pending) AND type:user");

string debug = DebugQueryVisitor.Run(result);
Console.WriteLine(debug);

Output:

Group:
  Operator: And
  Left - Group:
    HasParens: True
    Operator: Or
    Left - Term:
        Field: status
        Term: active
    Right - Term:
        Field: status
        Term: pending
  Right - Term:
      Field: type
      Term: user

FieldResolverQueryVisitor

Resolves field aliases:

var parser = new LuceneQueryParser();
var result = await parser.ParseAsync("user:john");

var fieldMap = new FieldMap {
    { "user", "data.user.identity" }
};

var resolved = await FieldResolverQueryVisitor.RunAsync(result, fieldMap);
// Result: data.user.identity:john

IncludeVisitor

Expands query includes/macros:

var parser = new LuceneQueryParser();
var result = await parser.ParseAsync("@include:active");

var includes = new Dictionary<string, string> {
    { "active", "status:active AND deleted:false" }
};

var expanded = await IncludeVisitor.RunAsync(result, includes);
// Result: (status:active AND deleted:false)

ValidationVisitor

Validates queries against rules:

var parser = new LuceneQueryParser();
var result = await parser.ParseAsync("status:active");

var context = new QueryVisitorContext();
context.SetValidationOptions(new QueryValidationOptions {
    AllowedFields = { "status", "name" }
});

await ValidationVisitor.RunAsync(result, context);

var validation = context.GetValidationResult();
Console.WriteLine($"Valid: {validation.IsValid}");

InvertQueryVisitor

Inverts query negation:

var parser = new LuceneQueryParser();
var result = await parser.ParseAsync("status:active");

var invertVisitor = new InvertQueryVisitor();
var inverted = await invertVisitor.AcceptAsync(result, new QueryVisitorContext());

string query = inverted.ToString();
// Output: "(NOT status:active)"

With Non-Inverted Fields

// Some fields should not be inverted
var invertVisitor = new InvertQueryVisitor(
    nonInvertedFields: new[] { "organization_id", "tenant_id" });

var result = await parser.ParseAsync("status:active organization_id:123");
var inverted = await invertVisitor.AcceptAsync(result, context);
// Output: "(NOT status:active) organization_id:123"

RemoveFieldsQueryVisitor

Removes specific fields from queries:

var parser = new LuceneQueryParser();
var result = await parser.ParseAsync("status:active AND secret:value AND name:john");

var removeVisitor = new RemoveFieldsQueryVisitor(new[] { "secret" });
var cleaned = await removeVisitor.AcceptAsync(result, new QueryVisitorContext());

string query = cleaned.ToString();
// Output: "status:active AND name:john"

CleanupQueryVisitor

Simplifies and cleans up the AST:

var parser = new LuceneQueryParser();
var result = await parser.ParseAsync("((status:active))");

// Returns the cleaned query string
string cleaned = await CleanupQueryVisitor.RunAsync(result);
// Removes unnecessary nesting

TermToFieldVisitor

Converts standalone terms to field queries (used for sort expressions):

var parser = new LuceneQueryParser();
var result = await parser.ParseAsync("-created +name");

await TermToFieldVisitor.RunAsync(result);
// Converts terms to field nodes for sort processing

Chained Visitors

Multiple visitors can be chained together with priority ordering:

using Foundatio.Parsers.LuceneQueries.Visitors;

var chainedVisitor = new ChainedQueryVisitor();

// Add visitors with priority (lower runs first)
chainedVisitor.AddVisitor(new FieldResolverQueryVisitor(fieldResolver), priority: 10);
chainedVisitor.AddVisitor(new IncludeVisitor(), priority: 20);
chainedVisitor.AddVisitor(new ValidationVisitor(), priority: 30);

// Run all visitors
var result = await chainedVisitor.AcceptAsync(ast, context);

Managing Chained Visitors

var chain = new ChainedQueryVisitor();

// Add visitor
chain.AddVisitor(new MyVisitor(), priority: 100);

// Remove visitor by type
chain.RemoveVisitor<MyVisitor>();

// Replace visitor
chain.ReplaceVisitor<OldVisitor>(new NewVisitor(), newPriority: 50);

// Add before/after specific visitor
chain.AddVisitorBefore<ValidationVisitor>(new MyVisitor());
chain.AddVisitorAfter<FieldResolverQueryVisitor>(new MyVisitor());

Visitor Context

Visitors receive a context object for sharing data:

var context = new QueryVisitorContext();

// Set values
context.SetValue("UserId", "123");
context.SetValue("IsAdmin", true);

// Get values
string userId = context.GetValue<string>("UserId");
bool isAdmin = context.GetValue<bool>("IsAdmin");

// Default operator
context.DefaultOperator = GroupOperator.And;

// Default fields for unqualified terms
context.DefaultFields = new[] { "title", "description" };

// Query type
context.QueryType = QueryTypes.Query; // or Aggregation, Sort

Context Extensions

// Field resolver
context.SetFieldResolver(async (field, ctx) => {
    return fieldMap.GetValueOrDefault(field);
});
var resolver = context.GetFieldResolver();

// Include resolver
context.SetIncludeResolver(async name => includes.GetValueOrDefault(name));
var includeResolver = context.GetIncludeResolver();

// Validation
context.SetValidationOptions(options);
var validation = context.GetValidationResult();
context.AddValidationError("Error message", position: 5);
context.ThrowIfInvalid();

AST Node Types

Visitors can handle different node types:

Node Type	Description	Key Properties
GroupNode	Boolean group	Left, Right, Operator, HasParens
TermNode	Single term	Field, Term, IsQuotedTerm, Boost, Proximity
TermRangeNode	Range query	Field, Min, Max, MinInclusive, MaxInclusive
ExistsNode	Existence check	Field
MissingNode	Missing check	Field

Node Properties

// Common to all field nodes
string field = node.Field;
string unescapedField = node.UnescapedField;
bool? isNegated = node.IsNegated; // the NOT keyword; also set by tree-rewriting visitors
string prefix = node.Prefix; // +, -, !, or null

// TermNode specific
string term = termNode.Term;
bool isQuoted = termNode.IsQuotedTerm;
bool isRegex = termNode.IsRegexTerm;
string boost = termNode.Boost;
string proximity = termNode.Proximity;

// TermRangeNode specific
string min = rangeNode.Min;
string max = rangeNode.Max;
bool minInclusive = rangeNode.MinInclusive;
bool maxInclusive = rangeNode.MaxInclusive;

// GroupNode specific
IQueryNode left = groupNode.Left;
IQueryNode right = groupNode.Right;
GroupOperator op = groupNode.Operator; // And, Or, Default
bool hasParens = groupNode.HasParens;

Negation and Prefix Operators

Negation is stored in two different places depending on the syntax used:

Query	IsNegated	Prefix
NOT field:value	true	null
NOT [1 TO 2]	true	null
-field:value	null	"-"
!field:value	null	"!"
+field:value	null	"+"
field:value	null	null
field:[1 TO 2]	false	null
_exists_:field	false	null

Why IsNegated is a bool? and not a bool

The name reads like a boolean, so the nullability invites the question of whether null means something false does not. For query rendering it does not: flipping any non-negated node between null and false across a corpus of nested, grouped and negated queries produces byte-identical output from both CleanupQueryVisitor and GenerateQueryVisitor, including through the group-collapse and double-negative folding. Only true changes the result. Every consumer in the repo tests HasValue && Value or is true, so none of them distinguishes the two.

Two places do observe the difference, so null and false are not strictly interchangeable:

• DebugQueryVisitor prints an IsNegated line only when the value is set, so false adds a line that null omits.
• CopyTo copies the value only when it is set, so copying a null source over a true target leaves true, while copying a false source overwrites it with false.

Neither is a semantic distinction worth relying on, and which value a non-negated node receives varies by grammar rule (see the table above) with no meaning attached. The practical rules are:

• Compare against true (node.IsNegated is true); never treat it as a plain boolean and never branch on false versus null.
• Never call .Value without checking HasValue.
• Better still, call IsExcluded(), which handles all of this along with the - and ! prefixes.

Narrowing the property to a non-nullable bool would be a reasonable cleanup, but it is a breaking change to the public IFieldQueryNode interface and to every node type and custom visitor implementing it, so it belongs in a major version rather than here.

Visitors that rewrite the tree (InvertNegation, CleanupQueryVisitor) also set this value, including to false.

The split between the two properties is intentional: keeping the operator that was actually written means GenerateQueryVisitor and ToString() re-emit -value as -value rather than rewriting it to NOT value. As a result, IsNegated alone is never a complete negation check.

Round-tripping preserves the operator, not always its position. A prefix or NOT written inside a scoped field group is re-emitted in the canonical leading position, so field:-(value) renders as -field:(value) and field:NOT (value) as NOT field:(value). These are semantically equivalent.

In query contexts, use the extension methods instead of inspecting the properties directly:

• IsExcluded() - node-local negation, covering NOT, -, and !
• IsRequired() - the + prefix
• IsNodeOrGroupNegated() - IsExcluded() plus negation on the nearest enclosing parenthesized group. When called on a GroupNode that already has parens, GetGroupNode() returns that same node, so only the group's own negation is considered and an excluded parent group is not inspected.

// Wrong: misses the ! prefix and any negation on the enclosing group
bool isNegated = node.IsNegated.GetValueOrDefault() || node.Prefix == "-";

// Right
bool isNegated = node.IsExcluded();

// Right, when negation on the enclosing group should also apply
// e.g. the value term in -field:(value)
bool isNegated = node.IsNodeOrGroupNegated();

Two caveats worth knowing:

• IsNodeOrGroupNegated() walks only up to the nearest parenthesized group, not the whole ancestor chain, so the inner group in NOT (a:(b)) reports false. It also returns false when the node carries a + prefix even if NOT is also present.

• Outside of query contexts these operators are interpreted as ordering, not negation, and the set of operators that is honored differs:

  • Sort: DefaultSortNodeExtensions calls IsNodeOrGroupNegated(), so -field, !field, and NOT field all sort descending. Because that helper ignores negation when + is present, NOT +field sorts ascending.
  • Aggregations: CombineAggregationsVisitor reads Prefix directly and only honors - (descending) and + (ascending) on a sub-aggregation. ! and the NOT keyword produce no order at all.

  Do not use IsExcluded() to interpret sort or aggregation direction.

Node Data Dictionary

Nodes have a data dictionary for storing metadata:

// Store custom data
node.Data["CustomKey"] = "CustomValue";

// Retrieve data
var value = node.Data.GetValueOrDefault("CustomKey");

// Extension methods for common data
node.SetOriginalField("alias");
string original = node.GetOriginalField();

node.SetTimeZone("America/New_York");
string tz = node.GetTimeZone();

Visitor Base Classes

Base Class	Description
QueryNodeVisitorBase	Non-mutating visitor
MutatingQueryNodeVisitorBase	Can modify nodes
QueryNodeVisitorWithResultBase<T>	Returns a result
ChainableQueryVisitor	Can be chained
ChainableMutatingQueryVisitor	Chainable and mutating

Next Steps

• Nested Queries and Visitor Traversal - Nested query handling and traversal details
• Custom Visitors - Create your own visitors
• Elasticsearch Integration - Elasticsearch-specific visitors
• Field Aliases - Field resolution