CodeQue basic documentation

CodeQue is structural code search tool for JavaScript and TypeScript projects.

CodeQue can be used to search for any code, from simple symbol search to complex multiline patterns.

It can be used as a linter by asserting search results to be found or not found in the codebase.

It's designed to be used on your device and help you during code development.

It's available as:

• Handy CLI tool for code search
• Visual Studio Code extension for code search and refactoring
• ESLint plugin for creating custom rules in zero time

Looking for CLI docs? Check GitHub readme. This is general CodeQue documentation.

In syntax aware search modes the query has to be a valid TypeScript code.

Once you type your query, it's then parsed by babel to AST.

The code pattern represented by AST is then matched within all TypeScript or JavaScript source files (excluding git-ignored).

So it's simply(?) trying to find matching sub tree (query) in given file AST.

CodeQue also offers text search mode which is generating a regular expression based on the query. Text search mode is useful for initial discovery, but it is not that accurate as syntax aware search

CodeQue contains a few search modes which can be used to achieve different search results.

Include is a default search mode. It's the less strict and allows to match code which contains more properties than query.

It also ignores order of the enumerable properties like keys in objects, properties in react components or statements in blocks of code.

It also matches code that has missing optional properties from query

Ok, I need an example...

A hypothetical query:

Can match this code:

Explanation:

• console.log()
  call expression and
  if
  statement are enumerable elements of block statement, so they can be in different order,
• function body contains return statement which is missing in query, but include search mode allows for additional nodes in the matched code,
• parameters declaration is missing type annotation in query, but types annotations are optional in TypeScript, so the code is matched.

If you wonder which other elements of the language are optional, you can explore

import { NODE_FIELDS } from '@babel/types'

in @babel/types

It works in the same way as include mode, but checks for the order of the statements.

However forced order doesn't mean there can't be any additional property in between! Previous query will also match this code:

As the name suggest, exact mode matches only exactly the same piece of code.

What makes conventional search useless for me are:

• lack of easy way of adding wildcards
• matching whitespaces exactly as they are defined

CodeQue address these two problems by generating a regular expression out of query. RegExp is then used to match patterns across source files and - unlike in the syntax aware modes - treat code as string.

Such a RegExp matches any whitespace in place of a specific whitespace from a query and matches optional whitespaces between keywords and punctuators in the code.

What ?

It basically means that code can have optional whitespaces in places where they are not affecting the code logic.

So i.e you can look for a given variable declaration and find it on different levels of indentation. Unlike in normal search, where indentation matters.

If you type a query like

it will be transformed into regexp (pseudo code):

So both

and

will be found!

But not the following, as it includes comma

The simplest way to start writing query is to copy some existing piece of code.

Then replacing some identifiers or statements with wildcards, which makes "gaps" in the query, so more similar code patterns can be found.

Wildcards is what makes CodeQue search so powerful!

They acts differently depending on whether they are used outside or inside strings.

Non string wildcards are:

• a identifier wildcard represented by
  $$
• a statement/expression wildcard represented by
  $$$

String wildcards are:

• optional string content which can be used as
  "$$"
• required string content (one or more chars) which is
  "$$$"

For number wildcards use

Example

Query

will match any variable declaration initialized with non-empty string, so say:

• const someVar = "value"
• const someNewVar = "new value"
• const x = "another value"

Example 2

Wildcards can be combined with other characters.

Query

will match the following code from previous example:

• const someVar = "value"
• const someNewVar = "new value"

but will not match the last one, due to mismatch in variable identifier.

You've probably noticed that each wildcard in code snippets is red and bold.

Example 3

Let's say you found out that developers in your project often assign a result of

The query might be

Regardless of how the provided callback would look like, this query would match all of them:

• callback might be a variable identifier
• callback might be anonymous function

The statement wildcard

Example 4

Wildcards are also working for TypeScript types.

You can look for some duplicated types with such queries:

Due to the fact a query is a valid code, it might be confusing to prepare some of them.

While looking for objects with some keys, you might try to write such a query:

Which is a valid code, but it's not an object. Parser interprets that as block of code!

Paste that in AST Explorer and check the output

To look for object, you have to add additional brackets:

Remember

Most of us, including me, do not actually remember what it does 😀.

If you write query with just apostrophes and content, parser would interpret that as a directive instead of a string.

So the desired query has to have brackets - like in the object example:

I've come up with these four patterns to act as "match anything" expressions:

• $$
  - 0 or more chars in the same line,
• $$m
  - 0 or more chars multiline,
• $$$
  - 1 or more chars in the same line,
• $$$m
  - 1 or more chars multiline.

Wildcards in text mode can match more than you would expect. They are useful for initial discovery, but once you need more accuracy, it's better to use syntax aware search modes.

Avoid Using very generic text queries, eg. just

$$m

, because it will result in huge performance degradation. It's also not very useful 🤷‍♂️.

For each search you can enable case insensitivity.

It will affect matching both identifiers and contents inside the strings.

CodeQue is in early stage and not all codebases might be supported.

I had to choose some settings for the parser to support most of the projects.

These language features (according to babel settings) are currently not supported:

• flow & flowComments (flow would probably never be supported, I hate flow)
• v8intrinsic
• asyncDoExpressions
• decimal
• doExpressions
• exportDefaultFrom
• functionBind
• importAssertions
• moduleBlocks
• partialApplication
• pipelineOperator
• recordAndTuple
• throwExpressions

You probably don't use these features anyway.

In future it's planned to infer parser configuration out of project config files like

For the time being CodeQue offers quite comprehensive search API

In nearest future I plan to implement features like

For me It's a must have. CLI interface is usable, but not ergonomic at all.

Editor integration is the first priority just after initial release

Perhaps some other code editors will be supported in the future

CodeQue ESlint plugin will be super powerful tool for enforcing code standards.

Implementing custom eslint plugins it super time consuming and usually it's not worth doing for temporal issues.

I want to simplify writing ESLint plugins to providing code pattern to match - a CodeQue query!

It requires some refactoring though, ESlint is using different parser and AST nodes than babel.

Each search needs a replace functionality.

Replacing part of trees is quite tricky task.

I hope I will manage to implement that.

It would allow to make ad-hoc codemods without needs to manual implementations which usually takes more time than doing the replace manually.

I plan to add several query extensions like:

• linking wildcards in query so you can indicate the same identifier
• allowing for looking for statements in nested structures
• changing search modes on different levels
• adding some kind of boolean conditions like not, and, or

I'm not sure about exact API yet, but I have use cases for them.

I'm super excited for all those features!

Check project roadmap for more context.