How to Query Elasticsearch in Kibana: A Quick Start Guide to Luce and Kibana Query Languages

How to Query Elasticsearch in Kibana

Updated November 2020

Kibana Query Syntax

When querying Elasticsearch in Kibana you can either use the traditional Lucene query syntax or the newer Kibana Query Language (KQL).  If you are using Kibana 7.0 or later, Kibana Query Language is included as a default.  In this article we provide the basics for both approaches and provide example searches.

Depending on the nature of a search or your personal experience, one language might be easier for your needs than the other.  For instance, Kibana Query Language can make queries easier with the autocomplete function (available with Elastic licensed version) and some simplifications in the syntax that make inversions easier.  However, KQL has some limitations such as not supporting fuzzy or regex searches, but we expect Elastic to focus on developing KQL in the future.

Switching Between KQL and Lucene in Kibana

You can switch between Kibana Query Language and Lucene Syntax by clicking on the square on the right end of the search bar in Kibana.  It will either read KQL or Lucene depending on which is activated. Once clicked, you can toggle the Kibana Query Language button either on or off.

An alternative way to switch between KQL and Lucene is by clicking on the management button (gear icon) on the left hand side of the Kibana window and then choosing Advanced Settings.  The query language option is about the 30th setting down on the page. 

Lucene Query Language

Elasticsearch initially worked solely off of Lucene syntax since Elasticsearch is built on top of Lucene.  The sections below serve as a reference guide to Elasticsearch query syntax using Lucene.  

First, a note on documents for those new to Elasticsearch.  Below in the examples we talk about “returning documents”.  In Elasticsearch individual entries such as purchases, customers, episodes, or any other item or event recorded are referred to as documents.  For more information on the structure of Elasticsearch check out our article:  How to Index Elasticsearch.

Lucene: Bool query ➝ AND, NOT, OR

Roland AND Jocelyn ➝ returns documents that include both Roland and Jocelyn

Moira NOT Johnny ➝ returns documents that include Moira, but not those documents that include both Moira and Johnny

David OR Alexis ➝ returns documents that include either David or Alexis

Lucene: Search by Field ➝ Field name:

name: Twyla ➝ returns documents that have Twyla in the name field

location: “Bob’s Garage” ➝ returns documents that have Bob’s Garage in the location field. Here the value “Bob’s Garage” is in quotes so that the search includes the full name with space

Lucene: Ranges ➝ [ ], { }, :>, :>=, :<, :<=

season:[1 TO 6] ➝  returns documents in any season from 1 through 6, including 1 and 6

season:{1 TO 6} ➝ returns documents in any season from 2 through 5

season:>2 ➝ returns documents in any season greater than 2

Lucene: Wildcards ➝ *, ?

mu* ➝ will return documents that include values that start with “mu”, such as Mutt and Mullens

mu*s ➝ will return documents that include values that start with “mu” and end in “s”, such as Mullens

mu?s ➝ will return documents that include values that start with “mu”, end in “s”, and have one character in between, such as mugs

Lucene: Regex ➝ / [ ] /, / < > /

/w[ia]g/ ➝ will return documents that include either wig or wag

Lucene: Fuzzy ➝ ~

Stevie~ ➝ will return documents that include values similar to “Stevie” such as Stevee, Stevey, Skevy, Stephy

Ronnie~2 ➝ adding a number after the tilde sets the distance away from the word. In the case of 2, then up to two characters can be changed and it still be considered a match, such as donnie and ronney.

Lucene: Free Text

Simplest kind of query, just like typing into a Google search.

Patrick ➝ returns documents that include “Patrick” in any field

“Blouse Barn” ➝ returns documents that include “Blouse Barn” in any field

Kibana Query Language (KQL)

Kibana Query Language (KQL) was first introduced in version 6.3 and became available as a default starting with version 7.0.  This new language was built to provide scripted field support and to simplify the syntax compared to the Lucene language discussed above.

When using the Elastic licensed versions of ELK, even the basic version, autocomplete is standard for KQL queries.  Autocomplete is not available at this time with either KQL syntax with the Apache version of Elasticsearch or for Lucene syntax with either the Elastic or Apache licensed versions of the Elastic Stack.

Changes for how the language works compared to Lucene are shown in bold font and examples follow below.  Now let’s take the examples we looked at above but now using KQL. 

KQL: Bool query ➝ AND, and • OR, or • AND NOT, and not

Boolean operators are not case sensitive with Kibana Query Language. AND ↔ and, OR ↔ or, AND NOT ↔ and not are all interchangeable.

Roland AND Jocelyn ➝ returns documents that include both Roland and Jocelyn

Roland and Jocelyn ➝ returns documents that include both Roland and Jocelyn

David OR Alexis ➝ returns documents that include either David or Alexis

David or Alexis ➝ returns documents that include either David or Alexis

NOT is changed to AND NOT or and not.

Moira AND NOT Johnny ➝ returns documents that include Moira, but not those documents that include both Moira and Johnny

Moira and not Johnny ➝ returns documents that include Moira, but not those documents that include both Moira and Johnny

By default and has higher precedence than or. Parentheses can be used to override this default.

David and (Patrick or Stevie) ➝ returns documents that include David and either Patrick or Stevie

Using not before a search term will invert its meaning.

not location:”Blouse Barn” ➝ returns documents that do not have Blouse Barn listed as their location

Entire groups can be inverted by using parentheses.

not (name:Patrick or location:”Rose Apothecary”) ➝ returns documents that do not have Patrick as the name or Rose Apothecary as the location

KQL: Search by Field ➝ Field name:

name: Twyla ➝ returns documents that have Twyla in the name field

location: “Bob’s Garage” ➝ returns documents that have Bob’s Garage in the location field. Here the value “Bob’s Garage” is in quotes so that the search includes the words Bob’s and Garage in that order.  Without the quotes, then results would also include Garage Bob’s. 

Kibana query language has a shorthand for searching a single field for multiple values.

location: (“Bob’s Garage” or “Café Tropical”) ➝ returns documents that have either Bob’s Garage or Café Tropical listed as the location

tags: (black and white and formal) ➝ returns documents that have all three black, white, and formal listed as tags

KQL: Ranges ➝ >, >=, <, <=

The colons are removed before the greater than, less than, etc. signs for searches.

season>2 ➝ returns documents in any season greater than 2

KQL: Wildcard ➝ *

The * wildcard is used in KQL.

mu* ➝ will return documents that include values that start with “mu”, such as Mutt and Mullens

mu*s ➝ will return documents that include values that start with “mu” and end in “s”, such as Mullens

Exist queries are carried out by adding the asterisk after the field name.

room_number:* ➝ returns documents where a response field exists for the field “room_number”

Can use wildcards to search text and keyword versions of a field simultaneously.

For the below example we are searching both the plot and plot.keyword fields for Galápagos.

plot*:Galápagos ➝ returns documents that have Galápagos for both the plot and plot.keyword fields

KQL: Free Text

Free text in KQL works the same as Lucene syntax.  It’s the simplest kind of query, just like typing into a Google search.

Patrick ➝ returns documents that include “Patrick” in any field

“Blouse Barn” ➝ returns documents that include “Blouse Barn” in any field

To Sum Up

Both KQL and Lucene syntax can be used to query Elasticsearch in Kibana.  The above sections provide a foundation for building searches in Kibana.  If you have questions for us, feel welcome to reach out to our RSAs or leave a comment below.

External references

Elasticsearch Support with Elastic Certified Engineers

Dattell’s Elastic Certified Engineers work one-on-one with companies to design, implement, manage, and improve their Elasticsearch deployments.  Pricing for Elasticsearch support services starts at $2,400.

Leave a Reply