Query messages using full-text search
All messages sent to elmah.io, are indexed in Elasticsearch. Storing messages in a database like Elasticsearch, opens up a world of possibilities. This article explains how to query your log messages using full-text search, Search Filters and Lucene Query Syntax.
The easiest approach to start searching your log messages, is by inputting search terms in the Search field on elmah.io:
We don't want to get into too much details on how full-text work in Elasticsearch. In short, Elasticsearch breaks the query into the terms implement and icontroller and tries to match all log messages including those terms. Full-text search work on analyzed fields in Elasticsearch, which means that wildcards and other constructs are fully supported.
Full-text queries work great. when you want to do a quick search for some keywords like part of an exception message or stacktrace. Remember that the entire log message is search, why a search for 500 would hit both log messages with status code 500 and the term 500 in the stacktrace.
Search filters are built exclusively for elmah.io. They are built on top of Lucene Query Syntax (which we'll discuss in a minute), but much easier to write. Search filters are available through either the Add filter button below the search field or using various links and icons on the elmah.io UI.
Let's say we want to find all errors with a status code of 500:
Adding the two filters is possible using a few clicks.
As mentioned previously, search filters are available througout the UI too. In this example, a filter is used to find messages not matching a specified URL:
Search filters can be used in combination with full-text queries for greater flexibility.
Lucene Query Syntax
Elasticsearch is implemented on top of Lucene; a high-performance search engine, written entirely in Java. While Elasticsearch supports a lot of nice abstractions on top of Lucene, sometime you just want close to the metal. This is when we need to introduce you to Lucene Query Syntax. The query syntax is a query language similar to the WHERE part of a SQL statement. Unlike SQL, the query syntax supports both filters (similar to SQL) and full-text queries.
Basically all Lucene queries are made up of strings containing one or more terms and operators:
term AND term OR (term AND NOT term)
NOT you can use operators known from C#:
term && term || (term && !term)
NOT speaks for itself, terms needs a bit of explanation. A term can be a single term or a phrase. We've already seen two single terms in the full-text search example. The query in the example corresponds to this Lucene query:
implement AND IController
Looking at term phrases, things get really interesting. With phrases, you can query on specific fields, perform range queries and much more. Examples are worth a thousand words, why the rest of this document is examples of frequently used queries. If you think that examples are missing or have a problem with a custom queries, let us know. We will extend this tutorial with the examples you need.
Find messages with type
Find messages with status codes
statusCode:[500 TO 599]
Find messages with URL and method
url:"/tester/" AND method:get
Find messages with URL starting with
Forward slash in the beginning needs to be escaped, since Lucene will understand it as the start of a regex otherwise.
Find messages by IP
Find messages by IP's
The examples above can be achieved using Search Filters as well. We recommend using Search Filters where possible and fall back to Lucene Query Syntax when something isn't supported through filters. An example is using OR which currently isn't possible using filters.
This article was brought to you by the elmah.io team. elmah.io is the best error management system for .NET web applications. We monitor your website, alert you when errors start happening and help you fix errors fast.