Skip to main content

RegExTractor

RegExTractor is a tool for Microsoft Windows. It's main purpose is to generate structured xml files from mostly unstructured application log files. For example, these xml files could be processed within Microsoft Excel (or Libre Office Calc) to generate graphical reports about the frequency of a special log entry, maybe a special error message.




The logs are split by regular expressions you have to provide. Each match will result in a new tag in the xml result file.


User documentation is still growing but I've created some quick tutorials:

Getting Started - Part 1
Getting Started - Part 2


Remark: You will need knowledge about regular expression and xsl transformation in order to use RegExTractor. A use case how to create a diagram like the one enough you can find here:

Postprocessing RegExTratcor output for analysis

Comments

Popular posts from this blog

RegExTractor: Getting Started (Part 1)

In this tutorial I like to show you the main functionalities of RegExTractor. RegExTractor is build for complex searches. Keep in mind not to use RegExTractor if there is an easier way to accomplish your task!

You may download the latest release of RegExTractor on GitHub.

For this example we have a file folder containing some application log files:



These files look like this one here and we're interested in how often the application has been started.

Create a search term file A search term file is a simple text file and as the name implies this file will contain all our search terms. In our simple example we will search for a single text term: "Application Started".

Search with RegExTractor
Now we open RegExTractor and choose the file folder which contains our example files. Decide if you like to search recursive in sub folders or if you like to search the top folder only. You may also apply a filter for files, if maybe just files with the extension *.log are in scope.



RegExTractor: Getting Started (Part 2)

In this post I will show how RegExTractor will use regular expressions as search terms.

As seen in part 1 of this tutorial RegExTractors search result show us all findings of the provided search term "Application started". But this is not our goal. We'd like to know all dates and times when our application was started.

RegExTractor supports regular expressions. It's assumed that you're familiar with regular expressions.

The things we are interested in is the date, the time and the text "Application started". So we build our regular expressions using brackets to define our match groups.
(\d{2}.\d{2}.\d{2}) (\d{2}:\d{2}:\d{2}).+?(Application Started) We create a search term file as described in part 1 using this more complex regular expression as search term instead of just the simple search string. The result looks like this:

Doing the regular expression with .NET Framework functions the search will return the whole match of our regular expression as <…

Postprocessing RegExTratcor output for analysis

In order to analyze your data in a chart tool like Excel, you have to to manage five steps.
Define your search termsSearch your files with RegExTractorCreate a transformation fileTransform your xmlImport the transformed xml into Excel and analyze your data. RegExTractor don't want to reinvent the wheel. It's just closing a gap. It enables you to "convert" a text file (or a part of it) into xml. I've already explained the main principles of point 1 and 2 in the "Getting Started" tutorials. And this is all RegExTractor is doing for you.

For the next steps we'd like to use mature tools and technologies instead of inventing new ones. In this post I'd like to show, how to go the whole way to get an Excel chart out of your data.

Remember the example log from the "Getting Started" tutorials. As this is a log from one of my applications I know, that every message contains the class name and the method from which the log entry was written.


What…