FreeMarker Expressions

DarkLight uses the Apache FreeMarker Template Language as a way to store and retrieve values from variables. FreeMarker can be very powerful, but many uses of it in DarkLight are basic and used to refer to values in the package table.

You'll see these words mentioned when we talk about making playbooks.

  • Package: A JSON-formatted file that DarkLight creates when an event enters the system. The package passes through the steps in one or more playbooks and contains all of the information from the originating event plus any variables and graphs the playbooks add. The playbook has a section for variables (the miscData) and a section for graphs. The primary graph in a package is named _default_.
  • Variable: The name given to something you want to refer to later. Variables are stored in the Package miscData. When content arrives to package from a Data Feed, it is put into a variable called rawInput by default. If a step asks for a variable, it does not need to use Freemarker syntax.
  • Value: The contents of a variable, regardless of how many individual items are in the variable. Values can be in the form of a single item, or as an array of elements in rows and columns. If a step asks for a Value, it must be written in FreeMarker syntax ${likeThis} ${orThis[0]} ${orSometimesThis[0][0] and will have the FreeMarker icon next to it.

Many steps save their results in an output variable. Common steps used to store values in variables are Step: Query Package, Step: Query Knowledge Base, and most of the steps in the Transform and Query categories.

Variables can be one of three types, and many of the steps will indicate which type they output:

Icon Output Type FreeMarker Version
Single Value (Scalar) ${examplevar}
List (One-Dimensional Array) ${examplevar[0]}
Table (Two-Dimensional Array) ${examplevar[0][0]
This SELECT query will output a Table variable, even if there is only one result. When referring to this variable later, you would use the form department[0][0]

Anywhere you see the icon next to an input box, you can use FreeMarker syntax to have that expression replaced by a variable's data in your package.

FreeMarker Template References

The Inventory View can help you know which FreeMarker templates you need to use to reference a specific value in a variable. In addition to the label hints, you can click on a value and copy the generated FreeMarker expression from the Template field.

  • Single Value (Scalar) ${ }
  • List (One-Dimensional Array) ${[*]}
  • Table (Multi-Dimensional Array) ${[*][*]}
  • ${trigger} - the IRI of the incoming event, the one that triggered the playbook
  • ${uuid()} - creates a randomly generated UUID
  • ${pkgdata("myprefix:dataProperty")} - returns the value of the data property from the default graph (more details below)
  • ${packageAsJSON()} - returns the complete DarkLight package from the current playbook. Can be used with Web Request step to send a package to another instance of DarkLight (via the HTTP Post Data Feed).
  • ${prefix("myprefix")} - Use at the top of a SPARQL query to convert a prefix into an IRI. For example, ${prefix("attack")} is equivalent to PREFIX attack: <tag:darklight:attack#> and then in the body of the query you can use the prefix and predicate (attack:T1037) instead of the IRI version (<tag:darklight:attack#T1037>). Note that the separator between prefix and predicate is a colon : when using a prefix and a the number sign # when using an IRI. Also note that when using a prefix in the query, you do not use the angle brackets.

FreeMarkers calls these built-ins

  • To represent a string as a number, add ?number ( details)
    • ${downloadsize[0][0]?number}
    • ${downloadsize[0][0]?number / 1024} (divides downloadsize by 1024)
  • To use a large number in calculations (i.e. removing thousands commas), add ?c ( details)
    • ${downloadsize[0][0]?c}
  • To encode a string for use in a URL, e.g. in the Web Request Step, add ?url to the end (DarkLight uses UTF-8 by default) ( details)
    • /my/path/?param=${somestringwithspaces?url}
  • To work with dates and times, you can do the following:
    • ${.now} returns the current date: Sep 11, 2018 2:59:02 PM
    • ${.now?string.iso} returns the date in ISO: 2018-09-11T15:00:25.876-07:00
    • ${.now?string["dd"]} returns just the day: 11
    • ${myDateVar?datetime["yyyy-M-d'T'HH:mm:ss.SSSXXX"]} takes a string input like "2018-09-11T15:00:25.876-07:00" and outputs it as "Sep 11, 2018 3:00:25 PM"
    • ${myDateVar?long} takes a datetime input and outputs Unix epoch time in seconds. For example, "2018-09-11T15:00:25.876-07:00" becomes 1536703225
    • Using the ?long built-in, you can do math with dates, too:
      #subtract 15 minutes
      ${(.now?long - 15 * 1000 * 60)?number_to_datetime?string.iso}
      
      #subtract 2 hours
      ${(.now?long - 2 * 1000 * 60 * 60)?number_to_datetime?string.iso}
      
      #subtract 7 days
      ${(.now?long - 7 * 1000 * 60 * 60 * 24)?number_to_datetime?string.iso}

FreeMarker is sometimes too helpful when it comes to escaping special characters in strings. In many cases, you'll appreciate it automatically escaping an angle bracket "<" into \< or a curly brace into \{ but other times that character might be something you actually want.

To prevent FreeMarker from escaping the output, add ?no_esc to the end. For instance, ${myvar[0]?no_esc}

You can also add it to the end even if you're already using another modifier, like replace${eventDetails[0][1]?replace("\\", "\\\\")?no_esc}

This FreeMarker Expression lets you check to see if a variable exists before you try to use it. This prevents the expression from exiting early with an error, negating the entire message (and step!).

This example checks for the variables username and computerName before it uses them. If the variable does not exist, it will print out the text that comes after the #else.

Alert on the demo feed by <#if username??>${username[0][0]}<#else> (unknown user)</#if> on <#if computerName??>${computerName[0][0]}<#else> (unknown computer)</#if>

(See: If/Else FreeMarker Directive)

Sometimes you will do a query and get back a list of things that you want to express using FreeMarker. This example uses a Package Query of select ?type where {<${trigger}> a ?type} to store all of the classes assigned to the incoming event in a variable called "types". Then, the Post to Slack or E-mail step can use a FreeMarker Directive like this:

CTCI Alert by <#if username??>${username[0][0]}<#else> (unknown user)</#if>
 on <#if computerName??>${computerName[0][0]}<#else> (unknown computer)</#if>

<#list types as type>
• ${type[0]?keep_after_last("#")}
<#else>
(no types)
</#list>

Event ID: ${trigger?keep_after("#")}

DarkLight Server: 192.168.2.222
(See: List Directive)

The Package Query returns a list of full IRI's, like http://www.champtc.com/ontologies/winevent#T4688 so it also uses the ?keep_after_last option to only print out what comes after the pound sign. (See: Keep After Last)

IF and LIST can be used together To ensure that the step runs, you may want to wrap a List directive with an If directive. For example:

<#if queryResult??>
<#list queryResult as item>
<li> ${item}
</#list>
<#else>No items
</#if>

The above example will print out each value in the list of the queryResult variable if one exists, or it will print out "No items" if the variable does not exist.

In many cases you'll use a pattern where you use a Query Package step to pull a value out of a data property (like username, or event time) and store it as a variable to use in another step (like a Query Database, or Date Normalize).

With the pkgdata() function, you can get the value without needing the Query Package step first. It's also very useful in the notification steps, like Send E-mail or Post to Slack.

Syntax

The basic syntax is ${pkgdata("prefix:dataProperty")}

Notes and Best Practices for pkgdata()

  • It can only match the data properties attached to the primary individual of a graph. For example, if you have an event (primary individual) with an employee object attached to it via an object property, you'll only be able to retrieve the data properties from the event.
  • If the primary object has multiple properties with the same name, only one value will be returned.
  • Each time it is used, a query runs in the background which uses system resources. If the playbook will be using the value of the chosen data property many times in the playbook, you probably will want to use the Set Value step to assign the property to a package variable and then use the variable in the remaining steps.
  • When used in a SPARQL statement, a text string will be returned, so make sure the whole expression is surrounded by quotes. "${pkgdata("")}"

Examples

In a SPARQL Query

#Query Database step example showing a SPARQL query to return the graph
#that represents the employee who's account name matches the event's subject user name.
#Note that ent: and winevent: must be prefixes used in your Ontology Manger

construct {?s ?p ?o} where { 
  ?s a ent:Employee. 
  ?s ent:hasAccountName "${pkgdata("winevent:hasSubjectUserName")}"
}

As the Input Value for a Step

It can also be used anywhere you can use other FreeMarker expressions.

In a Notification like E-Mail or Slack

*PowerShell Launched With Parameters*
>PowerShell was either launched from itself, or used one of the potentially malicious parameters

Command Line: *${pkgdata("winevent:hasCommandLine")}*

Hostname: *${pkgdata("winevent:hasComputer")}*

Accountname: *${pkgdata("winevent:hasSubjectUserName")}*

Core ID: *${trigger?keep_after_last("#")}*
Event Time: *${pkgdata("core:hasEventTime")}*
was used in the Slack step to create

  • tips/freemarker-templates
  • Last modified: 2019/04/02 00:31