3. Next Steps
After the installation you can test the service using one of the Thunderstorm collectors or the Python API client.
3.1. Configuration
The installation script for Linux system installs a service that passes
the parameter -t /etc/thunderstorm/thunderstorm.yml
to initialize
the default config stored at that location.
The default configuration file on Linux looks like this:
# License path
license-path: /etc/thunderstorm
# Write all outputs to the following directory
logfile: /var/log/thunderstorm/thunderstorm.log
appendlog: True
# Listen on all possible network interfaces
server-host: 0.0.0.0
server-port: 8080
# Pure YARA scanning
pure-yara: False
# SSL/TLS
# SSL/TLS Server Certificate
#server-cert: /path/to/file
# SSL/TLS Server Certificate Private Key
#server-key: /path/to/file
# File Submissions
# Directory to which the samples get stored in asynchronous mode
server-upload-dir: /tmp/thunderstorm
# Permanently store the submitted samples (valied values: none/all/malicious)
server-store-samples: none
# Tuning
# Server Result Cache
# This is the number of cached results from asynchronous submission
# available for remote queries (default: 10000)
#server-result-cache-size: 10000
You can use all of THOR's flags in that configuration. Be advised that you always have to use their long form.
This page lists all of THOR command line flags:
https://github.com/NextronSystems/nextron-helper-scripts/tree/master/thor-help
The following chapters list some of the most useful command line flags when using THOR Thunderstorm.
3.1.1. Forward Logs to SIEM or Analysis Cockpit
syslog: mysiem.local
Config entry to forward logs to a SIEM
We recommend reading chapter
Syslog or TCP/UDP Output <https://thor-manual.nextron-systems.com/en/latest/usage/output-options.html#syslog-or-tcp-udp-output>
in the THOR User Manual for details on the SYSLOG forwarding flags.
You can find it in the folder /opt/nextron/thunderstorm/docs
after a successful Thunderstorm installation on Linux or in the
"Downloads" section in the customer portal.
3.1.1.1. Keep Samples on the Thunderstorm Server
Keep samples with findings
server-store-samples: malicious
Keep all samples
server-store-samples: all
3.2. Log Output
The scan results and startup messages can be found in:
user@unix:~$ sudo less /var/log/thunderstorm/thunderstorm.log
You could open another command line window and monitor new messages with:
user@unix:~$ sudo tail -f /var/log/thunderstorm/thunderstorm.log
3.3. Thunderstorm API Documentation
An API documentation is integrated into the web service.
Simply visit the service URL, e.g.: http://my-server:8080/

Thunderstorm API Documentation
3.4. Test Submission
To test the Thunderstorm service, you can create a tiny webshell sample and submit it to the service using the following commands.
#!/bin/bash
echo "<%eval request(" > test.txt
curl -X POST "http://my-server:8080/api/check?pretty=true" -F "file=@test.txt"
This should produce the following output in the current command line.
[
{
"level": "Alert",
"module": "Filescan",
"message": "Malware file found",
"score": 350,
"context": {
"ext": ".txt",
"file": "test.txt",
"firstBytes": "3c256576616c2072657175657374280a / \\u003c%eval request(\\n",
"md5": "2481bc6bb2d063522ef8b5d579fd97d7",
"sha1": "4d40de75d7c8591d2ea59d3a000fb6cf58d97896",
"sha256": "3b435df5076f6b1df7f2bc97cd86fbf7b479352e8c33960dfc4f1cbbe9b14fa7",
"size": 16,
"type": "JSP"
}
}
]
Output of test sample submission
Be aware that this has been a "synchronous" submission to the API endpoint "check". The collection of high amounts of samples in collector scripts and tools uses the endpoint "checkAsync", which doesn't return a result to the submitting source.
3.4.1. Test Submission Using the API Documentation
The web GUI running on Port 8080 contains an interactive API documentation, which you can use to submit a first test sample.

Link to API Documentation on Start Page
Select the API function /api/check
, then click "Try it out" and then
select and submit a sample using the enabled form.

Test Sample Submission via API Documentation
The result appears in a separate text field. Use the "pretty" flag to get a prettified JSON response.
3.5. Thunderstorm Collectors
You can find a Thunderstorm collector for numerous different operating systems and architecture in our Github repository. We recommend using the collectors written in Go.
https://github.com/NextronSystems/thunderstorm-collector
You find pre-compiled collector binaries in the release section of the repository.
See the README on Github for more information.
3.5.1. Run the Collectors
We highly recommend using the config.yml as a configuration during the collection. It limits the samples the collector selects for a submission to relevant file types and sizes. Otherwise the collector would transmit every possible file, which is usually not recommended.
To retrieve the latest config.yml
file, you can use the URL in the following listing
or download it using wget
.
user@unix:~$ wget https://github.com/NextronSystems/thunderstorm-collector/releases/latest/download/config.yml
You would then start a collector run using the following command line:
Windows (64 bit):
C:\nextron\thunderstorm>amd64-windows-thunderstorm-collector.exe -t config.yml
Linux (64 bit):
user@unix:~$ ./amd64-linux-thunderstorm-collector -t config.yml
(Replace the collector binary name with the one you plan to use)
3.5.2. Performance Considerations for the Collection
In a THOR Thunderstorm setup, the system load moves from the end systems to the Thunderstorm server.
In cases in which you don't use the default configuration file provided
with the collectors (config.yml
) and collect all files from an end
system, the Thunderstorm server requires a much higher amount of time to
process the samples.
E.g. A Thunderstorm server with 40 CPU Cores (40 threads) needs 1 hour to process all 400,000 files sent from a Windows 10 end system. Sending all files from 200 Windows 10 end systems to a Thunderstorm server with that specs would take up to 10 days to process all the samples.
As a rule of thumb, when using the hardware recommended in
Hardware Requirements, calculate
with a processing speed of 250 samples per core per minute
.
We highly recommend using the default configuration file named
config.yml
provided with the collectors. See the README on Github
for more information.
3.6. Thunderstorm API Client
We provide a free and open source command line client written in Python to communicate with the Thunderstorm service.
https://github.com/NextronSystems/thunderstormAPI
It can be installed with:
user@unix:~$ pip install thunderstormAPI
3.7. Source Identification
The log file generated by THOR Thunderstorm doesn't contain the current host as hostname in each line. By default, it contains the sending source's FQDN or IP address if a name cannot be resolved using the locally configured DNS server.
However, every source can set a "source" value in the request and overwrite the automatically evaluated hostname. This way users can use custom values that are evaluated or set on the sending on the end system.
user@unix:~$ curl -X POST "http://myserver:8080/api/check?source=test" -F "file=@sample.exe"
3.8. Synchronous and Asynchronous Mode
It is also important to mention that THOR Thunderstorm supports two ways to submit samples, a synchronous and an asynchronous mode.
The default is synchronous submission. In this mode, the sender waits for the scan result, which can be empty in case of no detection or contains match elements in cases in which a threat could be identified.
In asynchronous mode, the submitter doesn't wait for the scan result but always gets a send receipt with an id, which can just be discarded or used to query the service at a later point in time. This mode is best for use cases in which the submitter doesn't need to know the scan results and batch submission should be as fast as possible.
Synchronous |
Asynchronous |
|
---|---|---|
Server API Endpoint |
/api/check |
/api/checkAsync |
ThunderstormAPI Client Parameter |
--asyn |
|
Advantage |
Returns Scan Result |
Faster Submission |
Disadvantage |
Client waits for result of each sample |
No immediate scan result on the client side |
In asynchronous mode, the Thunderstorm service keeps the samples in a
queue on disk and processes them one by one as soon as a thread has time
to scan them. The number of files in this queue can be queried at the
status endpoint /api/status
and checked on the landing page of the
web GUI.
In environments in which the Thunderstorm service is used to handle synchronous and asynchronous requests at the same time, it is possible that all threads are busy processing cached asynchronous samples and not more synchronous requests are possible.
In this case use the --sync-only-threads
flag to reserve a number of
threads for synchronous requests. (e.g. --threads 40 --sync-only-threads 10
)
3.9. Performance Tests
Performance tests showed the differences between the two submission modes.
In Synchronous mode, sample transmission and server processing take exactly the same time since the client always waits for the scan result. In asynchronous mode, the sample transmission takes much less time, but the processing on the server takes a bit longer, since the sever caches the samples on disk.
Synchronous |
Asynchronous |
|
---|---|---|
Client Transmission |
40 minutes |
18 minutes |
Server Processing |
46 minutes |
|
Total Time |
40 minutes |
46 minutes |
3.10. SSL/TLS
We do not recommend the use of SSL/TLS since it impacts the submission performance. In cases in which you transfer files through networks with IDS/IPS appliances, the submission in an SSL/TLS protected tunnel prevents IDS alerts and connection resets by the IPS.
Depending on the average size of the samples, the submission frequency and the number of different sources that submit samples, the transmission could take up to twice as much time.
Note
The thunderstormAPI client doesn't verify the server's certificate
by default as in this special case, secrecy isn't important. The main
goal of the SSL/TLS encryption is an obscured method to transport
potentially malicious samples over network segments that could be
monitored by IDS/IPS systems. You can activate certificate checks with
the --verify
command line flag or verify
parameter in API
library's method respectively.