Ingesting Okta logs in to Graylog

After many failed attempts to import Okta logs in to Graylog (using some PowerShell scripts I found online) I decided to take a different approach. Here is what my final Dashboard and view ended up looking like:


Okta Graylog Dashboard


Graylog Okta Table View


Prerequisites

To ingest Okta logs in to Graylog you will need the following:

  1. *Sumo Janus
  2. Graylog Sidecar
  3. Okta API Key

**You don’t need to have a Sumo Logic instances or subscription to use SumoJanus. We will just use it to download the Okta logs*

Installation

I am using Centos 7.x but this should work with other Linux distributions.

This installation assumes the following:

# Logs stored here for okta
/opt/okta-logs

# SumoJanus installation directory 
/opt/sumojanus-okta

Create path to store Okta logs

mkdir -p /opt/okta-logs

Install Java

cd /tmp
curl -L -O https://corretto.aws/downloads/latest/amazon-corretto-8-x64-linux-jdk.rpm
yum localinstall amazon-corretto-8-x64-linux-jdk.rpm
rm amazon-corretto-8-x64-linux-jdk.rpm

SumoJanus Install

mkdir /opt && cd /opt
curl -O https://script-collection.s3.amazonaws.com/okta/r1.0.1/sumojanus-okta-dist.1.0.1.tar.gz
tar xzvf sumojanus-okta-dist.1.0.1.tar.gz
rm tar xzvf sumojanus-okta-dist.1.0.1.tar.gz

Modify the sumologic.properties file to your Okta enviornment

# /opt/sumojanus-okta/conf/sumologic.properties
# replace REPLACE_WITH_YOUR_API_KEY and COMPANY_PREFIX
[generic]
path = .
[oktacollector]
api_token = REPLACE_WITH_YOUR_API_KEY
okta_org_url = https://COMPANY_PREFIX.okta.com
stream_pos_path = /opt/sumojanus-okta/conf/okta_checkpoint.dat
pagination_limit = 1000

Modify /opt/sumojanus-okta/bin/SumoJanus_Okta.bash to look like this:

#!/bin/bash

# If you run this script manually to test, call it right under the main sumojanus folder, for example go to \"sumojanus\", then call "bin\\SumoJanus_Okta.bash"

if [ "$1" = "-s" ]; then
        echo "Setup mode"
        runMode="-s"
else
        runMode="-r"
fi

JAVA_HOME=/usr/lib/jvm/java-1.8.0-amazon-corretto/jre
# Make sure JAVAPATH below is set to the JRE folder under the SumoCollector directory, default is set to the environment variable JAVA_HOME
export JAVAPATH="${JAVA_HOME}"

SUMOJANUS_JAR_FILE=`cd /opt/sumojanus-okta && ls januscore*.jar`
echo "Found SumoJanus core jar file: ${SUMOJANUS_JAR_FILE}, current path: ${PWD}, whoami: $(whoami), javapath: ${JAVAPATH}, runmode: ${runMode}"

cd /opt/sumojanus-okta && ${JAVAPATH}/bin/java -jar ${SUMOJANUS_JAR_FILE} ${runMode} OktaCollector-1.0.1.jar -e 1800

Create a script to run SumoJanus at /opt/okta-logs/getOktaLogs.sh


#!/bin/bash
# Runs the SumoJanus ingestion script

if pgrep -f "januscore-1.0.1.jar" ; then
    echo "SumoJanus is already running"
    exit 0
else
    echo "SumoJanus was not running.. attempting to start"
    cd /opt/sumojanus-okta && nohup ./bin/SumoJanus_Okta.bash >> /opt/okta-logs/okta.log 2> /opt/okta-logs/okta.error 

Modify Crontab to keep SumoJanus Always Running

Make sure you update the path in the crontab and add /opt/sumojanus-okta

SHELL=/bin/bash
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin:/opt/sumojanus-okta

* * * * * /usr/bin/flock -n /tmp/getOktaLogs.lockfile /opt/okta-logs/getOktaLogs.sh

Graylog Setup

I will partially cover how to setup a Graylog Sidecar on the machine

Configure a Beats Listener

Follow instructions here to create a beats listener


Graylog Beats Listener


Install Sidecar on your Linux box

Install sidecar on the same machine that you installed SumoJanus

mkdir -p /tmp/sidecar && cd /tmp/sidecar
curl -L -O https://github.com/Graylog2/collector-sidecar/releases/download/1.0.2/graylog-sidecar-1.0.2-1.x86_64.rpm
rpm -i graylog-sidecar-1.0.2-1.x86_64.rpm

Modify the sidecar config file to point to your environment

/etc/graylog/sidecar/sidecar.yml

Set Sidecar to Start at Boot

sudo graylog-sidecar -service install
sudo systemctl start graylog-sidecar

Install File Beats on the same machine that you installed SumoJanus

# setup yum repo
sudo rpm --import https://packages.elastic.co/GPG-KEY-elasticsearch
sudo cat <<'EOF' > /etc/yum.repos.d/elastic.repo
[elastic-7.x]
name=Elastic repository for 7.x packages
baseurl=https://artifacts.elastic.co/packages/7.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md

EOF

# install
yum install filebeat

# run at boot
systemctl enable filebeat

Step by Step Guide to Creating First Sidecar

Follow this guide to complete your sidecar setup

Your collector will look something like this:


Graylog Collector Config


Setup your extractors in Graylog

I am using the following for my extractors

{
  "extractors": [
    {
      "title": "Okta",
      "extractor_type": "json",
      "converters": [],
      "order": 0,
      "cursor_strategy": "copy",
      "source_field": "message",
      "target_field": "",
      "extractor_config": {
        "list_separator": ", ",
        "kv_separator": "=",
        "key_prefix": "",
        "key_separator": "_",
        "replace_key_whitespace": false,
        "key_whitespace_replacement": "_"
      },
      "condition_type": "string",
      "condition_value": "{\"actor\":{\"id\""
    },
    {
      "title": "Okta2",
      "extractor_type": "json",
      "converters": [],
      "order": 0,
      "cursor_strategy": "copy",
      "source_field": "message",
      "target_field": "",
      "extractor_config": {
        "list_separator": ", ",
        "kv_separator": "=",
        "key_prefix": "",
        "key_separator": "_",
        "replace_key_whitespace": false,
        "key_whitespace_replacement": "_"
      },
      "condition_type": "string",
      "condition_value": "{\"actor\":{\"id\""
    },
    {
      "title": "Okta - VPN Parse",
      "extractor_type": "grok",
      "converters": [],
      "order": 0,
      "cursor_strategy": "copy",
      "source_field": "target",
      "target_field": "",
      "extractor_config": {
        "grok_pattern": "id=%{DATA:actor_id2}, type=%{DATA:transaction_type2}, alternateId=%{DATA:actor_alternateId}, displayName=%{DATA:actor_displayName}, detailEntry=%{DATA:detail_entry}, \\{id=%{DATA:actor_id3}, type=%{DATA:transaction_type3}, alternateId=%{DATA:transaction_type}, displayName=%{DATA:transaction_type4}, detailEntry=%{GREEDYDATA:detail_entry2}\\}"
      },
      "condition_type": "string",
      "condition_value": "displayName=Cisco ASA VPN (RADIUS)"
    }
  ],
  "version": "3.1.4"
}

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.