LogInsight – data extraction script – api

Here is powershell script example to extract data out of LogInsight into CSV files for multiple search items where /text/CONTAINS is required… you can easily adapt ‘/text/CONTAINS‘ in this script to use different matching criteria… just check out the documentation here.

So to the script..

So that I don’t keep being prompted for credentials I save the account user and password with the powershell Get-Credential command.

$cred = Get-Credential
$cred | Export-Clixml -Path "D:\LogExtract\Config\LogInsight.xml"

 

Script usage:

./Get-LogInsight -Provider 'ActiveDirectory' -LogInsightAddress 'log.vMan.ch' -creds 'LogInsight' -SearchText 'WINSRV2','WINSRV4' -StartDate '2017/05/12 13:00' -EndDate '2017/05/13 13:05' -interval 5

 

-Provider ‘ActiveDirectory’ for Active Directory users or ‘Local’ for Loginsight users

-LogInsightAddress FQDN or IP of Loginsight

-creds credentials file to use.

-SearchText array of items to search

-StartDate start date for the query

-EndDate end date for the query

-interval interval in minutes to extract logs (batches which helps get around the 20000 row export limit)

 

<#

    Get Events from LogInsight via the API, extracting more than the 20000 rows can be a pain so this allows the calls to be extracted in batches by using X minute intervals as a "work around" for the export limit.

Usage:

   ./Get-LogInsight -Provider 'ActiveDirectory' -LogInsightAddress 'log.vMan.ch' -creds 'LogInsight' -SearchText '192.168.16.130','192.168.16.131' -StartDate '2017/02/12 13:00' -EndDate '2017/02/13 13:05' -interval 5

    Use -Provider 'ActiveDirectory' for active directory credentials or -Provider 'Local' for LogInsight users

#>



param
(
    [String]$LogInsightAddress = 'log.vman.ch',
    [String]$creds = 'LogInsight',
    [String]$Provider = 'ActiveDirectory',
    [Array]$SearchText = @('WINSRV2','WINSRV4'),
    [DateTime]$StartDate = '2017/02/12 14:10',
    [DateTime]$EndDate = '2017/02/12 20:20',
    [Int]$interval = 5
)

$ScriptPath = (Get-Item -Path ".\" -Verbose).FullName

if($creds -gt ""){

    $cred = Import-Clixml -Path "$ScriptPath\config\$creds.xml"

    $User = $cred.GetNetworkCredential().Username
    $Password = $cred.GetNetworkCredential().Password
    }
    else
    {
    echo "Credentials not specified, bye bye!"
    Exit
    }


#Take all certs.
add-type @"
    using System.Net;
    using System.Security.Cryptography.X509Certificates;
    public class TrustAllCertsPolicy : ICertificatePolicy {
        public bool CheckValidationResult(
            ServicePoint srvPoint, X509Certificate certificate,
            WebRequest request, int certificateProblem) {
            return true;
        }
    }
"@
[System.Net.ServicePointManager]::CertificatePolicy = New-Object TrustAllCertsPolicy






#Start Script

$lookupintervalMS = ($interval * 60 * 1000)

$ContentType = "application/json"
$header = New-Object "System.Collections.Generic.Dictionary[[String],[String]]"
$header.Add("Accept", 'application/json')
$header.Add("Content-Type", 'application/json')
$header.Add("User-Agent", 'vManCHExtractor/1.0')

$Authurl = 'https://'+$LogInsightAddress+'/api/v1/sessions'

$AuthBody = @"
{
"username": "$User",
"password": "$Password",
"provider": "$Provider"
}
"@

$Auth = Invoke-RestMethod -Method POST -uri $Authurl -ContentType $ContentType -Headers $header -Body $Authbody


$header.Add('Authorization',"Bearer $($auth.sessionId)")

[int64]$StartDateEpoc = Get-Date -Date $StartDate.ToUniversalTime() -UFormat %s
$StartDateEpoc = $StartDateEpoc*1000 

$StartDateFile = $StartDate.tostring("yyyyMMdd-HHmmss")            
$EndDateFile = $EndDate.tostring("yyyyMMdd-HHmmss")


ForEach ($Lookup in $SearchText){

        [int64]$EndDateEpoc = Get-Date -Date $EndDate.ToUniversalTime() -UFormat %s
        $LookupDateEndEpoc = $EndDateEpoc*1000 
        
        $LookupDateStartEpoc = ($LookupDateEndEpoc - $lookupintervalMS)


            While ($LookupDateEndEpoc -ne $StartDateEpoc) {

                $QueryURL = 'https://'+$LogInsightAddress+'/api/v1/events/timestamp/>='+$LookupDateStartEpoc+'/timestamp/<='+$LookupDateEndEpoc + '/text/CONTAINS '+$Lookup +'?limit=20000&timeout=300000'


                $LogInsightResults = Invoke-RestMethod -Method GET -uri $QueryURL -ContentType $ContentType -Headers $header
                Write-host "Log Collection incremented by 5 min for $Lookup, waiting for LookupEnd $LookupDateEndEpoc to hit StartTime $StartDateEpoc"

                $LookupDateStartEpoc = ($LookupDateStartEpoc - $lookupintervalMS)
                $LookupDateEndEpoc = ($LookupDateEndEpoc - $lookupintervalMS)

                $LogInsightResults.events.text | Out-file -FilePath "$ScriptPath\Results\$Lookup-$StartDateFile-$EndDateFile.log"-Append

                  }



}

Remove-Variable *  -Force -ErrorAction SilentlyContinue

 

Hope this helps someone 🙂

vMan