Convert HashVocab Values to upper case when parsing
Danko90 opened this issue · comments
Hi,
I had to poll data from a STIX repository but I got an error because it has the hash information of SSDEEP only in uppercase. I added the line
TERM_SSDEEP_LOWER = 'SSDeep'
in the file vocabs.py
Everything worked well.
Could you please let me know if I did it correctly or not?
Thank you
Danilo
Thanks, @Danko90.
The CybOX XML schemas define SSDEEP as being in upper-case only. If you're getting XML with the mixed case SSDeep, you should bring that up with the producer of the data.
There might be something we do to work around this in python-cybox. For example, since all of the values in HashNameEnum-1.0 are upper-case, we can call upper() during parsing. I would not recommend adding a separate vocabulary value, since then "SSDeep" and "SSDEEP" will be treated as different values, which is probably not what you want.
Can you paste the exact error you're getting (without adding the TERM_SSDEEP_LOWER = 'SSDeep' line)? That will help us track down the best way to solve this.
Hi @gtback,
Thanks for replying!
Yes, below the traceback:
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.4/dist-packages/misp_stix_converter-0.2.9-py3.4.egg/misp_stix_converter/converters/convert.py", line 156, in manage_stix
stix_package = STIXPackage.from_xml(stix)
File "/usr/local/lib/python3.4/dist-packages/stix/core/stix_package.py", line 249, in from_xml
return entity_parser.parse_xml(xml_file, encoding=encoding)
File "/usr/local/lib/python3.4/dist-packages/mixbox/parser.py", line 187, in parse_xml
entity = self.get_entity_class(xml_root_node.tag).from_obj(entity_obj)
File "/usr/local/lib/python3.4/dist-packages/mixbox/entities.py", line 361, in from_obj
val = transformer.from_obj(val)
File "/usr/local/lib/python3.4/dist-packages/mixbox/entities.py", line 359, in from_obj
val = [transformer.from_obj(x) for x in val]
File "/usr/local/lib/python3.4/dist-packages/mixbox/entities.py", line 359, in <listcomp>
val = [transformer.from_obj(x) for x in val]
File "/usr/local/lib/python3.4/dist-packages/mixbox/entities.py", line 361, in from_obj
val = transformer.from_obj(val)
File "/usr/local/lib/python3.4/dist-packages/mixbox/entities.py", line 361, in from_obj
val = transformer.from_obj(val)
File "/usr/local/lib/python3.4/dist-packages/mixbox/entities.py", line 184, in from_obj
return klass.from_obj(cls_obj)
File "/usr/local/lib/python3.4/dist-packages/mixbox/entities.py", line 361, in from_obj
val = transformer.from_obj(val)
File "/usr/local/lib/python3.4/dist-packages/mixbox/entities.py", line 359, in from_obj
val = [transformer.from_obj(x) for x in val]
File "/usr/local/lib/python3.4/dist-packages/mixbox/entities.py", line 359, in <listcomp>
val = [transformer.from_obj(x) for x in val]
File "/usr/local/lib/python3.4/dist-packages/mixbox/entities.py", line 361, in from_obj
val = transformer.from_obj(val)
File "/usr/local/lib/python3.4/dist-packages/mixbox/entities.py", line 184, in from_obj
return klass.from_obj(cls_obj)
File "/usr/local/lib/python3.4/dist-packages/cybox/common/vocabs.py", line 142, in from_obj
obj = super(VocabString, cls).from_obj(cls_obj)
File "/usr/local/lib/python3.4/dist-packages/cybox/common/attribute_groups.py", line 109, in from_obj
obj = super(PatternFieldGroup, cls).from_obj(cls_obj)
File "/usr/local/lib/python3.4/dist-packages/mixbox/entities.py", line 363, in from_obj
field.__set__(entity, val)
File "/usr/local/lib/python3.4/dist-packages/mixbox/fields.py", line 213, in __set__
self.preset_hook(instance, value)
File "/usr/local/lib/python3.4/dist-packages/cybox/common/vocabs.py", line 29, in validate_value
raise ValueError(error)
ValueError: Value for vocab <class 'cybox.common.vocabs.HashName'> must be one of ('MD5', 'SHA1', 'SHA384', 'SHA256', 'SHA224', 'MD6', 'SSDEEP', 'SHA512'). Received 'SSDeep'
Thanks @Danko90. The error you're getting is what I expected. I'll have to look around a bit to see the best place to fix it, since we don't want to blindly upper-case all vocabularies, just the HashName ones.
@Danko90, in addition to fixing this in python-cybox, I would encourage you to tell the producer of the data that they are producing invalid CybOX. The fix for it here is really just a workaround to "do the right thing" in a case where we're pretty sure what the producer meant to do.