Does not work on IDA 7.0 (mac)

Question

Does not work on IDA 7.0 (mac)

CaledoniaProject opened this issue 5 years ago · comments

CaledoniaProject commented 5 years ago

1. Can't load karta_identifier in IDA 7.0 on Mac

Steps to reproduce

Install Homebrew Python 2.7
Run pip install -r requirements.txt

Use sys.path.append to add PYTHONPATH in karta_identifier.py:

import sys
sys.path.append('/usr/local/homebrew/lib/python2.7/site-packages/')

Load it in IDA and it reports error:

/tmp/Karta-master/src/karta_identifier.py: 'NoneType' object has no attribute 'logHandler'
Traceback (most recent call last):
  File "/Applications/IDA Pro 7.0/ida64.app/Contents/MacOS/python/ida_idaapi.py", line 553, in IDAPython_ExecScript
    execfile(script, g)
  File "/tmp/Karta-master/src/karta_identifier.py", line 180, in <module>
    pluginMain()
  File "/tmp/Karta-master/src/karta_identifier.py", line 164, in pluginMain
    initUtils(logger, createDisassemblerHandler(logger))
  File "/tmp/Karta-master/src/config/utils.py", line 129, in initUtils
    global_logger.linkHandler(disas_layer.logHandler())
AttributeError: 'NoneType' object has no attribute 'logHandler'

2. karta_manual_identifier.py does not work at all

From strings xxx I can tell that target is linked against OpenSSL 1.0.1h

...
OpenSSL 1.0.1h 5 Jun 2014
...

Now if I try to identify the library:

%> python karta_manual_identifier.py /tmp/target.i64
[+] Starting the Script
[+] Starting the input loop
    [+] Enter the details for the current manually identified library:
    [+] Library Name (case sensitive): OpenSSL
    [+] Library Version (case sensitive): 1.0.1h
    [+] Do you want to identify another library version? <Y/N>:
[+] Storing the data to the knowledge configuration file
[+] Finished Successfully

Karta shows nothing. Since there is a config/OpenSSL_1.0.1j.json available, I tried to use 1.0.1j as well, nothing changed.

Can you help?

Jonas · Answer 1 · Tue Apr 02 2019 18:42:14 GMT+0800 (China Standard Time)

@CaledoniaProject

You must also add the karta src path to sys.path (a tip is to install the dependencies in a virtualenv). Pseudo code:

import sys
sys.path += ["/path/to/venv/lib/python2.7/site-packages", "/path/to/Karta/src"]

I'm not sure if this will help but you could try to enable ida logging by adding -L/tmp/ida.log to https://github.com/CheckPointSW/Karta/blob/master/src/disassembler/IDA/ida_cmd_api.py#L57

CaledoniaProject · Answer 2 · Tue Apr 02 2019 22:02:30 GMT+0800 (China Standard Time)

Problem 1 is resolved, but no library names is generated

%> cat xxx.i64_libraries.txt
Karta Identifier - xxx:
====================================

Identified Open Sources:
------------------------
zlib: 1.2.8
OpenSSL: 1.0.1h
libxml2: 20903

Identified Closed Sources:
--------------------------
(none)

Missing Open Sources:
---------------------
libpng: Was not found
OpenSSH: Was not found
net-snmp: Was not found
gSOAP: Was not found
libtiff: Was not found
mDNSResponder: Was not found
MAC-Telnet: Was not found
libjpeg-turbo: Was not found
libjpeg: Was not found
icu: Was not found
libvpx: Was not found

Missing Closed Sources:
-----------------------
Treck: Was not found

Final Note - Karta
------------------
If you encountered any bug, or wanted to add a new extension / feature, don't hesitate to contact us on GitHub:
https://github.com/CheckPointSW/Karta

Eyal Itkin · Answer 3 · Wed Apr 03 2019 15:05:45 GMT+0800 (China Standard Time)

Hi @CaledoniaProject, I'm happy to see that the first issue was already solved. Regarding the second issue, I'm having trouble to understand what isn't working.

Karta consists of 3 parts:

Compiling a new configuration - for a specific version of a given open source
Identifying the used open source versions in a given binary
Matching the functions for these open sources in the given binary

As can be found in the Read-The-Docs, the identification part is done using karta_identifier.py which gives you the list of used open sources, as can be seen in your output. karta_manual_identifier.py should only be used if the regular identifier failed to find a library, and you still want karta_matcher.py to try and match functions from it in your binary.

Only the matcher will match the functions in your binary, the identifier's role is to output a detailed list of used open sources, which in your example seems to work perfectly.

Could you please detail what was expected? and what isn't working for you?

Eyal Itkin · Answer 4 · Wed Apr 03 2019 15:21:01 GMT+0800 (China Standard Time)

One more note, the karta_identifier.py script should be executed from within IDA (as detailed in the documentation), and this is why it failed to create an IDA-based logger in the first issue you presented.

CaledoniaProject · Answer 5 · Wed Apr 03 2019 15:29:17 GMT+0800 (China Standard Time)

I'm executing the script in IDA. No errors and a dialog popped up telling me that the result is written to xxx.i64_libraries.txt at last.

Although Karta says OpenSSL is identified, but no symbols are actually identified, their names stay as sub_XXX

Eyal Itkin · Answer 6 · Wed Apr 03 2019 15:31:55 GMT+0800 (China Standard Time)

The symbols will only be matched during the matcher script. the Identifier script is used for the recon phase, and is only supposed to give you the list that you indeed received.

CaledoniaProject · Answer 7 · Wed Apr 03 2019 15:36:09 GMT+0800 (China Standard Time)

How can I use Karta to name OpenSSL symbols? I thought Karta is an alternative to FLAIR solution (which does not work).

Eyal Itkin · Answer 8 · Wed Apr 03 2019 15:39:03 GMT+0800 (China Standard Time)

I ask that you'll read the documentation, and my previous comments, before asking more questions.
It is quite straightforward:
karta_identifier.py - outputs a list of used open sources, and that's it.
karta_matcher.py - matches the functions from open sources with configurations, and shows you the matching results. In the GUI you can select to import these matches and rename the functions accordingly.

I'm closing this issue, as it seems there is no bug in the library. If you have any problems with the karta_matcher.py script, please open a new issue.

CaledoniaProject · Answer 9 · Wed Apr 03 2019 16:43:22 GMT+0800 (China Standard Time)

You're right, I should execute karta_matcher instead. Finally got some names.

CaledoniaProject · Answer 10 · Wed Apr 03 2019 17:08:20 GMT+0800 (China Standard Time)

Karta finished analysis in 20min, a few function are matched:

3/04/2019 16:41:39] - Karta - INFO: Anchor function - asn1_parse2: Matched at 0xa5ae40 (sub_A5AE40)
[03/04/2019 16:41:39] - Karta - INFO: Anchor function - X509_CERT_AUX_print: Matched at 0xa5de70 (sub_A5DE70)
[03/04/2019 16:41:39] - Karta - INFO: Anchor function - X509_print_ex: Matched at 0xa5d670 (sub_A5D670)

However, sub_a5d670 is still named as sub_a5d670, was it a bug?

Eyal Itkin · Answer 11 · Wed Apr 03 2019 17:57:08 GMT+0800 (China Standard Time)

I'm happy to see that it indeed matched the functions (20 minutes in OpenSSL usually means it matched > 2,000 functions).
karta_matcher.py shows the matched functions in a GUI window in IDA. Inside this window (or View) you can Right-Click and import the matches. You can import only the selected functions, or you can import all of the matches. The import operation will rename the functions so sub_a5d670 will be renamed OpenSSL_X509_print_ex.

In the future, please read the documentation as all of the things I've written in my comments are clearly written there.

Chris Evans · Answer 12 · Wed Apr 17 2019 02:32:48 GMT+0800 (China Standard Time)

Similar unresolved issue that @CaledoniaProject seemed to fix?
I uninstalled all the local Darwin pip modules and am just installing via Homebrew python (setup.py install as well as pip install -r requirements.txt).

Following edits to karta_identifier.py:

import sys
sys.path.insert(0, "/usr/local/lib/python2.7/site-packages")
sys.path.insert(0, "/tmp/Karta/src")

It is complaining of elementals module not being installed, although it is installed @ /usr/local/lib/python2.7/site-packages

If I add the .egg path explicitly (also requiring hexdump) the following error appears:

import sys
sys.path.insert(0, "/usr/local/lib/python2.7/site-packages")
sys.path.insert(0, "/tmp/Karta/src")
sys.path.insert(0, "/usr/local/lib/python2.7/site-packages/elementals-1.2.6-py2.7.egg")
sys.path.insert(0, "/usr/local/lib/python2.7/site-packages/hexdump-3.3-py2.7.egg")
print(sys.path)

/tmp/Karta/src/karta_identifier.py: 'NoneType' object has no attribute 'logHandler'
Traceback (most recent call last):
  File "/Applications/IDA Pro 7.2/ida.app/Contents/MacOS/python/ida_idaapi.py", line 572, in IDAPython_ExecScript
    execfile(script, g)
  File "/tmp/Karta/src/karta_identifier.py", line 184, in <module>
    pluginMain()
  File "/tmp/Karta/src/karta_identifier.py", line 168, in pluginMain
    initUtils(logger, createDisassemblerHandler(logger))
  File "/tmp/Karta/src/config/utils.py", line 129, in initUtils
    global_logger.linkHandler(disas_layer.logHandler())
AttributeError: 'NoneType' object has no attribute 'logHandler'

Eyal Itkin · Answer 13 · Thu Apr 18 2019 13:44:48 GMT+0800 (China Standard Time)

It looks like there is some mess regarding the paths that are used by python, ida-python and the combination of the two. It is complicated to recreate this issue, as we can't figure out why it works for some and not for others.

We are currently investigating the root cause of this issue, and we will issue a proper patch once it is resolved.