/*
IDA Pro Collaboration/Synchronization Plugin
Copyright (C) 2018 Chris Eagle <cseagle at gmail d0t com>
Copyright (C) 2018 Tim Vidas <tvidas at gmail d0t com>
This program is free software; you can redistribute it and/or modify it
under the terms of the GNU General Public License as published by the Free
Software Foundation; either version 2 of the License, or (at your option)
any later version.
This program is distributed in the hope that it will be useful, but WITHOUT
ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
more details.
You should have received a copy of the GNU General Public License along with
this program; if not, write to the Free Software Foundation, Inc., 59 Temple
Place, Suite 330, Boston, MA 02111-1307 USA
*/
Version 1.0.0
BACKGROUND:
IdaPro has an event notification mechanism that plugins may utilize to be
notified of many different types of changes being made to a database. The
primary idea behind CollabREate is to hook these change notifications and
broadcast changes to a central database server that records each change
and forwards the associated change parameters to other interested users
working on the same binary file. The theory being that one user's changes
will be applied to a second user's database effectively allowing multiple
users to simultaneously edit and annotate their databases.
Please not that no single master copy of a particular IDB is maintained
anywhere. A central database that records all events generated by all
registered users is the only record of changes. It is entirely possible
that the state of two databases diverges if one user were to make
modifications without publishing those modifications to the database server.
A major difficulty in this approach arises from the manner in which Ida
notifications are generated. Because of the manner in which the Ida auto
analysis subsystem operates, what appears to be a single action to the user
may in fact translated into many (tens, hundreds, or even thousands)
individual notification messages which all get pushed to the database. This
makes conflict resolution between users and events extremely difficult
because it is virtually impossible to associate a specific notification
message with a specific user action. In fact some notification messages may,
at first glance seem entirely unrelated to the user action which resulted in
the generation of those notifications.
Another challenge that arises is to supress the generation of duplicate
notifications when processing incoming notifications. The current version of
CollabREate simply de-registers itself for notifications while applying any
changes indicated by each incoming message. This seems to work well unless
the incoming messages results in the generation of more than one additional
notification in the receiving IDB. In such cases, it is extremely difficult
to determine when the last notification associated with the application of
a newly received message has be generated, and therefore difficult to
determine exactly when to re-register for receipt of local notifications.
This is suspected to be one of the primary causes of a CollabREate database
getting flooded with notifications and associated IDBs diverging in state.
In practice, this problem seems to arise from actions that generate large
numbers of notifications such as undefining large blocks of code or data
or patching large numbers of bytes.
RELEASE NOTES version 1.0.0
Removed all Qt dependeicies, changed to use IDA API user interface functions.
Notification protocol completely changed from a binary protocol to a
json based protocol. Server and database updated accordingly.
Support for IDA versions 6.5 through 7.0, older version no longer supported.
Binaries now land in bin directory
RELEASE NOTES version 0.4.0
Removed dependency on QtNetwork. Reworked asynchronous networking
for IDA versions 5.5 and later to make use of the IDA SDK execute_sync
function. Updated MSVC build files so that a single build file accomodates
multiple versions of IDA.
RELEASE NOTES version 0.3.0
This version of the plugin is NOT compatible with older (< 0.3.0) collabreate
databases. This is a result of the addtion of 64 bit support. If you need
to use old databases, then you should continue to use version 0.2.0. We may
publish an upgrade utility that will upgrade 0.2.0 databases to 0.3.0 databases.
For the purposes of this discussion we are talking about the SQL database that
stores all of the IDA update notifications.
BUILDING THE PLUGIN
The IDA Pro plugin requires the IDA Pro SDK, which is distributed along with
Pro. You need the version of the SDK that matches the version of IDA with
which you intend to use the plugin.
First you clone the repo, you should do this within
the SDK's "plugins" directory:
cd <SDKDIR>/plugins
git clone https://github.com/cseagle/collabREate.git
BUILDING FOR WINDOWS
Using Visual Studio 2013 or later, open collabrete.sln
Based on your IDA version select and build the corresponding project
IDA Version Project to build Output file
<= 6.95 idaq Release / Win32 bin/collabreate.plw
<= 6.95 idaq64 Release64 / Win32 bin/collabreate.p64
>= 7.0 ida Release / x64 bin/collabreate.dll
>= 7.0 ida64 Release64 / x64 bin/collabreate64.dll
BUILDING FOR LINUX / OSX
Use the supplied Makefile to build the plugin
$ make
You may need to adjust paths to your SDK an IDA install in the Makefile
see IDA_SDK and IDAHOME in the Makefile.
Compiled binairs should land in the bin directory.
The plugin depends on the json-c library from github:
https://github.com/json-c/json-c
NOTE that the plugin uses features that are not available in the older
versions of json-c that install with most package managers, so build
and install from the github sources.
INSTALLING THE PLUGIN
As with any IDA plugin, simply copy the compiled plugin file into collabreate's
bin directory to <idadir>/plugins, where <idadir> represents the location in
which IDA is installed.
Due to Windows' file locking, you will likely need to make sure all Ida databases
are closed before copying to this directory.
USING THE PLUGIN
It is intended that a new collabreate session is used for each binary loaded
into IDA Pro. Once a binary is loaded and the autoanalysis phase has completed
you can activate CollabREate by via the hotkey ALT-F6 or the Edit/Plugins menu.
Upon activation you will be presented with a series of dialogs prompting you to
connect to a CollabREate server, authenticate, start a new project, etc. The
exact series of steps will very upon the running mode of the server (such as if
it is connected to a backend database or not).
Typical Project Join Actions:
Connect: Connects to a collabreate server (default port is 5042)
Authenticate: Provide a User / Password to connect to the server
Choose Project: Provides a list of 'related' projects (1)
New Project: Request that the server creates a new project (2)
Existing Project: Join an existing project
Project Snapshot: Request a new project that is a Fork of an existing project
at a particular, previously saved point (2)
(1) only projects that match the MD5 of the binary loaded in IDA are displayed
(2) when starting a new project you must provide a textual description
When connecting to an existing project for the first time, you automatically
recieve all updates that have been made to the project prior to you joining.
On subsequent connections you will automatically receive any updates that have
been made since the last time you were connected to the project.
Once connected to a project, you can use IDA normally. Updates you make are
sent to the server, updates other collabreators make are reflected in your
database.
Reactivating the plugin while the plugin is already activated will bring up a menu of
collabreate specific commands.
Collaboration Actions:
Disconnect: Disconnect from the server.
Fork: Immediatly fork the project with the updates that IDA has
recieved and join this new project. (3)
Snapshot: Save a snapshot (point in time) (3)
Manage Req Perms: Allows the user to change the permissions requested when
joining the project
Manage Proj Perms: Allows the user (only if the user is the owner of the
project) to alter the project permissions.
(3) the user must provide a textual description
Forking creates a new projet that can have further updates, and that others can
participate in (join). Snapshots are simply 'projects at a point in time' - not
actual projects. You can not 'join' a snapshot. You must fork a new project
from a snapshot. Overall, snapshots take up much less space than forked
projects.
If the server is connected to a database backend, a unique project id is stored
in the idb file and subsequent connections to a server will automatically
connect to the correct project.