SMS Import / Export is a simple Android app that imports and exports SMS and MMS messages, call logs, and contacts from and to (ND)JSON files. (Contacts import and export are currently functional but considered experimental.) Root is not required.
Version 2.0.0 introduced a major rewrite of the SMS and MMS messages import / export code, implementing a new message storage format (v2
):
-
The messages are now stored in a Newline-delimited JSON file (always named
messages.ndjson
), as opposed to the standard JSON previously used. -
Binary MMS data is now stored separately from message text data and metadata; the
messages.ndjson
file, along with adata/
directory containing the MMS binary data files copied directly from the Android filesystem (with their original filenames), are both encapsulated in a ZIP file. -
All (ND)JSON tags added by SMS Import / Export are now prefixed with a double underscore (e.g.,
__display_name
,__parts
), to clearly indicate that they have been added by the app.
For a discussion of the advantages and disadvantages of the new format over the old one (v1
), see here.
The NDJSON file is not as human-readable as the previous pretty-printed JSON file, due to the necessary absence of newlines within each JSON message record, but this is easily rectified by feeding the NDJSON to the jq tool, which will pretty-print it:
~$ jq < messages.ndjson
These format changes unfortunately render versions of the app from 2.0.0 and on incompatible with JSON message files produced by earlier versions of the app. Several solutions to this incompatibility are possible:
-
An earlier version of the app (with a 1.x.x version number) can be used to import messages in
v1
format. -
Where feasible, a current version of the app can be used to re-export the messages to
v2
format. -
A conversion tool to convert message files from
v1
tov2
format is available here (documented here). This tool is experimental, and has not been extensively tested.
The above applies only to SMS and MMS messages; the format for call logs and contacts is currently unchanged, although they may be switched to the new format in the future.
SMS Import / Export is available from Github. The repository can be cloned and built locally, from the command line (e.g., by issuing gradlew assembleDebug
in the root directory of the project) or within Android Studio. Releases, which include pre-built APK packages, can be downloaded from the Releases page, and are also available at F-Droid. Automatically built packages of the latest code pushed to the repository are available here (click on the latest workflow run, then click on com.github.tmo1.sms_ie
in the Artifacts
section).
-
Import or export messages, call log, or contacts: Click the respective button, then select an import or export source or destination.
-
Wipe messages: Click the
Wipe Messages
button, then confirm by pressing theWipe
button in the pop-up dialog box.
These operations may take some time for large numbers of messages, calls, or contacts. The app will report the total number of SMS and MMS messages, calls, or contacts imported or exported, and the elapsed time, upon successful conclusion.
By default, binary MMS data (such as images and videos) are exported. The user can choose to exclude them, which will often result in a much smaller ZIP file.
Note that upon import or wipe, message apps present on the system may not immediately correctly reflect the new state of the message database due to app caching and / or local storage. This can be resolved by clearing such cache and storage, e.g. Settings / Apps / Messaging / Storage & cache / Clear storage | Clear cache
.
To enable the scheduled export of messages, call logs and / or contacts, enable the feature in the app's Settings, and select a time to export at and a directory to export to. (Optionally, select which of the various data types to export.) The app will then attempt to export the selected data to a new, datestamped file or files in the selected directory every day at the selected time. (See the TODO section below.)
When scheduled exports are enabled, the following options can be used to control retention:
-
Delete old exports
- If this option is not enabled (the default), then any old exports will be left untouched (i.e., all exports are retained indefinitely). If it is enabled, then for each data type (contacts, call log, and messages), upon successful export, the app will try to delete any old exports (i.e., all files with names of the form<data-type>-<yyyy-MM-dd>.[zip|json]
, where<data-type>
is the data type successfully exported, and<yyyy-MM-dd>
is a datestamp). Selective retention of a subset of old exports can be accomplished by enabling this option in conjunction with the use of external software with snapshotting and selective retention functionality, such as rsnapshot or borg, running either on the local device, or on a system to which the exports are synced via software such as Syncthing. This software should be scheduled to run between exports, and configured to preserve copies of the previous exports before the app deletes them following its next scheduled exports. -
Remove datestamps from filenames
- Scheduled exports are always initially created with filenames of the form<data-type>-<yyyy-MM-dd>.[zip|json]
. If this option is enabled (in addition to the previous one), then after attempting to delete all old exports (of the relevant data type), the app will then attempt to remove the datestamp from the current export's filename by renaming it to<data-type>.[zip|json]
. This is intended to make successive exports appear to be different versions of the same file, which may be useful in conjunction with external software that implements some form of file versioning, such as Syncthing or Nextcloud.
To export messages, permission to read SMSs and Contacts is required (the need for the latter is explained below). The app will ask for these permissions on startup, if it does not already have them.
To import or wipe messages, SMS Import / Export must be the default messaging app. This is due to an Android design decision.
To export call logs, permission to read Call Logs and Contacts is required (the need for the latter is explained below). Currently, the app does not ask permission to read Call Logs, and it must be granted by the user on his own initiative.
To import call logs, permission to read and write Call Logs is required.
To export contacts, permission to read Contacts is required.
To import contacts, permission to write Contacts is required. (Granting the app permission to access Contacts grants both read and write permission, although if the app is upgraded from an earlier version which did not declare that it uses permission to write Contacts, then it may be necessary to deny and re-grant Contacts permission in order to enable permission to write Contacts.)
SMS and MMS messages include phone numbers ("addresses") but not the names of the communicating parties. The contact information displayed by Android is generated by cross-referencing phone numbers with the device's Contacts database. When exporting messages, SMS Import / Export does this cross-referencing in order to include the contact names in its output; this is why permission to read Contacts in necessary. When importing, included contact names are ignored, since the app (at least currently) does not add entries to or modify the Android Contacts database during message import. The best way to maintain the association of messages with contacts is to separately transfer contacts to the device into which SMS Import / Export is importing messages, via either SMS Import / Export's contacts export / import functionality or Android's built in contacts export / import functionality. Contacts cross-referencing is performed for call log export as well, despite the fact that call log metadata will often already include the contact name; see below for a discussion of this point.
Following is the structure of the (ND)JSON currently exported by SMS Import / Export; this is subject to change in future versions of the app.
The exported NDJSON is a series of lines, each consisting of a JSON object representing a message, SMSs followed by MMSs. Each JSON message object contains a series of tag-value pairs taken directly from Android's internal message data / metadata structures, documented in the Android API Reference: SMS, MMS. In addition, SMS Import / Export adds some other tag-value pairs and child JSON objects, as described below. (All tags added by the app to message JSON objects and their children are prefixed with a double underscore ("__") to clearly indicate that they have been added by the app and are not present in Android's message structures.)
In SMS messages, the value of type
specifies (among other things) the direction of the message: the two most common values are 1
, denoting "inbox" (i.e., received), and 2
, denoting "sent".
SMS messages contain a single address
tag; depending on the message direction, this is either the sender or receiver address. SMS Import / Export attempts to look up the address in the Android Contacts database. If this is successful, a tag-value pair of the form "__display_name": "Alice"
is added to the SMS message object.
MMS message objects have the following additions to the tag-value pairs of their internal Android MMS representation:
-
A tag-value pair of the form
"__sender_address": { ... }
-
A tag-value pair of the form
"__recipient_addresses": [ { ... }, { ... } ]
. The child JSON objects associated with__sender_address
and__recipient_addresses
contain a series of tag-value pairs taken directly from Android's internal MMS address structure, documented here, plus possibly a single added tag-value pair of the form"__display_name": "Alice"
, as with SMS messages. -
A tag-value pair of the form
"__parts": [ { ... }, { ... }]
, where the child JSON objects contain a series of tag-value pairs taken directly from Android's internal MMS part structure, documented here.
Android stores binary data of MMS parts as individual files in its filesystem. SMS Import / Export copies these files directly into a data/
directory in the ZIP file, retaining their original filenames (without the full path). The association of these files with MMS parts is based on the values of the _DATA
tags of the MMS parts. (SMS Import / Export utilizes only the actual filename (the last segment of the path) for this association. If there is a problem accessing the binary data, then the data may not be present.)
The exported JSON is an array of JSON objects representing calls. Each JSON call object contains a series of tag-value pairs taken directly from Android's internal call metadata structures, documented in the Android API Reference. In addition, SMS Import / Export will try to add a display-name
tag, as with SMS and MMS messages. The call logs may already have a CACHED_NAME
(name
) field, but the app will still try to add a display-name
, since the documentation of the CACHED_NAME
field states:
The cached name associated with the phone number, if it exists.
This value is typically filled in by the dialer app for the caching purpose, so it's not guaranteed to be present, and may not be current if the contact information associated with this number has changed.
As explained in the official documentation, Android stores contacts in a complex system of three related database tables:
-
ContactsContract.Contacts
: Rows representing different people, based on aggregations of raw contact rows. -
ContactsContract.RawContact
: Rows containing a summary of a person's data, specific to a user account and type. -
ContactsContract.Data
: Rows containing the details for raw contact, such as email addresses or phone numbers.
SMS Import / Export simply dumps these tables in structured JSON format, resulting in a rather cluttered representation of the data with a great deal of repetition and redundancy. This is in accordance with the design principles of the app, which prioritize making sure that no useful information is excluded from the export, and avoiding the code complexity and coding time that would be necessary to filter and / or reorganize the raw data.
The exported JSON is an array of JSON objects representing aggregated contacts, each containing a series of tag-value pairs taken directly from the Contacts
table. To each contact JSON object, a tag-value pair of the form "raw_contacts": [ { ... }, { ... }]
is added, where the child JSON objects represent the (aggregated) contacts' associated raw contacts, and each contain a series of tag-value pairs taken directly from the RawContacts
table. To each raw contact JSON object, a tag-value pair of the form "contacts_data": [ { ... }, { ... }]
is added, where the child JSON objects represent the raw contacts' associated data (i.e., the actual details of the contacts, such as phone numbers, postal mail addresses, and email addresses), and each contain a series of tag-value pairs taken directly from the Data
table.
Currently, social stream data, contact groups, and contact photos are not exported.
Contacts import and export is currently considered experimental, and the JSON format is subject to change.
Note: Currently, when contacts are exported and then imported, the app may report a larger total of contacts imported than exported. This is due to the fact that when exporting, the total number of Contacts
exported is reported (since this is a logical and straightforward thinng to do), whereas when importing, the total number of Raw Contacts
imported is reported (since as per the documentation, applications are not allowed to add Contacts
, only Raw Contacts
, and as noted above, a Contact
may consist of an aggregation of multiple Raw Contacts
).
Currently, no deduplication is done. For example, if messages are exported and then immediately reimported, the device will then contain two copies of every message. To avoid this, the device can be wiped of all messages before importing by using the Wipe Messages
button.
Contacts import only imports basic contact data (name, phone numbers, email and postal addresses, etc.), but not the contacts metadata that Android stores. Additionally, imported contacts are not associated with the accounts with which they had been associated on the system from which they were exported, and the user has no control over which account they will be associated with on the target system; all contacts are inserted into the target system's default account.
Although this is apparently not publicly officially documented, Android's Call Log has a fixed maximum number of calls that it will store (500 in many / most versions of Android, 1000 in API 30 (version 11) on a Pixel [my own experience, corroborated here]).
Earlier versions of this document stated that:
Attempting to import calls when the log is full may fail, in which case the app will not report an error, but the reported number of imported calls will be lower then the number of calls provided for import. E.g., if calls are exported from a phone with a full log, and the output file is then imported to the same phone, the app will report 0 calls imported.
This was a misinterpretation of observed call import failures, which were actually caused by a bug in the app, which has since been fixed.
Bugs, feature requests, and other issues can be filed at the SMS Import / Export issue tracker. When reporting any problem with the app, please try to reproduce the problem with the latest release of the app, and please specify the versions of the app used for export and / or import, as applicable.
When reporting a problem with import or export functionality, please try to include the (ND)JSON file involved, in accordance with the following guidelines:
Please try to reproduce the problem with as small a (ND)JSON file as possible. The simplest way to reduce the size of the file is to use the app's Settings / Debugging options / Maximum records ...
option to export only a small number of messages.
It is strongly recommended to redact any posted (ND)JSON and remove any sensitive information. To help automate this process (currently, for message collections only), a Python script redact-messages.py
is available. It has no external dependencies beyond a standard Python environment. It expects a collection of messages in the NDJSON format used by SMS Import / Export on standard input, and writes a redacted version of the same to standard output:
~$ ./redact-messages.py < messages.ndjson > messages-redacted.ndjson
When reporting a crash, particularly a reproducible one, please attach a logcat (a collection of log messages produced by Android - see here and here). Instructions for doing so (with increasing level of detail) can be found here, here, and here.
When importing messages that have been exported from a MIUI system into a MIUI system, the following error may be encountered:
java.lang.IllegalArgumentException: The non-sync-callers AND non-blocked-url should not specify DELETED for inserting.
For a possible solution, see here.
SMS Import / Export has been translated (from the original English) into the following languages (note that some of these translations may contain inaccuracies, due to changes to the app's original text since they were made):
To add a translation into a new language, or to correct, update, or improve an existing translation, see here.
The following are various features and improvements to the app that have been suggested and may be implemented in the future:
- Greater flexibility of scheduled exporting, including intervals other than daily, incremental / differential exporting, and retention handling (discussion in issue #7)
For information about contributing to SMS Import / Export, and a list of contributors, see here.
SMS Import / Export does no tracking, advertising, or phoning home. No user data is stored or transmitted anywhere except as explicitly designated by the user.
SMS Import / Export is a sibling project to sms-db, a Linux tool to build an SQLite database out of collections of SMS and MMS messages in various formats. sms-db will hopefully eventually be able to import ZIP files created by SMS Import / Export, and to export its database to ZIP files that can be imported by SMS Import / Export.
Coming from a procedural, command line interface, synchronous, Linux, Perl and Python background, the development of SMS Import / Export served as a crash course in object-oriented, graphical user interface, asynchronous, Android, Kotlin programming, and consequently entailed a fair amount of amateurishness and cargo cult programming. After much work and learning, however, the app does seem to function correctly and effectively.
SMS Import / Export is absolutely free software, and there is no expectation of any sort of compensation or support for the project. That being said, if anyone wishes to donate (to Thomas More, the app's primary author), this can be done via the Ko-fi platform.
SMS Import / Export is free / open source software, released under the terms of the GNU GPLv3 or later.