rclone cloud sync utility for ArkOS
Watches save directories for RetroArch, and select standalone games.
This project offers no warranties or guarantees. Always make extra backup copies of your data. Use at your own risk!
arklone is released under a GNU GPLv3 license. See LICENSE.md for more information.
rclone, RetroArch, EmulationStation, and ArkOS are the properties of their respective owners.
Table of Contents
- Installation
- rclone Configuration
- First Run
- Settings Dialog
- Syncing with the Cloud
- Advanced RetroArch Configuration
- Recommended RetroArch Configuration
- Advanced arklone Configuration
- Troubleshooting
- Developers
- FAQ
This module is not yet integrated into ArkOS. See this pull request for updates.
To test, install manually by downloading the installation script, and run it from a terminal:
wget https://github.com/ridgekuhn/arkos/raw/cloudbackups/10092021/install.sh -O installArklone.sh
chmod a+x installArklone.sh
./installArklone.sh
rm ./installArklone.sh
wget https://github.com/ridgekuhn/arkos/raw/cloudbackups/10092021/uninstall.sh -O uninstallArklone.sh
chmod a+x uninstallArklone.sh
./uninstallArklone.sh
rm ./uninstallArklone.sh
To begin using arklone, you must create an rclone config file. For most cloud providers, this will involve installing rclone to a computer with a web browser, like your desktop or laptop. See the rclone docs for more information on how to do this for your specific provider. Make sure you install the latest version of rclone, (1.56.2 as of this writing) to your desktop or laptop. If you use a package manager like apt, the repository version will be outdated.
Once you have completed this process, copy the rclone.conf file from your computer to your ArkOS device. Your rclone.conf can be located by running:
rclone config file
On your ArkOS SD card, copy rclone.conf to:
EASYROMS/backup/rclone/rclone.conf
If you already had rclone installed on your device, your rclone.conf has been moved to EASYROMS for easier access, and symlinked to its original location. arklone will restore the original arrangement if uninstalled.
From EmulationStation, navigate to Options -> Cloud Settings
On first run, you will be greeted by a prompt asking if you'd like to change your RetroArch configurations to the recommended settings.
Obviously, this is recommended!
This will set the following settings in your retroarch.cfg (and your retroarch32/retroarch.cfg):
savefile_directory = "~/.config/retroarch/saves"
savefiles_in_content_dir = "false"
sort_savefiles_enable = "false"
sort_savefiles_by_content_enable = "true"
savestate_directory = "~/.config/retroarch/saves"
savestates_in_content_dir = "false"
sort_savestates_enable = "false"
sort_savestates_by_content_enable = "true"
This will result in savefiles and savestates being stored in the same directory hierarchy as your RetroArch content root, in ~/.config/retroarch/saves
eg,
~/.config/retroarch/saves/nes/TheLegendOfZelda.srm
~/.config/retroarch/saves/nes/TheLegendOfZelda.savestate0
- Set cloud remote
Allows you to select from the remotes you set up in
rclone.conf - Manually sync saves Manually send/receive from the cloud remote
- Enable/Disable automatic saves sync Watches directories for changes and syncs them to your selected remote.
- Manual backup/sync ArkOS settings Runs the ArkOS backup script and uploads the file to the selected remote.
- Regenerate RetroArch path units Re-scans for new RetroArch directories to watch and generates path units for them.
- View log file Shows the log file.
Keeping multiple devices synced can be difficult. arklone tries to do its best, but you should always keep an extra backup copy just in case.
If you enable automatic syncing in the settings dialog, arklone assumes the copy of your data stored in the cloud is the canonical and "always correct" version. On system boot, arklone will run before EmulationStation and attempt to receive updates from the cloud remote. If the remote contains a newer version of a file, it will overwrite the local copy. On this initial sync, arklone only receives updates and does not send anything back.
If the boot sync process succeeds, arklone will begin watching all your save directories, and send updates any time a write is detected, overwriting older versions on the cloud remote. It will also receive updates at the interval set when you enabled automatic syncing.
If the boot process fails or is cancelled by the user, the user is given the chance to try again, or the dirty boot state is set.
The manual sync dialog allows you to send or receive any directory which has a path module registered with arklone.
If automatic syncing is enabled and the boot sync process fails, the dirty boot state is set. Automatic syncing is disabled for the rest of the session, and will resume after the next boot. On the next boot, the user is shown a warning message about potential data loss before the boot sync proceeds.
To manually reset the dirtyboot state, delete the lock file located at:
~/.config/arklone/.dirtyboot
This section is for users who wish to have more control over their retroarch.cfg settings and save directories.
ArkOS includes 64-bit and 32-bit builds of RetroArch.
The configuration files are stored at
/home/ark/.config/retroarch/retroarch.cfg and
/home/ark/.config/retroarch32/retroarch.cfg.
The following settings are supported:
savefile_directory
savefiles_in_content_dir
sort_savefiles_enable
sort_savefiles_by_content_enable
savestate_directory
savestates_in_content_dir
sort_savestates_enable
sort_savestates_by_content_enable
For the next examples, filetype refers to either savefile or savestate.
If filetypes_in_content_dir = "true", it will override the other related settings, and create save data next to the content file.
Otherwise, if sort_filetypes_enable = "true", save data will be organized by libretro core inside filetype_directory.
eg,
/path/to/filetype_directory/FCEUmm/TheLegendOfZelda.srm
If sort_filetypes_by_content_enable = "true", save data will be organized by the parent directory of the content file.
eg,
/path/to/filetype_directory/nes/TheLegendOfZelda.srm
If both sort_filetypes_enable = "true" and sort_filetypes_by_content_enable = "true", save data will be organized by the parent directory of the content file, then by libretro core.
eg,
/path/to/filetype_directory/nes/FCEUmm/TheLegendOfZelda.srm
ArkOS currently contains a bug which prevents systemd path units from watching subdirectories of exFAT partitions. (See issue #289.) This means that savefiles/savestates can not be watched (and automatically synced) if filetypes_in_content_dir = "true".
Until this bug is resolved, if you wish to store your saves next to the content, you must manually sync your saves from the arklone dialog.
Since the bug only applies to exFAT partitions, advanced users who really want to use automatic syncing and keep savefiles/savestates in the content directories can re-format the EASYROMS partition to FAT32, ext4, etc and edit the mount entry in /etc/fstab.
savefile_directory = "~/.config/retroarch/saves"
savefiles_in_content_dir = "false"
sort_savefiles_enable = "false"
sort_savefiles_by_content_enable = "true"
savestate_directory = "~/.config/retroarch/states"
savestates_in_content_dir = "false"
sort_savestates_enable = "false"
sort_savestates_by_content_enable = "true"
Arklone has a few settings that can be changed by the user, mostly paths where arklone looks for various files. The user configuration file is stored at ~/.config/arklone/arklone.cfg.
Setting remote to an empty string forces the settings dialog to show the "first run" screen again.
arklone.cfg
remote = ""
Where filetype refers to either savefile or savestate:
If your retroarch.cfg contains the settings filetypes_in_content_dir = "true" or sort_filetypes_by_content_enable = "true", arklone expects your RetroArch content to be organized in a directory hierarchy with one level of subdirectories, where each contains all content for a particular platform.
eg,
retroarchContentRoot/nes/TheLegendOfZelda.rom
arklone.cfg
retroarchContentRoot = "/absolute/path/to/retroarchContentRoot"
arklone also supports select standalone software and "ports". See the systemd/units for a list, and the Path Units section of the developer docs for more info.
arklone supports multiple instances of RetroArch, in case your distro has both 64-bit and 32-bit builds installed. Set retroarchCfg to a space-delimited list of absolute paths to each retroarch.cfg.
arklone.cfg
retroarchCfg = "/home/user/.config/retroarch/retroarch.cfg /home/user/.config/retroarch32/retroarch.cfg"
arklone passes various filter lists to rclone when a sync script is run. See the Path Units and rclone Filters sections of the developer docs for more info.
arklone only watches the RetroArch save directories it knew about when it first generated the corresponding path units. If you selected "Set Recommended Settings" on your first run, arklone will automatically generate path units for all your RetroArch content directories which are not empty. If you've added games since then and some of your save directories are not being synced automatically, try manually regenerating them from the arklone dialog menu.
If you change any of the above settings in retroarch.cfg, you must also manually regenerate the path units.
ArkOS is constantly updated with new apps and ports, and we probably haven't caught up to them yet. Please create a new issue so we can include it in a future update.
To save unnecessary writes to your SD card or hard drive, arklone writes logs to the RAM filesystem at /dev/shm/arklone.log. This file disappears when the system is powered down, but you can view it by opening the arklone settings dialog and selecting "View log file".
Contributions are welcome! Please see the developer docs.
Linux
A RetroPie release is planned soon. arklone is written in bash, and relies on tools like apt, dpkg, and inotify-tools. It should theoretically work on any Debian-based distro, as long as your content is organized in the expected directory hierarchy. See Advanced arklone Configuration for more info.
Windows and MacOS
If your cloud provider offers a desktop client, you should install and use that instead.
See the Path Units section of the developer docs.
Not unless you want to set it up yourself. Many users' game libraries are massive and would probably exceed the storage limit on your cloud account several times over. There are also system performance implications for keeping this much data synced on low-power devices, like the ones ArkOS is designed for, where the background sync operations may affect gameplay. It's probably much faster/efficient to transfer your game libraries from device-to-device via USB or over your LAN.
If you have automatic syncing enabled, arklone attempts to download all the different save directories it knows about from the cloud remote. If they don't exist on the cloud remote, these messages are generated for logging and debugging purposes. If there are any actual problems downloading save data from the cloud, you will be presented with a dialog screen, and asked if you want to proceed or view the log file. If you don't see this dialog screen and your device boots straight into EmulationStation, then everything is ok!