ODK Aggregate
ODK Aggregate provides a ready-to-deploy server and data repository to:
- provide blank forms to ODK Collect (or other OpenRosa clients),
- accept finalized forms (submissions) from ODK Collect and manage collected data,
- visualize the collected data using maps and simple graphs,
- export data (e.g., as CSV files for spreadsheets, or as KML files for Google Earth), and
- publish data to external systems (e.g., Google Spreadsheets or Google Fusion Tables).
ODK Aggregate can be deployed on a Tomcat server (or any servlet 2.5-compatible (or higher) web container) backed with a PostgreSQL or a MySQL database server.
- ODK website: https://opendatakit.org
- ODK Aggregate usage instructions: https://opendatakit.org/use/aggregate/
- ODK forum: https://forum.opendatakit.org
- ODK developer Slack chat: http://slack.opendatakit.org
- ODK developer Slack archive: http://opendatakit.slackarchive.io
- ODK developer wiki: https://github.com/opendatakit/opendatakit/wiki
Getting the code
-
Fork the Aggregate project (why and how to fork)
-
Install Git LFS
-
Clone your fork of the project locally. At the command line:
git clone https://github.com/YOUR-GITHUB-USERNAME/aggregate
Setting up the database
Aggregate supports a variety of DBs, but we strongly recommend you use PostgreSQL first to ensure everything is working. If you wish to use another DB (e.g., MySQL, or SQLServer databases) after that see database configurations.
PostgreSQL with Docker
-
Install Docker and Docker Compose
-
To start the development server run the gradle task
postgresqlComposeUp
with the command:./gradlew postgresqlComposeUp
- Make sure port 5432 is not used by any other service in your computer. You can change this editing the
ports
section of thedb/postgresql/docker-compose.yml
configuration file. Be sure to check the documentation: Compose file version 3 reference - Ports section.
- Make sure port 5432 is not used by any other service in your computer. You can change this editing the
-
To stop the server run the gradle task
postgresqlComposeDown
with the command:./gradlew postgresqlComposeDown
.
Alternatively, you can use the docker-compose
command line tool to start & stop the development server. All configuration files regarding the development server are at db/postgresql
path. Check the online documentation.
Local PostgreSQL server
-
Download and install PostgreSQL 9 or later.
- If you are a macOS user, we recommend Postgres.app. If you are a Windows user, we recommend BigSQL.
-
In the command-line interface connect to the database. Assuming the user is postgres and the server is installed on your local machine, the command will be:
psql -U postgres -h localhost
. -
Setup your database with these commands. You must use
psql
or the\connect
command will not work.CREATE USER "odk_unit" WITH PASSWORD 'test'; CREATE DATABASE odk_db WITH OWNER odk_unit; GRANT ALL PRIVILEGES ON DATABASE odk_db TO odk_unit; \connect odk_db; CREATE SCHEMA odk_db; GRANT ALL PRIVILEGES ON SCHEMA odk_db TO odk_unit;
Running the project
Aggregate is built using Gradle and Gretty, but we strongly recommend you use IntelliJ IDEA first to ensure everything is working. If you wish to use another development environment after that, run ./gradlew tasks
to get a sense of your options.
Import
-
On the welcome screen, click
Import Project
, navigate to your aggregate folder, and select thebuild.gradle
file. -
Make sure you check
Use auto-import
option in theImport Project from Gradle
dialog -
Once the project is imported, IntelliJ may ask you configure any detected GWT, Spring or web Facets, you can ignore these messages
Configure
-
If you are running Java 9, make sure you also have Java 8 installed. Aggregate will not compile with Java 9.
-
Copy
gradle.properties.example
to agradle.properties
file at the root of the project and fill in your setup configurationKey Valid values Description warMode
complete
orinstaller
Use installer
if you are going to produce an installer with the resulting WAR artifact andcomplete
otherwiseorg.gradle.java.home
Set path to a Java 8 install. This is only required if you have Java 9 installed. - Any property can be overwritten by passing
-Pkey=value
arguments to any Gradle task.
- Any property can be overwritten by passing
Run
-
In the
Run
menu, selectEdit Configurations...
-
Press the + button to add a
Gradle
configuration- Name:
appStartWar
(or whatever you'd like) - Gradle project:
odk-aggregate
- Tasks:
appStartWar
- Name:
-
Press
OK
-
To run Aggregate, go to the
Run
menu, then toRun...
andRun
theappStartWar
configuration. This will start Aggregate. -
You should now be able to browse http://localhost:8080
Run the tests
-
Copy one of the example
jdbc.properties.example_*
andodk-settings.xml.example_*
files onsrc/test/resources/
into the same place removing the.example_*
extension. Choose the variant corresponding to your test database engine. -
Make sure your jdbc settings are correct. The examples use the default database name, user and password settings.
-
Right click on the
src/test/java
folder on IntelliJ and selectRun all Tests
option.
Debug
-
In the
Run
menu, selectEdit Configurations...
-
Press the + button to add a
Gradle
configuration- Name:
appStartWarDebug
(or whatever you'd like) - Gradle project:
odk-aggregate
- Tasks:
appStartWarDebug
- Name:
-
Press
Apply
and then press the + button to add aRemote
configuration- Name:
appServer
(or whatever you'd like) - Host:
localhost
- Port:
5005
- Search sources using module's classpath:
aggregate
- Name:
-
Press
OK
-
To debug Aggregate, go to the
Run
menu, then toRun...
andRun
(not Debug!) theappStartWarDebug
configuration. This will start Aggregate in debug mode and wait for a debugging session to be connected to the server's debugging listener. -
Now, go to the
Run
menu, then toRun...
andDebug
theappServer
. This will connect the debugger. -
You should now be able to browse http://localhost:8080 and debug
Connect from an external device
By default, Gretty will launch a server using a localhost
address which will not be accessible by external devices (e.g., ODK Collect in an emulator, ODK Briefcase on another computer). To set a non-localhost address, edit the following files:
- In
src/main/resources/security.properties
, changesecurity.server.hostname
to the address - In
build.gradle
, inside thegretty
block, changehost
to the same address
Extended topics
There is a /docs
directory in the repo with more documentation files that expand on certain topics:
- Configuration files
- Supported database configurations
- IntelliJ Setup
- Build the Installer app
- Build and run a Docker image
- Build and run with Docker Compose
- Build and run a Virtual Machine
Contributing
Any and all contributions to the project are welcome. ODK Aggregate is used across the world primarily by organizations with a social purpose so you can have real impact!
If you're ready to contribute code, see the contribution guide.
The best way to help us test is to build from source! We are currently focusing on stabilizing the build process.
Troubleshooting
-
We enabled Git LFS on the Aggregate codebase and reduced the repo size from 700 MB to 34 MB. No code was changed, but if you cloned before December 11th, 2017, you'll need to reclone the project.
-
If you get an Invalid Gradle JDK configuration found error importing the code, you might not have set the
JAVA_HOME
environment variable. Try these solutions. -
If you are having problems with hung Tomcat/Jetty processes, try running the
appStop
Gradle task to stop running all instances. -
If you're using Chrome and are seeing blank pages or refreshing not working, connect to Aggregate with the dev tools window open. Then in the
Network
tab, checkDisable cache
.