esys / go-amadeus-trip

Pipeline to extract travel information from emails using Amadeus Trip API

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Trip Parser

This project illustrates the use of Amadeus Trip API to parse emails and extract travel information from it.

How does it work

The go process retrieve mails by polling a Gmail inbox and then process them using the Amadeus TRIP API, first by creating a parsing job, then by querying the status to eventually get the results.

Extracted travel information is stored into a SQLite3 database and made available through a REST API.

alt text

Requirements

You will need:

Configuration

Configure the application through a file

  • this file must be called config.(yaml|json) and put at the project root
  • a sample config.sample.yaml with dummy values is available, copy and rename it to config.yaml before editing

Or use environment variables (they take precedence over the configuration file)

Name Description Example
API_LISTEN server:port on which API listens :1323
PARSER_KEY Amadeus API key yRveyxreiof83ID2FlldsfgIW95
PARSER_SECRET Amadeus API secret d5Gtof7Q4pxlI8KGH
PARSER_URL Amadeus API endpoint https://test.api.amadeus.com
MAIL_CREDENTIALS GMail client credentials JSON file client_credentials.json
MAIL_TOKEN GMail token JSON file gmail_token.json
STORAGE_NAME SQLite database name :memory:

Running

Build and run it directly

$ make run

Or Build and run it as a docker image

$ make docker-build docker-run

Check logs in stdout to see if processing is going OK.

2:02PM DBG trip 95ed6a4c-3910-4bce-8f06-0d2b2ea1d344 (ref: UCFRMZ) written in repository

To check for results, query the API

$ curl "http://localhost:1323"
[{"ID":"95ed6a4c-3910-4bce-8f06-0d2b2ea1d344","Reference":"UCFRMZ" .... }]

$ curl "http://localhost:1323/trip?ref=UCFRMZ"
{"ID":"95ed6a4c-3910-4bce-8f06-0d2b2ea1d344","Reference":"UCFRMZ","Start":"0001-01-01T00:00:00Z","End":"0001-01-01T00:00:00Z","TripSteps":[{"ID":"1ef923bb-10af-44b7-8022-65c7aae805b3","TripID":"95ed6a4c-3910-4bce-8f06-0d2b2ea1d344","Type":"flight-end","DateTime":"2020-04-12T15:30:00Z","Location":"PARIS","Description":"Flight end with TRANSAVIA FRANCE"},{"ID":"49c85155-4ad5-493e-973b-ec4a939b7a18","TripID":"95ed6a4c-3910-4bce-8f06-0d2b2ea1d344","Type":"flight-start","DateTime":"2020-04-12T11:55:00Z","Location":"TUNIS","Description":"Flight start with TRANSAVIA FRANCE"},{"ID":"df9b3b22-e797-467c-9e71-7ee6d13bb787","TripID":"95ed6a4c-3910-4bce-8f06-0d2b2ea1d344","Type":"flight-end","DateTime":"2020-04-06T17:45:00Z","Location":"TUNIS","Description":"Flight end with TRANSAVIA FRANCE"},{"ID":"ef983fa6-1222-4e2f-9315-9355895570a3","TripID":"95ed6a4c-3910-4bce-8f06-0d2b2ea1d344","Type":"flight-start","DateTime":"2020-04-06T16:10:00Z","Location":"PARIS","Description":"Flight start with TRANSAVIA FRANCE"}]}

Code

Code organization follows the Clean Architecture guidelines.

  • domain contains only models and interfaces
  • usecase contains application logic
  • adapter is for interface implementations and communication with external world
.
├── adapter
│   ├── api
│   │   ├── rest.go
│   │   └── rest_test.go
│   ├── backend
│   │   ├── mail
│   │   │   └── gmail
│   │   │       ├── gmail.go
│   │   │       └── gmail_test.go
│   │   └── parser
│   │       └── amadeus
│   │           ├── amadeus.go
│   │           ├── amadeus_test.go
│   │           ├── converter.go
│   │           ├── converter_test.go
│   │           ├── dto.go
│   │           ├── dto_test.go
│   │           └── testdata
│   │               ├── air.json
│   │               ├── hotel.json
│   │               └── msg-encoded
│   └── repository
│       ├── sqlite.go
│       └── sqlite_test.go
├── domain
│   ├── backend.go
│   ├── mocks
│   │   ├── TripFinder.go
│   │   └── TripRepository.go
│   ├── model
│   │   ├── email.go
│   │   └── trip.go
│   ├── repository.go
│   └── usecase.go
└── usecase
    ├── processor.go
    ├── tripfinder.go
    └── tripfinder_test.go

Dependencies

This project makes use of:

About

Pipeline to extract travel information from emails using Amadeus Trip API


Languages

Language:Go 97.0%Language:Makefile 2.8%Language:Dockerfile 0.2%