NiFI SQL Lookup Service is a SQL-based lookup service for Apache NiFi. It allows you to enrich your flowfiles with any jdbc-compliant data store, including:
- PostgreSQL
- Oracle
- MySQL
- MS SQL Server
- SQLite
- ... and many more
It includes both LookupRecord and LookupAttribute controllers.
These controllers were designed to be flexible and fast.
- Download Apache NiFi
- Compile and install
nifi-sqllookup-bundle
> cd nifi-sqllookup-bundle
> mvn package
> cp ./nifi-sqllookup-services-nar/target/nifi-sqllookup-services-nar-1.12.0.nar /NIFI_INSTALL/lib/
> cp ./nifi-sqllookup-services-api-nar/target/nifi-sqllookup-services-api-nar-1.12.0.nar /NIFI_INSTALL/lib/- Start NiFi
This service supports multiple query types:
- Named Parameters:
SELECT name FROM foo WHERE value = :value - SQL IN queries:
SELECT name FROM foo WHERE value IN(:values) - Multiple lookup values:
SELECT name FROM foo WHERE value IN(:values) AND sequence = :sequence AND catalog = :catalog
The goal of this service is to return values quickly. It has a built-in cache so that your database doesn't get overwhelmed.
This is configurable in the controller settings.
These caches are all built-in to this service. Select your preferable cache in the controller settings.
You can also select the number of items that you want to cache. The caches all keep the most accessed items available by default.
If you don't know which to choose, just go with the default.
https://github.com/mrcsparker/nifi-sqllookup-services-bundle/releases/latest