karl-run / finndiff

A node web scraper with a React front-end to look at changes in apartment listings over time

finndiff

Unofficial, not affiliated with Finn.no, Schibsted or Polaris Media.

What is it

Finndiff itself is in Norwegian, I'll keep this documentation in English.

Finndiff aims to do two things: Reduce the number of spam calls from realtors, and let you see how a apartment listing evolves over time.

Less spam

Whenever you "heart" an apartment listing the realtor that posted it gets your contact information, they might use this information to contact you unsolicited. If you use Finndiff to keep track of the listings you are interested in, no one will know.

Keep track

Apartment listings change over time, but on the the live listing only contains the most recent edit. Finndiff stores diffs to let you keep track of price changes, or any other edit to the listing.

Caveat: It only starts tracking when you add the listing to Finndiff.

Development

This project has two modules, api and web, with backend and frontend respectively. For development you can run them both at the same time from the root folder.

Running both from root folder

1. npm i (installs dependencies for both modules)
2. npm run dev

This starts the api with nodemon, and the development build of the web project. The web project is proxied through CRA's proxy functionality. Meaning that you should browse the application through localhost:3000 and use graphiql through localhost:4000/api/graphql. This is not true for the production build, where the api module statically serves the web build.

Run only api

1. npm run dev-api

Run only web

1. npm run dev-web

Bump the version

1. From root: npm run tag
2. lerna creates a commit, push the commit and the tag with git push --follow-tags

Deployment

This project has been configured to be deployed to now.sh.

Prerequisites

1. A now.sh account
2. now.sh cli installed
3. A mongodb database and its connection string
4. (Optional) For authentication, an auth0 account.

Deploying

Run from the root folder

0. now --public -e MONGO_URL=<url-to-mongo-db> (you can omit the --public flag if you are on a paid plan)

You can omit the -e flag if you configure environment variables in the now.json file. It's best to use secrets so you don't commit full access to your DB.

What happens

When deploying to now.sh it runs npm i and npm build in the build step. This creates a production build of the web module. It then runs npm start which starts the api node server with the production flag, which is the API and hosts the static files in the web folder.