spugachev / SpotChum

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

SpotChum Bot

What is this repository for?

Docker containers deployment

docker build -t gcr.io/serp-main/spotchumbot:v1 .
docker run --name spotchumbot -d gcr.io/serp-main/spotchumbot:v1
gcloud docker push gcr.io/serp-main/spotchumbot:v1

docker build -t gcr.io/serp-main/instcrawler:v1 .
docker run --name instcrawler -d gcr.io/serp-main/instcrawler:v1
gcloud docker push gcr.io/serp-main/instcrawler:v1

docker build -t gcr.io/serp-main/cognitive:v1 .
docker run --name cognitive -d gcr.io/serp-main/cognitive:v1
gcloud docker push gcr.io/serp-main/cognitive:v1

gcr.io/serp-main/spotchumbot:v1
gcr.io/serp-main/instcrawler:v1
gcr.io/serp-main/cognitive:v1

Kubernetes cluster config

Kubernetes manual Kubernetes cluster

gcloud config set project serp-main
gcloud config set compute/zone europe-west1-d
gcloud config set container/cluster cluster-serp
gcloud container clusters get-credentials cluster-serp

kubectl run spotchumbot --image=gcr.io/serp-main/spotchumbot:v1
kubectl get deployments
kubectl get pods
kubectl delete service,deployment spotchumbot
kubectl cluster-info
kubectl config view

Scala and Spark

Write and run Spark Scala jobs on a Cloud Dataproc cluster BasicParseJson.scala SparkPageRank.scala

sbt run
sbt package

gs://scala-jars/SpotChumJob.jar

About


Languages

Language:XSLT 54.0%Language:JavaScript 31.8%Language:CSS 11.2%Language:Scala 3.0%