Giters

OryxProject / oryx

Oryx 2: Lambda architecture on Apache Spark, Apache Kafka for real-time large scale machine learning

Home Page:http://oryx.io

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Reseting model

stiv-yakovenko opened this issue · comments

commented

I have some recommendation data in oryx. How can I remove everything I had sent there without reformatting hdfs?

commented

You can just delete data and or models from the data and model dirs you set up on HDFS. You may also need to delete the Kafka topics to be sure they are empty.

commented

I am still not able to reset model. Initially model is in place:

image

Now I kill hdfs, yarn, oryx-batch, oryx-serving, oryx-speed:

image

Now I delete all hdfs files and format hdfs the partition:

image

And recreate kafka topics:

image

After restarting yarn, hdfs, orix-* I still see model not being reset:

image

It looks like there is some inner cache somewhere inside one of layers, how can I reset it?

commented

There are still a number of possibilities, and I think they're specific to your deployment.

  • Are you sure you killed the serving process? I'm not clear from the text above. That isn't using the script from the project.
  • Are you sure you deleted the topics? this shows they existed already, not deleted.
  • Are you sure you are deleting the files and topics your config references?