facebookarchive / flashback

Capture and replay real mongodb workloads

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Allow sharded clusters to self discover

dbmurphy opened this issue · comments

If using a sharded system, it could be useful to have it self discover nodes and have something like

auto_config: true,
// profiler could be [ all, secondaries, primary]
// additional options could be added later
auto_config_options: { profiler : "all" }

One question is due the pyMongos change in 3.0 should we support only the new MongoClient which include the replica_set_connection class, or should we detect the version and then build the connection based on which driver version is installed.

@charity : Thoughts on this?

Detecting the version and building the connection based on that sounds better to me. @tredman any thoughts?

I think I found a pretty backward compatible way to do this. Currently about to test this out :)

The new config is below and it will copy (user,password,auth_db) to the profiler/oplog lists as it builds them:

import logging


DB_CONFIG = {
        "auto_config" : True,
        "auto_config_options": {
            "user": "admin",
            "password": "password",
            "auth_db": "admin",
            "use_secondaries": False,
            #mongo_uri in this case is a mongos
            "mongo_uri" : "mongos_hostname:mongos_port"
        },
        "target_databases": [ ],
        "target_collections": None,
        "oplog_servers": [ ],
        "profiler_servers": [   ],
        "oplog_output_file": "./OPLOG_OUTPUT",
        "output_file": "./OUTPUT",
        "duration_secs": 60
    }

APP_CONFIG = {
    "logging_level": logging.DEBUG
}

Any suggestions or asks before I finalize on this pattern?

This is now tied to the pull request btw :) I am going to close this and reference things there.