boto / boto

For the latest version of boto, see https://github.com/boto/boto3 -- Python interface to Amazon Web Services

Home Page:http://docs.pythonboto.org/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Invalid device name /dev/sdf1 on EC2 spot request

jldonovan3 opened this issue · comments

I'm using the brenda tool to try to spin up ec2 spot instances to use as render nodes for a Blender project. It utilizes boto as its interface for aws, and I am requesting spot instances that have an EBS snapshot attached that contains project files to render. I get a bad-parameters failure on the spot request with the following state: failed: Invalid device name /dev/sdf1 The output from the brenda-run command gives me my Snapshot ID, and what appears to be the device mapping:
Project EBS snapshot: [('snap-xxxxxxxx', '/dev/sdf1')]

Would the spot requests be failing because boto is device mapping to something AWS is saying is invalid? brenda is assigning block_device_map to bdm , and in brenda/aws.py, bdm is using boto.ec2.blockdevicemapping.BlockDeviceMapping()

from brenda/aws.py:

def blk_dev_map(opts, conf, itype, snapshots):
    if not int(conf.get('NO_EBS', '0')):
        bdm = boto.ec2.blockdevicemapping.BlockDeviceMapping()
        snap = project_ebs_snapshot(conf)
        snap_id = translate_snapshot_name(conf, snap, snapshots)
        snap_description = []
        if snap_id:
            dev = utils.blkdev(0)
            bdm[dev] = boto.ec2.blockdevicemapping.EBSBlockDeviceType(snapshot_id=snap_id, delete_on_termination=True)
            snap_description.append((snap, snap_id, dev))
        i = 0
        for k in additional_ebs_iterator(conf):
            i += 1
            snap = parse_ebs_url(conf[k].split(',')[0])
            snap_id = translate_snapshot_name(conf, snap, snapshots)
            if snap_id:
                dev = utils.blkdev(i)
                bdm[dev] = boto.ec2.blockdevicemapping.EBSBlockDeviceType(snapshot_id=snap_id, delete_on_termination=True)
                snap_description.append((snap, snap_id, dev))
        istore_dev = add_instance_store(opts, conf, bdm, itype)
        return bdm, snap_description, istore_dev
    else:
        return None, None, None

In the file used to start up the instances, brenda/run.py, bdm is given as the block_device_map

from brenda/run.py:

def spot(opts, conf):
    ami_id = utils.get_opt(opts.ami, conf, 'AMI_ID', default=AMI_ID, must_exist=True)
    price = utils.get_opt(opts.price, conf, 'BID_PRICE', must_exist=True)
    reqtype = 'persistent' if opts.persistent else 'one-time'
    itype = brenda_instance_type(opts, conf)
    snapshots = aws.get_snapshots(conf)
    bdm, snap_description, istore_dev = aws.blk_dev_map(opts, conf, itype, snapshots)
    script = startup_script(opts, conf, istore_dev)
    user_data = None
    if not opts.idle:
        user_data = script
    ssh_key_name = conf.get("SSH_KEY_NAME", "brenda")
    sec_groups = (conf.get("SECURITY_GROUP", "brenda"),)
    run_args = {
        'image_id'      : ami_id,
        'price'         : price,
        'type'          : reqtype,
        'count'         : opts.n_instances,
        'instance_type' : itype,
        'user_data'     : user_data,
        'key_name'      : ssh_key_name,
        'security_groups' : sec_groups,
        'block_device_map' : bdm,
        }

    print "----------------------------"
    print "AMI ID:", ami_id
    print "Max bid price", price
    print "Request type:", reqtype
    print "Instance type:", itype
    print "Instance count:", opts.n_instances
    if snap_description:
        print "Project EBS snapshot:", snap_description
    if istore_dev:
        print "Instance store device:", istore_dev
    print "SSH key name:", ssh_key_name
    print "Security groups:", sec_groups
    print_script(opts, conf, script)
    aws.get_done(opts, conf) # sanity check on DONE var
    if not opts.dry_run:
        ec2 = aws.get_ec2_conn(conf)
        reservation = ec2.request_spot_instances(**run_args)
        print reservation

I would have asked this question on the brenda issues page, but the tool is old and obscure by now, and nobody seems to be around anymore to answer questions.

Full disclosure, I have no experience with boto, so I may be way off in pinpointing the issue. (I'm more of a vfx guy than a technologist...I'm hobbling along trying to get these render nodes working) It seems similar to what's going on in this thread, but I'm not sure how to implement the solution in my case.

Version: boto 2.49.0

Appreciate any help!