alibaba / pipcook

Machine learning platform for Web developers

Home Page:https://alibaba.github.io/pipcook/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

pipcook run json 失败

holanlan opened this issue · comments

commented

node:v14.15.0

pipcook init 成功

pipcook daemon start 成功

pipcook run json 文件失败

{
  "plugins": {
    "dataCollect": {
      "package": "@pipcook/plugins-object-detection-pascalvoc-data-collect",
      "params": {
        "url": "http://ai-sample.oss-cn-hangzhou.aliyuncs.com/pipcook/datasets/component-recognition-detection/component-recognition-detection.zip"
      }
    },
    "dataAccess": {
      "package": "@pipcook/plugins-coco-data-access"
    },
    "modelDefine": {
      "package": "@pipcook/plugins-detectron-fasterrcnn-model-define"
    },
    "modelTrain": {
      "package": "@pipcook/plugins-detectron-model-train",
      "params": {
        "steps": 100000
      }
    },
    "modelEvaluate": {
      "package": "@pipcook/plugins-detectron-model-evaluate"
    }
  }
}

info如下

plugin @pipcook/plugins-object-detection-pascalvoc-data-collect@1.1.0 has already been installed
✔ plugin @pipcook/plugins-coco-data-access@1.1.2 has already been installed
✔ plugin @pipcook/plugins-detectron-fasterrcnn-model-define@1.1.1 has already been installed
✔ plugin @pipcook/plugins-detectron-model-train@1.1.1 has already been installed
✔ plugin @pipcook/plugins-detectron-model-evaluate@1.1.3 has already been installed
ℹ start to create pipeline
✔ pipeline is created: qohrlo3y, installing
✔ pipeline installed successfully, start to run job
✔ job is created: 07ltu05m, running
ℹ [job] running
ℹ [job] running dataCollect start
ℹ start loading plugin @pipcook/plugins-object-detection-pascalvoc-data-collect
ℹ downloading dataset ...
ℹ unzip and collecting data...
ℹ [job] running dataCollect end
ℹ [job] running dataAccess start
ℹ start loading plugin @pipcook/plugins-coco-data-access
ℹ create a result "33x4v0s7" for plugin "@pipcook/plugins-coco-data-access@1.1.2"
ℹ [job] running dataAccess end
ℹ [job] running modelDefine start
ℹ start loading plugin @pipcook/plugins-detectron-fasterrcnn-model-define
ℹ [job] fails
✖ job is failed: Error: Could not locate the bindings file. Tried:
 → /Users/holan/.pipcook/plugins/node_modules/@pipcook/plugins-detectron-fasterrcnn-model-define/node_modules/@pipcook/boa/build/boa.node
 → /Users/holan/.pipcook/plugins/node_modules/@pipcook/plugins-detectron-fasterrcnn-model-define/node_modules/@pipcook/boa/build/Debug/boa.node
 → /Users/holan/.pipcook/plugins/node_modules/@pipcook/plugins-detectron-fasterrcnn-model-define/node_modules/@pipcook/boa/build/Release/boa.node
 → /Users/holan/.pipcook/plugins/node_modules/@pipcook/plugins-detectron-fasterrcnn-model-define/node_modules/@pipcook/boa/out/Debug/boa.node
 → /Users/holan/.pipcook/plugins/node_modules/@pipcook/plugins-detectron-fasterrcnn-model-define/node_modules/@pipcook/boa/Debug/boa.node
 → /Users/holan/.pipcook/plugins/node_modules/@pipcook/plugins-detectron-fasterrcnn-model-define/node_modules/@pipcook/boa/out/Release/boa.node
 → /Users/holan/.pipcook/plugins/node_modules/@pipcook/plugins-detectron-fasterrcnn-model-define/node_modules/@pipcook/boa/Release/boa.node
 → /Users/holan/.pipcook/plugins/node_modules/@pipcook/plugins-detectron-fasterrcnn-model-define/node_modules/@pipcook/boa/build/default/boa.node
 → /Users/holan/.pipcook/plugins/node_modules/@pipcook/plugins-detectron-fasterrcnn-model-define/node_modules/@pipcook/boa/compiled/14.15.0/darwin/x64/boa.node
 → /Users/holan/.pipcook/plugins/node_modules/@pipcook/plugins-detectron-fasterrcnn-model-define/node_modules/@pipcook/boa/addon-build/release/install-root/boa.node
 → /Users/holan/.pipcook/plugins/node_modules/@pipcook/plugins-detectron-fasterrcnn-model-define/node_modules/@pipcook/boa/addon-build/debug/install-root/boa.node
 → /Users/holan/.pipcook/plugins/node_modules/@pipcook/plugins-detectron-fasterrcnn-model-define/node_modules/@pipcook/boa/addon-build/default/install-root/boa.node
 → /Users/holan/.pipcook/plugins/node_modules/@pipcook/plugins-detectron-fasterrcnn-model-define/node_modules/@pipcook/boa/lib/binding/node-v83-darwin-x64/boa.node
    at bindings (/Users/holan/.pipcook/plugins/node_modules/@pipcook/plugins-detectron-fasterrcnn-model-define/node_modules/bindings/bindings.js:126:9)
    at Object.<anonymous> (/Users/holan/.pipcook/plugins/node_modules/@pipcook/plugins-detectron-fasterrcnn-model-define/node_modules/@pipcook/boa/lib/index.js:5:35)
    at Module._compile (internal/modules/cjs/loader.js:1063:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1092:10)
    at Module.load (internal/modules/cjs/loader.js:928:32)
    at Function.Module._load (internal/modules/cjs/loader.js:769:14)
    at Module.require (internal/modules/cjs/loader.js:952:19)
    at require (internal/modules/cjs/helpers.js:88:18)
    at Object.<anonymous> (/Users/holan/.pipcook/plugins/node_modules/@pipcook/plugins-detectron-fasterrcnn-model-define/dist/index.js:34:13)
    at Module._compile (internal/modules/cjs/loader.js:1063:30)
commented

是不是install 的时候中断过,第二次 install 的时候忽略已存在 boa package,就没有再执行编译 script 了,可以看下 /Users/holan/.pipcook/plugins/node_modules/@pipcook/plugins-detectron-fasterrcnn-model-define/node_modules/@pipcook/boa 的目录结构吗?

$ cd /Users/holan/.pipcook/plugins/node_modules/@pipcook/plugins-detectron-fasterrcnn-model-define/node_modules/@pipcook/boa
$ tree

或者在这个目录:

$ cd /Users/holan/.pipcook/plugins/node_modules/@pipcook/boa
$ tree
commented

好像确实是这个问题,我把plugins-detectron-fasterrcnn-model-define下的node_moduldes删除重新install就可以了

不过现在有其他问题,例如会报一些[job] fails,例如

job is failed: Error: Cannot find module '/Users/holan/.pipcook/plugins/node_modules/@pipcook/plugins-detectron-model-train/dist/index'. Please verify that the package.json has a valid "main" entry
    at tryPackage (internal/modules/cjs/loader.js:303:19)
    at Function.Module._findPath (internal/modules/cjs/loader.js:516:18)
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:867:27)
    at Function.Module._load (internal/modules/cjs/loader.js:725:27)
    at Module.require (internal/modules/cjs/loader.js:952:19)
    at require (internal/modules/cjs/helpers.js:88:18)
    at Object.default_1 [as default] (/Users/holan/.pipcook/server/node_modules/@pipcook/costa/dist/src/client/loaders/nodejs.js:5:14)
    at Object.default_1 [as default] (/Users/holan/.pipcook/server/node_modules/@pipcook/costa/dist/src/client/loaders/index.js:14:28)
    at /Users/holan/.pipcook/server/node_modules/@pipcook/costa/dist/src/client/entry.js:113:41
    at Generator.next (<anonymous>)

目前的解决办法是当github上找到对应的库,源码下载下来在对应目录手动build

commented

好像确实是这个问题,我把plugins-detectron-fasterrcnn-model-define下的node_moduldes删除重新install就可以了

不过现在有其他问题,例如会报一些[job] fails,例如

job is failed: Error: Cannot find module '/Users/holan/.pipcook/plugins/node_modules/@pipcook/plugins-detectron-model-train/dist/index'. Please verify that the package.json has a valid "main" entry
    at tryPackage (internal/modules/cjs/loader.js:303:19)
    at Function.Module._findPath (internal/modules/cjs/loader.js:516:18)
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:867:27)
    at Function.Module._load (internal/modules/cjs/loader.js:725:27)
    at Module.require (internal/modules/cjs/loader.js:952:19)
    at require (internal/modules/cjs/helpers.js:88:18)
    at Object.default_1 [as default] (/Users/holan/.pipcook/server/node_modules/@pipcook/costa/dist/src/client/loaders/nodejs.js:5:14)
    at Object.default_1 [as default] (/Users/holan/.pipcook/server/node_modules/@pipcook/costa/dist/src/client/loaders/index.js:14:28)
    at /Users/holan/.pipcook/server/node_modules/@pipcook/costa/dist/src/client/entry.js:113:41
    at Generator.next (<anonymous>)

目前的解决办法是当github上找到对应的库,源码下载下来在对应目录手动build

这个报错信息比较奇怪,你刚刚说的

我把plugins-detectron-fasterrcnn-model-define下的node_moduldes删除重新install就可以了

是怎么操作的?

commented

这个操作的前提条件是我的大部分插件例如plugins-detectron-fasterrcnn-model-define,都是本地安装的,因为远程直接安装经常会出现很多问题,所以我都是找到库以后,通过pipcook plugin install 本地目录来安装的,以上是前提条件

pipcook run json时,就会出现上面这个回复的错误,我看了下@pipcook目录下对应插件的目录,是缺少了dist目录导致的,推断是本地安装插件没有自动build,因此先npm install 依赖下来,然后再执行npm run build

而boa的问题应该是我在install 的时候,觉得太慢了,切换了淘宝源再次install的时候,有一些文件没有安装到,所以我把@pipcook/plugins-detectron-fasterrcnn-model-define/node_moduls删除后,重新在@pipcook/plugins-detectron-fasterrcnn-model-define目录下执行install就可以了

commented

这个操作的前提条件是我的大部分插件例如plugins-detectron-fasterrcnn-model-define,都是本地安装的,因为远程直接安装经常会出现很多问题,所以我都是找到库以后,通过pipcook plugin install 本地目录来安装的,以上是前提条件

那这个可以解释得通,本地插件没有 build 引起的找不到 dist/index

commented

另外还想请问一下,现在基本已经run起来了,但是卡在

[11/18 14:58:31 d2.checkpoint.c2_model_loading]: Some model parameters or buffers are not found in the checkpoint:
ℹ   pixel_mean
ℹ   pixel_std
ℹ   proposal_generator.anchor_generator.cell_anchors.0
ℹ   proposal_generator.rpn_head.anchor_deltas.{bias, weight}
ℹ   proposal_generator.rpn_head.conv.{bias, weight}
ℹ   proposal_generator.rpn_head.objectness_logits.{bias, weight}
ℹ   roi_heads.box_predictor.bbox_pred.{bias, weight}
ℹ   roi_heads.box_predictor.cls_score.{bias, weight}
ℹ [11/18 14:58:31 d2.checkpoint.c2_model_loading]: The checkpoint state_dict contains keys that are not used by the model:
ℹ   fc1000_b
ℹ   fc1000_w
ℹ   conv1_b
ℹ [11/18 14:58:31 d2.engine.train_loop]: Starting training from iteration 0

这个步骤已经半小时了,用的是官方的json例子,这个时长是正常的吗?

commented

目前有些 mac 上跑 tensorflow 相关的 pipeline 会卡住,这个问题还在修复中,可以跟踪这个 issue: #631

commented

好的,非常感谢

commented

请问现在把pipeline放在服务器上跑,CentOS 7.8,fetching package一直卡住是什么原因呢?下载到本地build完以后,从本地安装插件一样卡住。

commented
[root@VM-0-5-centos pipcook]# pipcook plugin list
┌─────────┬────────────┬────────────────────────────────────────────────────────────┬─────────┬───────────────┬──────────┬─────────────┐
│ (index) │     id     │                            name                            │ version │   category    │ datatype │   status    │
├─────────┼────────────┼────────────────────────────────────────────────────────────┼─────────┼───────────────┼──────────┼─────────────┤
│    0    │ '60trpxer' │ '@pipcook/plugins-object-detection-pascalvoc-data-collect' │ '1.1.0' │ 'dataCollect' │ 'image'  │ 'installed' │
│    1    │ 'tpodulcq' │            '@pipcook/plugins-coco-data-access'             │ '1.1.2' │ 'dataAccess'  │ 'image'  │ 'installed' │
│    2    │ 'haf46q8b' │    '@pipcook/plugins-detectron-fasterrcnn-model-define'    │ '1.1.1' │ 'modelDefine' │ 'image'  │  'pending'  │
│    3    │ 'pa4bnkus' │          '@pipcook/plugins-detectron-model-train'          │ '1.1.1' │ 'modelTrain'  │ 'image'  │  'pending'  │
└─────────┴────────────┴────────────────────────────────────────────────────────────┴─────────┴───────────────┴──────────┴─────────────┘
[root@VM-0-5-centos pipcook]# pipcook pipeline list
no pipeline is created.
[root@VM-0-5-centos pipcook]# clear
[root@VM-0-5-centos pipcook]# pipcook run ./component-group.json --tuna
✔ plugin @pipcook/plugins-object-detection-pascalvoc-data-collect@1.1.0 has already been installed
✔ plugin @pipcook/plugins-coco-data-access@1.1.2 has already been installed
⠏ fetching package info @pipcook/plugins-detectron-fasterrcnn-model-define

这个是因为网络原因导致有时候从 NPM 获取包信息导致的,可以尝试 pipcook plugin install @pipcook/plugins-detectron-fasterrcnn-model-define 进行单独安装。

另外这个安装包本身也比较大,可以在安装命令加一个 --tuna 参数用于加速 Python 依赖在国内的安装。

commented

已经把安装包本地build本地化了,安装的时候选择本地路径,也是一样需要从网络获取的吗?

安装包本地build本地化了

这个操作的具体操作描述一下?

安装的时候选择本地路径,也是一样需要从网络获取的吗

fetching package 执行的是获取元信息的操作,我们现在是需要每次安装和执行前获取最新的包元信息来做一些判断。

commented
git clone https://github.com/imgcook/pipcook-plugin-detectron-model-train.git pipcook-plugin-detectron-model-train
cd pipcook-plugin-detectron-model-train
npm install
npm run build
cd ..
pipcook plugin install ./pipcook-plugin-detectron-model-train
commented

如上操作应该是从本地安装的方式,但是一样会卡住

那这个需要 @FeelyChau 来看下具体怎么操作了,我在我本地机器上使用 pipcook plugin install @pipcook/plugins-detectron-fasterrcnn-model-define --tuna 貌似可以安装成功。

这个可能是 detectron 安装需要编译一些东西,需要等待一段时间。

commented

安装插件时卡住,经过定位是卡在pipcook/packages/sdk/base.ts

traceEvent(traceId: string, eventCallback: EventCallback): Promise<void> {
    return new Promise((resolve, reject) => {
      // TODO(feely): listen all event and transfer out
      listen(`${this.route}/event/${traceId}`, undefined, {
        'log': (e: MessageEvent) => {
          const eventObj = JSON.parse(e.data) as LogEvent;
          if (typeof eventCallback === 'function') {
            eventCallback('log', eventObj);
          }
        },
        'job_status': (e: MessageEvent) => {
          const eventObj = JSON.parse(e.data) as JobStatusChangeEvent;
          if (typeof eventCallback === 'function') {
            eventCallback('job_status', eventObj);
          }
        },
        'error': (e: MessageEvent) => {
          reject(new Error(e.data));
        },
        'close': () => {
          resolve();
        }
      });
    });
}

无法进入close()回调,导致pipcook/packages/cli/src/service/plugin.ts内部

export async function installFromRemote(uriOrName: string, opts: any): Promise<PluginResp> {
  const client = initClient(opts.hostIp, opts.port);
  logger.start(`fetching package info ${uriOrName}`);
  const resp = await client.plugin.createByName(uriOrName, opts.tuna ? tunaMirrorURI : undefined);
  if (resp.status === PluginStatus.INSTALLED) {
    logger.success(`plugin ${resp.name}@${resp.version} has already been installed`);
    return resp;
  }
  if (resp.traceId) {
    await traceInstallEvent(resp, opts); // 此处一直无法执行完毕
  }
  const plugin = await client.plugin.get(resp.id);
  if (plugin.status !== PluginStatus.INSTALLED) {
    throw new TypeError(`Plugin ${plugin.name} install failed`);
  }
  logger.success(`install ${resp.name}@${resp.version} successfully`);
  return plugin;
}

请问上述问题可能是什么导致的?本地环境node 14.15.1,python版本3.7.6,npm走的淘宝源

commented

是安装哪个插件?

$ pipcook plugin list

看看

commented

是安装哪个插件?

$ pipcook plugin list

看看

plugins-detectron-model-train

commented

是安装哪个插件?

$ pipcook plugin list

看看

plugins-detectron-model-train

安装的命令也发一下吧,还有当前的插件状态: pipcook plugin list,还有安装的日志没有吗?从前几天发的插件状态来看,有几个插件安装状态一直是 pending,没有真正运行安装。

commented

是安装哪个插件?

$ pipcook plugin list

看看

plugins-detectron-model-train

安装的命令也发一下吧,还有当前的插件状态: pipcook plugin list,还有安装的日志没有吗?从前几天发的插件状态来看,有几个插件安装状态一直是 pending,没有真正运行安装。

# pipcook plugin list
┌─────────┬────────────┬──────────────────────────────────────────────────────┬─────────┬───────────────┬──────────┬──────────────┐
│ (index) │     id     │                         name                         │ version │   category    │ datatype │    status    │
├─────────┼────────────┼──────────────────────────────────────────────────────┼─────────┼───────────────┼──────────┼──────────────┤
│    0    │ '3d92ytze' │ '@pipcook/plugins-detectron-fasterrcnn-model-define' │ '1.1.1' │ 'modelDefine' │ 'image'  │ 'installed'  │
│    1    │ 'zt6jrfc1' │       '@pipcook/plugins-detectron-model-train'       │ '1.1.1' │ 'modelTrain'  │ 'image'  │ 'installing' │
└─────────┴────────────┴──────────────────────────────────────────────────────┴─────────┴───────────────┴──────────┴──────────────┘

安装的命令

pipcook plugin install @pipcook/plugins-detectron-model-train --tuna

⠇ fetching package info @pipcook/plugins-detectron-model-train
ℹ PYTHON: Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
ℹ PYTHON: Collecting git+https://gitee.com/pipcook/detectron2.git (from -r /root/.pipcook/plugins/conda_envs/@pipcook/plugins-detectron-model-train@1.1.1/requirements.txt (line 6))
ℹ PYTHON:   Cloning https://gitee.com/pipcook/detectron2.git to /tmp/pip-req-build-ju_g5y6l
⚠ NODE: npm WARN plugins@1.0.0 No description
⚠ NODE: npm WARN plugins@1.0.0 No repository field.
⚠ NODE:
ℹ NODE: + @pipcook/plugins-detectron-model-train@1.1.1
ℹ NODE: added 2 packages and audited 135 packages in 4.209s
ℹ NODE:
ℹ NODE: 2 packages are looking for funding
ℹ NODE:   run `npm fund` for details
ℹ NODE:
ℹ NODE: found 0 vulnerabilities
ℹ NODE:

然后就是一直无返回,从源码定位就是上面的描述方法没有进行下一步

commented

知道原因了,线上的版本有一个 bug,在 beta 版本中已经修复,可以升级下 daemon:

$ pipcook init beta --tuna
commented

知道原因了,线上的版本有一个 bug,在 beta 版本中已经修复,可以升级下 daemon:

$ pipcook init beta --tuna

这个可行,插件已经正常安装了,感谢~

不过接下来的模型训练过程报了一个错,信息如下:

ℹ [job] fails
✖ job is failed: costa runtime is destroyed(SIGKILL)

然后训练中断~

commented

补充一个插件信息

# pipcook plugin list
┌─────────┬────────────┬────────────────────────────────────────────────────────────┬─────────┬─────────────────┬──────────┬─────────────┐
│ (index) │     id     │                            name                            │ version │    category     │ datatype │   status    │
├─────────┼────────────┼────────────────────────────────────────────────────────────┼─────────┼─────────────────┼──────────┼─────────────┤
│    0    │ 'fk8fpcc8' │          '@pipcook/plugins-detectron-model-train'          │ '1.1.1' │  'modelTrain'   │ 'image'  │ 'installed' │
│    1    │ '11piitkq' │ '@pipcook/plugins-object-detection-pascalvoc-data-collect' │ '1.1.0' │  'dataCollect'  │ 'image'  │ 'installed' │
│    2    │ 'aqbx4cza' │            '@pipcook/plugins-coco-data-access'             │ '1.1.2' │  'dataAccess'   │ 'image'  │ 'installed' │
│    3    │ '0blapc9m' │        '@pipcook/plugins-detectron-model-evaluate'         │ '1.1.3' │ 'modelEvaluate' │ 'image'  │ 'installed' │
│    4    │ 'o9jjwh4g' │    '@pipcook/plugins-detectron-fasterrcnn-model-define'    │ '1.1.1' │  'modelDefine'  │ 'image'  │ 'installed' │
└─────────┴────────────┴────────────────────────────────────────────────────────────┴─────────┴─────────────────┴──────────┴─────────────┘
commented

确认下内存和显存?看起来是 OOM 了。

commented

用的是阿里云的GPU实例

实例规格:ecs.gn6i-c4g1.xlarge
vCPU: 4
内存(GiB):15
GPU:T4
GPU显存(GB): 16

commented

错误信息贴长一点吧,现在看不出具体哪个插件出问题了。

commented

从开始到结束的信息如下,有点长~

# pipcook run config.json --tuna
✔ plugin @pipcook/plugins-object-detection-pascalvoc-data-collect@1.1.0 has already been installed
✔ plugin @pipcook/plugins-coco-data-access@1.1.2 has already been installed
✔ plugin @pipcook/plugins-detectron-fasterrcnn-model-define@1.1.1 has already been installed
✔ plugin @pipcook/plugins-detectron-model-train@1.1.1 has already been installed
✔ plugin @pipcook/plugins-detectron-model-evaluate@1.1.3 has already been installed
ℹ start to create pipeline
✔ pipeline is created: wa6ath9k, installing
ℹ plugin @pipcook/plugins-object-detection-pascalvoc-data-collect@1.1.0 already installed
ℹ plugin @pipcook/plugins-coco-data-access@1.1.2 already installed
ℹ plugin @pipcook/plugins-detectron-fasterrcnn-model-define@1.1.1 already installed
ℹ plugin @pipcook/plugins-detectron-model-train@1.1.1 already installed
ℹ plugin @pipcook/plugins-detectron-model-evaluate@1.1.3 already installed
✔ pipeline installed successfully, start to run job
✔ job is created: njcpmzoz, running
ℹ [job] running
ℹ [job] running dataCollect start
ℹ start loading plugin @pipcook/plugins-object-detection-pascalvoc-data-collect
ℹ downloading dataset ...
ℹ unzip and collecting data...
ℹ [job] running dataCollect end
ℹ [job] running dataAccess start
ℹ start loading plugin @pipcook/plugins-coco-data-access
ℹ create a result "yqhgjq2v" for plugin "@pipcook/plugins-coco-data-access@1.1.2"
ℹ [job] running dataAccess end
ℹ [job] running modelDefine start
ℹ start loading plugin @pipcook/plugins-detectron-fasterrcnn-model-define
ℹ WARNING [11/27 11:51:23 d2.config.compat]: Config '/root/.pipcook/plugins/node_modules/@pipcook/plugins-detectron-fasterrcnn-model-define/dist/config/faster_rcnn_R_50_C4_3x.yaml' has no VERSION. Assuming it to be compatible with latest v2.
ℹ create a result "oma52tkh" for plugin "@pipcook/plugins-detectron-fasterrcnn-model-define@1.1.1"
ℹ [job] running modelDefine end
ℹ [job] running modelTrain start
ℹ start loading plugin @pipcook/plugins-detectron-model-train
ℹ [11/27 11:51:23 d2.engine.defaults]: Model:
ℹ GeneralizedRCNN(
ℹ   (backbone): ResNet(
ℹ     (stem): BasicStem(
ℹ       (conv1): Conv2d(
ℹ         3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False
ℹ         (norm): FrozenBatchNorm2d(num_features=64, eps=1e-05)
ℹ       )
ℹ     )
ℹ     (res2): Sequential(
ℹ       (0): BottleneckBlock(
ℹ         (shortcut): Conv2d(
ℹ           64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False
ℹ           (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
ℹ         )
ℹ         (conv1): Conv2d(
ℹ           64, 64, kernel_size=(1, 1), stride=(1, 1), bias=False
ℹ           (norm): FrozenBatchNorm2d(num_features=64, eps=1e-05)
ℹ         )
ℹ         (conv2): Conv2d(
ℹ           64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
ℹ           (norm): FrozenBatchNorm2d(num_features=64, eps=1e-05)
ℹ         )
ℹ         (conv3): Conv2d(
ℹ           64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False
ℹ           (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
ℹ         )
ℹ       )
ℹ       (1): BottleneckBlock(
ℹ         (conv1): Conv2d(
ℹ           256, 64, kernel_size=(1, 1), stride=(1, 1), bias=False
ℹ           (norm): FrozenBatchNorm2d(num_features=64, eps=1e-05)
ℹ         )
ℹ         (conv2): Conv2d(
ℹ           64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
ℹ           (norm): FrozenBatchNorm2d(num_features=64, eps=1e-05)
ℹ         )
ℹ         (conv3): Conv2d(
ℹ           64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False
ℹ           (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
ℹ         )
ℹ       )
ℹ       (2): BottleneckBlock(
ℹ         (conv1): Conv2d(
ℹ           256, 64, kernel_size=(1, 1), stride=(1, 1), bias=False
ℹ           (norm): FrozenBatchNorm2d(num_features=64, eps=1e-05)
ℹ         )
ℹ         (conv2): Conv2d(
ℹ           64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
ℹ           (norm): FrozenBatchNorm2d(num_features=64, eps=1e-05)
ℹ         )
ℹ         (conv3): Conv2d(
ℹ           64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False
ℹ           (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
ℹ         )
ℹ       )
ℹ     )
ℹ     (res3): Sequential(
ℹ       (0): BottleneckBlock(
ℹ         (shortcut): Conv2d(
ℹ           256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False
ℹ           (norm): FrozenBatchNorm2d(num_features=512, eps=1e-05)
ℹ         )
ℹ         (conv1): Conv2d(
ℹ           256, 128, kernel_size=(1, 1), stride=(2, 2), bias=False
ℹ           (norm): FrozenBatchNorm2d(num_features=128, eps=1e-05)
ℹ         )
ℹ         (conv2): Conv2d(
ℹ           128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
ℹ           (norm): FrozenBatchNorm2d(num_features=128, eps=1e-05)
ℹ         )
ℹ         (conv3): Conv2d(
ℹ           128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False
ℹ           (norm): FrozenBatchNorm2d(num_features=512, eps=1e-05)
ℹ         )
ℹ       )
ℹ       (1): BottleneckBlock(
ℹ         (conv1): Conv2d(
ℹ           512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False
ℹ           (norm): FrozenBatchNorm2d(num_features=128, eps=1e-05)
ℹ         )
ℹ         (conv2): Conv2d(
ℹ           128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
ℹ           (norm): FrozenBatchNorm2d(num_features=128, eps=1e-05)
ℹ         )
ℹ         (conv3): Conv2d(
ℹ           128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False
ℹ           (norm): FrozenBatchNorm2d(num_features=512, eps=1e-05)
ℹ         )
ℹ       )
ℹ       (2): BottleneckBlock(
ℹ         (conv1): Conv2d(
ℹ           512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False
ℹ           (norm): FrozenBatchNorm2d(num_features=128, eps=1e-05)
ℹ         )
ℹ         (conv2): Conv2d(
ℹ           128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
ℹ           (norm): FrozenBatchNorm2d(num_features=128, eps=1e-05)
ℹ         )
ℹ         (conv3): Conv2d(
ℹ           128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False
ℹ           (norm): FrozenBatchNorm2d(num_features=512, eps=1e-05)
ℹ         )
ℹ       )
ℹ       (3): BottleneckBlock(
ℹ         (conv1): Conv2d(
ℹ           512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False
ℹ           (norm): FrozenBatchNorm2d(num_features=128, eps=1e-05)
ℹ         )
ℹ         (conv2): Conv2d(
ℹ           128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
ℹ           (norm): FrozenBatchNorm2d(num_features=128, eps=1e-05)
ℹ         )
ℹ         (conv3): Conv2d(
ℹ           128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False
ℹ           (norm): FrozenBatchNorm2d(num_features=512, eps=1e-05)
ℹ         )
ℹ       )
ℹ     )
ℹ     (res4): Sequential(
ℹ       (0): BottleneckBlock(
ℹ         (shortcut): Conv2d(
ℹ           512, 1024, kernel_size=(1, 1), stride=(2, 2), bias=False
ℹ           (norm): FrozenBatchNorm2d(num_features=1024, eps=1e-05)
ℹ         )
ℹ         (conv1): Conv2d(
ℹ           512, 256, kernel_size=(1, 1), stride=(2, 2), bias=False
ℹ           (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
ℹ         )
ℹ         (conv2): Conv2d(
ℹ           256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
ℹ           (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
ℹ         )
ℹ         (conv3): Conv2d(
ℹ           256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False
ℹ           (norm): FrozenBatchNorm2d(num_features=1024, eps=1e-05)
ℹ         )
ℹ       )
ℹ       (1): BottleneckBlock(
ℹ         (conv1): Conv2d(
ℹ           1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False
ℹ           (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
ℹ         )
ℹ         (conv2): Conv2d(
ℹ           256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
ℹ           (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
ℹ         )
ℹ         (conv3): Conv2d(
ℹ           256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False
ℹ           (norm): FrozenBatchNorm2d(num_features=1024, eps=1e-05)
ℹ         )
ℹ       )
ℹ       (2): BottleneckBlock(
ℹ         (conv1): Conv2d(
ℹ           1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False
ℹ           (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
ℹ         )
ℹ         (conv2): Conv2d(
ℹ           256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
ℹ           (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
ℹ         )
ℹ         (conv3): Conv2d(
ℹ           256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False
ℹ           (norm): FrozenBatchNorm2d(num_features=1024, eps=1e-05)
ℹ         )
ℹ       )
ℹ       (3): BottleneckBlock(
ℹ         (conv1): Conv2d(
ℹ           1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False
ℹ           (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
ℹ         )
ℹ         (conv2): Conv2d(
ℹ           256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
ℹ           (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
ℹ         )
ℹ         (conv3): Conv2d(
ℹ           256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False
ℹ           (norm): FrozenBatchNorm2d(num_features=1024, eps=1e-05)
ℹ         )
ℹ       )
ℹ       (4): BottleneckBlock(
ℹ         (conv1): Conv2d(
ℹ           1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False
ℹ           (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
ℹ         )
ℹ         (conv2): Conv2d(
ℹ           256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
ℹ           (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
ℹ         )
ℹ         (conv3): Conv2d(
ℹ           256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False
ℹ           (norm): FrozenBatchNorm2d(num_features=1024, eps=1e-05)
ℹ         )
ℹ       )
ℹ       (5): BottleneckBlock(
ℹ         (conv1): Conv2d(
ℹ           1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False
ℹ           (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
ℹ         )
ℹ         (conv2): Conv2d(
ℹ           256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
ℹ           (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
ℹ         )
ℹ         (conv3): Conv2d(
ℹ           256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False
ℹ           (norm): FrozenBatchNorm2d(num_features=1024, eps=1e-05)
ℹ         )
ℹ       )
ℹ     )
ℹ   )
ℹ   (proposal_generator): RPN(
ℹ     (rpn_head): StandardRPNHead(
ℹ       (conv): Conv2d(1024, 1024, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
ℹ       (objectness_logits): Conv2d(1024, 15, kernel_size=(1, 1), stride=(1, 1))
ℹ       (anchor_deltas): Conv2d(1024, 60, kernel_size=(1, 1), stride=(1, 1))
ℹ     )
ℹ     (anchor_generator): DefaultAnchorGenerator(
ℹ       (cell_anchors): BufferList()
ℹ     )
ℹ   )
ℹ   (roi_heads): Res5ROIHeads(
ℹ     (pooler): ROIPooler(
ℹ       (level_poolers): ModuleList(
ℹ         (0): ROIAlign(output_size=(14, 14), spatial_scale=0.0625, sampling_ratio=0, aligned=True)
ℹ       )
ℹ     )
ℹ     (res5): Sequential(
ℹ       (0): BottleneckBlock(
ℹ         (shortcut): Conv2d(
ℹ           1024, 2048, kernel_size=(1, 1), stride=(2, 2), bias=False
ℹ           (norm): FrozenBatchNorm2d(num_features=2048, eps=1e-05)
ℹ         )
ℹ         (conv1): Conv2d(
ℹ           1024, 512, kernel_size=(1, 1), stride=(2, 2), bias=False
ℹ           (norm): FrozenBatchNorm2d(num_features=512, eps=1e-05)
ℹ         )
ℹ         (conv2): Conv2d(
ℹ           512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
ℹ           (norm): FrozenBatchNorm2d(num_features=512, eps=1e-05)
ℹ         )
ℹ         (conv3): Conv2d(
ℹ           512, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False
ℹ           (norm): FrozenBatchNorm2d(num_features=2048, eps=1e-05)
ℹ         )
ℹ       )
ℹ       (1): BottleneckBlock(
ℹ         (conv1): Conv2d(
ℹ           2048, 512, kernel_size=(1, 1), stride=(1, 1), bias=False
ℹ           (norm): FrozenBatchNorm2d(num_features=512, eps=1e-05)
ℹ         )
ℹ         (conv2): Conv2d(
ℹ           512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
ℹ           (norm): FrozenBatchNorm2d(num_features=512, eps=1e-05)
ℹ         )
ℹ         (conv3): Conv2d(
ℹ           512, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False
ℹ           (norm): FrozenBatchNorm2d(num_features=2048, eps=1e-05)
ℹ         )
ℹ       )
ℹ       (2): BottleneckBlock(
ℹ         (conv1): Conv2d(
ℹ           2048, 512, kernel_size=(1, 1), stride=(1, 1), bias=False
ℹ           (norm): FrozenBatchNorm2d(num_features=512, eps=1e-05)
ℹ         )
ℹ         (conv2): Conv2d(
ℹ           512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
ℹ           (norm): FrozenBatchNorm2d(num_features=512, eps=1e-05)
ℹ         )
ℹ         (conv3): Conv2d(
ℹ           512, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False
ℹ           (norm): FrozenBatchNorm2d(num_features=2048, eps=1e-05)
ℹ         )
ℹ       )
ℹ     )
ℹ     (box_predictor): FastRCNNOutputLayers(
ℹ       (cls_score): Linear(in_features=2048, out_features=15, bias=True)
ℹ       (bbox_pred): Linear(in_features=2048, out_features=56, bias=True)
ℹ     )
ℹ   )
ℹ )
ℹ WARNING [11/27 11:51:23 d2.data.datasets.coco]:
ℹ Category ids in annotations are not in [1, #categories]! We'll apply a mapping for you.
ℹ run pipeline from pipeline wa6ath9k
ℹ [11/27 11:51:23 d2.data.datasets.coco]: Loaded 3004 images in COCO format from /root/.pipcook/datasets/@pipcook/plugins-object-detection-pascalvoc-data-collect@1.1.0/train/annotation.json
ℹ [11/27 11:51:23 d2.data.build]: Removed 0 images with no usable annotations. 3004 images left.
ℹ [11/27 11:51:24 d2.data.build]: Distribution of instances among all 14 categories:
ℹ |   category   | #instances   |  category   | #instances   |  category  | #instances   |
ℹ |:------------:|:-------------|:-----------:|:-------------|:----------:|:-------------|
ℹ |     tags     | 3114         |    input    | 2756         |  buttons   | 3075         |
ℹ | imagesUpload | 316          |    links    | 3055         |   select   | 2861         |
ℹ |    radio     | 317          |  textarea   | 292          | datePicker | 316          |
ℹ |     rate     | 292          | rangePicker | 315          |   switch   | 303          |
ℹ |  timePicker  | 293          |  checkbox   | 293          |            |              |
ℹ |    total     | 17598        |             |              |            |              |
ℹ [11/27 11:51:24 d2.data.common]: Serializing 3004 elements to byte tensors and concatenating them all ...
ℹ [11/27 11:51:24 d2.data.common]: Serialized dataset takes 1.38 MiB
ℹ [11/27 11:51:24 d2.data.detection_utils]: Augmentations used in training: [ResizeShortestEdge(short_edge_length=(640, 672, 704, 736, 768, 800), max_size=1333, sample_style='choice'), RandomFlip()]
ℹ [11/27 11:51:24 d2.data.build]: Using training sampler TrainingSampler
ℹ [11/27 11:51:24 d2.checkpoint.c2_model_loading]: Remapping C2 weights ......
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res2.0.conv1.norm.bias                      loaded from res2_0_branch2a_bn_beta           of shape (64,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res2.0.conv1.norm.running_mean              loaded from res2_0_branch2a_bn_running_mean   of shape (64,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res2.0.conv1.norm.running_var               loaded from res2_0_branch2a_bn_running_var    of shape (64,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res2.0.conv1.norm.weight                    loaded from res2_0_branch2a_bn_gamma          of shape (64,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res2.0.conv1.weight                         loaded from res2_0_branch2a_w                 of shape (64, 64, 1, 1)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res2.0.conv2.norm.bias                      loaded from res2_0_branch2b_bn_beta           of shape (64,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res2.0.conv2.norm.running_mean              loaded from res2_0_branch2b_bn_running_mean   of shape (64,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res2.0.conv2.norm.running_var               loaded from res2_0_branch2b_bn_running_var    of shape (64,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res2.0.conv2.norm.weight                    loaded from res2_0_branch2b_bn_gamma          of shape (64,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res2.0.conv2.weight                         loaded from res2_0_branch2b_w                 of shape (64, 64, 3, 3)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res2.0.conv3.norm.bias                      loaded from res2_0_branch2c_bn_beta           of shape (256,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res2.0.conv3.norm.running_mean              loaded from res2_0_branch2c_bn_running_mean   of shape (256,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res2.0.conv3.norm.running_var               loaded from res2_0_branch2c_bn_running_var    of shape (256,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res2.0.conv3.norm.weight                    loaded from res2_0_branch2c_bn_gamma          of shape (256,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res2.0.conv3.weight                         loaded from res2_0_branch2c_w                 of shape (256, 64, 1, 1)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res2.0.shortcut.norm.bias                   loaded from res2_0_branch1_bn_beta            of shape (256,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res2.0.shortcut.norm.running_mean           loaded from res2_0_branch1_bn_running_mean    of shape (256,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res2.0.shortcut.norm.running_var            loaded from res2_0_branch1_bn_running_var     of shape (256,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res2.0.shortcut.norm.weight                 loaded from res2_0_branch1_bn_gamma           of shape (256,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res2.0.shortcut.weight                      loaded from res2_0_branch1_w                  of shape (256, 64, 1, 1)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res2.1.conv1.norm.bias                      loaded from res2_1_branch2a_bn_beta           of shape (64,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res2.1.conv1.norm.running_mean              loaded from res2_1_branch2a_bn_running_mean   of shape (64,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res2.1.conv1.norm.running_var               loaded from res2_1_branch2a_bn_running_var    of shape (64,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res2.1.conv1.norm.weight                    loaded from res2_1_branch2a_bn_gamma          of shape (64,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res2.1.conv1.weight                         loaded from res2_1_branch2a_w                 of shape (64, 256, 1, 1)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res2.1.conv2.norm.bias                      loaded from res2_1_branch2b_bn_beta           of shape (64,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res2.1.conv2.norm.running_mean              loaded from res2_1_branch2b_bn_running_mean   of shape (64,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res2.1.conv2.norm.running_var               loaded from res2_1_branch2b_bn_running_var    of shape (64,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res2.1.conv2.norm.weight                    loaded from res2_1_branch2b_bn_gamma          of shape (64,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res2.1.conv2.weight                         loaded from res2_1_branch2b_w                 of shape (64, 64, 3, 3)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res2.1.conv3.norm.bias                      loaded from res2_1_branch2c_bn_beta           of shape (256,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res2.1.conv3.norm.running_mean              loaded from res2_1_branch2c_bn_running_mean   of shape (256,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res2.1.conv3.norm.running_var               loaded from res2_1_branch2c_bn_running_var    of shape (256,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res2.1.conv3.norm.weight                    loaded from res2_1_branch2c_bn_gamma          of shape (256,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res2.1.conv3.weight                         loaded from res2_1_branch2c_w                 of shape (256, 64, 1, 1)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res2.2.conv1.norm.bias                      loaded from res2_2_branch2a_bn_beta           of shape (64,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res2.2.conv1.norm.running_mean              loaded from res2_2_branch2a_bn_running_mean   of shape (64,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res2.2.conv1.norm.running_var               loaded from res2_2_branch2a_bn_running_var    of shape (64,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res2.2.conv1.norm.weight                    loaded from res2_2_branch2a_bn_gamma          of shape (64,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res2.2.conv1.weight                         loaded from res2_2_branch2a_w                 of shape (64, 256, 1, 1)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res2.2.conv2.norm.bias                      loaded from res2_2_branch2b_bn_beta           of shape (64,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res2.2.conv2.norm.running_mean              loaded from res2_2_branch2b_bn_running_mean   of shape (64,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res2.2.conv2.norm.running_var               loaded from res2_2_branch2b_bn_running_var    of shape (64,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res2.2.conv2.norm.weight                    loaded from res2_2_branch2b_bn_gamma          of shape (64,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res2.2.conv2.weight                         loaded from res2_2_branch2b_w                 of shape (64, 64, 3, 3)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res2.2.conv3.norm.bias                      loaded from res2_2_branch2c_bn_beta           of shape (256,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res2.2.conv3.norm.running_mean              loaded from res2_2_branch2c_bn_running_mean   of shape (256,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res2.2.conv3.norm.running_var               loaded from res2_2_branch2c_bn_running_var    of shape (256,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res2.2.conv3.norm.weight                    loaded from res2_2_branch2c_bn_gamma          of shape (256,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res2.2.conv3.weight                         loaded from res2_2_branch2c_w                 of shape (256, 64, 1, 1)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res3.0.conv1.norm.bias                      loaded from res3_0_branch2a_bn_beta           of shape (128,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res3.0.conv1.norm.running_mean              loaded from res3_0_branch2a_bn_running_mean   of shape (128,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res3.0.conv1.norm.running_var               loaded from res3_0_branch2a_bn_running_var    of shape (128,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res3.0.conv1.norm.weight                    loaded from res3_0_branch2a_bn_gamma          of shape (128,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res3.0.conv1.weight                         loaded from res3_0_branch2a_w                 of shape (128, 256, 1, 1)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res3.0.conv2.norm.bias                      loaded from res3_0_branch2b_bn_beta           of shape (128,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res3.0.conv2.norm.running_mean              loaded from res3_0_branch2b_bn_running_mean   of shape (128,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res3.0.conv2.norm.running_var               loaded from res3_0_branch2b_bn_running_var    of shape (128,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res3.0.conv2.norm.weight                    loaded from res3_0_branch2b_bn_gamma          of shape (128,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res3.0.conv2.weight                         loaded from res3_0_branch2b_w                 of shape (128, 128, 3, 3)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res3.0.conv3.norm.bias                      loaded from res3_0_branch2c_bn_beta           of shape (512,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res3.0.conv3.norm.running_mean              loaded from res3_0_branch2c_bn_running_mean   of shape (512,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res3.0.conv3.norm.running_var               loaded from res3_0_branch2c_bn_running_var    of shape (512,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res3.0.conv3.norm.weight                    loaded from res3_0_branch2c_bn_gamma          of shape (512,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res3.0.conv3.weight                         loaded from res3_0_branch2c_w                 of shape (512, 128, 1, 1)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res3.0.shortcut.norm.bias                   loaded from res3_0_branch1_bn_beta            of shape (512,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res3.0.shortcut.norm.running_mean           loaded from res3_0_branch1_bn_running_mean    of shape (512,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res3.0.shortcut.norm.running_var            loaded from res3_0_branch1_bn_running_var     of shape (512,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res3.0.shortcut.norm.weight                 loaded from res3_0_branch1_bn_gamma           of shape (512,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res3.0.shortcut.weight                      loaded from res3_0_branch1_w                  of shape (512, 256, 1, 1)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res3.1.conv1.norm.bias                      loaded from res3_1_branch2a_bn_beta           of shape (128,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res3.1.conv1.norm.running_mean              loaded from res3_1_branch2a_bn_running_mean   of shape (128,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res3.1.conv1.norm.running_var               loaded from res3_1_branch2a_bn_running_var    of shape (128,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res3.1.conv1.norm.weight                    loaded from res3_1_branch2a_bn_gamma          of shape (128,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res3.1.conv1.weight                         loaded from res3_1_branch2a_w                 of shape (128, 512, 1, 1)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res3.1.conv2.norm.bias                      loaded from res3_1_branch2b_bn_beta           of shape (128,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res3.1.conv2.norm.running_mean              loaded from res3_1_branch2b_bn_running_mean   of shape (128,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res3.1.conv2.norm.running_var               loaded from res3_1_branch2b_bn_running_var    of shape (128,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res3.1.conv2.norm.weight                    loaded from res3_1_branch2b_bn_gamma          of shape (128,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res3.1.conv2.weight                         loaded from res3_1_branch2b_w                 of shape (128, 128, 3, 3)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res3.1.conv3.norm.bias                      loaded from res3_1_branch2c_bn_beta           of shape (512,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res3.1.conv3.norm.running_mean              loaded from res3_1_branch2c_bn_running_mean   of shape (512,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res3.1.conv3.norm.running_var               loaded from res3_1_branch2c_bn_running_var    of shape (512,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res3.1.conv3.norm.weight                    loaded from res3_1_branch2c_bn_gamma          of shape (512,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res3.1.conv3.weight                         loaded from res3_1_branch2c_w                 of shape (512, 128, 1, 1)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res3.2.conv1.norm.bias                      loaded from res3_2_branch2a_bn_beta           of shape (128,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res3.2.conv1.norm.running_mean              loaded from res3_2_branch2a_bn_running_mean   of shape (128,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res3.2.conv1.norm.running_var               loaded from res3_2_branch2a_bn_running_var    of shape (128,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res3.2.conv1.norm.weight                    loaded from res3_2_branch2a_bn_gamma          of shape (128,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res3.2.conv1.weight                         loaded from res3_2_branch2a_w                 of shape (128, 512, 1, 1)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res3.2.conv2.norm.bias                      loaded from res3_2_branch2b_bn_beta           of shape (128,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res3.2.conv2.norm.running_mean              loaded from res3_2_branch2b_bn_running_mean   of shape (128,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res3.2.conv2.norm.running_var               loaded from res3_2_branch2b_bn_running_var    of shape (128,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res3.2.conv2.norm.weight                    loaded from res3_2_branch2b_bn_gamma          of shape (128,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res3.2.conv2.weight                         loaded from res3_2_branch2b_w                 of shape (128, 128, 3, 3)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res3.2.conv3.norm.bias                      loaded from res3_2_branch2c_bn_beta           of shape (512,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res3.2.conv3.norm.running_mean              loaded from res3_2_branch2c_bn_running_mean   of shape (512,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res3.2.conv3.norm.running_var               loaded from res3_2_branch2c_bn_running_var    of shape (512,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res3.2.conv3.norm.weight                    loaded from res3_2_branch2c_bn_gamma          of shape (512,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res3.2.conv3.weight                         loaded from res3_2_branch2c_w                 of shape (512, 128, 1, 1)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res3.3.conv1.norm.bias                      loaded from res3_3_branch2a_bn_beta           of shape (128,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res3.3.conv1.norm.running_mean              loaded from res3_3_branch2a_bn_running_mean   of shape (128,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res3.3.conv1.norm.running_var               loaded from res3_3_branch2a_bn_running_var    of shape (128,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res3.3.conv1.norm.weight                    loaded from res3_3_branch2a_bn_gamma          of shape (128,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res3.3.conv1.weight                         loaded from res3_3_branch2a_w                 of shape (128, 512, 1, 1)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res3.3.conv2.norm.bias                      loaded from res3_3_branch2b_bn_beta           of shape (128,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res3.3.conv2.norm.running_mean              loaded from res3_3_branch2b_bn_running_mean   of shape (128,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res3.3.conv2.norm.running_var               loaded from res3_3_branch2b_bn_running_var    of shape (128,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res3.3.conv2.norm.weight                    loaded from res3_3_branch2b_bn_gamma          of shape (128,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res3.3.conv2.weight                         loaded from res3_3_branch2b_w                 of shape (128, 128, 3, 3)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res3.3.conv3.norm.bias                      loaded from res3_3_branch2c_bn_beta           of shape (512,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res3.3.conv3.norm.running_mean              loaded from res3_3_branch2c_bn_running_mean   of shape (512,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res3.3.conv3.norm.running_var               loaded from res3_3_branch2c_bn_running_var    of shape (512,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res3.3.conv3.norm.weight                    loaded from res3_3_branch2c_bn_gamma          of shape (512,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res3.3.conv3.weight                         loaded from res3_3_branch2c_w                 of shape (512, 128, 1, 1)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.0.conv1.norm.bias                      loaded from res4_0_branch2a_bn_beta           of shape (256,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.0.conv1.norm.running_mean              loaded from res4_0_branch2a_bn_running_mean   of shape (256,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.0.conv1.norm.running_var               loaded from res4_0_branch2a_bn_running_var    of shape (256,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.0.conv1.norm.weight                    loaded from res4_0_branch2a_bn_gamma          of shape (256,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.0.conv1.weight                         loaded from res4_0_branch2a_w                 of shape (256, 512, 1, 1)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.0.conv2.norm.bias                      loaded from res4_0_branch2b_bn_beta           of shape (256,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.0.conv2.norm.running_mean              loaded from res4_0_branch2b_bn_running_mean   of shape (256,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.0.conv2.norm.running_var               loaded from res4_0_branch2b_bn_running_var    of shape (256,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.0.conv2.norm.weight                    loaded from res4_0_branch2b_bn_gamma          of shape (256,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.0.conv2.weight                         loaded from res4_0_branch2b_w                 of shape (256, 256, 3, 3)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.0.conv3.norm.bias                      loaded from res4_0_branch2c_bn_beta           of shape (1024,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.0.conv3.norm.running_mean              loaded from res4_0_branch2c_bn_running_mean   of shape (1024,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.0.conv3.norm.running_var               loaded from res4_0_branch2c_bn_running_var    of shape (1024,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.0.conv3.norm.weight                    loaded from res4_0_branch2c_bn_gamma          of shape (1024,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.0.conv3.weight                         loaded from res4_0_branch2c_w                 of shape (1024, 256, 1, 1)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.0.shortcut.norm.bias                   loaded from res4_0_branch1_bn_beta            of shape (1024,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.0.shortcut.norm.running_mean           loaded from res4_0_branch1_bn_running_mean    of shape (1024,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.0.shortcut.norm.running_var            loaded from res4_0_branch1_bn_running_var     of shape (1024,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.0.shortcut.norm.weight                 loaded from res4_0_branch1_bn_gamma           of shape (1024,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.0.shortcut.weight                      loaded from res4_0_branch1_w                  of shape (1024, 512, 1, 1)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.1.conv1.norm.bias                      loaded from res4_1_branch2a_bn_beta           of shape (256,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.1.conv1.norm.running_mean              loaded from res4_1_branch2a_bn_running_mean   of shape (256,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.1.conv1.norm.running_var               loaded from res4_1_branch2a_bn_running_var    of shape (256,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.1.conv1.norm.weight                    loaded from res4_1_branch2a_bn_gamma          of shape (256,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.1.conv1.weight                         loaded from res4_1_branch2a_w                 of shape (256, 1024, 1, 1)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.1.conv2.norm.bias                      loaded from res4_1_branch2b_bn_beta           of shape (256,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.1.conv2.norm.running_mean              loaded from res4_1_branch2b_bn_running_mean   of shape (256,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.1.conv2.norm.running_var               loaded from res4_1_branch2b_bn_running_var    of shape (256,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.1.conv2.norm.weight                    loaded from res4_1_branch2b_bn_gamma          of shape (256,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.1.conv2.weight                         loaded from res4_1_branch2b_w                 of shape (256, 256, 3, 3)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.1.conv3.norm.bias                      loaded from res4_1_branch2c_bn_beta           of shape (1024,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.1.conv3.norm.running_mean              loaded from res4_1_branch2c_bn_running_mean   of shape (1024,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.1.conv3.norm.running_var               loaded from res4_1_branch2c_bn_running_var    of shape (1024,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.1.conv3.norm.weight                    loaded from res4_1_branch2c_bn_gamma          of shape (1024,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.1.conv3.weight                         loaded from res4_1_branch2c_w                 of shape (1024, 256, 1, 1)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.2.conv1.norm.bias                      loaded from res4_2_branch2a_bn_beta           of shape (256,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.2.conv1.norm.running_mean              loaded from res4_2_branch2a_bn_running_mean   of shape (256,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.2.conv1.norm.running_var               loaded from res4_2_branch2a_bn_running_var    of shape (256,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.2.conv1.norm.weight                    loaded from res4_2_branch2a_bn_gamma          of shape (256,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.2.conv1.weight                         loaded from res4_2_branch2a_w                 of shape (256, 1024, 1, 1)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.2.conv2.norm.bias                      loaded from res4_2_branch2b_bn_beta           of shape (256,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.2.conv2.norm.running_mean              loaded from res4_2_branch2b_bn_running_mean   of shape (256,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.2.conv2.norm.running_var               loaded from res4_2_branch2b_bn_running_var    of shape (256,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.2.conv2.norm.weight                    loaded from res4_2_branch2b_bn_gamma          of shape (256,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.2.conv2.weight                         loaded from res4_2_branch2b_w                 of shape (256, 256, 3, 3)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.2.conv3.norm.bias                      loaded from res4_2_branch2c_bn_beta           of shape (1024,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.2.conv3.norm.running_mean              loaded from res4_2_branch2c_bn_running_mean   of shape (1024,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.2.conv3.norm.running_var               loaded from res4_2_branch2c_bn_running_var    of shape (1024,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.2.conv3.norm.weight                    loaded from res4_2_branch2c_bn_gamma          of shape (1024,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.2.conv3.weight                         loaded from res4_2_branch2c_w                 of shape (1024, 256, 1, 1)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.3.conv1.norm.bias                      loaded from res4_3_branch2a_bn_beta           of shape (256,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.3.conv1.norm.running_mean              loaded from res4_3_branch2a_bn_running_mean   of shape (256,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.3.conv1.norm.running_var               loaded from res4_3_branch2a_bn_running_var    of shape (256,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.3.conv1.norm.weight                    loaded from res4_3_branch2a_bn_gamma          of shape (256,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.3.conv1.weight                         loaded from res4_3_branch2a_w                 of shape (256, 1024, 1, 1)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.3.conv2.norm.bias                      loaded from res4_3_branch2b_bn_beta           of shape (256,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.3.conv2.norm.running_mean              loaded from res4_3_branch2b_bn_running_mean   of shape (256,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.3.conv2.norm.running_var               loaded from res4_3_branch2b_bn_running_var    of shape (256,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.3.conv2.norm.weight                    loaded from res4_3_branch2b_bn_gamma          of shape (256,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.3.conv2.weight                         loaded from res4_3_branch2b_w                 of shape (256, 256, 3, 3)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.3.conv3.norm.bias                      loaded from res4_3_branch2c_bn_beta           of shape (1024,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.3.conv3.norm.running_mean              loaded from res4_3_branch2c_bn_running_mean   of shape (1024,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.3.conv3.norm.running_var               loaded from res4_3_branch2c_bn_running_var    of shape (1024,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.3.conv3.norm.weight                    loaded from res4_3_branch2c_bn_gamma          of shape (1024,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.3.conv3.weight                         loaded from res4_3_branch2c_w                 of shape (1024, 256, 1, 1)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.4.conv1.norm.bias                      loaded from res4_4_branch2a_bn_beta           of shape (256,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.4.conv1.norm.running_mean              loaded from res4_4_branch2a_bn_running_mean   of shape (256,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.4.conv1.norm.running_var               loaded from res4_4_branch2a_bn_running_var    of shape (256,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.4.conv1.norm.weight                    loaded from res4_4_branch2a_bn_gamma          of shape (256,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.4.conv1.weight                         loaded from res4_4_branch2a_w                 of shape (256, 1024, 1, 1)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.4.conv2.norm.bias                      loaded from res4_4_branch2b_bn_beta           of shape (256,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.4.conv2.norm.running_mean              loaded from res4_4_branch2b_bn_running_mean   of shape (256,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.4.conv2.norm.running_var               loaded from res4_4_branch2b_bn_running_var    of shape (256,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.4.conv2.norm.weight                    loaded from res4_4_branch2b_bn_gamma          of shape (256,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.4.conv2.weight                         loaded from res4_4_branch2b_w                 of shape (256, 256, 3, 3)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.4.conv3.norm.bias                      loaded from res4_4_branch2c_bn_beta           of shape (1024,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.4.conv3.norm.running_mean              loaded from res4_4_branch2c_bn_running_mean   of shape (1024,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.4.conv3.norm.running_var               loaded from res4_4_branch2c_bn_running_var    of shape (1024,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.4.conv3.norm.weight                    loaded from res4_4_branch2c_bn_gamma          of shape (1024,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.4.conv3.weight                         loaded from res4_4_branch2c_w                 of shape (1024, 256, 1, 1)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.5.conv1.norm.bias                      loaded from res4_5_branch2a_bn_beta           of shape (256,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.5.conv1.norm.running_mean              loaded from res4_5_branch2a_bn_running_mean   of shape (256,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.5.conv1.norm.running_var               loaded from res4_5_branch2a_bn_running_var    of shape (256,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.5.conv1.norm.weight                    loaded from res4_5_branch2a_bn_gamma          of shape (256,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.5.conv1.weight                         loaded from res4_5_branch2a_w                 of shape (256, 1024, 1, 1)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.5.conv2.norm.bias                      loaded from res4_5_branch2b_bn_beta           of shape (256,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.5.conv2.norm.running_mean              loaded from res4_5_branch2b_bn_running_mean   of shape (256,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.5.conv2.norm.running_var               loaded from res4_5_branch2b_bn_running_var    of shape (256,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.5.conv2.norm.weight                    loaded from res4_5_branch2b_bn_gamma          of shape (256,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.5.conv2.weight                         loaded from res4_5_branch2b_w                 of shape (256, 256, 3, 3)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.5.conv3.norm.bias                      loaded from res4_5_branch2c_bn_beta           of shape (1024,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.5.conv3.norm.running_mean              loaded from res4_5_branch2c_bn_running_mean   of shape (1024,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.5.conv3.norm.running_var               loaded from res4_5_branch2c_bn_running_var    of shape (1024,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.5.conv3.norm.weight                    loaded from res4_5_branch2c_bn_gamma          of shape (1024,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.res4.5.conv3.weight                         loaded from res4_5_branch2c_w                 of shape (1024, 256, 1, 1)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.stem.conv1.norm.bias                        loaded from res_conv1_bn_beta                 of shape (64,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.stem.conv1.norm.running_mean                loaded from res_conv1_bn_running_mean         of shape (64,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.stem.conv1.norm.running_var                 loaded from res_conv1_bn_running_var          of shape (64,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.stem.conv1.norm.weight                      loaded from res_conv1_bn_gamma                of shape (64,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: backbone.stem.conv1.weight                           loaded from conv1_w                           of shape (64, 3, 7, 7)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: roi_heads.res5.0.conv1.norm.bias                     loaded from res5_0_branch2a_bn_beta           of shape (512,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: roi_heads.res5.0.conv1.norm.running_mean             loaded from res5_0_branch2a_bn_running_mean   of shape (512,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: roi_heads.res5.0.conv1.norm.running_var              loaded from res5_0_branch2a_bn_running_var    of shape (512,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: roi_heads.res5.0.conv1.norm.weight                   loaded from res5_0_branch2a_bn_gamma          of shape (512,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: roi_heads.res5.0.conv1.weight                        loaded from res5_0_branch2a_w                 of shape (512, 1024, 1, 1)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: roi_heads.res5.0.conv2.norm.bias                     loaded from res5_0_branch2b_bn_beta           of shape (512,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: roi_heads.res5.0.conv2.norm.running_mean             loaded from res5_0_branch2b_bn_running_mean   of shape (512,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: roi_heads.res5.0.conv2.norm.running_var              loaded from res5_0_branch2b_bn_running_var    of shape (512,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: roi_heads.res5.0.conv2.norm.weight                   loaded from res5_0_branch2b_bn_gamma          of shape (512,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: roi_heads.res5.0.conv2.weight                        loaded from res5_0_branch2b_w                 of shape (512, 512, 3, 3)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: roi_heads.res5.0.conv3.norm.bias                     loaded from res5_0_branch2c_bn_beta           of shape (2048,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: roi_heads.res5.0.conv3.norm.running_mean             loaded from res5_0_branch2c_bn_running_mean   of shape (2048,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: roi_heads.res5.0.conv3.norm.running_var              loaded from res5_0_branch2c_bn_running_var    of shape (2048,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: roi_heads.res5.0.conv3.norm.weight                   loaded from res5_0_branch2c_bn_gamma          of shape (2048,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: roi_heads.res5.0.conv3.weight                        loaded from res5_0_branch2c_w                 of shape (2048, 512, 1, 1)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: roi_heads.res5.0.shortcut.norm.bias                  loaded from res5_0_branch1_bn_beta            of shape (2048,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: roi_heads.res5.0.shortcut.norm.running_mean          loaded from res5_0_branch1_bn_running_mean    of shape (2048,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: roi_heads.res5.0.shortcut.norm.running_var           loaded from res5_0_branch1_bn_running_var     of shape (2048,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: roi_heads.res5.0.shortcut.norm.weight                loaded from res5_0_branch1_bn_gamma           of shape (2048,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: roi_heads.res5.0.shortcut.weight                     loaded from res5_0_branch1_w                  of shape (2048, 1024, 1, 1)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: roi_heads.res5.1.conv1.norm.bias                     loaded from res5_1_branch2a_bn_beta           of shape (512,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: roi_heads.res5.1.conv1.norm.running_mean             loaded from res5_1_branch2a_bn_running_mean   of shape (512,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: roi_heads.res5.1.conv1.norm.running_var              loaded from res5_1_branch2a_bn_running_var    of shape (512,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: roi_heads.res5.1.conv1.norm.weight                   loaded from res5_1_branch2a_bn_gamma          of shape (512,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: roi_heads.res5.1.conv1.weight                        loaded from res5_1_branch2a_w                 of shape (512, 2048, 1, 1)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: roi_heads.res5.1.conv2.norm.bias                     loaded from res5_1_branch2b_bn_beta           of shape (512,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: roi_heads.res5.1.conv2.norm.running_mean             loaded from res5_1_branch2b_bn_running_mean   of shape (512,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: roi_heads.res5.1.conv2.norm.running_var              loaded from res5_1_branch2b_bn_running_var    of shape (512,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: roi_heads.res5.1.conv2.norm.weight                   loaded from res5_1_branch2b_bn_gamma          of shape (512,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: roi_heads.res5.1.conv2.weight                        loaded from res5_1_branch2b_w                 of shape (512, 512, 3, 3)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: roi_heads.res5.1.conv3.norm.bias                     loaded from res5_1_branch2c_bn_beta           of shape (2048,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: roi_heads.res5.1.conv3.norm.running_mean             loaded from res5_1_branch2c_bn_running_mean   of shape (2048,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: roi_heads.res5.1.conv3.norm.running_var              loaded from res5_1_branch2c_bn_running_var    of shape (2048,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: roi_heads.res5.1.conv3.norm.weight                   loaded from res5_1_branch2c_bn_gamma          of shape (2048,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: roi_heads.res5.1.conv3.weight                        loaded from res5_1_branch2c_w                 of shape (2048, 512, 1, 1)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: roi_heads.res5.2.conv1.norm.bias                     loaded from res5_2_branch2a_bn_beta           of shape (512,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: roi_heads.res5.2.conv1.norm.running_mean             loaded from res5_2_branch2a_bn_running_mean   of shape (512,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: roi_heads.res5.2.conv1.norm.running_var              loaded from res5_2_branch2a_bn_running_var    of shape (512,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: roi_heads.res5.2.conv1.norm.weight                   loaded from res5_2_branch2a_bn_gamma          of shape (512,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: roi_heads.res5.2.conv1.weight                        loaded from res5_2_branch2a_w                 of shape (512, 2048, 1, 1)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: roi_heads.res5.2.conv2.norm.bias                     loaded from res5_2_branch2b_bn_beta           of shape (512,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: roi_heads.res5.2.conv2.norm.running_mean             loaded from res5_2_branch2b_bn_running_mean   of shape (512,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: roi_heads.res5.2.conv2.norm.running_var              loaded from res5_2_branch2b_bn_running_var    of shape (512,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: roi_heads.res5.2.conv2.norm.weight                   loaded from res5_2_branch2b_bn_gamma          of shape (512,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: roi_heads.res5.2.conv2.weight                        loaded from res5_2_branch2b_w                 of shape (512, 512, 3, 3)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: roi_heads.res5.2.conv3.norm.bias                     loaded from res5_2_branch2c_bn_beta           of shape (2048,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: roi_heads.res5.2.conv3.norm.running_mean             loaded from res5_2_branch2c_bn_running_mean   of shape (2048,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: roi_heads.res5.2.conv3.norm.running_var              loaded from res5_2_branch2c_bn_running_var    of shape (2048,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: roi_heads.res5.2.conv3.norm.weight                   loaded from res5_2_branch2c_bn_gamma          of shape (2048,)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: roi_heads.res5.2.conv3.weight                        loaded from res5_2_branch2c_w                 of shape (2048, 512, 1, 1)
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: Some model parameters or buffers are not found in the checkpoint:
ℹ   pixel_mean
ℹ   pixel_std
ℹ   proposal_generator.anchor_generator.cell_anchors.0
ℹ   proposal_generator.rpn_head.anchor_deltas.{bias, weight}
ℹ   proposal_generator.rpn_head.conv.{bias, weight}
ℹ   proposal_generator.rpn_head.objectness_logits.{bias, weight}
ℹ   roi_heads.box_predictor.bbox_pred.{bias, weight}
ℹ   roi_heads.box_predictor.cls_score.{bias, weight}
ℹ [11/27 11:51:25 d2.checkpoint.c2_model_loading]: The checkpoint state_dict contains keys that are not used by the model:
ℹ   fc1000_b
ℹ   fc1000_w
ℹ   conv1_b
ℹ [11/27 11:51:25 d2.engine.train_loop]: Starting training from iteration 0
ℹ [job] fails
✖ job is failed: costa runtime is destroyed(SIGKILL).
commented

补充一个,用的json是官方文档上提供的示例~,识别图片中的前端组件

commented

请问你在 Pipcook 的社区群里吗? 方便的话加下我们具体聊下,目前相同路径下复现不了这个问题。
社区群二维码:
image

commented

好的,我这边之前不用钉钉,我注册一个加一下。

commented

排查后发现是 cuda 未安装,导致内存不足而 OOM,我们需要考虑下这种情况的报错处理,错误信息应该更友善。

commented

好的,谢谢~

commented

问题已解决,issue 关闭。有其他问题欢迎提 issue 和 PR。 @holanlan