tensorflow / profiler

A profiling and performance analysis tool for TensorFlow

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

What is _FusedMulAdd of profiling result?

alphaRGB opened this issue · comments

I'm profiling a NLP Transformer model. In the Transformer module, this module contains div mul, but can't find them in profiler result. Only found _FusedMulAdd. I used @tf.function + Eager

Code

matmul_qk = ...
mask = ...
dk = tf.cast(tf.shape(k)[-1], tf.float32)
matmul_qk = matmul_qk / tf.math.sqrt(dk)
matmul_qk += (mask * -1e9)