about attention

[xinsheng]

Hello, about the problem of calculating attention, the attention of node i and its adjacent nodes is calculated in the formula, but I find that the final calculation is the attention of all nodes, and it does not distinguish whether the nodes are connected. Is there a problem with my understanding?

[Vijay Prakash Dwivedi]

Hi @lao-sheng,

The code section you highlight give the outputs of Q, K, V projections.

The implementation for node to attend to its local neighborhood is carried out by this function, which is just followed after the projections.

graphtransformer/layers/graph_transformer_layer.py

Line 73 in 3c83b4b

 self.propagate_attention(g) 

graphtransformer/layers/graph_transformer_layer.py

Lines 51 to 59 in 3c83b4b

 def propagate_attention(self, g): 
     # Compute attention score 
     g.apply_edges(src_dot_dst('K_h', 'Q_h', 'score')) #, edges) 
     g.apply_edges(scaled_exp('score', np.sqrt(self.out_dim))) 
  
     # Send weighted values to target nodes 
     eids = g.edges() 
     g.send_and_recv(eids, fn.src_mul_edge('V_h', 'score', 'V_h'), fn.sum('V_h', 'wV')) 
     g.send_and_recv(eids, fn.copy_edge('score', 'score'), fn.sum('score', 'z')) 

[xinsheng]

oh,i see.thanks！