Name: Functions and operators for Dot and Matrix multiplication and Element-wise calculation in PyTorch
Rating: 1.4 (960 reviews)
Author: hyperkai

Buy Me a Coffee☕

*Memos:

My post explains Matrix and Element-wise multiplication in PyTorch.
My post explains Dot and Matrix-vector multiplication in PyTorch.
My post explains matmul() and dot()
My post explains mv(), mm() and bmm().
My post explains add().
My post explains sub() and mul().
My post explains div().
My post explains remainder() and fmod().

<Dot multiplication>

dot() can multiply 1D tensors:

import torch

tensor1 = torch.tensor([2, 7, 4]) # 1D tensor
tensor2 = torch.tensor([6, 3, 5]) # 1D tensor

torch.dot(tensor1, tensor2)
tensor1.dot(tensor2)
# tensor(53)

matmul() or @ can multiply 1D or more D tensors:

import torch

tensor1 = torch.tensor([2, 7, 4]) # 1D tensor
tensor2 = torch.tensor([6, 3, 5]) # 1D tensor

torch.matmul(tensor1, tensor2)
tensor1.matmul(tensor2)
tensor1 @ tensor2
# tensor(53)

<Matrix-vector multiplication>

mv() can multiply a 2D and 1D tensor:

import torch

tensor1 = torch.tensor([[2, 7, 4], [8, 3, 2]]) # 2D tensor
tensor2 = torch.tensor([5, 0, 8]) # 1D tensor

torch.mv(tensor1, tensor2)
tensor1.mv(tensor2)
# tensor([42, 56])

matmul() or @ can multiply 1D or more D tensors:

import torch

tensor1 = torch.tensor([[2, 7, 4], [8, 3, 2]]) # 2D tensor
tensor2 = torch.tensor([5, 0, 8]) # 1D tensor

torch.matmul(tensor1, tensor2)
tensor1.matmul(tensor2)
tensor1 @ tensor2
# tensor([42, 56])

<Matrix multiplication>

mm() can multiply 2D tensors:

import torch

tensor1 = torch.tensor([[2, 7, 4], [8, 3, 2]]) # 2D tensor
tensor2 = torch.tensor([[5, 0, 8, 6], # 2D tensor
                        [3, 6, 1, 7],
                        [1, 4, 9, 2]])
torch.mm(tensor1, tensor2)
tensor1.mm(tensor2)
# tensor([[35, 58, 59, 69], [51, 26, 85, 73]])

bmm() can multiply 3D tensors:

import torch

tensor1 = torch.tensor([[[2, 7]], [[8, 3]]]) # 3D tensor
tensor2 = torch.tensor([[[5, 9], [3, 6]], # 3D tensor
                        [[7, 2], [1, 4]]])

torch.bmm(tensor1, tensor2)
tensor1.bmm(tensor2)
# tensor([[[31, 60]], [[59, 28]]])

matmul() or @ can multiply 1D or more D tensors by dot or matrix multiplication:

import torch

tensor1 = torch.tensor([[2, 7], [8, 3]]) # 2D tensor
tensor2 = torch.tensor([[[[5, 9], [3, 6]], [[7, 2], [1, 4]]],
                        [[[6, 0], [4, 6]], [[2, 9], [8, 1]]]])
                       # 4D tensor
torch.matmul(tensor1, tensor2)
tensor1.matmul(tensor2)
tensor1 @ tensor2
# tensor([[[[31, 60], [49, 90]], [[21, 32], [59, 28]]],
#         [[[40, 42], [60, 18]], [[60, 25], [40, 75]]]])

<Element-wise calculation>

mul() or * can do multiplication with 0D or more D tensors. *mul() and multiply() are the same because multiply() is the alias of mul():

import torch

tensor1 = torch.tensor([2, 7, 4]) # 1D tensor
tensor2 = torch.tensor([6, 3, 5]) # 1D tensor

torch.mul(tensor1, tensor2)
tensor1.mul(tensor2)
tensor1 * tensor2
# tensor([12, 21, 20])

div() or / can do division with 0D or more D tensors: *Memos:
divide() is the alias of div().
true_divide() is the alias of div() with rounding_mode=None.
floor_divide() is the same as div() with rounding_mode="trunc" as long as I experimented:

import torch

tensor1 = torch.tensor([2, 7, 4]) # 1D tensor
tensor2 = torch.tensor([6, 3, 5]) # 1D tensor

torch.div(tensor1, tensor2)
tensor1.div(tensor2)
tensor1 / tensor2
# tensor([0.3333, 2.3333, 0.8000])

remainder() or % can do modulo(mod) calculation with 0D or more D tensors:

import torch

tensor1 = torch.tensor([2, 7, 4]) # 1D tensor
tensor2 = torch.tensor([6, 3, 5]) # 1D tensor

torch.remainder(tensor1, tensor2)
tensor1.remainder(tensor2)
tensor1 % tensor2
# tensor([2, 1, 4])

add() or + can do addition with 0D or more D tensors:

import torch

tensor1 = torch.tensor([2, 7, 4]) # 1D tensor
tensor2 = torch.tensor([6, 3, 5]) # 1D tensor

torch.add(tensor1, tensor2)
tensor1.add(tensor2)
tensor1 + tensor2
# tensor([8, 10, 9])

sub() or - can do subtraction with 0D or more D tensors. *sub() and subtract() are the aliases of sub():

import torch

tensor1 = torch.tensor([2, 7, 4]) # 1D tensor
tensor2 = torch.tensor([6, 3, 5]) # 1D tensor

torch.subtract(tensor1, tensor2)
tensor1.subtract(tensor2)
tensor1 - tensor2
# tensor([-4, 4, -1])