·research·paper
Multi-graph former — a transformer over heterogeneous graph structure
Architecture sketch for a transformer variant that operates over multiple, related graphs simultaneously and learns the morphism between them.
Jacob Valdez
workshop submission
cite key:
valdez2023multigraphIdea
Most graph transformers operate over a single graph at a time. The generalization is unsurprising — operate over a stack of graphs that share nodes — but the interesting structure is in learning the alignment across graphs as a first-class operation, rather than treating graphs as independent inputs.
Concretely, attention is computed over a tensor indexed by (graph, node, graph, node), and a low-rank factorization keeps the cost reasonable.
Status
Draft figure done; experiments started but not finished. Filed here as a node so it doesn't fall out of the graph.
Neighborhood