Documentation

Formalizations.Analysis.TransformerAttention.CayleyMap

The Cayley map γ(z) = (2 − z)/(2 + z) and its fixed point #

The Padé decay exponent γ_Padé(z) = −(z − 2)/(z + 2) is a Cayley map. Its positive fixed point is z* = (√17 − 3)/2, with susceptibility χ = (5 + √17)/4 and Floquet multiplier μ = γ'(z*) = (√17 − 9)/8.

Main results #

References #

theorem TransformerAttention.CayleyMap.cayley_fixed_point_iff (z : ℝ) (h : z + 2 ≠ 0) :
-(z - 2) / (z + 2) = z ↔ z ^ 2 + 3 * z - 2 = 0

γ(z) = −(z − 2)/(z + 2) = z ↔ z² + 3z − 2 = 0.

theorem TransformerAttention.CayleyMap.cayley_root :
((√17 - 3) / 2) ^ 2 + 3 * ((√17 - 3) / 2) - 2 = 0

z* = (√17 − 3)/2 is a root of z² + 3z − 2.

theorem TransformerAttention.CayleyMap.chi_root :
2 * ((5 + √17) / 4) ^ 2 - 5 * ((5 + √17) / 4) + 1 = 0

χ = (5 + √17)/4 satisfies 2χ² − 5χ + 1 = 0.

z* = (√17 − 3)/2 > 0: the physical root.

theorem TransformerAttention.CayleyMap.floquet_multiplier :
(√17 - 9) / 8 * (2 + (√17 - 3) / 2) ^ 2 = -4

Floquet multiplier in cleared form: ((√17 − 9)/8) (2 + z*)² = −4, i.e. γ'(z*) = −4/(2 + z*)² = (√17 − 9)/8.

The table's form of the Floquet multiplier: −4(√17 + 1)/(5√17 + 13) = (√17 − 9)/8.

Stability of the fixed point: |(√17 − 9)/8| < 1.

γ(0) = 1 (the Hagedorn point).

γ(2) = 0.