Credits

How Transformers Attend · A field guide and reference — from tokens to long context.

First edition, 2026.

No part of this work may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means —electronic, mechanical, photocopy, recording, or otherwise— without the prior written permission of the author, except for brief quotation in reviews, teaching, or academic work with appropriate attribution.

The trademarks, model names, libraries, and products mentioned belong to their respective owners and are cited solely for descriptive and educational purposes, with no intent of infringement.

The content is offered “as is,” with the greatest possible diligence and verification; even so, it may contain errors. The author welcomes corrections.

Colophon. This guide was composed from a single source in Quarto, with a serif typeface for the body and Candara for the cover; the figures are generated by code (Matplotlib) and are reproducible; the formal identities are verified in Lean and the data published as open datasets. Interactive companion: tafagent.