Context window

A context window, used in large language models, is an amount of context attended to by an attention mechanism, which analyzes each word's relationships with the other words in the scope of a single conversation.

Size

The shortcomings of making a context window larger include higher computational cost and diluting the focus on local context, while making it smaller can cause a model to miss an important long-range dependency. Balancing them is a matter of experimentation and domain-specific considerations.

This article "Context window" is from Wikipedia. The list of its authors can be seen in its historical and/or the page Edithistory:Context window. Articles copied from Draft Namespace on Wikipedia could be seen on the Draft Namespace of Wikipedia and not main one.

---