SUI

https://arxiv.org/abs/2307.02486 LongNet: Scaling Transformers to 1,000,000,000 Tokens Scaling sequence length has become a critical demand in the era of large language models. However, existing methods struggle with either computational complexity or model expressivity, rendering the maximum sequence length restricted. In this work, we intr arxiv.org https://github.com/microsoft/torchscale GitH..

https://arxiv.org/abs/2306.14892 Supervised Pretraining Can Learn In-Context Reinforcement Learning Large transformer models trained on diverse datasets have shown a remarkable ability to learn in-context, achieving high few-shot performance on tasks they were not explicitly trained to solve. In this paper, we study the in-context learning capabilities o arxiv.org 1. 다양한 데이터셋에서 훈련된 대형 트랜스포머 모델은 ..

https://arxiv.org/abs/2306.14824 Kosmos-2: Grounding Multimodal Large Language Models to the World We introduce Kosmos-2, a Multimodal Large Language Model (MLLM), enabling new capabilities of perceiving object descriptions (e.g., bounding boxes) and grounding text to the visual world. Specifically, we represent refer expressions as links in Markdown, i arxiv.org https://github.com/microsoft/uni..