SUI

https://arxiv.org/abs/2306.04031 Certified Reasoning with Language Models Language models often achieve higher accuracy when reasoning step-by-step in complex tasks. However, their reasoning can be unsound, inconsistent, or rely on undesirable prior assumptions. To tackle these issues, we introduce a class of tools for language arxiv.org 1. 언어 모델은 복잡한 작업에서 단계별로 추론을 할 때 종종 더 높은 정확도를 보입니다. 그러나 그들의..

https://arxiv.org/abs/2306.01711 OMNI: Open-endedness via Models of human Notions of Interestingness Open-ended algorithms aim to learn new, interesting behaviors forever. That requires a vast environment search space, but there are thus infinitely many possible tasks. Even after filtering for tasks the current agent can learn (i.e., learning progress), c arxiv.org https://twitter.com/jeffclune/..

https://arxiv.org/abs/2305.19466 The Impact of Positional Encoding on Length Generalization in Transformers Length generalization, the ability to generalize from small training context sizes to larger ones, is a critical challenge in the development of Transformer-based language models. Positional encoding (PE) has been identified as a major factor influencing l arxiv.org 설명: https://twitter.com..

https://arxiv.org/abs/2306.00323 Thought Cloning: Learning to Think while Acting by Imitating Human Thinking Language is often considered a key aspect of human thinking, providing us with exceptional abilities to generalize, explore, plan, replan, and adapt to new situations. However, Reinforcement Learning (RL) agents are far from human-level performance in any arxiv.org https://twitter.com/jef..

https://arxiv.org/abs/2305.20010 Human or Not? A Gamified Approach to the Turing Test We present "Human or Not?", an online game inspired by the Turing test, that measures the capability of AI chatbots to mimic humans in dialog, and of humans to tell bots from other humans. Over the course of a month, the game was played by over 1.5 million arxiv.org 1. 우리는 튜링 테스트에 영감을 받은 온라인 게임인 "Human or Not?"..

Chinchilla 스케일링 법칙 확장 설명: https://twitter.com/Muennighoff/status/1661895337248686081 트위터에서 즐기는 Niklas Muennighoff “How to keep scaling Large Language Models when data runs out? 🎢 We train 400 models with up to 9B params & 900B tokens to create an extension of Chinchilla scaling laws for repeated data. Results are interesting… 🧐 📜: https://t.co/586bWwvpba twitter.com 1. 이 연구에서는 데이터 제한 조건에서 언어 모델을..

abs: https://arxiv.org/abs/2305.19476 Accelerating Reinforcement Learning with Value-Conditional State Entropy Exploration A promising technique for exploration is to maximize the entropy of visited state distribution, i.e., state entropy, by encouraging uniform coverage of visited state space. While it has been effective for an unsupervised setup, it tends to struggle in a su arxiv.org code: ht..

abs: https://arxiv.org/abs/2305.19370 Blockwise Parallel Transformer for Long Context Large Models Transformers have emerged as the cornerstone of state-of-the-art natural language processing models, showcasing exceptional performance across a wide range of AI applications. However, the memory demands posed by the self-attention mechanism and the large arxiv.org 1. 트랜스포머 모델은 다양한 AI 응용 분야에서 최첨단 자..

https://penn-pal-lab.github.io/LIV/ LIV LIV as Representation for Language-Conditioned BC We use LIV's frozen multi-modal representation as backbone for LCBC and achieve impressive performance (46% success rate, absolute ~30% better than the second best baseline) on a challenging real-world muli penn-pal-lab.github.io 설명: https://twitter.com/JasonMa2020/status/1663618652778942464 트위터에서 즐기는 Jason..

abs: https://arxiv.org/abs/2305.18390 github: https://github.com/THUNLP/modularity-analysis Emergent Modularity in Pre-trained Transformers This work examines the presence of modularity in pre-trained Transformers, a feature commonly found in human brains and thought to be vital for general intelligence. In analogy to human brains, we consider two main characteristics of modularity: (1) functi a..