Skip to content

soraw-ai/Awesome-Robotics-And-Agentics-Work

BranchesTags

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

71 Commits
expert_roles
expert_roles
 
 
papers
papers
 
 
products
products
 
 
.DS_Store
.DS_Store
 
 
.gitignore
.gitignore
 
 
README.md
README.md
 
 

Repository files navigation

Awesome-Robotics-And-Agentics-Work Awesome

Areas covered in this repo: Robotics, AI Agents, Autonomous Agents, Multi-agents, LLM

Table of Content

Agentics Papers

For details please refer to Papers List

LLM-Based Autonomous Agents

  • COMBO: Compositional World Models for Embodied Multi-Agent Cooperation Multi-agents Co-op
    Hongxin Zhang, Zeyuan Wang, Qiushi Lyu, Zheyuan Zhang, Sunli Chen, Tianmin Shu, Yilun Du, Chuang Gan
    arXiv, 2024.04 [Paper], [PDF], [Code], [Home Page], [Demo (Video)]
  • ResearchAgent: Iterative Research Idea Generation over Scientific Literature with Large Language Models
    Jinheon Baek, Sujay Kumar Jauhar, Silviu Cucerzan, Sung Ju Hwang
    arXiv, 2024.04 [Paper], [PDF]
  • An Incomplete Loop: Deductive, Inductive, and Abductive Learning in Large Language Models Workflow Reasoning
    Emmy Liu, Graham Neubig, Jacob Andreas
    arXiv, 2024.04 [Paper], [PDF], [Code (TBD)]
  • Scaling Instructable Agents Across Many Simulated Worlds
    Team: DeepMind
    SIMA Team, Maria Abi Raad, Arun Ahuja, et al., Nick Young
    arXiv, 2024.03 [Paper], [PDF]
  • Mora: Enabling Generalist Video Generation via A Multi-Agent Framework Text2Video Generation
    Zhengqing Yuan, Ruoxi Chen, Zhaoxu Li, et al., Lichao Sun
    arXiv, 2024.03 [Paper], [PDF], [Code (TBD)]
  • LLM Agent Operating System Agents OS
    Kai Mei, Zelong Li, Shuyuan Xu, et al., Yongfeng Zhang
    arXiv, 2024.03 [Paper], [PDF], [Code]
  • Learning to Use Tools via Cooperative and Interactive Agents
    Zhengliang Shi, Shen Gao, Xiuyi Chen, et al., Zhaochun Ren
    arXiv, 2024.03 [Paper], [PDF]
  • LoTa-Bench: Benchmarking Language-oriented Task Planners for Embodied Agents Embodied Agents
    Jae-Woo Choi, Youngwoo Yoon, Hyobin Ong, Jaehong Kim, Minsu Jang
    ICLR'24, arXiv, 2024.02 [Paper], [PDF], [Code], [Home Page]
  • AgentTuning: Enabling Generalized Agent Abilities for LLMs
    Aohan Zeng, Mingdao Liu, Rui Lu, et al., Jie Tang
    arXiv, 2023.10 [Paper], [Code]
  • MusicAgent: An AI Agent for Music Understanding and Generation with Large Language Models
    Dingyao Yu, Kaitao Song, Peiling Lu, et al., Jiang Bian
    Team: Microsoft
    arXiv, 2023.10 [Paper], [Code]
  • Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Models
    Andy Zhou, Kai Yan, Michal Shlapentokh-Rothman, Haohan Wang, Yu-Xiong Wang
    arXiv, 2023.10 [Paper], [Code]
  • SmartPlay : A Benchmark for LLMs as Intelligent Agents
    Yue Wu, Xuan Tang, Tom M. Mitchell, Yuanzhi Li
    Team: Microsoft
    arXiv, 2023.10 [Paper], [Code]
  • A Survey on Large Language Model based Autonomous Agents
    Lei Wang, Chen Ma, Xueyang Feng, et al., Ji-Rong Wen
    arXiv, 2023.08 [Paper], [Code]
  • AgentSims: An Open-Source Sandbox for Large Language Model Evaluation
    Jiaju Lin, Haoran Zhao, Aochi Zhang, et al., Qin Chen
    arXiv, 2023.08. [Paper], [Code]
  • MetaGPT: Meta Programming for Multi-Agent Collaborative Framework
    Sirui Hong, Xiawu Zheng, Jonathan Chen, et al., Chenglin Wu
    arXiv, 2023.08. [Paper], [Code]
  • WebArena: A Realistic Web Environment for Building Autonomous Agents
    Shuyan Zhou, Frank F. Xu, Hao Zhu, et al., Graham Neubig
    arXiv, 2023.07. [Paper], [Code]
  • Unleashing Cognitive Synergy in Large Language Models: A Task-Solving Agent through Multi-Persona Self-Collaboration
    Zhenhailong Wang, Shaoguang Mao, Wenshan Wu, et al., Heng Ji
    arXiv, 2023.07. [Paper], [Code]
  • Minimum Levels of Interpretability for Artificial Moral Agents
    Avish Vijayaraghavan, Cosmin Badea
    arXiv, 2023.07. [Paper]
  • Reflexion: Language Agents with Verbal Reinforcement Learning
    Noah Shinn, Federico Cassano, Edward Berman, et al., Shunyu Yao
    NeurIPS'23, arXiv, 2023.05. [Paper], [PDF], [Code]
  • Decision-Oriented Dialogue for Human-AI Collaboration
    Jessy Lin, Nicholas Tomlin, Jacob Andreas, Jason Eisner
    arXiv, 2023.05. [Paper], [Code]
  • HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face
    Yongliang Shen, Kaitao Song, Xu Tan, et al., Yueting Zhuang
    arXiv, 2023.05. [Paper]
  • Interactive Natural Language Processing
    Zekun Wang, Ge Zhang, Kexin Yang, et al., Jie Fu
    arXiv, 2023.05. [Paper]
  • Introspective Tips: Large Language Model for In-Context Decision Making
    Liting Chen, Lu Wang, Hang Dong, et al., Dongmei Zhang
    arXiv, 2023.05. [Paper]
  • ExpeL: LLM Agents Are Experiential Learners
    Andrew Zhao, Daniel Huang, Quentin Xu, et al., Gao Huang
    arXiv, 2023.08. [Paper]
  • Communicative Agents for Software Development
    Team: OpenBMB
    Chen Qian, Xin Cong, Wei Liu, et al., Maosong Sun
    arXiv, 2023.07 [Paper], [PDF], [Code]
  • Cognitive Architectures for Language Agents
    Theodore Sumers, Shunyu Yao, Karthik Narasimhan, Thomas L. Griffiths
    arXiv, 2023.09 [Paper]
  • AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviors in Agents
    Weize Chen, Yusheng Su, Jingwei Zuo, et al., Jie Zhou
    arXiv, 2023.08 [Paper]
  • Agents: An Open-source Framework for Autonomous Language Agents
    Wangchunshu Zhou, Yuchen Eleanor Jiang, Long Li, et al., Mrinmaya Sachan
    arXiv, 2023.08 [Paper]
  • The Rise and Potential of Large Language Model Based Agents: A Survey
    Zhiheng Xi, Wenxiang Chen, Xin Guo, et al., Tao Gui
    Team: NLP group, Fudan University
    arXiv, 2023.08 [Paper], [Code]

Traditional Autonomous Agents

  1. [IJCAI’22] Forming Effective Human-AI Teams: Building Machine Learning Models that Complement the Capabilities of Multiple Experts
  2. [EMNLP’21] MindCraft: Theory of Mind Modeling for Situated Dialogue in Collaborative Tasks
  3. [arXiv 2017.03] It Takes Two to Tango: Towards Theory of AI's Mind
  4. [arXiv 2021.03] Human-AI Symbiosis: A Survey of Current Approaches
  5. [arXiv 2023.08] AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation Framework

LLM-based Swarm Intelligence

  • Exploring Large Language Models for Communication Games: An Empirical Study on Werewolf
    Yuzhuang Xu, Shuo Wang, Peng Li, et al., Yang Liu
    arXiv, 2023.09 [Paper]

Consciousness

  • It Takes Two to Tango: Towards Theory of AI's Mind
    Patrick Butlin, Robert Long, Eric Elmoznino, et al., Rufin VanRullen
    arXiv, 2023.08. [Paper]

Dataset/Benchmark/Simulator/Tool

  • Towards better Human-Agent Alignment: Assessing Task Utility in LLM-Powered Applications
    Negar Arabzadeh, Julia Kiseleva, Qingyun Wu, et al., Charles Clarke
    Team: Univerity of Waterloo, Microsoft
    arXiv, 2024.02 [Paper], [PDF], [Code (Notebook)]

Foundation Model

  • An Interactive Agent Foundation Model
    Zane Durante, Bidipta Sarkar, Ran Gong, et al., Qiuyuan Huang
    Team: Stanford University Fei-Fei Li, Microsoft
    arXiv, 2024.02 [Paper], [PDF], [Home Page]

Survey

  • The Landscape of Emerging AI Agent Architectures for Reasoning, Planning, and Tool Calling: A Survey
    Zhiheng Xi, Wenxiang Chen, Xin Guo, et al., Tao Gui
    Team: IBM
    arXiv, 2024.04 [Paper], [PDF]
  • The Rise and Potential of Large Language Model Based Agents: A Survey
    Tula Masterman, Sandi Besen, Mason Sawtell, Alex Chao
    Team: Fudan University
    arXiv, 2023.09 [Paper], [PDF], [Paper List]
  • A Survey on Large Language Model based Autonomous Agents
    Lei Wang, Chen Ma, Xueyang Feng, et al., Ji-Rong Wen
    Team: Renmin University
    arXiv, 2023.08 [Paper], [PDF], [Paper List]

Robotics Papers

For details please refer to Papers List

Data/Simulation Argument

  • cuRobo: Parallelized Collision-Free Minimum-Jerk Robot Motion Generation
    Balakumar Sundaralingam, Siva Kumar Sastry Hari, Adam Fishman, Caelan Garrett, et al., Dieter Fox
    Team: NVlabs NVIDIA
    arXiv, 2023.10 [Paper], [PDF], [Code]
  • GenSim: Generating Robotic Simulation Tasks via Large Language Models
    Lirui Wang, Yiyang Ling, Zhecheng Yuan, et al., Xiaolong Wang
    arXiv, 2023.10 [Paper], [Code]
  • RoboGen: Towards Unleashing Infinite Data for Automated Robot Learning via Generative Simulation
    Yufei Wang, Zhou Xian, Feng Chen, et al., Chuang Gan
    arXiv, 2023.11 [Paper], [Code]

Manipulation

  • Embodied AI with Two Arms: Zero-shot Learning, Safety and Modularity Safty
    Team: Google Deep Mind Robotics, UNC Chapel Hill.
    Jake Varley, Sumeet Singh, Deepali Jain, et al., Vikas Sindhwani
    arXiv, 2024.04 [Paper], [PDF]
  • Robo-ABC: Affordance Generalization Beyond Categories via Semantic Correspondence for Robot Manipulation Grasp
    Yuanchen Ju, Kaizhe Hu, Guowei Zhang, et al., Huazhe Xu
    arXiv, 2024.01 [Paper], [PDF], [Code (TBD)], [Home Page]
  • Visual Robotic Manipulation with Depth-Aware Pretraining
    Wanying Wang, Jinming Li, Yichen Zhu, et al., Jian Tang
    arXiv, 2024.01 [Paper], [PDF]
  • ManipLLM: Embodied Multimodal Large Language Model for Object-Centric Robotic Manipulation
    Xiaoqi Li, Mingxu Zhang, Yiran Geng, et al., Hao Dong
    arXiv, 2023.11 [Paper], [PDF], [Home Page]
  • M2T2: Multi-Task Masked Transformer for Object-centric Pick and Place Grasp
    Wentao Yuan, Adithyavairavan Murali, Arsalan Mousavian, Dieter Fox
    CoRL'23, arXiv, 2023.12 [Paper], [PDF], [Code], [Home Page]
  • Make a Donut: Language-Guided Hierarchical EMD-Space Planning for Zero-shot Deformable Object Manipulation
    Yang You, Bokui Shen, Congyue Deng, et al., Leonidas Guibas
    arXiv, 2023.11 [Paper], [PDF]
  • D3Fields: Dynamic 3D Descriptor Fields for Zero-Shot Generalizable Robotic Manipulation Multimodal
    Team: UIUC, Stanford University, Fei-Fei Li.
    Yixuan Wang, Zhuoran Li, Mingtong Zhang, et al., Li Fei-Fei, Yunzhu Li
    CoRL'23, arXiv, 2023.10 [Paper], [PDF], [Code], [Home Page], [Demo]
  • Gen2Sim: Scaling up Robot Learning in Simulation with Generative Models
    Pushkal Katara, Zhou Xian, Katerina Fragkiadaki
    arXiv, 2023.10 [Paper], [PDF], [Code], [Home Page]
  • How to Prompt Your Robot: A PromptBook for Manipulation Skills with Code as Policies Prompt Engineering
    Team: Google.
    Montserrat Gonzalez Arenas, Ted Xiao, Sumeet Singh, et al., Andy Zeng
    CoRL'23 workshop, 2023.10 [Paper], [PDF]
  • RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control Multimodal Robotic Control VLA
    Team: Google DeepMind
    Anthony Brohan, Noah Brown, Justice Carbajal, et al., Brianna Zitkovich
    arXiv, 2023.07 [Paper], [PDF], [Home Page]
  • RVT: Robotic View Transformer for 3D Object Manipulation Grasp
    Team: NVIDIA Labs
    Ankit Goyal, Jie Xu, Yijie Guo, et al., Dieter Fox
    CoRL'23 (Oral), arXiv, 2023.06 [Paper], [PDF], [Code], [Home Page], [Demo]
  • Distilled Feature Fields Enable Few-Shot Language-Guided Manipulation Grasp
    Team: MIT CSAIL
    William Shen, Ge Yang, Alan Yu, et al., Phillip Isola
    CoRL'23 (Best Paper), arXiv, 2023.06 [Paper], [PDF], [Code], [Home Page], [Demo]
  • RT-1: Robotics Transformer for Real-World Control at Scale Multimodal Robotic Control
    Team: Robotics at Google, Everyday Robotics
    Anthony Brohan, Noah Brown, Justice Carbajal, et al., Brianna Zitkovich
    arXiv, 2022.12 [Paper], [PDF], [Code], [Home Page], [Demo]
  • Predicting Stable Configurations for Semantic Placement of Novel Objects Grasp
    Team: NVIDIA.
    Chris Paxton, Chris Xie, Tucker Hermans, Dieter Fox
    CoRL'22, arXiv, 2021.08 [Paper], [PDF]

Navigation

  • MapGPT: Map-Guided Prompting for Unified Vision-and-Language Navigation
    Jiaqi Chen, Bingqian Lin, Ran Xu, et al., Kwan-Yee K. Wong
    arXiv, 2024.01 [Paper], [PDF]
  • Visual Language Maps for Robot Navigation
    Chenguang Huang, Oier Mees, Andy Zeng, Wolfram Burgard
    ICRA'23, arXiv, 2022.10 [Paper], [PDF], [Code], [Home Page]

Planning

  • SUGAR: Pre-training 3D Visual Representations for Robotics
    Shizhe Chen, Ricardo Garcia, Ivan Laptev, Cordelia Schmid
    CVPR'24, arXiv, 2024.04 [Paper], [PDF], [Home Page]
  • DexCap: Scalable and Portable Mocap Data Collection System for Dexterous Manipulation Motion Planning
    Team: Fei-Fei Li, Stanford University.
    Chen Wang, Haochen Shi, Weizhuo Wang, Ruohan Zhang, Li Fei-Fei, C. Karen Liu
    arXiv, 2024.03 [Paper], [PDF], [Code], [Home Page], [Demo]
  • RT-H: Action Hierarchies Using Language Task Planning
    Team: Google DeepMind, Stanford University.
    Suneel Belkhale, Tianli Ding, Ted Xiao, et al., Dorsa Sadigh
    arXiv, 2024.03 [Paper], [PDF], [Home Page], [Demo]
  • RePLan: Robotic Replanning with Perception and Language Models Motion Planning,Multimodal
    Marta Skreta, Zihan Zhou, Jia Lin Yuan, et al., Animesh Garg
    arXiv, 2024.01 [Paper], [PDF], [Home Page], [Demo]
  • Human Demonstrations are Generalizable Knowledge for Robots Task Planning, Human Demo
    Guangyan Chen, Te Cui, Tianxing Zhou, et al., Yufeng Yue
    arXiv, 2023.12 [Paper], [PDF]
  • Look Before You Leap: Unveiling the Power of GPT-4V in Robotic Vision-Language Planning Motion Planning
    Team: Tsinghua University.
    Yingdong Hu, Fanqi Lin, Tong Zhang, Li Yi, Yang Gao
    arXiv, 2023.11 [Paper], [PDF], [Home Page], [Demo]
  • GPT-4V(ision) for Robotics: Multimodal Task Planning from Human Demonstration Task Planning, Human Demo
    Team: Microsoft.
    Naoki Wake, Atsushi Kanehira, Kazuhiro Sasabuchi, Jun Takamatsu, Katsushi Ikeuchi
    arXiv, 2023.11 [Paper], [PDF], [Code], [Home Page]
  • VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Models Motion Planning, Multimodal, PoT
    Team: Stanford University, Fei-Fei Li.
    Wenlong Huang, Chen Wang, Ruohan Zhang, et al., Li Fei-Fei
    CoRL'23(Oral), arXiv, 2023.10 [Paper], [PDF], [Code], [Home Page], [Demo]
  • Vision-Language Models are Zero-Shot Reward Models for Reinforcement Learning Reward Desgin, Multimodal
    Juan Rocamonde, Victoriano Montesinos, Elvis Nava, Ethan Perez, David Lindner
    NeurIPS'23 workshop, arXiv, 2023.10 [Paper], [PDF], [Code], [Home Page]
  • Learning Reward for Physical Skills using Large Language Model Reward Desgin
    Yuwei Zeng, Yiqing Xu
    CoRL'23 workshop, arXiv, 2023.10 [Paper], [PDF]
  • Eureka: Human-Level Reward Design via Coding Large Language Models Reward Desgin
    Team: NVIDIA, UPenn.
    Yecheng Jason Ma, William Liang, Guanzhi Wang, et al., Anima Anandkumar
    arXiv, 2023.10 [Paper], [PDF], [Code], [Home Page]
  • RoboCLIP: One Demonstration is Enough to Learn Robot Policies Learning From Demo
    Team: UC Berkeley, Stanford University, Google
    Sumedh A Sontakke, Jesse Zhang, Sébastien M. R. Arnold, et al., Laurent Itti
    NeurIPS'23, arXiv, 2023.10 [Paper], [PDF], [Code], [Home Page], [Demo]
  • Text2Reward: Automated Dense Reward Function Generation for Reinforcement Learning Reward Desgin
    Tianbao Xie, Siheng Zhao, Chen Henry Wu, et al., Tao Yu
    arXiv, 2023.09 [Paper], [PDF], [Code], [Home Page]
  • ConceptGraphs: Open-Vocabulary 3D Scene Graphs for Perception and Planning
    Qiao Gu, Alihusein Kuwajerwala, Sacha Morin, et al., Liam Paull
    CoRL'23 workshop, arXiv, 2023.09 [Paper], [PDF], [Code], [Home Page]
  • Prompt a Robot to Walk with Large Language Models
    Yen-Jen Wang, Bike Zhang, Jianyu Chen, Koushil Sreenath
    arXiv, 2023.09 [Paper], [PDF], [Code], [Home Page], [Demo]
  • SayPlan: Grounding Large Language Models using 3D Scene Graphs for Scalable Robot Task Planning Task Planning
    Krishan Rana, Jesse Haviland, Sourav Garg, et al., Niko Suenderhauf
    CoRL'23(Oral), arXiv, 2023.07 [Paper], [PDF], [Home Page], [Demo]
  • Scaling Up and Distilling Down: Language-Guided Robot Skill Acquisition Reward Desgin
    Team: Columbia University, Google Deepmind.
    Huy Ha, Pete Florence, Shuran Song
    CoRL'23, arXiv, 2023.07 [Paper], [PDF], [Code], [Home Page], [Demo]
  • LARG, Language-based Automatic Reward and Goal Generation Reward Desgin
    Julien Perez, Denys Proux, Claude Roux, Michael Niemaz
    arXiv, 2023.06 [Paper], [PDF]
  • Language to Rewards for Robotic Skill Synthesis Reward Desgin
    Team: Google Deepmind.
    Wenhao Yu, Nimrod Gileadi, Chuyuan Fu, etal., Fei Xia
    ICLR'23, arXiv, 2023.06 [Paper], [PDF], [Code], [Home Page], [Demo]
  • Affordances From Human Videos as a Versatile Representation for Robotics Learn from Demo
    Team: CMU, Meta.
    Shikhar Bahl, Russell Mendonca, Lili Chen, Unnat Jain, Deepak Pathak
    CVPR'23, arXiv, 2023.04 [Paper], [PDF], [Code], [Home Page], [Demo (Video)]
  • PaLM-E: An Embodied Multimodal Language Model Task Planning, Multimodal
    Team: Robotics at Google.
    Danny Driess, Fei Xia, Mehdi S. M. Sajjadi, et al., Pete Florence
    ICML'23, arXiv, 2023.03 [Paper], [PDF], [Home Page]
  • Grounded Decoding: Guiding Text Generation with Grounded Models for Embodied Agents Multimodal
    Team: Stanford University, Robotics at Google.
    Wenlong Huang, Fei Xia, Dhruv Shah, et al., Brian Ichter
    arXiv, 2023.03 [Paper], [PDF], [Home Page], [Demo]
  • Reward Design with Language Models Reward Desgin
    Team: Stanford University, Deepmind.
    Minae Kwon, Sang Michael Xie, Kalesha Bullard, Dorsa Sadigh
    ICLR'23, arXiv, 2023.02 [Paper], [PDF], [Code]
  • Code as Policies: Language Model Programs for Embodied Control Task Planning, PoT
    Team: Robotics at Google.
    Jacky Liang, Wenlong Huang, Fei Xia, et al., Andy Zeng
    ICRA'23, CoRL openreview, 2022,11 [Paper], [PDF], [Code], [Home Page]
  • Correcting Robot Plans with Natural Language Feedback Motion Planning
    Team: NVIDIA, MIT.
    Pratyusha Sharma, Balakumar Sundaralingam, Valts Blukis, et al., Dieter Fox
    arXiv, 2022.04 [Paper], [PDF]
  • Do As I Can, Not As I Say: Grounding Language in Robotic Affordances Task Planning, Multimodal
    Team: Robotics at Google, Everyday Robots.
    Michael Ahn, Anthony Brohan, Noah Brown, et al., Andy Zeng
    arXiv, 2022.04 [Paper], [PDF], [Code], [Home Page], [Demo]
  • Sequence-of-Constraints MPC: Reactive Timing-Optimal Control of Sequential Manipulation Manipulation Planning
    Marc Toussaint, Jason Harris, Jung-Su Ha, Danny Driess, Wolfgang Hönig
    IROS'22, arXiv, 2022.03 [Paper], [PDF]
  • Visually-Grounded Planning without Vision: Language Models Infer Detailed Plans from High-level Instructions Language Model Only
    Peter A. Jansen
    EMNLP'20, arXiv, 2020.09 [Paper], [PDF], [Code]

Task Adaptation

  • Bootstrap Your Own Skills: Learning to Solve New Tasks with Large Language Model Guidance Learn New Tasks
    Team: Robotics at Google, Everyday Robots.
    Jesse Zhang, Jiahui Zhang, Karl Pertsch, et al., Joseph J. Lim
    CoRL'23(Oral), arXiv, 2023.10 [Paper], [PDF], [Code], [Home Page]

Survey

  • Real-World Robot Applications of Foundation Models: A Review
    Kento Kawaharazuka, Tatsuya Matsushima, Andrew Gambardella, et al., Andy Zeng
    arXiv, 2024.02 [Paper], [PDF]
  • Large Language Models for Robotics: Opportunities, Challenges, and Perspectives
    Jiaqi Wang, Zihao Wu, Yiwei Li, et al., Shu Zhang
    arXiv, 2024.01 [Paper], [PDF]
  • Toward General-Purpose Robots via Foundation Models: A Survey and Meta-Analysis
    Yafei Hu, Quanting Xie, Vidhi Jain, etal., Yonatan Bisk
    arXiv, 2023.12 [Paper], [PDF]
  • Language-conditioned Learning for Robotic Manipulation: A Survey
    Hongkuan Zhou, Xiangtong Yao, Yuan Meng, et al., Alois Knoll
    arXiv, 2023.12 [Paper], [PDF]
  • Toward General-Purpose Robots via Foundation Models: A Survey and Meta-Analysis
    Yafei Hu, Quanting Xie, Vidhi Jain, et al., Yonatan Bisk
    arXiv, 2023.12 [Paper], [Code]
  • Foundation Models in Robotics: Applications, Challenges, and the Future
    Roya Firoozi, Johnathan Tucker, Stephen Tian, et al., Mac Schwager
    arXiv, 2023.12 [Paper], [PDF], [Code]
  • Robot Learning in the Era of Foundation Models: A Survey
    Xuan Xiao, Jiahang Liu, Zhipeng Wang, et al., Shuo Jiang
    arXiv, 2023.11 [Paper], [PDF]
  • Recent Advances in Robot Learning from Demonstration
    *Harish Ravichandar, Athanasios S. Polydoros, Sonia Chernova, Aude Billard
    Annual Review of Control, Robotics, and Autonomous Systems, 2020.05 [Paper], [PDF]

Dataset/Benchmark/Simulator/Tool

  • DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset Dataset
    Team: Stanford University, UC Berkeley, et al.
    Alexander Khazatsky, Karl Pertsch, Suraj Nair, et al., Chelsea Finn
    arXiv, 2024.03 [Paper], [PDF], [Home Page], [Dataset Visualizer], [Colab Demo]
  • BEHAVIOR-1K: A Human-Centered, Embodied AI Benchmark with 1,000 Everyday Activities and Realistic Simulation Benchmark
    Team: Stanford University, Fei-fei Li
    Chengshu Li, Ruohan Zhang, Josiah Wong, et al., Li Fei-Fei
    CoRL'22 (preliminary version), arXiv, 2024.03 [Paper], [PDF], [Home Page]
  • Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robots Data Collection Tool
    Team: Stanford University
    Cheng Chi, Zhenjia Xu, Chuer Pan, et al., Shuran Song
    arXiv, 2024.02 [Paper], [PDF], [Code], [Home Page]
  • Open X-Embodiment: Robotic Learning Datasets and RT-X Models
    Team: DeepMind
    Open X-Embodiment Collaboration, et al.
    arXiv, 2023.10 [Paper], [PDF], [Code], [Home Page]
  • ARNOLD: A Benchmark for Language-Grounded Task Learning With Continuous States in Realistic 3D Scenes
    Ran Gong, Jiangyong Huang, Yizhou Zhao, et al., Siyuan Huang
    ICCV'23, 2023.04 [Paper], [PDF], [Code]
  • RLBench: The Robot Learning Benchmark & Learning Environment
    Team: NVlabs NVIDIA
    Stephen James, Zicong Ma, David Rovick Arrojo, Andrew J. Davison
    IEEE Robotics and Automation Letters, 2019.09 [Paper], [Code]

Foundation Model

  • 3D-VLA: A 3D Vision-Language-Action Generative World Model VLA
    Haoyu Zhen, Xiaowen Qiu, Peihao Chen, et al., Chuang Gan
    arXiv, 2024.03 [Paper], [PDF], [Code], [Home Page]

Autonomous Agents Product

For details, please refer to Products List

AI Agents in 2024, by Jan 02, 2024 E2B

Category Project Team Code Stars Last Commit
R&D XAgent OpenBMB; Tsinghua University Code Stars Last Commit
R&D AIWaves Agents - Code Stars Last Commit
R&D CoALA - Code Stars Last Commit
R&D AgentVerse OpenBMB Code Stars Last Commit
R&D ChatDev OpenBMB Code Stars Last Commit
R&D GPT Researcher - Code Stars Last Commit
R&D Lagent - Code Stars Last Commit
R&D AgentSims - Code Stars Last Commit
R&D AI Town a16z-infra Code Stars Last Commit
R&D WebArena - Code Stars Last Commit
R&D Generative Agents - Code Stars Last Commit
R&D MetaGPT - Code Stars Last Commit
R&D Auto-GPT - Code Stars Last Commit
R&D Langchain - Code Stars Last Commit
R&D BabyAG - Code Stars Last Commit
Business AutoGen Microsoft Site Code Stars Last Commit
Business Council - Code Stars Last Commit
Business SuperAGI - Code Stars Last Commit
Business AgentGPT - Code Stars Last Commit
Business AI Agent - Site - -

Expert Roles LLM

For Coding

  • Based LLaMa 2 Code Llama By Meta AI · August 24, 2023

Tools

  • TensorRT-LLM
    By NVIDIA [Code]
  • llama_ros
    [Code] This repository provides a set of ROS 2 packages to integrate llama.cpp into ROS 2. By using the llama_ros packages, you can easily incorporate the powerful optimization capabilities of llama.cpp into your ROS 2 projects.

Useful Blogs and Resources

Acknowledgement

Citation

If you find this repository useful, please consider citing this list:

@misc{rui2023roboticsandagneticslist,
    title = {Awesome-Robotics-And-Agentics-Work},
    author = {Rui Sun},
    journal = {GitHub repository},
    url = {https://github.com/soraw-ai/Awesome-Robotics-And-Agentics-Work},
    year = {2023},
}

Abbreviations List

Symbol Full Name Description
LLM Large Language Model
VLM Vision Language Model
PoT Program of Thoughts

References

  1. MetaGPT作者深度解析直播回放
  2. 从生成式AI到合成式AI ，MarTech下一步如何进化by 晓晓 · Aug 10, 2023

About

All about Robotics and AI Agents you need are here

soraw.ai

Resources

Readme
Activity
Custom properties

Stars

21 stars

Watchers

1 watching

Forks

1 fork
Report repository

Releases

No releases published

Packages

No packages published