简介 Brief Introduction
耶稣通用大模型V1是基于LLaMa的130亿参数的大规模预训练模型，具备翻译，编程，文本分类，信息抽取，摘要，文案生成，常识问答和数学计算等能力。目前耶稣通用大模型已完成大规模预训练、多任务有监督微调和人类反馈学习三阶段的训练过程。
The YesU-LLaMA-v1 is a large-scale pre-trained model based on LLaMA with 13 billion parameters. It has the ability to perform tasks such as translation, programming, text classification, information extraction, summarization, copywriting, common sense Q&A, and mathematical calculation. YesU-LLaMA-v1 has undergone three stages of training: large-scale continual pre-training (PT), multi-task supervised fine-tuning (SFT), and human feedback learning (RM, PPO).
软件依赖
待发布
模型分类 Model Taxonomy
模型信息 Model Information
继续预训练 Continual pretraining
原始数据包含英文和中文，其中英文数据来自openwebtext、Books、Wikipedia和Code，中文数据来自清洗后的悟道数据集、自建的中文数据集。在对原始数据进行去重、模型打分、数据分桶、规则过滤、敏感主题过滤和数据评估后，最终得到125B tokens的有效数据。
耶稣大模型正在探索与企业应用结合的应用场景，目前在以下两个场景进行研发和训练工作：
智慧网管：在耶稣通用大模型的基础上，加入网络管理知识库语料、交换机异常库、交换机指令库、交换机配置库等数据集，实现智能化监控、分析和管理网络和设备，并通过语音和文字进行故障管理、性能管理、配置管理、计费管理、安全管理、客户管理等日常网络管理功能。
模型对抗：准备中…
The original data contains both English and Chinese, with English data from openwebtext, Books, Wikipedia, and Code, and Chinese data from the cleaned Wudao dataset and self-built Chinese dataset. After deduplication, model scoring, data bucketing, rule filtering, sensitive topic filtering, and data evaluation, we finally obtained 125 billion tokens of valid data.
Autonomous network management：On the basis of the YesU-LLaMA-v1, data sets such as network management knowledge base corpus, switch exception library, switch instruction library, and switch configuration library are added to achieve intelligent monitoring, analysis, and management of networks and devices. Daily network management functions such as fault management, performance management, configuration management, billing management, security management, and customer management are carried out through voice and text.
Model Perturbations：Preparing...