跳到正文
GitHub Changelog· Allison·· 5 天前精选AI 评分84

GitHub Copilot Evals 开启公测

GitHub Copilot Evals is now in public preview

AI 导读

GitHub 宣布 Copilot Evals 正式开启公测,面向 Individual、Business 和 Enterprise 用户开放。该功能基于本地评测引擎 Vally,允许开发者通过自然语言或 `/eval` 命令为 Skill、自定义 Agent 及 MCP 工具创建可重复测试,对比配置调整前后的输出并排查回归。

推荐理由

原文给出了评估机制与具体接入方式,开发者可以据此为自定义智能体和MCP工具建立回归测试工作流。

正文 · 原文

You can now use GitHub Copilot Evals to optimize GitHub Copilot for yourself. As your prompts, models, skills, and tools change, are you still getting the most out of Copilot? Evals helps you answer that question for the skills, agents, and MCP tools in your setup. Create an eval, inspect how they behave, and use the results to decide what to improve.

What’s an eval?

An eval is a repeatable test for AI behavior. You give an agent a realistic task, describe what a good result looks like, and check both the result and how the agent got there.

AI output can change from one run to the next, so one good answer doesn’t tell you whether a customization works well for you. After changing a skill, agent, or MCP in your Copilot setup, rerun the same eval to check the behavior you care about and catch regressions.

What you can do

Ask Copilot in your own words or use the /eval command to:

  • Create an eval for a skill, custom agent, or MCP tool in your project.
  • Run it and see which checks passed and failed. You approve each run before it starts.
  • Find out why a check failed and decide what to change in your Copilot setup.
  • Compare runs from before and after a customization change to see whether it helped.

GitHub Copilot Evals uses Vally, an extensible local eval tool, as its evaluation engine.

Get started

Install GitHub Copilot Evals where you already use Copilot:

  • Visual Studio Code: Install the GitHub Copilot Evals extension from the Visual Studio Marketplace.
  • Copilot CLI: Install the GitHub Copilot Evals plugin by following the installation instructions.
  • GitHub Copilot app: Open Customize, add the github/evals marketplace, and install the GitHub Copilot Evals plugin.

GitHub Copilot Evals is in public preview for Copilot Individual, Copilot Business, and Copilot Enterprise plans.

To learn more, see the GitHub Copilot Evals documentation. Share feedback by opening an issue or joining the discussion in GitHub Community.

The post GitHub Copilot Evals is now in public preview appeared first on The GitHub Blog.

来源:GitHub Changelog · github.blog

EVAOO 主站实战落地 · AI Agent 与研发提效

看完情报?回到 EVAOO 主站直接上手工具与完整 SOP

返回 EVAOO 主站