ci: verify examples and Copilot integration tests

[Michelle Clayton (michelle-clayton-work)]

Summary

Add scheduled and manually dispatched CI that runs every example supported by the configured credentials. Add GitHub Copilot provider integration tests to the normal test workflow.

The first provider covered is GitHub Copilot. Both workflows use the short-lived ${{ github.token }} with copilot-requests: write, so they require no repository secret, API key, Foundry project, or Azure setup.

Example verification

live-examples.yml runs the existing cmd/verifyexamples command without a name filter. Examples for providers without configured credentials skip with their existing reasons.

Two fixed-input number-guessing examples now use their existing deterministic output checks instead of requiring Foundry semantic grading.

A validated run executed 24 available examples and skipped 83 unavailable examples with explicit reasons:
https://github.com/microsoft/agent-framework-go/actions/runs/33197392893

Copilot integration tests

Port all tests from the .NET Agent Framework GitHubCopilotAgentTests.cs into provider/copilotprovider/agent_e2e_test.go:

• nonstreaming response
• streaming response
• function-tool invocation
• approval-required function tool
• same-session context
• cross-client session resume
• shell command permission
• URL fetch permission
• local MCP filesystem server
• remote MCP server, explicitly skipped to match its upstream disabled category

These tests run in a dedicated Linux job in .github/workflows/test.yml. Ordinary test runs skip them. GitHub Actions runs them when GITHUB_TOKEN is available, and developers can opt in locally with RUN_COPILOT_INTEGRATION_TESTS=true.

The integration job does not run for fork pull requests because it executes checked-out code with Copilot access.

Authentication and safety

• Checkouts do not persist Git credentials.
• Copilot receives the job-scoped GitHub Actions token only in live test steps.
• Integration tests disable config discovery, file hooks, git operations, session storage, and skills.
• The normal test workflow's conclusion job includes the Copilot integration job.
• The examples workflow uploads reports even when verification fails.

Validation

• go test -count=1 ./...
• go vet ./cmd/verifyexamples ./provider/copilotprovider
• YAML parsing and git diff --check
• Standard Tests workflow, including Copilot integration: https://github.com/microsoft/agent-framework-go/actions/runs/33421954501

Nine active Copilot integration tests passed. The remote MCP case skipped as intended.

Upstream reference

• https://github.com/microsoft/agent-framework/blob/main/dotnet/tests/Microsoft.Agents.AI.GitHub.Copilot.IntegrationTests/GitHubCopilotAgentTests.cs

[Copilot]

Pull request overview

Adds a scheduled GitHub Actions workflow that runs a minimal GitHub Copilot-backed example end-to-end via the existing cmd/verifyexamples harness, to continuously validate the Copilot provider integration without external secrets or Azure/Foundry setup.

Changes:

• Add a new Copilot “function tool canary” example that exposes a fixed tool result and prompts Copilot to include it in the final response.
• Register the new example in cmd/verifyexamples with output assertions to validate the run.
• Add a new scheduled/manual GitHub Actions workflow to execute the example and upload logs/reports as artifacts.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

File	Description
examples/02-agents/providers/github-copilot/function_tool/main.go	New minimal Copilot-backed agent example with a single fixed function tool and locked-down session config.
examples/02-agents/providers/github-copilot/function_tool/main_test.go	Unit test to verify the function tool returns the expected canary string.
cmd/verifyexamples/examples.go	Adds the Copilot canary example to the 02-agents verifyexamples set with must-contain / must-not-contain assertions.
.github/workflows/live-examples.yml	New scheduled/manual workflow that installs Copilot CLI, runs verifyexamples for the canary, writes a job summary, and uploads artifacts.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Quim Muntal (qmuntal)]

Like the direction 😸

[github-actions]

API Consistency Review

Scope: internal-only (CI workflows and tests)

Changed Go contract: None — no exported Go identifiers were added, removed, or modified. The PR adds two GitHub Actions workflow files (.github/workflows/), refactors internal test-tool verification logic in cmd/verifyexamples/examples.go (unexported helpers), and adds a new integration test file provider/copilotprovider/agent_e2e_test.go (test package, not part of the public API surface).

Upstream evidence reviewed: The PR description cites dotnet/tests/Microsoft.Agents.AI.GitHub.Copilot.IntegrationTests/GitHubCopilotAgentTests.cs as the upstream reference. The three ported test cases (non-streaming response, streaming response, function-tool invocation) map directly to the upstream test class. No public API or behavioral contract is being introduced or changed; this is a test coverage port.

Result: Out of scope — the change is limited to CI configuration and tests with no user-visible Go API or behavior change. No parity issues identified.

Generated by Go API Consistency Review Agent for #931 · sonnet46 · 23.1 AIC · ⌖ 4.06 AIC · ⊞ 6.4K · ◷

[Quim Muntal (qmuntal)]

Why the changes in this file?

[Quim Muntal (qmuntal)]

Add a new line at the end of the file.

[Quim Muntal (qmuntal)]

Don't hardcode timeouts. The testing framework already cancels the t.Context() when the test should time out. Adding another source of cancellation only adds noise. Same for subsequent WithTimeout Calls.

[Quim Muntal (qmuntal)]

Can we make this version handled by dependabot?

[Quim Muntal (qmuntal)]

Can we make this version handled by dependabot?