Add multi-grader support for reinforcement fine-tuning#6765
Add multi-grader support for reinforcement fine-tuning#6765saanikaguptamicrosoft merged 10 commits intomainfrom
Conversation
Co-authored-by: saanikaguptamicrosoft <[email protected]>
Co-authored-by: saanikaguptamicrosoft <[email protected]>
Co-authored-by: saanikaguptamicrosoft <[email protected]>
There was a problem hiding this comment.
Pull request overview
Adds support for multi-grader configurations in the Azure AI fine-tuning extension’s reinforcement fine-tuning (RFT) job creation flow, working around an OpenAI SDK v3.2.0 type mismatch by sending multi-grader payloads via extraBody.
Changes:
- Detect
grader.type: multifor RFT and route grader submission throughextraBodyinstead of the SDK-typed params. - Add helper builders to construct the multi-grader JSON payload (including sub-graders).
- Add unit tests covering existing grader conversions and the new multi-grader
extraBodybehavior.
Reviewed changes
Copilot reviewed 5 out of 6 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| cli/azd/extensions/azure.ai.finetune/internal/providers/openai/conversions.go | Adds multi-grader detection and JSON construction helpers; routes multi-grader via ExtraBody. |
| cli/azd/extensions/azure.ai.finetune/internal/providers/openai/conversions_test.go | Adds unit tests for grader conversion and RFT multi-grader extraBody serialization. |
| cli/azd/go.mod | Promotes google.golang.org/genproto/googleapis/rpc to a direct dependency. |
| cli/azd/extensions/azure.ai.agents/go.mod | Promotes github.com/stretchr/testify to a direct dependency. |
| cli/azd/extensions/microsoft.azd.concurx/go.mod | Updates indirect dependencies (adds survey/v2, removes keyboard). |
| cli/azd/extensions/microsoft.azd.concurx/go.sum | Updates sums consistent with the module dependency changes. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
cli/azd/extensions/azure.ai.finetune/internal/providers/openai/conversions.go
Outdated
Show resolved
Hide resolved
cli/azd/extensions/azure.ai.finetune/internal/providers/openai/conversions.go
Outdated
Show resolved
Hide resolved
Azure Dev CLI Install InstructionsInstall scriptsMacOS/Linux
bash: pwsh: WindowsPowerShell install MSI install Standalone Binary
MSI
Documentationlearn.microsoft.com documentationtitle: Azure Developer CLI reference
|
…serialized method fields
ConvertGraderMapToSDKParamdid not supportmultigrader type for reinforcement fine-tuning. Multi-graders combine multiple graders using a formula (e.g.,0.9 * exact + 0.1 * fuzzy).Implementation
buildMultiGraderData- Constructs multi-grader JSON with nested sub-graders mapbuildGraderData- Builds individual grader structures for all sub-grader types:string_check,text_similarity,python,score_model,label_modelconvertInternalJobParamToOpenAiJobParams- Detects multi-grader and routes toextraBody(SDK v3.2.0 typesGradersas single union instead ofmap[string]MultiGraderGradersUnionParam)ConvertGraderMapToSDKParam- Returns empty union for multi-grader since it's handled via extraBodyExample
Technical Note
OpenAI SDK v3.2.0 defines
MultiGraderParam.Gradersas singularMultiGraderGradersUnionParam, but the API expectsmap[string]MultiGraderGradersUnionParam. UsingextraBodybypasses this type mismatch.Warning
Firewall rules blocked me from connecting to one or more addresses (expand for details)
I tried to connect to the following addresses, but was blocked by firewall rules:
platform.openai.com/home/REDACTED/work/_temp/ghcca-node/node/bin/node /home/REDACTED/work/_temp/ghcca-node/node/bin/node --enable-source-maps /home/REDACTED/work/_temp/copilot-developer-action-main/dist/index.js(dns block)If you need me to access, download, or install something from one of these locations, you can either:
Original prompt
✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.