On October 30, 2026, Microsoft Research described a platform that builds synthetic enterprise workspaces and industry-specific test queries for evaluating Microsoft Copilot. Microsoft reported a 39% improvement in its composite synthetic-data quality metric compared with its initial generation approach. That figure describes the quality of the test environments; a separate Microsoft-documented workflow evaluates agent responses.
What Microsoft’s synthetic evaluation platform does
The Industry Evaluation Data Generation Platform brings together synthetic enterprise environments, automated query-generation pipelines and structured quality frameworks. Microsoft says the simulated workspaces include users, meetings, emails, documents, organizational structures and evolving work histories. It describes environments for pharmaceuticals, capital markets, and IT and software, with work artifacts and workflows tailored to each field. Microsoft says no real customer data enters the pipeline. Microsoft Research describes the platform and its design.
The goal is to give Copilot-related evaluations repeatable workplace settings and questions. Microsoft says it placed the resulting platform in an internal shared catalog for teams to reuse.
How Microsoft turns work tasks into test queries
Microsoft names O*NET occupational data, online job postings and SEC filings as grounding inputs for its synthetic environments. These sources inform simulated roles, work activities, organizational structures and business contexts.
The query pipeline starts with general workplace-task templates. Microsoft says it filters and adapts tasks for a specific industry and its synthetic data, generates queries grounded in that setting, rewrites them to sound natural, then checks them for accuracy and uniqueness. The sequence turns a general task, such as analyzing information, into a question shaped by a particular workplace context.
What Microsoft’s reported results measure
Microsoft reported a 39% improvement in a composite measure of synthetic-data quality over its initial generation approach. It also said some industry-specific environments scored above 0.9 out of 1.0. The framework lists structural fidelity, contextual relevance, behavioral realism and content safety as quality criteria.
The results concern the generated environments and evaluation process. Microsoft also reported hundreds of users in each environment and a shift from multi-day manual dataset creation to automated runs. Its estimates put savings at about $100,000 per evaluation dataset and $300,000 across the project.
A separate way to evaluate agent responses
Microsoft Learn documents repeatable, scenario-based response tests for Employee Self-Service agents in Copilot Studio. The test sets use different user profiles; people with edit access can run them, and results can be exported for people without Copilot Studio access. The workflow assesses agent responses in that Employee Self-Service context. Microsoft Learn’s evaluation documentation describes this response-testing approach separately from the synthetic-data platform.
