GH-300 sample questions with answers

1. A.

   Search the dependency's current documentation, since a client library's retry behaviour is settled early and rarely changes between releases, so the latest description will apply to the pinned release too.

2. B.

   Check the documentation for the pinned release, and confirm the behaviour with a small test against the version the project actually resolves.

3. C.

   Ask Copilot Chat to cite the source of its claim and accept the answer if a plausible link is produced.

4. D.

   Raise an issue with the dependency's maintainers asking them to confirm the behaviour before the change is merged.

1. A.

   Disable Copilot for the repositories with the largest pull requests until review throughput recovers to the level it was at before the rollout.

2. B.

   Constrain the size of changes so review stays meaningful, because the harm is review capacity being outpaced rather than the tool itself.

3. C.

   Add more reviewers to each pull request so that the increased volume of generated code is spread across a larger group and no single reviewer is overwhelmed by it.

4. D.

   Set a target for review turnaround time so the team can show that quality has not fallen.

1. A.

   A defect in the model, which should be reported so the vendor can retrain it before the code is used.

2. B.

   Bias present in the material the model learned from can be reproduced in generated logic, so rules that classify people's data need deliberate review against the organisation's own requirements.

3. C.

   A context problem, fixed by opening the project's validation tests before asking again.

4. D.

   A filtering problem, fixed by enabling the public code matching filter for the repository.

1. A.

   Record the incident against the tool's vendor so a pattern can be established across future reports.

2. B.

   Keep Copilot but require that every suggestion longer than five lines is rewritten by hand before it is committed.

3. C.

   Handle it as an ordinary engineering incident: establish which control failed to catch the defect, and fix that control.

4. D.

   Withdraw Copilot from the team involved until the incident review has finished and a written assurance has been given that the same class of defect cannot occur again in their repositories.

1. A.

   The disputed line will have been reformatted by the repository's linter, so the original no longer exists.

2. B.

   Generation is probabilistic, so the same request need not produce the same suggestion again, and no regeneration can evidence what was originally offered.

3. C.

   Regeneration would work, but only a repository administrator holds the permission needed to replay another developer's suggestion, so the request has to be routed through them instead.

4. D.

   Copilot never produces the same suggestion twice by design, so regeneration always returns something different.

1. A.

   Stop Copilot operating on that repository through an administrator-configured control, rather than relying on each consultant to remember.

2. B.

   Allow the consultants to continue, provided each of them reviews every suggestion carefully and records in their timesheet which client files they had open while Copilot was active.

3. C.

   Allow it, because a contract restricting third-party processing does not cover a coding assistant that only returns suggestions.

4. D.

   Ask each consultant to close client files before invoking Copilot and to reopen them afterwards.

1. A.

   An automatic block on the file type the suggestion related to.

2. B.

   A monthly survey asking developers how satisfied they are with suggestion quality.

3. C.

   A rule that the developer stops using Copilot until the vendor confirms a fix.

4. D.

   A route for reporting a bad suggestion, a record of what was reported, and a decision about whether guidance or configuration should change as a result.

Choose 2.

1. A.

   Require a human reviewer to read the generated change with the same care as hand-written code.

2. B.

   Ask Copilot Chat to confirm that the suggestion it has just produced contains no vulnerabilities.

3. C.

   Enable the public code matching filter, which withholds suggestions matching publicly available code.

4. D.

   Run the organisation's existing security scanning over the branch before it can be merged.

5. E.

   Rely on Copilot's content filtering, which is described as blocking insecure code before display.

1. A.

   The tool's behaviour is not fixed, so an assessment of its output needs repeating rather than being treated as a permanent finding.

2. B.

   Nothing is wrong with it, provided the organisation recorded which developers took part and how long the pilot ran.

3. C.

   The assessment is invalid because a pilot measures acceptance rate, which is the only meaningful measure of a coding assistant.

4. D.

   A pilot cannot produce a valid assessment at all, because the developers who volunteer for one are more capable than the organisation's average and their results never generalise to anyone else.

1. A.

   Let the team enable the feature for its own repositories, since the team understands its own risk better than a central function does and can reverse the change if it causes a problem.

2. B.

   Decide it through the documented exception route, so the reason and the decision are recorded centrally.

3. C.

   Enable it organisation-wide, since a feature one team needs is likely to be useful to the others.

4. D.

   Decline it, because a policy that admits exceptions cannot be enforced consistently across an organisation.