mirror of
https://github.com/github/awesome-copilot.git
synced 2026-10-15 16:21:51 +00:00
fix(skills): clarify batch output constraints
This commit is contained in:
@@ -33,7 +33,7 @@ For an unknown total, report counts without a fabricated percentage, ETA, or rem
|
||||
|
||||
## 3. Choose output streams and cadence
|
||||
|
||||
Reserve stdout for the established machine-readable result, pipeline objects, or protocol. Put progress and diagnostics on a separate channel; stderr is the portable default for standalone CLI scripts. Stderr output alone does not imply failure: callers must use the agreed exit code and result contract. Verify callers that treat any stderr as failure before changing their output.
|
||||
Reserve stdout for the established machine-readable result, pipeline objects, or protocol. Put progress and diagnostics on a separate channel; stderr is the portable default for standalone CLI scripts. Stderr output alone does not imply failure: callers must use the agreed exit code and result contract. Verify callers that treat any stderr as failure before changing their output. If stdout is reserved and stderr is incompatible, use an existing compatible logger or sink only when the caller already supports it. Do not move progress to stdout or another stream with changed semantics. If no compatible channel exists, stop and ask whether the output contract may change, or report that progress cannot be added safely.
|
||||
|
||||
Use a bounded cadence, such as every configured number of items **or** elapsed interval, rather than unconditionally logging every item. Emit start, failures, and summary regardless of that cadence. Preserve required per-item audit records separately. Avoid unbounded in-memory failure lists and repeated stack traces. Redact credentials, query parameters, personal data, and sensitive paths before formatting; do not dump raw exceptions or whole input records by default.
|
||||
|
||||
@@ -45,7 +45,7 @@ Use monotonic time for intervals and duration. Ensure output is flushed or drain
|
||||
|
||||
Read [language notes and safe examples](references/language-notes.md) for the selected runtime. The examples are executable output-contract probes, not production processors; adapt their emitter and accounting to the existing code rather than copying their failure policy.
|
||||
|
||||
Keep reporting separate from processing decisions. In concurrent code, serialize reporting through the coordinator and count completed outcomes, not submissions. On handled failure or cancellation, attempt one final summary and preserve the original outcome; a summary-write error must not replace the original processing exception. Do not swallow cancellation as an ordinary item failure.
|
||||
Keep reporting separate from processing decisions. In concurrent code, serialize reporting through the coordinator and count completed outcomes, not submissions. On handled failure or cancellation, attempt one final summary and preserve the original outcome; a summary-write error must not replace the original processing exception. Do not swallow cancellation as an ordinary item failure. For host-managed cancellation, including PowerShell host stops, preserve the existing host behavior; do not convert stop exceptions into item failures or promise a final summary after host termination. Use `finally` for applicable cleanup, but verify the actual cancellation path in the supported host when graceful cancellation is part of the contract.
|
||||
|
||||
Do not force successful termination to deliver buffered output. In Node.js prefer setting `process.exitCode` and draining writes over an immediate `process.exit()`. Respect existing host/library semantics in PowerShell: a reusable function should not exit its caller. Forced termination, process crashes, and broken output pipes can prevent final reporting; do not claim otherwise.
|
||||
|
||||
@@ -53,7 +53,7 @@ Do not force successful termination to deliver buffered output. In Node.js prefe
|
||||
|
||||
Read [verification cases](references/verification.md), then run focused tests for the changed script with harmless fixtures. Capture stdout and stderr separately and check the real exit status. Verify start/progress ordering, monotonic counters, sanitized failures, exactly one summary on supported paths, and unchanged result contents and processing policy.
|
||||
|
||||
Prove liveness by reading the child process output **while it is still running**, not by examining output collected after exit. Check both terminal and redirected behavior when both are supported. A controlled pause after a progress event is a useful deterministic probe: receive the event, confirm the child has not exited, then release it. A watchdog may fail the test; it must not manufacture a successful result.
|
||||
Prove liveness by reading the child process output **while it is still running**, not by examining output collected after exit. Check both terminal and redirected behavior when both are supported. A controlled pause after a progress event is a useful deterministic probe: receive the event, confirm the child has not exited, then release it. Keep such gates in harmless test fixtures or the test harness; do not add stdin waits, sleeps, or cancellation hooks to normal production processing solely to make a test pass. Prefer an existing callback or injectable processing boundary. If no safe deterministic seam exists, test the real path only with harmless inputs or report liveness as unverified. A watchdog may fail the test; it must not manufacture a successful result.
|
||||
|
||||
The bundled [cross-language verifier](tests/verify-examples.mjs) runs the [Python](examples/batch.py), [PowerShell](examples/batch.ps1), and [Node.js](examples/batch.mjs) probes with normal, partial-failure, preserved partial-failure exit-zero, empty, unknown-total, fail-fast, simulated-cancellation, and live-output cases. It checks the probes only; it does not prove that an arbitrary generated script or automatic skill selection works.
|
||||
|
||||
|
||||
@@ -15,6 +15,7 @@ The [Python probe](../examples/batch.py) uses the standard library and never acc
|
||||
|
||||
- `Write-Output` writes to the success stream, so it can corrupt returned objects or JSON. Use the existing information/error streams when the host contract supports them; for a standalone native CLI, `[Console]::Error.WriteLine(...)` and `Flush()` provide explicit stderr output.
|
||||
- `Write-Progress` depends on the host and preferences. It may be invisible in CI or redirection; do not use it as the only evidence of progress.
|
||||
- Ctrl+C and host stop are host-managed, and behavior can differ by host and version. Preserve the existing stop behavior; do not catch broad exceptions and count host cancellation as an item failure. Use `finally` for cleanup where applicable, but do not promise a final summary after the host stops the pipeline. If graceful cancellation reporting is part of the contract, test the real cancellation path in each supported host; a simulated fixture does not establish it.
|
||||
- Use `[System.Diagnostics.Stopwatch]` for elapsed intervals. Avoid a blocking `Start-Sleep` heartbeat in the same processing loop.
|
||||
- Check terminating errors and non-terminating errors separately. Use `-ErrorAction Stop` at a specific cmdlet boundary when that matches the established policy; changing `$ErrorActionPreference` globally can change processing behavior.
|
||||
- Capture native `$LASTEXITCODE` immediately after the native command. On Windows PowerShell 5.1, do not assume a nonzero native exit is a catchable terminating exception.
|
||||
|
||||
@@ -9,9 +9,9 @@ Use harmless inputs and the actual caller contract. Fix expectations before runn
|
||||
- **Zero items:** zero counts, start and summary, no false progress or percentage arithmetic.
|
||||
- **Unknown total:** lazy input is consumed only once; total remains unknown; no invented percentage, remaining count, or ETA.
|
||||
- **Fail-fast:** the failing item is counted once; not-started items stay unprocessed; the first failure is not lost when reporting the summary.
|
||||
- **Cancellation:** verify the supported cancellation mechanism and cleanup; do not count incomplete work as successful. Simulated cancellation is not proof of SIGINT or host cancellation behavior.
|
||||
- **Long item:** distinguish between-item cadence from mid-item notification. Verify callbacks or heartbeats with the operation still active, or report the known notification gap.
|
||||
- **Redirection:** parse stdout using its real format, retain newline-delimited diagnostics separately, and read progress before child exit. Also verify a caller that interprets stderr as failure, if present.
|
||||
- **Cancellation:** verify the supported cancellation mechanism and cleanup; do not count incomplete work as successful. For PowerShell, preserve host-managed stop behavior and do not count a host stop as an item failure. Test Ctrl+C or another real host cancellation path only when graceful handling is part of the existing contract. Simulated cancellation is not proof of SIGINT or host cancellation behavior.
|
||||
- **Long item:** distinguish between-item cadence from mid-item notification. Verify callbacks or heartbeats with the operation still active, or report the known notification gap. Keep deterministic gates in harmless fixtures or the harness; do not add test-only waits or hooks to the production path.
|
||||
- **Redirection:** parse stdout using its real format, retain newline-delimited diagnostics separately, and read progress before child exit. Also verify a caller that interprets stderr as failure, if present. If stderr is incompatible, use only an already-supported logger or sink; if none exists, stop rather than changing stdout or another stream's semantics without approval.
|
||||
- **Secrets and volume:** a synthetic secret must not appear in either stream. Failure records must not require retaining every raw exception or input in memory.
|
||||
- **Retries, transactions, or concurrency, when present:** count terminal outcomes once; distinguish attempts, submissions, and committed results. Check a rollback before accepting a success summary.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user