Track reviewed HD families and guard catalog registration
This commit is contained in:
raptorreplica committed
1 parent
88fa7b570c
commit
cd493c4eb6
9 files changed
+3986
-1436
No files matched your search
@@ -2,41 +2,56 @@
|
||||
|
||||
The Rust Options screen includes **HD art**. It starts disabled. Enable it to use reviewed artwork from `../data/hd/hd.json`; each missing or rejected frame keeps the selected original / 2x / 3x / 4x crisp treatment. The scale setting still determines the indexed rendering resolution, and the HD toggle can be changed while paused. HD art is presentation only: simulation, collision sizes, banking positions and save content IDs stay unchanged.
|
||||
|
||||
The local base-game pack currently supplies **63 reviewed replacement frames**. Complete additions cover all seven light and dark player banking poses, two first-mission fighters, its industrial boss, three ground-defense targets and a five-frame opening bunker, five equipment/pickup definitions, the two-frame Plasma projectile, the two-frame enemy cannon pulse, the twelve-frame energy ring and sixteen canonical terrain tiles. The firing-flash poses still use original artwork. The complete catalog backlog and generation status are listed in [hd-backlog.md](hd-backlog.md).
|
||||
The local base-game pack currently supplies **97 reviewed replacement frames**. Complete additions cover all twenty-one light, dark and firing-flash player poses, four fighters, four industrial bosses, four ground-defense targets and a five-frame opening bunker, fifteen equipment/pickup families, five complete projectile families, the twelve-frame energy ring and sixteen canonical terrain tiles. The complete catalog backlog and generation status are listed in [hd-backlog.md](hd-backlog.md).
|
||||
|
||||
| Reviewed artwork | Original asset IDs | Supplied / complete frames |
|
||||
| --- | --- | ---: |
|
||||
| Light player banking | `0x1011a`–`0x10120` | 7 / 7 |
|
||||
| Dark player banking | `0x20045`–`0x2004b` | 7 / 7 |
|
||||
| Firing-flash player banking | `0x2004c`–`0x20052` | 7 / 7 |
|
||||
| First-mission fighter `SHIP01G1_PIC` | `0x102f8` | 1 / 1 |
|
||||
| First-mission fighter `SHIP27G1_PIC` | `0x10330` | 1 / 1 |
|
||||
| Fighters `SHIP02G1_PIC`, `SHIP14G1_PIC` | `0x102f9`, `0x1030a` | 2 / 2 |
|
||||
| First-mission industrial boss `SHIP10G1_PIC` | `0x10306` | 1 / 1 |
|
||||
| Industrial bosses `SHIP12G1_PIC`, `SHIP17G1_PIC` | `0x10308`, `0x1030f` | 2 / 2 |
|
||||
| Industrial boss `SHIP11G1_PIC` | `0x10307` | 1 / 1 |
|
||||
| Ground-defense target `TARGT1G1_PIC` | `0x10138` | 1 / 1 |
|
||||
| Ground-defense targets `TARGT2G1_PIC`, `TARGT3G1_PIC` | `0x10139`, `0x1013a` | 2 / 2 |
|
||||
| Ground-defense target `TARGT5G1_PIC` | `0x1013c` | 1 / 1 |
|
||||
| Opening ground-defense bunker `TARGT6G1_PIC` | `0x1013d`–`0x10141` | 5 / 5 |
|
||||
| Forward Guns equipment | `0x102b0` | 1 / 1 |
|
||||
| Plasma Guns equipment | `0x102b1`–`0x102b2` | 2 / 2 |
|
||||
| Dumb Missile equipment | `0x102b5` | 1 / 1 |
|
||||
| Air Missile equipment | `0x102bf` | 1 / 1 |
|
||||
| Micro Missile equipment | `0x102b3`–`0x102b4` | 2 / 2 |
|
||||
| MiniGun equipment rotation | `0x102b6`–`0x102b9` | 4 / 4 |
|
||||
| Missile Pods equipment | `0x102be` | 1 / 1 |
|
||||
| Mega Bomb equipment | `0x102c5` | 1 / 1 |
|
||||
| Super Shield pickup | `0x102d0` | 1 / 1 |
|
||||
| Detector and additional shop/pickup images | `0x102d5`–`0x102da` | 6 / 6 |
|
||||
| Plasma projectile | `0x101cf`–`0x101d0` | 2 / 2 |
|
||||
| Enemy cannon pulse | `0x101d6`–`0x101d7` | 2 / 2 |
|
||||
| Dumb Missile projectile | `0x101ba`–`0x101bc` | 3 / 3 |
|
||||
| Missile Pods / Air Missile 2 projectile | `0x101bd`–`0x101be` | 2 / 2 |
|
||||
| Ground Missile 2 projectile | `0x101bf`–`0x101c0` | 2 / 2 |
|
||||
| Energy ring | `0x101ae`–`0x101b9` | 12 / 12 |
|
||||
| Canonical terrain set 1, first shard | `0x10360`–`0x1036f` | 16 / 16 |
|
||||
|
||||
Every registered frame passed the exact original source hash, logical canvas, aspect, visible alpha and path checks. Complete contact sheets were inspected at source size and the 4× presentation size. Fighter/boss silhouettes have approximately 84–93% overlap with their source masks at logical resolution; their logical collision sizes are unchanged. Small contours and interpreted metal detail differ. Missing artwork keeps the crisp original fallback. HD remains an opt-in preview while the dark and firing player families and game screenshots are reviewed.
|
||||
Every registered frame passed the exact original source hash, logical canvas, aspect, visible alpha and path checks. Complete contact sheets were inspected at source size and the 4× presentation size; per-frame geometry metrics and review evidence remain with each family. Their logical collision sizes are unchanged. Small contours and interpreted metal detail differ. Missing artwork keeps the crisp original fallback. HD remains an opt-in preview while the remaining catalog and game screenshots are reviewed.
|
||||
|
||||
An actual Rust `HdAssets::sprite` pass also checked all registered frames at scales 1–4 with the game palette: every output has the expected dimensions and visible pixels, and the pack reports no diagnostics. Complete 4× source/HD sheets from that runtime path were reviewed for readable projectiles, equipment alignment, banking progression and energy-ring timing stages. These sprite checks complement live gameplay captures and do not establish full-game visual completion.
|
||||
|
||||
The complete light banking sequence uses matching silver/gunmetal materials, dark red accents and the seven distinct original roll poses. Its safe square crop rectangles remove transparent canvas margin without excluding any pixel with alpha at least 32. Independent Rust `image::Lanczos3` checks use the runtime's two resampling stages, first to the resident 128×128 canvas and then to the original 32×32 logical size. All seven frames have source silhouette overlap of 87.7–92.3%, bounds within one logical pixel and centroid displacement below 0.74 logical pixels. Full-sequence visual review also checked the nose, canopies, wing ends and engine exits; numeric overlap alone does not approve a frame.
|
||||
|
||||
Equipment visible material masks overlap their originals by approximately 93–100%, and both Plasma icon frames share the same aligned alpha footprint. Full alpha-mask overlap is lower, approximately 54–78%, because nearly black original opaque outlines become transparent around the detailed replacement. This difference is recorded separately from the visible metal silhouette. The ground-defense original and its replacement are both fully opaque, which the source-aware loader now accepts.
|
||||
The first four equipment families' visible material masks overlap their originals by approximately 93–100%, and both Plasma icon frames share the same aligned alpha footprint. Their full alpha-mask overlap is lower, approximately 54–78%, because nearly black original opaque outlines become transparent around the detailed replacement. This difference is recorded separately from the visible metal silhouette. The ten newly added equipment/pickup frames have full logical-mask overlap of approximately 87.5–99.3%. Ground-defense originals and their replacements are fully opaque, which the source-aware loader accepts.
|
||||
|
||||
Small-scale differences remain visible. At 1×, each Plasma projectile loses one tail row: the original 8/6-pixel heights become 7/5 pixels, while 4× retains the original logical extents. The first energy effect's fine cardinal rays also contract under binary alpha at small scales; its 1× bounds become `(7,5)..(34,34)` against original `(3,2)..(38,37)`. At 4× the rays remain visible but end approximately 1–2 logical pixels inside the source extremes. Later rings use smoother, thinner contours. These differences affect presentation and are follow-up candidates for per-asset fallback; source collision dimensions remain unchanged.
|
||||
|
||||
Generated drafts are not active coverage. Earlier player atlas attempts and shortened extreme-bank variants were rejected. The complete light family replaces the initial straight-frame proof; the dark family uses coherent lighting edits of those reviewed poses and the original dark palette. Its exact runtime logical-mask overlap is 87.3–91.4%, with centroid displacement below 0.68 pixels, bounds within one pixel and no clipped meaningful alpha. Other equipment families remain uncovered. A 3×3 shoreline prototype preserves the broad coastline and demonstrates the terrain renderer, but remains unregistered: repeated canonical tiles, neighbors and destroyed states need individual artwork and seam review. Original AGX cinematics retain their original artwork.
|
||||
|
||||
The first sixteen canonical terrain replacements passed independent review of every source/logical/4× frame, all 48 supplied mission contexts and all 755 declared cardinal neighbor pairs: 170 HD/HD and 585 HD/original. Actual native scene checks exercised all 32 cached scroll phases and content-hash/load-order invalidation. None of these sixteen IDs has a declared destroyed partner; the separate native destruction-cache proof therefore adds no HD debris coverage. Mixed detail contrast remains until neighboring tiles are remastered. The next terrain family remains held because tile `0x20111` invented a canopy on an originally treeless grass/dirt tile.
|
||||
The firing family preserves all seven original poses and their localized amber reflections. Its distinct straight-frame contour points were checked explicitly: `(19,8)` remains opaque, `(11,12)` and `(11,13)` remain transparent, and `(11,14)` stays opaque. The MiniGun's four rotation phases and the seven newly registered missile phases passed complete sequence inspection; each missile retains its original exhaust length and fin layout. Native scene captures remain a separate reachability check. The additional `SHIP11G1_PIC` boss interprets its dark perimeter: logical alpha overlap is 82.9%, with 373 removed original occupied pixels (252 very dark) and 23 added pixels. Its towers, side machinery, center weapon and lower pods remain readable at every scale, but its alpha contour is not identical to the source.
|
||||
|
||||
The first sixteen canonical terrain replacements passed independent review of every source/logical/4× frame, all 48 supplied mission contexts and all 755 declared cardinal neighbor pairs: 170 HD/HD and 585 HD/original. Actual native scene checks exercised all 32 cached scroll phases and content-hash/load-order invalidation. None of these sixteen IDs has a declared destroyed partner; the separate native destruction-cache proof therefore adds no HD debris coverage. Mixed detail contrast remains until neighboring tiles are remastered. The next terrain family has a corrected treeless `0x20111` and awaits renewed context review. Industrial terrain candidates retain source geometry but remain under review for recurring grain lines at some tile boundaries; they add no active coverage yet.
|
||||
|
||||
Original pixels and original-derived base-game artwork remain outside Git under `../data/hd/`. The separate authored mod packages include their generated SD/HD PNGs, sounds and provenance in Git. The new authored menu backdrop has its own packaged rendering path and is not counted as an original catalog sprite replacement.
|
||||
|
||||
@@ -87,7 +102,7 @@ The original picture header's two fields before width/height are `opt1` and `opt
|
||||
|
||||
The streaming runtime source-checks the complete manifest and validates each original generator-sized PNG/crop when used. It stores no more than the original logical canvas at 4× resolution, the highest supported presentation scale. Raw generated PNGs and provenance remain intact on disk. Three 32 MiB LRU caches hold decoded atlases, derived RGBA frames and palette-bound indexed sprites; eviction never removes a registered frame. A separate derived disk cache is capped at 512 MiB. The full catalog is no longer subject to a 128 MiB artwork admission limit. See [hd-streaming.md](hd-streaming.md) for validation ceilings, cache identities and corruption handling.
|
||||
|
||||
Original source locations also constrain palette classes: ordinary art uses indices 0–239, water stays in 240–244, fire in 245–249, and original blink/flash labels 250–255 remain exact. New HD coverage over an originally transparent location stays ordinary. Compact source-label metadata has a separate 32 MiB ceiling and is omitted for wholly ordinary sources. This prevents static metal or equipment highlights from acquiring fire cycles or blinking merely because a generated RGB value matches an animated palette entry. An actual all-scale recheck of the 63-frame pack confirmed every wholly ordinary source and output remains outside indices 240–255.
|
||||
Original source locations also constrain palette classes: ordinary art uses indices 0–239, water stays in 240–244, fire in 245–249, and original blink/flash labels 250–255 remain exact. New HD coverage over an originally transparent location stays ordinary. Compact source-label metadata has a separate 32 MiB ceiling and is omitted for wholly ordinary sources. This prevents static metal or equipment highlights from acquiring fire cycles or blinking merely because a generated RGB value matches an animated palette entry. The accepted frames passed all-scale checks of these reserved-index counts. Static cinematic cards use a separate palette mode with all 256 colors; source references bind Apogee, Cygnus and Taiwan cards to their authentic palettes. These cards still await authored HD artwork and are not counted as remasters.
|
||||
|
||||
HD frames are resampled directly to the selected presentation canvas and quantized to the game palette. They bypass ScaleNx filtering, preserve transparent pixels and use the normal layer order. Indexed output allows palette effects to apply consistently; subtle RGB shades and alpha edges are reduced by the original palette and binary sprite transparency. The validated [terrain renderer](hd-terrain.md) overlays canonical, full-opaque HD tiles on the joined original layer. It shares HD edges and corners where source material classes agree, transitions HD/original boundaries over two logical pixels, and preserves water/fire/flash palette classes. Intact and destroyed IDs are independent targets. The renderer adds no remaster coverage by itself; each terrain family needs the same contextual review before registration.
|
||||
|
||||
@@ -95,7 +110,9 @@ The metadata-only [completion ledger](hd-completion.json) tracks every one of th
|
||||
|
||||
Workers can validate an isolated proposal with `python3 tools/hd_candidate_review.py --candidate ../data/hd/jobs/JOB_ID/candidate.json`. The helper hard-links raw artwork into a temporary pack, links against the newest built Rust game library, and exports all scales 1–4 plus source/logical/4× contact sheets. It records raw artwork hashes and the library identity, warns when asset-loader source is newer than the build, and pages large reviews in groups of sixteen frames to avoid an unbounded giant image. Every page listed in `runtime-contact-sheets.json` requires inspection. It never edits the active pack or generated pixels. A successful gallery still needs coordinator visual approval and separate scene/context evidence.
|
||||
|
||||
Generation workers share a four-submission gate through `tools/hd_generation_gate.py`. They acquire before a new built-in request, report its actual outcome afterward, and prepare or review other jobs while all slots are reserved. Thirty-minute leases can be renewed; stale reservations expire after a two-minute grace period without interrupting a running tool. Connection failures retain their exact sanitized text and UTC timestamp, trigger a 60–600-second exponential cooldown, and permit at most three connection failures per job in a thirty-minute retry batch. Existing in-flight calls are allowed to finish. Logs and gate state stay under ignored `data/hd/jobs/.imagegen-gate/`.
|
||||
After explicit artistic and independent approval, the coordinator can run `python3 tools/hd_register_reviewed.py --job JOB_ID --dry-run`, then omit `--dry-run` to register the reviewed family. The command checks every original target, complete sequences, the exact approved frame bindings, all four runtime scales, the reviewed library and unchanged PNG hashes. Held artwork, stale evidence and duplicate registered IDs are rejected. It writes the complete manifest atomically and preserves provenance; registration does not claim a native scene occurrence or finish the catalog backlog.
|
||||
|
||||
Generation workers share a four-submission gate through `tools/hd_generation_gate.py`. They prepare references and save prompts before acquiring. One orchestration acquires, marks the request started, awaits built-in imagegen and reports its actual outcome immediately before returning to the model or doing local review. Started and legacy-unknown requests require explicit completion; only unstarted reservations expire. Status shows phases, ages and admissions, and supports owner recovery without cancelling tools. Connection failures retain exact sanitized text and UTC timestamps, impose a 60–600-second cooldown, and permit at most three connection failures per job in a thirty-minute retry batch. See the [mandatory workflow and wrapper](hd-generation-workflow.md). Logs and gate state remain ignored.
|
||||
|
||||
## Source and sequence review
|
||||
|
||||
|
||||
@@ -1,15 +1,15 @@
|
||||
# Original artwork HD backlog
|
||||
|
||||
The checked catalog contains **5,753 standalone picture frames**. The active local HD pack supplies **63**; **5,690** have no registered HD replacement. These counts include unnamed graphics outside `sprites/`. Scale2x/3x/4x filtering is crisp scaling, not a newly authored HD replacement. Authored mod SD/HD art is separate from this original-catalog audit.
|
||||
The checked catalog contains **5,753 standalone picture frames**. The active local HD pack supplies **97**; **5,656** have no registered HD replacement. These counts include unnamed graphics outside `sprites/`. Scale2x/3x/4x filtering is crisp scaling, not a newly authored HD replacement. Authored mod SD/HD art is separate from this original-catalog audit.
|
||||
|
||||
| Category | Picture frames | Registered HD | Missing HD | Declared sequences / terrain sets |
|
||||
| --- | ---: | ---: | ---: | ---: |
|
||||
| Player banking | 21 | 14 | 7 | 3 |
|
||||
| Bosses | 249 | 1 | 248 | 79 |
|
||||
| Enemies and ground objects | 419 | 10 | 409 | 183 |
|
||||
| Player and enemy projectiles | 51 | 4 | 47 | 18 |
|
||||
| Player banking | 21 | 21 | 0 | 3 |
|
||||
| Bosses | 249 | 4 | 245 | 79 |
|
||||
| Enemies and ground objects | 419 | 13 | 406 | 183 |
|
||||
| Player and enemy projectiles | 51 | 11 | 40 | 18 |
|
||||
| Explosions and effects | 281 | 12 | 269 | 25 |
|
||||
| Equipment and pickups | 47 | 6 | 41 | 23 |
|
||||
| Equipment and pickups | 47 | 20 | 27 | 23 |
|
||||
| Terrain | 4,535 | 16 | 4,519 | 4 |
|
||||
| Menus, portraits, HUD and other pictures | 150 | 0 | 150 | mixed static / animated |
|
||||
|
||||
@@ -52,6 +52,26 @@ Only registered frames passing canvas, source-hash, PNG and alpha checks count a
|
||||
| Terrain 1036e | `0x1036e` | 1 / 1 |
|
||||
| Terrain 1036f | `0x1036f` | 1 / 1 |
|
||||
| Super Shield pickup | `0x102d0` | 1 / 1 |
|
||||
| Micro Missile | `0x102b3` | 2 / 2 |
|
||||
| Missile Pods | `0x102be` | 1 / 1 |
|
||||
| Mega Bomb | `0x102c5` | 1 / 1 |
|
||||
| Detect / Itembuy1 | `0x102d5` | 1 / 1 |
|
||||
| Itembuy2 | `0x102d6` | 1 / 1 |
|
||||
| Itembuy3 | `0x102d7` | 1 / 1 |
|
||||
| Itembuy4 | `0x102d8` | 1 / 1 |
|
||||
| Itembuy5 | `0x102d9` | 1 / 1 |
|
||||
| Bomb | `0x102da` | 1 / 1 |
|
||||
| SHIP14G1_PIC | `0x1030a` | 1 / 1 |
|
||||
| SHIP02G1_PIC | `0x102f9` | 1 / 1 |
|
||||
| SHIP12G1_PIC | `0x10308` | 1 / 1 |
|
||||
| SHIP17G1_PIC | `0x1030f` | 1 / 1 |
|
||||
| TARGT5G1_PIC | `0x1013c` | 1 / 1 |
|
||||
| Mini Gun | `0x102b6` | 4 / 4 |
|
||||
| Player firing-flash banking | `0x2004c` | 7 / 7 |
|
||||
| Dumb Missile | `0x101ba` | 3 / 3 |
|
||||
| Missile Pods / Air Missile | `0x101bd` | 2 / 2 |
|
||||
| Grd Missile | `0x101bf` | 2 / 2 |
|
||||
| SHIP11G1_PIC source-faithful HD candidate | `0x10307` | 1 / 1 |
|
||||
|
||||
| Generation job | Status |
|
||||
| --- | --- |
|
||||
|
||||
+3663
-1412
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,70 @@
|
||||
# Shared HD generation workflow
|
||||
|
||||
Prepare and inspect every source reference, check the palette binding, and save the complete prompt before acquiring a slot. Acquire only when ready to submit. A slot must never cover local preparation, a model turn, artwork copying or visual review.
|
||||
|
||||
Use **one `functions.exec` orchestration** for acquire → start → built-in imagegen → outcome report. Submit only when acquisition succeeds and `start.changed` is `true`. Run the report immediately after the awaited tool returns or throws, before returning the result to the model. Then copy the saved raw PNG unchanged and perform artistic/runtime review. Successful generation does not mean accepted artwork.
|
||||
|
||||
Observed built-in successes have `image_url` and `output_hint` keys. `image_url` contains a PNG data URL; never print or serialize it to a log. `output_hint` identifies the saved raw PNG. Observed connection failures throw `image generation failed: connection failed: error sending request`. Also check returned `isError`, textual error fields and unexpected result shapes; a nonthrowing failure is still a failure.
|
||||
|
||||
The following skeleton runs inside `functions.exec`. `preparedArguments`, `owner`, `job`, `gatePath`, `projectRoot` and `errorFile` must already be known. Shell arguments are quoted individually; the error file is written without evaluating its contents.
|
||||
|
||||
```javascript
|
||||
const quote = s => "'" + String(s).replaceAll("'", "'\"'\"'") + "'";
|
||||
async function gate(...args) {
|
||||
const r = await tools.exec_command({
|
||||
cmd: ["python3", gatePath, ...args].map(quote).join(" "),
|
||||
workdir: projectRoot, max_output_tokens: 800
|
||||
});
|
||||
if (r.exit_code !== 0) throw new Error("Generation gate command failed");
|
||||
return JSON.parse(r.output);
|
||||
}
|
||||
const lease = await gate("acquire", "--owner", owner, "--job", job);
|
||||
if (!lease.acquired) { text(lease); exit(); }
|
||||
const started = await gate("start", "--owner", owner, "--lease", lease.lease_id);
|
||||
if (!started.changed) {
|
||||
if (["admissions_paused", "active_request_accounting_over_capacity"].includes(started.reason)) {
|
||||
// This newly acquired reservation has not submitted a request.
|
||||
text(await gate("release", "--owner", owner, "--lease", lease.lease_id));
|
||||
}
|
||||
text(started); exit();
|
||||
}
|
||||
let result, outcome = "failure", errorText = "";
|
||||
try {
|
||||
result = await tools.image_gen__imagegen(preparedArguments);
|
||||
const returnedError = result?.error?.message || result?.error || result?.message;
|
||||
if (result?.isError || returnedError) {
|
||||
errorText = typeof returnedError === "string" ? returnedError :
|
||||
result?.content?.filter(x => x.type === "text").map(x => x.text).join("\n") ||
|
||||
"Built-in imagegen returned an error";
|
||||
} else if (typeof result?.image_url === "string" &&
|
||||
result.image_url.startsWith("data:image/") &&
|
||||
typeof result?.output_hint === "string") {
|
||||
outcome = "success";
|
||||
} else {
|
||||
errorText = "Unexpected built-in imagegen result shape";
|
||||
}
|
||||
} catch (error) {
|
||||
errorText = String(error?.message || error);
|
||||
} finally {
|
||||
if (/connection failed|error sending request/i.test(errorText)) outcome = "connection_failure";
|
||||
const reportArgs = ["report", "--owner", owner, "--lease", lease.lease_id,
|
||||
"--outcome", outcome];
|
||||
if (errorText) {
|
||||
const code = "from pathlib import Path; import sys; Path(sys.argv[1]).write_text(sys.argv[2])";
|
||||
const written = await tools.exec_command({
|
||||
cmd: ["python3", "-c", code, errorFile, errorText].map(quote).join(" "),
|
||||
workdir: projectRoot, max_output_tokens: 200
|
||||
});
|
||||
if (written.exit_code !== 0) throw new Error("Could not save imagegen error; retain lease for recovery");
|
||||
reportArgs.push("--error-file", errorFile);
|
||||
}
|
||||
text(await gate(...reportArgs));
|
||||
}
|
||||
if (outcome === "success") generatedImage(result);
|
||||
```
|
||||
|
||||
The four-slot gate retains started requests and old requests with unknown status until explicit completion or recovery. Only unstarted reservations expire after thirty minutes plus a two-minute grace period. `start` refuses duplicate starts, unknown legacy requests, paused admissions and over-capacity accounting. A restored unknown request is immediately flagged for owner attention. No gate command starts, stops or terminates an imagegen tool.
|
||||
|
||||
After interruption, recover the original yielded cell or saved result before any new submission. Report a recovered completion, or release a reservation only after confirming no request was submitted/running. Do not infer completion from age. `status` reads lease phases, ages, admission state and recent outcomes without changing reservations. Root can pause new admissions while reconciling unknown calls; existing calls may still finish and report. Eligibility currently uses the first available slot with one request per owner, without a FIFO queue.
|
||||
|
||||
Exact sanitized service errors and UTC report times stay in ignored `data/hd/jobs/.imagegen-gate/events.jsonl`. Connection failures impose a 60–600-second cooldown and at most three failures per job in a thirty-minute retry batch. During denial or cooldown, prepare or review another job. Do not replace a failed remaster with scaled original art.
|
||||
@@ -62,6 +62,11 @@ def proposal_frames(proposal, expected=None):
|
||||
return rows
|
||||
|
||||
|
||||
def binding_digest(rows):
|
||||
"""Hash only renderer-bound fields, separately from artist review notes."""
|
||||
return hashlib.sha256(json.dumps(rows, sort_keys=True, separators=(",", ":")).encode()).hexdigest()
|
||||
|
||||
|
||||
def main():
|
||||
project = Path(__file__).resolve().parents[1]
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
@@ -73,7 +78,8 @@ def main():
|
||||
args = parser.parse_args()
|
||||
if not 1 <= args.sheet_rows <= 64:
|
||||
parser.error("sheet-rows must be 1..64")
|
||||
proposal = json.loads(args.candidate.read_text())
|
||||
candidate_at_start = args.candidate.read_bytes()
|
||||
proposal = json.loads(candidate_at_start)
|
||||
job_path = args.candidate.parent / "job.json"
|
||||
expected = None
|
||||
if job_path.is_file():
|
||||
@@ -82,6 +88,8 @@ def main():
|
||||
raise ValueError("candidate job identity differs")
|
||||
expected = {target["asset"]: target for target in job["targets"]}
|
||||
rows = proposal_frames(proposal, expected)
|
||||
bindings_at_start = binding_digest(rows)
|
||||
static_cards = any(int(identity, 16) in [0x1002b, 0x1002d, 0x1002f] for identity in rows)
|
||||
manifest = {"format_version": 1, "name": "Isolated candidate review", "description": "Unregistered diagnostic pack", "sequences": proposal["sequences"]}
|
||||
profiles = ["debug", "release"] if args.profile == "auto" else [args.profile]
|
||||
libraries = [library for profile in profiles for library in (project / "target" / profile / "deps").glob("libraptor_replica-*.rlib")]
|
||||
@@ -111,7 +119,10 @@ def main():
|
||||
shutil.copyfile(source, destination)
|
||||
(isolated / "hd.json").write_text(json.dumps(manifest))
|
||||
binary = isolated / "review-bin"
|
||||
subprocess.run(["rustc", "--edition=2021", str(project / "tools/hd_candidate_review.rs"), "--extern", f"raptor_replica={library}", "--extern", f"serde_json={serde}", "-L", f"dependency={dependency_directory}", "-L", "native=" + str(Path.home() / ".local/lib"), "-o", str(binary)], check=True)
|
||||
compiler_args = ["rustc", "--edition=2021", str(project / "tools/hd_candidate_review.rs"), "--extern", f"raptor_replica={library}", "--extern", f"serde_json={serde}", "-L", f"dependency={dependency_directory}", "-L", "native=" + str(Path.home() / ".local/lib"), "-o", str(binary)]
|
||||
if static_cards:
|
||||
compiler_args.extend(["--cfg", "static_card_api"])
|
||||
subprocess.run(compiler_args, check=True)
|
||||
environment = os.environ.copy()
|
||||
environment["LD_LIBRARY_PATH"] = str(Path.home() / ".local/lib") + os.pathsep + environment.get("LD_LIBRARY_PATH", "")
|
||||
subprocess.run([str(binary), str(args.data.resolve()), str(isolated), str(output)], check=True, env=environment)
|
||||
@@ -120,10 +131,17 @@ def main():
|
||||
raise ValueError("artwork changed during runtime review")
|
||||
qa_path = output / "runtime-qa.json"
|
||||
qa = json.loads(qa_path.read_text())
|
||||
candidate_at_finish = args.candidate.read_bytes()
|
||||
if binding_digest(proposal_frames(json.loads(candidate_at_finish), expected)) != bindings_at_start:
|
||||
raise ValueError("candidate renderer bindings changed during runtime review")
|
||||
qa["library"] = str(library.relative_to(project))
|
||||
qa["library_modified_ns"] = library.stat().st_mtime_ns
|
||||
qa["candidate_sha256"] = hashlib.sha256(args.candidate.read_bytes()).hexdigest()
|
||||
qa["candidate_sha256"] = hashlib.sha256(candidate_at_finish).hexdigest()
|
||||
qa["candidate_sha256_at_start"] = hashlib.sha256(candidate_at_start).hexdigest()
|
||||
qa["candidate_notes_changed_during_review"] = candidate_at_start != candidate_at_finish
|
||||
qa["frame_bindings_sha256"] = bindings_at_start
|
||||
qa["artwork_sha256"] = artwork_hashes
|
||||
qa["static_card_palette_mode"] = static_cards
|
||||
qa["source_newer_than_library"] = any((project / "src" / name).stat().st_mtime_ns > library.stat().st_mtime_ns for name in ["assets.rs", "hd.rs"])
|
||||
qa["any_game_source_newer_than_library"] = any(path.stat().st_mtime_ns > library.stat().st_mtime_ns for path in (project / "src").glob("*.rs"))
|
||||
qa_path.write_text(json.dumps(qa, indent=2) + "\n")
|
||||
|
||||
@@ -45,7 +45,14 @@ fn main() {
|
||||
ppm(&output.join(format!("{id:05x}-original.ppm")), &original, &palette);
|
||||
let mut masks = Vec::new(); let mut logical = None;
|
||||
for scale in 1..=4 {
|
||||
let sprite = hd.sprite(id, scale, &palette).unwrap().expect("candidate rejected");
|
||||
let sprite = if palette_name == "PALETTE_DAT" {
|
||||
hd.sprite(id, scale, &palette)
|
||||
} else {
|
||||
#[cfg(static_card_api)]
|
||||
{ hd.sprite_static_palette(id, scale, &palette) }
|
||||
#[cfg(not(static_card_api))]
|
||||
{ panic!("Static-card review requires the source-specific palette API in the selected library") }
|
||||
}.unwrap().expect("candidate rejected");
|
||||
assert_eq!((sprite.width, sprite.height), (frame.logical_width as usize * scale, frame.logical_height as usize * scale));
|
||||
assert!(sprite.pixels.iter().any(|p| *p != TRANSPARENT));
|
||||
ppm(&output.join(format!("{id:05x}-hd-{scale}.ppm")), &sprite, &palette);
|
||||
|
||||
@@ -179,6 +179,8 @@ def operate(directory, action, owner, job=None, lease_id=None, outcome=None, err
|
||||
response.update(changed=False, reason="request_already_started")
|
||||
elif lease.get("phase") != "reserved":
|
||||
response.update(changed=False, reason="legacy_request_requires_recovery_or_finish")
|
||||
elif len(state["leases"]) > CAPACITY:
|
||||
response.update(changed=False, reason="active_request_accounting_over_capacity")
|
||||
elif state.get("admissions_paused"):
|
||||
response.update(changed=False, reason="admissions_paused")
|
||||
else:
|
||||
|
||||
@@ -16,7 +16,7 @@ from pathlib import Path
|
||||
|
||||
from PIL import Image, ImageDraw, ImageFont
|
||||
from hd_art_source import decode
|
||||
from hd_candidate_review import proposal_frames
|
||||
from hd_candidate_review import proposal_frames, binding_digest
|
||||
from hd_art_audit import source_palette_name, checked_palette
|
||||
|
||||
CATEGORY_SLUGS = {
|
||||
@@ -171,7 +171,8 @@ def terrain_context(runtime):
|
||||
samples[asset].append(sample)
|
||||
return {asset: {"roles": sorted(roles[asset]),
|
||||
"state_partners": [f"{item:05x}" for item in sorted(partners[asset])],
|
||||
"neighbors": {direction: [f"{item:05x}" for item in sorted(items)] for direction, items in neighbors[asset].items()},
|
||||
"neighbors": {direction: [f"{item:05x}" for item in sorted(neighbors[asset].get(direction, []))]
|
||||
for direction in ["north", "south", "west", "east"]},
|
||||
"samples": samples[asset]}
|
||||
for asset in set(roles) | set(neighbors)}
|
||||
|
||||
@@ -180,7 +181,7 @@ def contract(family):
|
||||
terrain = family["kind"] == "contextual_terrain_shard"
|
||||
return {
|
||||
"generation": "Use built-in imagegen with inspected original source sheets. Produce newly authored detail, materials and lighting; nearest/ScaleNx enlargement alone is not HD completion.",
|
||||
"submission_gate": "Before each new built-in imagegen request, acquire one of four shared reservations using tools/hd_generation_gate.py acquire --owner YOUR_TASK --job THIS_JOB. Submit only when acquired:true; report the actual outcome and exact service error afterward. Thirty-minute leases are renewable and never terminate existing calls. If denied or cooling down, prepare/review other jobs instead of repeating requests. Do not substitute scaling for a failed remaster.",
|
||||
"submission_gate": "Prepare and inspect sources and save the prompt before acquiring. In one functions.exec orchestration, acquire one of four slots, start the reservation, submit built-in imagegen only when start.changed is true, then classify and report the actual result immediately after await before returning to the model or doing QA. See RustRaptor/docs/hd-generation-workflow.md. Started and legacy-unknown requests require explicit finish; only unstarted reservations expire. If denied or cooling down, prepare/review other jobs. Do not substitute scaling for a failed remaster.",
|
||||
"source_truth": "Keep every source orientation, silhouette, color identity, logical canvas, anchor and explicit source hash. Use each target's source_palette and source_palette_sha256; Apogee/Cygnus/Taiwan cards use their own palettes, not the gameplay palette. Inspect every frame before generating. Never invent poses or drop damaged/destroyed variants.",
|
||||
"ownership": "Only this job's target IDs may be proposed. Neighbor/context references and exact-source aliases are reference-only; never claim or overwrite another job's IDs.",
|
||||
"raw_output": "Keep raw generated PNG bytes, prompts, source references and provenance. Do not retouch generated RGB/alpha. Atlas/square margin crops are allowed only if they preserve all meaningful visible pixels and original aspect.",
|
||||
@@ -258,7 +259,11 @@ def build(args):
|
||||
candidates = {}
|
||||
candidate_error = str(error)
|
||||
coordinator_review = read_json(directory / "coordinator-review.json")
|
||||
if coordinator_review and coordinator_review.get("candidate_sha256") != candidate_digest:
|
||||
review_changed = coordinator_review and (
|
||||
coordinator_review.get("frame_bindings_sha256") != binding_digest(candidates)
|
||||
if coordinator_review.get("frame_bindings_sha256") else
|
||||
coordinator_review.get("candidate_sha256") != candidate_digest)
|
||||
if review_changed:
|
||||
# A corrected proposal needs fresh review; an old approval or hold must
|
||||
# never silently apply to newly submitted artwork.
|
||||
coordinator_review = None
|
||||
@@ -284,8 +289,11 @@ def build(args):
|
||||
renderer["availability"] = "pipeline_pending"
|
||||
if 0x10225 <= asset <= 0x10228 and renderer["availability"] in ["unverified", "pipeline_pending"]:
|
||||
renderer = {"availability": "alpha_shade_mask_path_available", "evidence": ["RustRaptor/src/desktop.rs:pickup glow mask path", "RustRaptor/src/render.rs:Canvas::shade"], "limits": "Shape-alpha and native light table semantics are supported; this adds no remastered mask artwork."}
|
||||
if source["source_palette"] != "PALETTE_DAT" and renderer["availability"] in ["unverified", "pipeline_pending"]:
|
||||
renderer = {"availability": "hd_card_playback_pending", "evidence": ["RustRaptor/docs/intro-validation.md"], "limits": "Classic card playback uses its authentic custom palette and original image. Correctly sourced HD artwork is a separate optional target; registration alone does not switch cinematics or implement a static custom-palette class policy."}
|
||||
if source["source_palette"] != "PALETTE_DAT" and renderer["availability"] in ["unverified", "pipeline_pending", "hd_card_playback_pending"]:
|
||||
opening_card = asset in [0x1002b, 0x1002d]
|
||||
renderer = {"availability": "static_palette_opening_card_path_available" if opening_card else "static_palette_api_available_scene_pending",
|
||||
"evidence": ["RustRaptor/docs/hd-opening-cards.md", "RustRaptor/src/hd.rs:HdAssets::sprite_static_palette"],
|
||||
"limits": "The source-specific palette API supports all256 static colors. Optional Apogee/Cygnus still-card routing is implemented; Taiwan artwork has no verified native optional-HD scene. These code paths add no remastered artwork or actual-art scene coverage."}
|
||||
if prior.get("status") == "renderreachable" and source["hd"]:
|
||||
status = "renderreachable"
|
||||
row = {"kind": "picture", "category": source["category"],
|
||||
@@ -311,8 +319,13 @@ def build(args):
|
||||
"status": status}
|
||||
if canonical[asset] != asset:
|
||||
target["reuse_dependency_job"] = asset_jobs[canonical[asset]]
|
||||
if asset in context:
|
||||
target["terrain_context"] = context[asset]
|
||||
if source["category"] == "Terrain":
|
||||
target["terrain_context"] = context.get(asset, {
|
||||
"roles": [], "state_partners": [],
|
||||
"neighbors": {direction: [] for direction in ["north", "south", "west", "east"]},
|
||||
"samples": [], "registry_or_layout_referenced": False,
|
||||
"limits": "Unused catalog tile: no native mission neighbors or state partners were found. Source/runtime gallery review does not establish a native scene occurrence."
|
||||
})
|
||||
target_rows.append(target)
|
||||
statuses = collections.Counter(row["status"] for row in target_rows)
|
||||
job = {"format_version": 1, "job_id": family["id"], "name": family["name"],
|
||||
|
||||
@@ -0,0 +1,152 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Register explicitly approved, source-bound complete HD families.
|
||||
|
||||
Generation and a successful loader probe never approve artwork. Each job must
|
||||
already have a coordinator-review.json approval bound to its exact frame map,
|
||||
plus complete current all-scale evidence. Raw images are read, never changed.
|
||||
"""
|
||||
import argparse
|
||||
import datetime
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
|
||||
from hd_candidate_review import binding_digest, proposal_frames
|
||||
|
||||
|
||||
def contained(root, relative):
|
||||
path = (root / relative).resolve()
|
||||
if not path.is_relative_to(root.resolve()):
|
||||
raise ValueError(f"path escapes HD root: {relative}")
|
||||
return path
|
||||
|
||||
|
||||
def approved_job(root, identity, project):
|
||||
directory = contained(root, Path("jobs") / identity)
|
||||
if directory.parent != (root / "jobs").resolve():
|
||||
raise ValueError("job must be one direct directory in jobs/")
|
||||
candidate_path = directory / "candidate.json"
|
||||
candidate = json.loads(candidate_path.read_text())
|
||||
job = json.loads((directory / "job.json").read_text())
|
||||
if job["job_id"] != identity or candidate.get("job_id") != identity:
|
||||
raise ValueError(f"job identity differs: {identity}")
|
||||
expected = {target["asset"]: target for target in job["targets"]}
|
||||
rows = proposal_frames(candidate, expected)
|
||||
if not candidate.get("complete_family") or set(rows) != set(expected):
|
||||
raise ValueError(f"incomplete family: {identity}")
|
||||
if any({frame["frame"] for frame in sequence["frames"]} != set(range(sequence["frame_count"])) for sequence in candidate["sequences"]):
|
||||
raise ValueError(f"complete-family claim contains a partial sequence: {identity}")
|
||||
review = json.loads((directory / "coordinator-review.json").read_text())
|
||||
if review.get("status") != "approved":
|
||||
raise ValueError(f"no artistic approval: {identity}")
|
||||
digest = binding_digest(rows)
|
||||
if review.get("frame_bindings_sha256") != digest:
|
||||
raise ValueError(f"artistic approval belongs to another frame map: {identity}")
|
||||
if not review.get("source_and_all_scale_visual_review") or not review.get("independent_review"):
|
||||
raise ValueError(f"missing complete visual/independent review: {identity}")
|
||||
qa_path = contained(root, review["runtime_qa"])
|
||||
qa = json.loads(qa_path.read_text())
|
||||
if qa.get("frame_bindings_sha256") != digest or qa.get("diagnostics"):
|
||||
raise ValueError(f"runtime evidence differs or reports errors: {identity}")
|
||||
if qa.get("source_newer_than_library"):
|
||||
raise ValueError(f"runtime evidence used stale asset/HD code: {identity}")
|
||||
library = (project / qa["library"]).resolve()
|
||||
if not library.is_relative_to(project / "target") or library.stat().st_mtime_ns != qa["library_modified_ns"]:
|
||||
raise ValueError(f"runtime library was replaced after review: {identity}")
|
||||
for name in ["assets.rs", "hd.rs"]:
|
||||
if (project / "src" / name).stat().st_mtime_ns > qa["library_modified_ns"]:
|
||||
raise ValueError(f"asset/HD code changed after review: {identity}")
|
||||
reports = {frame["asset"]: frame for frame in qa["frames"]}
|
||||
if set(reports) != set(rows):
|
||||
raise ValueError(f"runtime evidence omits family IDs: {identity}")
|
||||
for asset, frame in rows.items():
|
||||
report = reports[asset]
|
||||
if report["source_sha256"] != frame["source_sha256"] or report["logical_canvas"] != [frame["logical_width"], frame["logical_height"]]:
|
||||
raise ValueError(f"runtime source/canvas differs: {asset}")
|
||||
if {mask["scale"] for mask in report["hd_masks"]} != {1, 2, 3, 4}:
|
||||
raise ValueError(f"missing presentation scale: {asset}")
|
||||
source_reserved = report["source_mask"]["palette_indices_240_to_255_counts"]
|
||||
if not qa.get("static_card_palette_mode") and not any(source_reserved):
|
||||
if any(any(mask["mask"]["palette_indices_240_to_255_counts"]) for mask in report["hd_masks"]):
|
||||
raise ValueError(f"ordinary source gained animated palette labels: {asset}")
|
||||
artwork = contained(root, frame["file"])
|
||||
if hashlib.sha256(artwork.read_bytes()).hexdigest() != qa["artwork_sha256"][frame["file"]]:
|
||||
raise ValueError(f"raw artwork changed since review: {asset}")
|
||||
return candidate, review, rows
|
||||
|
||||
|
||||
def atomic_json(path, value):
|
||||
descriptor, temporary = tempfile.mkstemp(prefix=path.name + ".", dir=path.parent)
|
||||
try:
|
||||
with os.fdopen(descriptor, "w") as output:
|
||||
json.dump(value, output, indent=2)
|
||||
output.write("\n")
|
||||
output.flush()
|
||||
os.fsync(output.fileno())
|
||||
os.replace(temporary, path)
|
||||
finally:
|
||||
if os.path.exists(temporary):
|
||||
os.unlink(temporary)
|
||||
|
||||
|
||||
def main():
|
||||
project = Path(__file__).resolve().parents[1]
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument("--hd-root", type=Path, default=project.parent / "data/hd")
|
||||
parser.add_argument("--job", action="append", required=True)
|
||||
parser.add_argument("--dry-run", action="store_true")
|
||||
args = parser.parse_args()
|
||||
root = args.hd_root.resolve()
|
||||
manifest_path = root / "hd.json"
|
||||
provenance_path = root / "provenance.json"
|
||||
original_manifest = manifest_path.read_bytes()
|
||||
original_provenance = provenance_path.read_bytes()
|
||||
manifest = json.loads(original_manifest)
|
||||
provenance = json.loads(original_provenance)
|
||||
active = proposal_frames(manifest)
|
||||
additions = []
|
||||
for identity in args.job:
|
||||
candidate, review, rows = approved_job(root, identity, project)
|
||||
overlaps = set(rows) & set(active)
|
||||
if overlaps:
|
||||
if overlaps == set(rows) and all(rows[asset] == active[asset] for asset in rows):
|
||||
continue
|
||||
raise ValueError(f"duplicate registered ID; review a replacement explicitly: {sorted(overlaps)}")
|
||||
manifest["sequences"].extend(candidate["sequences"])
|
||||
active.update(rows)
|
||||
detail = f"jobs/{identity}/provenance.json"
|
||||
if not contained(root, detail).is_file():
|
||||
# Some artists keep generation provenance in their review or
|
||||
# family folder. Do not invent a nonexistent evidence reference.
|
||||
detail = f"jobs/{identity}/candidate.json"
|
||||
additions.append({
|
||||
"job_id": identity,
|
||||
"assets": sorted(rows),
|
||||
"frames": len(rows),
|
||||
"metadata": "hd.json",
|
||||
"detail": detail,
|
||||
"coordinator_review": f"jobs/{identity}/coordinator-review.json",
|
||||
"runtime_qa": review["runtime_qa"],
|
||||
"review": review["source_and_all_scale_visual_review"],
|
||||
"independent_review": review["independent_review"],
|
||||
"frame_bindings_sha256": review["frame_bindings_sha256"],
|
||||
})
|
||||
if additions and not args.dry_run:
|
||||
# Do not overwrite another registration completed during validation.
|
||||
if manifest_path.read_bytes() != original_manifest or provenance_path.read_bytes() != original_provenance:
|
||||
raise ValueError("active pack changed during validation; retry from the new snapshot")
|
||||
provenance.setdefault("reviewed_sequence_additions", []).extend(additions)
|
||||
provenance["registered_frame_count"] = len(active)
|
||||
provenance["status"] = f"{len(active)} complete-family reviewed original picture frames registered. Whole-catalog queues remain active; HD stays optional. Render reachability is recorded separately."
|
||||
provenance["registration_updated_at_utc"] = datetime.datetime.now(datetime.timezone.utc).isoformat()
|
||||
# Runtime uses hd.json alone. Write its ancillary evidence first, then
|
||||
# replace the complete manifest in one atomic rename.
|
||||
atomic_json(provenance_path, provenance)
|
||||
atomic_json(manifest_path, manifest)
|
||||
print(json.dumps({"dry_run": args.dry_run, "jobs_added": [row["job_id"] for row in additions], "frames_added": sum(row["frames"] for row in additions), "registered_frames": len(active)}))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
Reference in new issue
Block a user