floor-planner/benchmarks/README.md
Andras Schmelczer 8bf3e3b705
Some checks failed
Deploy to Pages / build (push) Failing after 10m45s
Simplify
2026-09-11 23:27:29 +01:00

147 lines
9.2 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# Optimiser measurements
Run `npm run benchmark` for **Minimise cuts**, or `npm run benchmark -- --random` for
**Maximise randomness**. Both are compared with the original stock-first optimiser's
frozen results in `baseline.json`. The 84 inputs in `cases.ts` cover rectangular
remainders, fireplaces in both directions, room-order permutations, tight and exact
stagger limits, fractional stock, large floors, and both supplied angled rooms.
`--output=/tmp/results.json` saves a run; `--compare=/tmp/results.json` compares
against a previous iteration. Do not overwrite the baseline to hide regressions.
`h-pattern-baseline.json` separately freezes both options immediately before the
visual H-pattern correction, evaluated using the broader alignment metric below.
## Fixed metrics
- **Saw passes:** crosscuts actually required by the allocation, long-edge rips,
and additional shaping edges. Skirting includes bevel and coping operations.
- **Cut difficulty:** shaping passes, then rip passes, used to break equal saw-count
ties. This is an explicit proxy, not an estimate of minutes or tool-specific effort.
- **Legacy repetition:** the original penalty, evaluated independently
from the output rows. It penalises nearby repeating joints, similar consecutive
steps and staircases, including steps that wrap across a board length. Raw repeat,
similar-step and staircase counts are also reported. This stays unchanged for
historical comparisons, but its 3%-of-board-length alignment tolerance missed visible H patterns.
- **Near-H row pairs:** joints two rows apart separated by less than one board width,
capped at one sixth of the board length. This catches the 40–150 mm misalignments
that still look like an H. Current appearance cost is the legacy penalty plus
12 per near-H pair, so avoiding an H outweighs several same-direction steps.
- **Waste:** purchased stock minus installed material, including kerf. The report
gives board counts and waste percentages weighted by purchased stock.
- **Validity and runtime:** errors, elapsed time, p95 and maximum case time. Tests
separately verify coverage, non-overlapping row bands, minimum sizes, stagger,
factory ends, rounded stock consumption, cut accounting, references and input purity.
Minimise cuts orders `(passes, shaping, rips, appearance cost, boards)`.
Maximise randomness orders `(appearance cost, passes, shaping, rips, boards)`.
Validity precedes either objective. Skirting orders `(passes, waste)`.
## Measured results
Totals across the same 84 cases, September 2026:
| Metric | Original | Minimise cuts | Maximise randomness |
| ------------------- | -------: | ------------: | ------------------: |
| Flooring saw passes | 2,177 | **1,741** | **2,122** |
| Shaping passes | 192 | 184 | 188 |
| Rip passes | 414 | 402 | 411 |
| Legacy penalty | 265 | 2,279 | 350 |
| Flooring boards | 3,561 | 3,548 | 3,550 |
| Flooring waste | 8.20% | 7.85% | 7.91% |
| Invalid cases | 0 | 0 | 0 |
Minimise cuts saves **20.0%** of saw passes. Maximise randomness saves **2.5%** of saw
passes against the original planner. For random flooring in the default
4,800 × 3,600 mm room, 38 passes become **24** or **31**, respectively; both options use 76
boards and have approximately 1.2% waste.
Skirting drops from **1,447 to 1,437 passes**. It uses 522 boards instead of 513:
cutting effort now takes precedence over waste, which rises from 9.28% to 10.85%.
Seven cases improve and 77 tie.
For flooring, 78 cut-priority cases improve and six tie against the original planner.
The bounded search does not establish global optimality.
## H-pattern correction
The first optimisation reported a 57% lower legacy penalty, but visual inspection
showed that the metric was too narrow. The default appearance-priority room still
had **11 near-H row pairs**, despite no repeats within the old 36 mm tolerance.
The candidate hash also generated almost identical offsets for adjacent candidate
numbers. A final bit-mixing step now spreads those candidates across the allowed range.
| Metric | Cuts before | Cuts now | Random before | Random now |
| ------------------------- | ----------: | -------: | ------------: | ---------: |
| Near-H row pairs | 515 | 269 | 557 | **6** |
| Saw passes | 1,741 | 1,741 | 1,994 | 2,122 |
| Boards | 3,537 | 3,548 | 3,579 | 3,550 |
| Waste | 7.57% | 7.85% | 8.66% | 7.91% |
| Default-room near-H pairs | 17 | 17 | 11 | **0** |
| Default-room saw passes | 24 | 24 | 30 | 31 |
Appearance priority removes **98.9%** of near-H pairs for **6.4%** more cuts than the
previous iteration. All six remaining pairs are in `forced-alternation`, whose
600 mm minimum pieces force alternating starts. It improves 79 cases and ties five
under the corrected objective. Ten additional room seeds also produce zero near-H pairs.
The tradeoff is explicit: the old direction-run/staircase score increases, and the
randomness option uses more cuts. Cuts-first retains its total saw count but three
individual cases use extra cuts (`two-rooms-reversed-6`, `angled`, `angled-f8`).
It can still show H patterns when avoiding them requires extra cuts; the UI explains
this beside the laying-pattern choice. Both frozen baselines remain unchanged.
## Iterations retained
1. Correct the objective order; measuring stock first contradicted the requested priorities.
2. Deduplicate the complete relevant state, including recent joint history. Stop
computing keys once the beam is full; share immutable exhausted stocks and avoid
searching offcuts for pieces requiring both factory ends.
3. Repack completed boundary blanks, reserving exact complementary factory ends
first. Preserve the original placement and cutting order within each source board.
4. Polish all surviving completed candidates, retaining the best result.
5. Preserve end remnants during search instead of trimming them immediately.
This alone improves 56 cut-priority cases and saves another 99 passes after the
earlier improvements. Final packing can still combine non-exact blanks.
6. When repetition remains, compare an immediate-reuse search as well. This improves
five appearance-priority cases. All searches share their clipping cache, and the
two UI choices select from the completed results. Selection itself does not rerun
the optimiser.
7. Add the board-scale H penalty and mix the offset hash. Search size and geometry
constraints stay unchanged. Before/after drawings were checked for the default
rectangle, fireplace and large floor, alongside the numeric metrics.
## Experiments rejected
Increasing beam widths from 5/2 to 12/4 roughly tripled search time, with regressions
in both modes. Carrying two room results into the next room worsened two appearance
cases for a single waste improvement. Expanding the offcut candidate pool with
distinct remnants had no cut-priority benefit and regressed a large-room result,
including after the end-preservation change. These variants were removed.
The original single-result benchmark took about 1.5 seconds for the corpus. Producing
both choices, including the conditional extra appearance search, takes about 3 seconds
on the same machine; p95 is around 110 ms and the most complex case around 0.7 seconds.
These are indicative timings, not CI thresholds. Ordinary option changes reuse the
already calculated plans. A separate 500-row, 2,000-board boundary check completes
without an error; temporary stock is repacked before enforcing the board limit.
Tests enforce aggregate cutting budgets, broad H avoidance and physical invariants.
The original metrics and cases remain fixed; the new H metric has its own frozen
baseline so the correction does not rewrite the earlier results.
## Simplification tradeoffs
Removing row-swap polishing preserves all 1,741 cuts-first passes. Across the 84
cases, appearance-first passes change from 2,113 to 2,122 (+0.43%), boards from 3,551
to 3,550, and the legacy penalty from 346 to 350. Cuts-first near-H pairs change
from 247 to 269; appearance-first remains at six, all in the forced-alternation case.
Removing the exact three-piece skirting packing variant changes 1,436 passes to
1,437 and 521 boards to 522. Both frozen historical baselines remain unchanged.
The conditional immediate-reuse search stays: removing it introduced a near-H pair
outside the forced-alternation case. Distinct room-label and cut-preview placement
strategies also stay; their bounds and collision primitives are shared. There is
not enough evidence to replace these strategies without losing useful behavior.
Tests retain physical invariants, the two option priorities, near-H avoidance,
and aggregate budgets relative to the original baseline (at least 19% fewer
cuts-first passes; appearance-first and skirting use fewer passes than originally).
They no longer freeze the exact totals of a particular heuristic iteration.