st-grok46

kaggle-spaceship-titanic · worker grok:grok-4.6 (xhigh) · finished (patience)

打开交互式搜索树 · REPORT.md · INSIGHTS.md · DEADENDS.md · rsi.toml

叙述

本次实验面向 Kaggle Spaceship Titanic，以官方验证准确率 score（越高越好）为优化目标。搜索在约 37 分 47 秒后因耐心阈值停止，共探索 16 个节点量级的树（宽度根 3、宽度 2、深度 4、并行 2）。根节点基线为 0.7717；最终最优节点是 n008，官方分为 0.8189，已略高于公开榜参考水平 solid_public_lb 0.81，held-out kaggle 种子集上最优为 0.8034。

真正拉开差距的是第一层从逻辑回归换到 LightGBM，并加入舱位甲板/编号/舷侧、团体规模与消费相关特征：n001 官方分直接到 0.8177。机制上，树模型能吃到甲板、舷侧、冷冻睡眠与消费长尾的非线性交互；同时把“有消费则不可能处于 CryoSleep”这类硬约束写进特征，也能理顺训练里约百行的矛盾。后续路径几乎都在这条 LightGBM 主干上打补丁：团体内缺失填充与甲板→星球映射（n004）、姓氏频率与舱位占用（n006），再到 n008 把剩余零消费的 CryoSleep 缺失填为 True，并加入奢侈消费占比与餐饮购物占比。官方验证上 n008 相对父节点 n006 的 0.8166 仅再抬约 0.0023，仍落在噪声带内，但成为整棵搜索里最高的官方分；worker 快速集上为 0.80748，相对未改动父节点快速分 0.80417 约 +0.00331，且五折全升。

未站住的方向同样清晰。团体运输率先验与固定概率混合（n003）在按团体划分、训练/测试团体不重叠时，预测端会塌成全局均值，无法把组内相关带到未见团体。全局姓氏计数容易跨团体干扰舱位信号；舱位占用与团体规模高度冗余。宽泛的位置填充、组内共享标志、更深 HGB 或线性模型，要么官方不升，要么只在 worker 快速集上有小幅抖动却带不走验证。同思路的花费占比在另一条支路（n009）甚至会因与已有 log1p 消费列共线而失效，说明特征价值依赖整条特征栈，而非单点技巧。

相对 0.81 的公开参考，当前最优已略超；相对更强公共方案仍可能只是小数点后第三位的空间。较有希望的下一步是：在保留 n008 冷冻填充与花费占比的前提下剔除易过拟合的整数 GroupId；用组内同姓计数替换全局 LastNameSize；并消融 LastNameSize/CabinOccupancy，确认它们是否只是在拖累已接近天花板的 LightGBM 栈。

结果

最佳路径

• root 0.7717 — baseline：Baseline starter code. Official score: 0.7717.
• n001 0.8177 — Gradient boosting + cabin/group features：Switched from logistic regression to LightGBM with cabin deck/num/side, group id/size, categoricals, log1p spends, Age median, and CryoSleep-from-spend. Official validation score 0.81771 (n=1739) vs parent 0.7717 (+0.046). Worker quick-set: 0.80331 vs baseline quick 0.76335.
• n004 0.8137 — Group-consistent missing imputation：Added per-frame unique group-mate fill for HomePlanet/Cabin/Destination plus hard deck→planet map (A/B/C/T→Europa, G→Earth) in LightGBM features. Official validation score 0.81369 (n=1739, sem=0.00934) vs parent 0.8177 (Δ−0.00401). Worker's quick-set: 0.80331→0.80732.
• n006 0.8166 — Surname and cabin occupancy：Added per-frame LastNameSize and CabinOccupancy to LightGBM features. Official validation score 0.81656 (n=1739, sem=0.00928) vs parent 0.8137; |Δ|=0.00286 within noise. Worker's quick-set: 0.80417 vs untouched parent 0.80732 (Δ=−0.00315).
• n008 0.8189 — CryoSleep remaining NA and spend mix：Added CryoSleep NA→True when spend=0 plus LuxuryShare and FoodShopShare in features.py; LightGBM unchanged. Official validation score 0.81886 (n=1739, sem=0.00924) vs parent 0.8166 (Δ+0.00226, within no_change band). Worker quick-set: 0.80748 vs untouched parent 0.80417 (+0.00331).

所有节点

Insights（审计员从各节点核实后记录）

• GBDT on cabin/group + spend features beats linear spend/age/CryoSleep because trees capture non-linear deck/side/cryo and spend-tail interactions. [n001, score 0.8177]
• Forcing CryoSleep=False when any spend>0 removes a real train contradiction (~100 rows) and clarifies the cryo–transport signal. [n001, score 0.8177]
• GroupSize and Deck/Side are strong tabular levers on this task; fixed 500 trees at lr=0.05 already landed in the hypothesized 0.80–0.81 band. [n001, score 0.8177]
• CryoSleep=True always has zero amenity spend, and missing CryoSleep splits by spend like awake vs cryo, so spend-based Cryo imputation aligns features with a hard domain rule. [n002, score 0.8125]
• Under the same cryo/one-hot features, logistic regression gains little (worker quick +0.006) while depth-3 HGB captures spend×planet/deck interactions and drives most of the lift. [n002, score 0.8125]
• Worker same-fold probes show cryo rules add only ~0.001–0.004 once HGB is used on baseline columns, so the booster—not imputation alone—explains most of the jump. [n002, score 0.8125]
• With group-aware CV and zero train/test group overlap, LOO group label priors collapse to the global mean at predict, so they cannot transfer the within-group outcome dependence they encode on train. [n003, score 0.8016]
• Most of the jump from worker quick-set 0.76 to 0.80 came from LightGBM plus GroupSize and group-filled planet/cabin location features, not from GroupTransportedMean. [n003, score 0.8016]
• A fixed predict-time blend of row probabilities toward the in-batch group mean is the only label-free way to use outcome correlation on unseen groups, but fixed weights 0.30/0.50 stayed inside fold noise. [n003, score 0.8016]
• Unique-only group fill plus hard A/B/C/T/G deck→planet cuts HomePlanet NA a lot (worker: 152→49) but ~55% singletons only get the deck map, so location NA-fill has limited leverage. [n004, score 0.8137]
• A worker quick-set lift of +0.004 below fold std (~0.008) did not carry to official validation—treat small quick gains on location imputation as noise unless validation moves. [n004, score 0.8137]
• Destination is mixed in many multi-groups; unique-only Destination fill barely moves score (worker ablation −0.00058) and is not the driver. [n004, score 0.8137]
• Global surname frequency can cancel a useful cabin signal because LightGBM spends splits on name counts that span unrelated groups (corr with GroupSize ~0.35). [n005, score 0.8097]
• CabinOccupancy ≠ GroupSize on many rows (groups split across cabins / missing Cabin), so room-sharing is a distinct co-travel cue from booking-group size. [n005, score 0.8097]
• Global surname frequency is a weak family proxy because many last names span multiple passenger groups, so the count mostly ranks common names and can raise fold variance. [n006, score 0.8166]
• CabinOccupancy is near-redundant with GroupSize (high correlation; occupancy never exceeds group size), so LightGBM gains little from the extra integer once GroupSize is present. [n006, score 0.8166]
• Sharing a last name inside the same group is a clearer family signal than global LastNameSize and was not tested here. [n006, score 0.8166]
• Within-group surname counts add little once GroupSize exists: FamilyInGroup equals GroupSize on most rows (worker: 84.9%, corr 0.830), so trees mostly relearn group size. [n007, score 0.8148]
• A small worker quick lift vs n006 can come from dropping LastNameSize/CabinOccupancy, not from FamilyInGroup itself (worker ablation without it: quick 0.80732 > 0.80603). [n007, score 0.8148]
• Spend composition shares can beat raw amenity magnitudes as signals: luxury share corr with Transported was −0.446 vs −0.275 for raw Spa+VRDeck, and LightGBM used both new shares (ranks 6 and 8). [n008, score 0.8189]
• Combining a small domain fill with composition features can stabilize folds even when the fill alone is weak: cryo-fill-only quick mixed folds/higher std, shares-only had two folds down, both together lifted all five. [n008, score 0.8189]
• Spend-mix shares can rank high in LightGBM splits yet still fail to raise accuracy when five log1p amenity columns already encode the same structure (collinearity). [n009, score 0.8108]
• After spend>0→CryoSleep=False, remaining zero-spend CryoSleep NA looks cryo-like on average but overlaps many awake non-spenders (esp. children), so a blanket True fill stays near noise. [n009, score 0.8108]
• Quick-set gains from cabin side and within-group sharing (~+0.008 worker quick) can fail to transfer to official validation when fold variance also rises (worker quick std 0.00745→0.00909), so treat share flags as provisional until validation confirms. [n010, score 0.8079]
• For depth-3 HGB, cabin side appears to carry most of the quick lift; GroupSize/IsAlone add little once Side is present (worker ablations). [n010, score 0.8079]
• Within-group sharing flags without target encoding still risk acting as noise on held-out validation, similar to prior group-fill/surname dead ends. [n010, score 0.8079]
• Luxury vs essential spend is a real axis (LuxurySpend dominates tree splits), but depth-3 HGB already recovered most of it from the five raw logs, so accuracy barely moves. [n011, score 0.8068]
• IsChild is a true domain fact (Age<13 never spend) but is redundant for shallow HGB once Age and ZeroSpend exist—ablations matched sums-only fold-for-fold. [n011, score 0.8068]
• Linear models harvest the composition (+0.017 worker LR) because they cannot recombine five logs; HGB still wins overall, so shipping LR for this alone is not worth it. [n011, score 0.8068]
• Integer GroupId is largely redundant with CabinNum (worker corr 0.681); after removal, LightGBM shifts splits onto CabinNum and other existing numerics, so accuracy stays flat. [n012, score 0.8154]
• Consecutive gggg groups do not share Deck/Side above chance (worker), so sequential GroupId is not a cabin-block proxy worth keeping as a numeric split feature. [n012, score 0.8154]
• Unique group-mate fill plus A/B/C/T/G→planet cuts location NA a lot (worker: HomePlanet 152→49, Side 164→80) but does not reliably raise LightGBM accuracy once GroupSize/CabinOccupancy exist, because ~55% of rows are singletons and NA itself is already a usable split. [n014, score 0.8148]
• Recomputing CabinOccupancy after filling Cabin changes many rows (worker: 226) and was the weakest occupancy timing on quick folds—reconstructed roommates are a noisier sharing signal than raw missing-cabin occupancy=1. [n014, score 0.8148]
• Leftover F-deck HomePlanet NA is Earth/Mars-separable by spend mix (Mars luxury ~0.15 vs Earth ~0.40; zeros mostly Mars), and filling only those 33 train rows moved official validation +0.00749 despite near-noise quick delta—small targeted location fills can still help holdout. [n013, score 0.8183]
• Spend-mix planet guesses largely duplicate amenity axes LightGBM already sees, so the fill’s ceiling is tiny (33/6954 rows) and can leak Transported signal via HomePlanet. [n013, score 0.8183]
• Unique last-name→planet among known F has perfect group-LOO planet accuracy but still spans unrelated groups, so it is a bad imputation feature even when planet labels look pure. [n013, score 0.8183]

Dead ends

• LogisticRegression on cryo-imputed + one-hot features: worker quick 0.76910 vs 0.76335; C tuning did not help. [n002]
• Raw vs log1p spends under HGB both scored worker quick 0.79655. [n002]
• Depth-4 HGB reached worker quick 0.80061 but raised fold std (0.010 vs 0.007) and was not shipped. [n002]
• LOO GroupTransportedMean / GroupOtherTransportedCount as tree features: worker local ablation 0.80073 (on) vs 0.80163 (off); no reliable gain because predict values are constant. [n003]
• Fixed group-proba blend strength 0.30 or 0.50: worker |delta| ≤ 0.002 vs no blend on the same folds. [n003]
• Group-consistent HomePlanet/Cabin/Destination fill + deck→planet map: official 0.81369 vs parent 0.8177 (Δ−0.00401 within no_change band); does not beat parent on validation. [n004]
• Hard-mapping deck F to Earth was correctly avoided; remaining HomePlanet NA are mostly F (Earth vs Mars), so majority guess is not an almost-hard rule. [n004]
• Frame-wide SurnameFreq with CabinOccupancy: worker quick both 0.80374 ≈ parent quick 0.80331; surname alone mixed folds—do not retry global surname counts. [n005]
• CabinOccupancy alone lifted worker quick (+0.00487) but official validation 0.80966 did not beat parent 0.8177. [n005]
• LastNameSize+CabinOccupancy together: worker quick 0.80417 vs parent 0.80732 (three folds down); official validation flat vs parent. [n006]
• CabinOccupancy alone: worker quick 0.80460 with all five folds down vs parent. [n006]
• LastNameSize alone: worker quick 0.80534 with fold-0 −0.01278 and higher fold std (0.01351 vs parent 0.00812). [n006]
• Integer FamilyInGroup on LightGBM: official 0.81484 vs parent 0.8166 (|Δ|<0.01085); worker's quick also below n004-like ablation (0.80603 vs 0.80732). [n007]
• Cryo remaining-NA→True alone: worker quick 0.80562 vs parent 0.80417 (+0.00145) with mixed folds and std 0.01274—extra cryo rules add little once spend features exist. [n008]
• Adding RoomService share is redundant: on spenders it equals 1−LuxuryShare−FoodShopShare (residual ~2e-16). [n008]
• Cryo zero-spend True fill + continuous Luxury/FoodShopping shares: official 0.81081 vs parent 0.8137; worker quick 0.80718 vs 0.80732. [n009]
• Continuous LuxuryShare+FoodShoppingShare alone: worker same-fold 0.80503 vs parent-like 0.80732 (Δ −0.00229). [n009]
• Full GroupSize+Side+SharesLastName+Shared{HomePlanet,Destination,Deck,CryoSleep} set: official validation 0.80794 vs parent 0.8125 (no meaningful improvement). [n010]
• Worker side-only quick 0.80304 missed their 0.007 bar vs parent quick 0.79655, so shipping side alone was not justified even on quick. [n010]
• Keeping five raw amenity logs plus the two sums (sums+raw): worker quick 0.79918 with higher fold std; collinear and not better than sums-only. [n011]
• Adding IsChild or AgeBin on top of Age/ZeroSpend under depth-3 HGB: identical folds to the matching spend config. [n011]
• Dropping TotalSpend once Luxury+Essential exist: worker quick 0.79859, slightly worse than keeping it. [n011]
• Ablating GroupId alone does not improve the official score beyond noise (0.81541 vs parent 0.8108, |Δ| 0.00461 < 0.009201); worker quick also flat (+0.00043). [n012]
• Shipped stack (unique HomePlanet/Cabin/Destination fill + deck→planet + occupancy-after-fill): worker quick 0.80460 vs parent 0.80818 (4/5 down); official 0.81484 stays within |Δ|<0.008682 of parent 0.8097—no meaningful gain to retry. [n014]
• Fill with occupancy counted on raw Cabin still lost on quick (worker 0.80530); skipping Destination fill (0.80617) or dropping occupancy under fill (0.80732) also stayed below the occupancy-only parent. [n014]
• Unique F-deck last-name→planet fill: worker same-fold quick 0.80475 vs parent quick 0.80718 (Δ −0.00243). [n013]
• Last-name map plus spend-mix LR: official quick 0.80660 vs parent quick 0.80718 (Δ −0.00058); last name was the drag. [n013]