ArchitectureRegression★ Featured
2-Layer Regression Transformer
embed → 2× attn → scalar head
✓PFN Studio Team
Smallest model that consistently solves linear-regression-shape priors. Good first pass for any 1D scalar-output task.
Architecture
tabular_embedder (d=64)transformer_encoder × 2scalar_head