Machine Learning & Color Pipeline

Text-to-Palette Benchmark & Local AI Architecture

How we designed an in-browser neural color pipeline that runs entirely on client devices, supports English and Russian prompts, and avoids the common pitfalls of direct neural color regression.

1. The Direct Regression Failure

In early prototypes, we trained a lightweight MLP regression head directly on text embeddings to predict OKLCH coordinates. This produced a well-known machine learning failure mode: regression to the mean.

The "Black → Muddy Brown" Problem:

Because training losses minimize squared error across thousands of scene descriptions, prompt inputs for extreme colors like "black" or "white" were continually pulled toward mid-lightness, mid-chroma averages (generating dark muddy brown instead of true neutral black).

2. Semantic Anchor Architecture

To guarantee high-quality, art-direction-friendly palettes without massive server-side models, we decoupled semantic interpretation from color math:

  1. Multilingual Embedding: The user's prompt is embedded locally using a compact multilingual model (ONNX WebAssembly) into a dense 384-dimensional vector space.
  2. Semantic Anchor Mapping: The embedding is matched against curated semantic anchors representing specific moods, materials, and lighting states.
  3. Literal Color Extraction: Explicit color tokens ("red", "синий", "emerald") have deterministic boundary guarantees.
  4. Perceptual Harmony Generation: The resolved base color is expanded into a full palette (Shadow, Base, Highlight, Accent) using OKLCH perceptual harmony and sRGB gamut fitting.

3. Semantic Quality Benchmark

Our automated test suite evaluates prompt-to-palette accuracy across 4 independent test suites:

Direct Colors

PASS · 100%

Literal color names in English and Russian (e.g. "black", "чёрный", "cyan", "золотой"). Tests strict neutrality and hue bounds.

Semantic Concepts

PASS · 95%+

Complex atmospheric scene descriptions ("winter forest", "rusty factory at sunset", "deep sea horror"). Tests mood-to-color mapping.

Synonym Pairs & Cross-Lingual Parity

PASS · 95%+

Equivalence across languages ("lava" vs "лава", "snow" vs "снег", "cyberpunk" vs "киберпанк"). Tests vector alignment.

Out-of-Distribution (OOD) Prompts

PASS · 90%+

Novel metaphors and abstract game concepts ("poisoned moonlight", "digital decay"). Tests graceful generalization.

4. In-Browser Privacy & Performance

• Zero Remote API Calls: No prompts or generated palettes are sent to external AI servers (no OpenAI, Gemini, or remote endpoints).

• Lazy Loading: Model assets are only downloaded when the user actively triggers generation — browsing pages and viewing palettes does not download model files.

• Offline Capable: Once cached, inference runs locally in browser memory using ONNX Runtime Web.

Try the AI Generator

Type any scene or idea and see the semantic mapping pipeline in real time.

AI Palette Generator