✨ Frontier UI

ChatGPT-style generative UI for Open WebUI — real photos, live street maps with routes, place cards, itineraries, charts and follow-ups, rendered right inside the chat.

Open WebUI answers in markdown. ChatGPT answers with an interface: photos of the places it mentions, a map with the route, cards you can scroll, and next questions you can tap. Frontier UI brings that to any model in Open WebUI.

The model writes one small JSON spec. The tool does the rest on the server — finds real photos, verifies places on the map and draws real street routes — then renders a polished, theme-aware card inline in the conversation.

<sub>Follow-up in the same chat: a live map around Union Station with pinned picks and photo cards.


A 7-day Cancún trip: day-by-day routes on a self-hosted planet map, budget sliders, live weather, link previews — then “build me a spreadsheet”.</sub>

1. Workspace → Tools → + in Open WebUI, paste frontier_ui.py and save.
2. Workspace → Models → your model → Tools: enable Frontier UI. Turn off other generative-UI tools so the model has one clear choice. Native function calling works best.
3. Ask for something visual — “Plan a day in San Francisco”, “Good Italian near Union Station under $120 for two”, “Compare the Arc Pro B70 and RTX 4090 for local LLMs”.

Per-user settings (chat › Tools › Frontier UI ⚙︎): units and map_app, so a European user can pick metric even if the admin default is imperial.

Photo sources. Wikipedia and Wikimedia Commons are great for landmarks, cities and topics; Openverse adds CC-licensed photos. Small businesses (a specific café) often have no free photo — the card then shows a tidy placeholder. For better coverage there, add SearXNG (free, self-hosted), a Brave Search key, a Google Programmable Search key + engine id, or Pexels / Unsplash for generic scenes. Bing's image API was retired in 2025, so it isn't offered. Attribution (author · licence · source) is kept on every photo.

Opening map / Gmail / Calendar links. Open WebUI renders tool cards in a sandboxed iframe without allow-popups-to-escape-sandbox, so a new tab opened from the card inherits the sandbox. Google Maps, Gmail, Calendar, Apple Maps and OpenStreetMap all send Cross-Origin-Opener-Policy, so Chromium shows ERR_BLOCKED_BY_RESPONSE (the same URL works in a fresh tab). Frontier therefore:

1. opens links through the Open WebUI window when Settings › Interface › iframe Sandbox Allow Same Origin is on — a normal tab;
2. opens directly on phones/tablets (the OS hands the link to the Maps / Gmail app) and for Open WebUI's own downloads;
3. otherwise copies the link and shows a small sheet with Copy link / Try opening anyway.

Turning on Allow Same Origin is an admin decision: it relaxes the sandbox for every embed, not just Frontier UI.

Requirements: Open WebUI ≥ 0.6 (tested on 0.12.5). Everything is keyless: the server needs outbound HTTPS to Wikipedia / Wikimedia Commons, Openverse, Photon / Nominatim, OSRM / FOSSGIS, Open-Meteo and Yahoo Finance; the browser loads map tiles and photos. Interactive maps need WebGL; without it (blocklisted GPU, hardened Firefox profiles, some VMs) maps fall back to a static sketch. The add-ons below are optional.

ArtCraft apps supported (all 16): WordCraft · GridCraft · DeckCraft · PdfCraft · PhotoCraft · LightCraft · VectorCraft · DesignCraft · FigCraft · FilmCraft · EffectCraft · SoundCraft · CADCraft · SolidCraft · TextureCraft · Craft Anything.

model ──► render_frontier_ui(spec JSON)
            │
            ├─ photos   ─► Wikipedia · Commons · Openverse  (+ SearXNG / API keys if set; ranked, cached)
            ├─ places   ─► Wikipedia coords / Photon / Nominatim  (snaps the model's estimate)
            ├─ routes   ─► FOSSGIS foot/bike/car · OSRM car (with fallbacks)
            ▼
         sandboxed HTML card  ─►  MapLibre map · galleries · cards · charts · follow-ups

{"title":"San Francisco in one day",
 "blocks":[
  {"type":"text","md":"I'd plan a **9 AM–7 PM route** along the waterfront [1]."},
  {"type":"images","items":[{"query":"Golden Gate Bridge from Baker Beach"},{"query":"Lombard Street"},{"query":"Palace of Fine Arts"}]},
  {"type":"map","area":"San Francisco, CA","route":true,"places":[
    {"name":"Ferry Building","lat":37.7955,"lng":-122.3937,"icon":"⛴️","group":"Morning","time":"9:00"},
    {"name":"Pier 39","lat":37.8087,"lng":-122.4098,"icon":"🦭","group":"Afternoon","time":"12:30"}]},
  {"type":"timeline","items":[{"time":"9:00 AM","title":"Ferry Building","place":"Ferry Building"}]}],
 "sources":[{"title":"Fleet Week schedule","url":"https://fleetweeksf.org/schedule/"}],
 "followups":["Make it a two-day itinerary"]}

The tool ships as one file, built from readable parts in src/: p1, p2, p6 are the Python resolvers, p3–p5 and p7 the HTML / CSS / JS renderer, p9 the tool class. See CHANGELOG.md.

./src/build.sh   # rebuilds frontier_ui.py

• Open WebUI shows a tool result when it completes, so cards appear whole rather than streaming token by token.
• Public map, routing and photo services have fair-use limits; results are cached and the tool falls back to the model's coordinates when geocoders are busy.

Inspired by ChatGPT's Intelligent UI and thesysdev/open-intelligent-ui (MIT), whose map palette and routing approach this follows. Map data © OpenStreetMap contributors · tiles © OpenFreeMap / OpenMapTiles.